Data warehousing Books

Book SynopsisA comprehensive guide to everything scientists need to know about data management, this book is essential for researchers who need to learn how to organize, document and take care of their own data. Researchers in all disciplines are faced with the challenge of managing the growing amounts of digital data that are the foundation of their research. Kristin Briney offers practical advice and clearly explains policies and principles, in an accessible and in-depth text that will allow researchers to understand and achieve the goal of better research data management. Data Management for Researchers includes sections on: * The data problem – an introduction to the growing importance and challenges of using digital data in research. Covers both the inherent problems with managing digital information, as well as how the research landscape is changing to give more value to research datasets and code. * The data lifecycle – a framework for data’s place within the research process and how data’s role is changing. Greater emphasis on data sharing and data reuse will not only change the way we conduct research but also how we manage research data. * Planning for data management – covers the many aspects of data management and how to put them together in a data management plan. This section also includes sample data management plans. * Documenting your data – an often overlooked part of the data management process, but one that is critical to good management; data without documentation are frequently unusable. * Organizing your data – explains how to keep your data in order using organizational systems and file naming conventions. This section also covers using a database to organize and analyze content. * Improving data analysis – covers managing information through the analysis process. This section starts by comparing the management of raw and analyzed data and then describes ways to make analysis easier, such as spreadsheet best practices. It also examines practices for research code, including version control systems. * Managing secure and private data – many researchers are dealing with data that require extra security. This section outlines what data falls into this category and some of the policies that apply, before addressing the best practices for keeping data secure. * Short-term storage – deals with the practical matters of storage and backup and covers the many options available. This section also goes through the best practices to insure that data are not lost. * Preserving and archiving your data – digital data can have a long life if properly cared for. This section covers managing data in the long term including choosing good file formats and media, as well as determining who will manage the data after the end of the project. * Sharing/publishing your data – addresses how to make data sharing across research groups easier, as well as how and why to publicly share data. This section covers intellectual property and licenses for datasets, before ending with the altmetrics that measure the impact of publicly shared data. * Reusing data – as more data are shared, it becomes possible to use outside data in your research. This chapter discusses strategies for finding datasets and lays out how to cite data once you have found it. This book is designed for active scientific researchers but it is useful for anyone who wants to get more from their data: academics, educators, professionals or anyone who teaches data management, sharing and preservation. "An excellent practical treatise on the art and practice of data management, this book is essential to any researcher, regardless of subject or discipline." —Robert Buntrock, Chemical Information BulletinTrade ReviewApparently, NASA lost much of the early data from space exploration, including high quality video footage of the first moon landing. All the more reason to do as it says in the sub-title to the book. -- Alan Crowden * BES Bulletin *For researchers and consumers of data who are often fraught with managing excess information, Briney's book offers valuable techniques, strategies and standards to help achieve proficient data management and successful outcomes. This book can be useful to both novice researchers and well-established scientists alike. -- Mary F. Miles * Medical Reference Services Quarterly *... recommended as a textbook for graduate-level research techniques courses. It's an important resource for academic and special library shelves and a vital reference for anyone working with data. -- Kristen LaBonte * Issues in Science and Technology Librarianship *Briney has written a useful primer on data management for researchers which provides practical advice throughout on managing data. It is easy to read and clearly structured. http://www.ariadne.ac.uk/issue75/cole -- Gareth Cole, Loughborough University Library * Ariadne *Kristin Briney’s Data Management for Researchers is a book that should be on the shelf (physical or virtual) of every librarian, researcher and research administrator. Scientists, engineers, social scientists, humanists — anyone who’s work involves generating and keeping track of digital data. This is the book for you. .... I recommend this book without hesitation for all academic libraries. Individual researchers, research administrators, funding agency employees and academic librarians would all find much useful information. Simply giving a copy to new graduate students is probably a worthwhile investment at any institution. http://scienceblogs.com/confessions/2016/01/11/reading-diary-data-management-for-researchers-organize-maintain-and-share-your-data-for-research-success-by-kristin-briney/ -- John Dupuis, York University Library, Toronto * ScienceBlogs *Briney takes the reader through a pragmatic and sensible route through the activities of data management. -- David Bawden, AlexandriaThis practical handbook can help bring new researchers quickly up-to-speed on the topic, as well as serve as a reference to meet specific data management needs they encounter throughout the data life cycle. * Enid Karr, Journal of eScience Librarianship *I cannot recommend this slender, seemingly innocent looking book enough – it will literally change how you think about data management. -- CBatt[I] intend to give a copy of this book to each graduate student / trainee that joins my lab -- Christie Bahlai, Practical Data Management for Bug Counters

Read more less

Book SynopsisNearly 80 recipes to help you collect and transform data from multiple sources into a single data source, making it way easier to perform analytics on the dataKey Features Build data pipelines from scratch and find solutions to common data engineering problems Learn how to work with Azure Data Factory, Data Lake, Databricks, and Synapse Analytics Monitor and maintain your data engineering pipelines using Log Analytics, Azure Monitor, and Azure Purview Book DescriptionThe famous quote 'Data is the new oil' seems more true every day as the key to most organizations' long-term success lies in extracting insights from raw data. One of the major challenges organizations face in leveraging value out of data is building performant data engineering pipelines for data visualization, ingestion, storage, and processing. This second edition of the immensely successful book by Ahmad Osama brings to you several recent enhancements in Azure data engineering and shares approximately 80 useful recipes covering common scenarios in building data engineering pipelines in Microsoft Azure.You'll explore recipes from Azure Synapse Analytics workspaces Gen 2 and get to grips with Synapse Spark pools, SQL Serverless pools, Synapse integration pipelines, and Synapse data flows. You'll also understand Synapse SQL Pool optimization techniques in this second edition. Besides Synapse enhancements, you'll discover helpful tips on managing Azure SQL Database and learn about security, high availability, and performance monitoring. Finally, the book takes you through overall data engineering pipeline management, focusing on monitoring using Log Analytics and tracking data lineage using Azure Purview.By the end of this book, you'll be able to build superior data engineering pipelines along with having an invaluable go-to guide.What you will learn Process data using Azure Databricks and Azure Synapse Analytics Perform data transformation using Azure Synapse data flows Perform common administrative tasks in Azure SQL Database Build effective Synapse SQL pools which can be consumed by Power BI Monitor Synapse SQL and Spark pools using Log Analytics Track data lineage using Microsoft Purview integration with pipelines Who this book is forThis book is for data engineers, data architects, database administrators, and data professionals who want to get well versed with the Azure data services for building data pipelines. Basic understanding of cloud and data engineering concepts will help in getting the most out of this book.Table of ContentsTable of Contents Creating and Managing Data in Azure Data Lake Securing and Monitoring Data in Azure Data Lake Building Data Ingestion Pipelines Using Azure Data Factory Azure Data Factory Integration Runtime Configuring and Securing Azure SQL Database Implementing High Availability and Monitoring in Azure SQL Database Processing Data Using Azure Databricks Processing Data Using Azure Synapse Analytics Transforming Data Using Azure Synapse Dataflows Building the Serving Layer in Azure Synapse SQL Pool Monitoring Synapse SQL and Spark Pools Optimizing and Maintaining Synapse SQL and Spark Pools Monitoring and Maintaining Azure Data Engineering Pipelines

Read more less

Book SynopsisLeverage BigQuery to understand and prepare your data to ensure that it's accurate, reliable, and ready for analysis and modeling Key Features Use mock datasets to explore data with the BigQuery web UI, bq CLI, and BigQuery API in the Cloud console Master optimization techniques for storage and query performance in BigQuery Engage with case studies on data exploration and preparation for advertising, transportation, and customer support data Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionData professionals encounter a multitude of challenges such as handling large volumes of data, dealing with data silos, and the lack of appropriate tools. Datasets often arrive in different conditions and formats, demanding considerable time from analysts, engineers, and scientists to process and uncover insights. The complexity of the data life cycle often hinders teams and organizations from extracting the desired value from their data assets. Data Exploration and Preparation with BigQuery offers a holistic solution to these challenges. The book begins with the basics of BigQuery while covering the fundamentals of data exploration and preparation. It then progresses to demonstrate how to use BigQuery for these tasks and explores the array of big data tools at your disposal within the Google Cloud ecosystem. The book doesn’t merely offer theoretical insights; it’s a hands-on companion that walks you through properly structuring your tables for query efficiency and ensures adherence to data preparation best practices. You’ll also learn when to use Dataflow, BigQuery, and Dataprep for ETL and ELT workflows. The book will skillfully guide you through various case studies, demonstrating how BigQuery can be used to solve real-world data problems. By the end of this book, you’ll have mastered the use of SQL to explore and prepare datasets in BigQuery, unlocking deeper insights from data.What you will learn Assess the quality of a dataset and learn best practices for data cleansing Prepare data for analysis, visualization, and machine learning Explore approaches to data visualization in BigQuery Apply acquired knowledge to real-life scenarios and design patterns Set up and organize BigQuery resources Use SQL and other tools to navigate datasets Implement best practices to query BigQuery datasets Gain proficiency in using data preparation tools, techniques, and strategies Who this book is forThis book is for data analysts seeking to enhance their data exploration and preparation skills using BigQuery. It guides anyone using BigQuery as a data warehouse to extract business insights from large datasets. A basic understanding of SQL, reporting, data modeling, and transformations will assist with understanding the topics covered in this book.Table of ContentsTable of Contents Introducing BigQuery and Its Components BigQuery Organization and Design Exploring Data in BigQuery Loading and Transforming Data Querying BigQuery Data Exploring Data with Notebooks Further Exploring and Visualizing Data An Overview of Data Preparation Tools Cleansing and Transforming Data Best Practices for Data Preparation, Optimization, and Cost Control Hands-On Exercise – Analyzing Advertising Data Hands-On Exercise Analyzing Transportation Data Hands-On Exercise – Analyzing Customer Support Data Summary and Future Directions

Read more less

Book SynopsisEverything you need to know to apply data contracts and build a truly data-driven organization that harnesses quality data to deliver tangible business value Purchase of the print or Kindle book includes a free PDF eBook Key Features Understand data contracts and their power to resolving the problems in contemporary data platforms Learn how to design and implement a cutting-edge data platform powered by data contracts Access practical guidance from the pioneer of data contracts to get expert insights on effective utilization Book DescriptionDespite the passage of time and the evolution of technology and architecture, the challenges we face in building data platforms persist. Our data often remains unreliable, lacks trust, and fails to deliver the promised value. With Driving Data Quality with Data Contracts, you’ll discover the potential of data contracts to transform how you build your data platforms, finally overcoming these enduring problems. You’ll learn how establishing contracts as the interface allows you to explicitly assign responsibility and accountability of the data to those who know it best—the data generators—and give them the autonomy to generate and manage data as required. The book will show you how data contracts ensure that consumers get quality data with clearly defined expectations, enabling them to build on that data with confidence to deliver valuable analytics, performant ML models, and trusted data-driven products. By the end of this book, you’ll have gained a comprehensive understanding of how data contracts can revolutionize your organization’s data culture and provide a competitive advantage by unlocking the real value within your data.What you will learn Gain insights into the intricacies and shortcomings of today’s data architectures Understand exactly how data contracts can solve prevalent data challenges Drive a fundamental transformation of your data culture by implementing data contracts Discover what goes into a data contract and why it’s important Design a modern data architecture that leverages the power of data contracts Explore sample implementations to get practical knowledge of using data contracts Embrace best practices for the successful deployment of data contracts Who this book is forIf you’re a data engineer, data leader, architect, or practitioner thinking about your data architecture and looking to design one that enables your organization to get the most value from your data, this book is for you. Additionally, staff engineers, product managers, and software engineering leaders and executives will also find valuable insights.Table of ContentsTable of Contents A Brief History of Data Platforms Introducing Data Contracts How to Get Adoption in Your Organization Bringing Data Consumers and Generators Closer Together Embedding Data Governance What Makes Up a Data Contract A Contract-Driven Data Architecture A Sample Implementation Implementing Data Contracts In Your Organization Data Contracts in Practice

Read more less

Book SynopsisThis book constitutes the refereed proceedings of the 8th EAI International Conference on Game Theory for Networks, GameNets 2019, held in Paris, France, in April 2019. The 8 full and 3 short papers presented were carefully reviewed and selected from 17 submissions. They are organized in the following topical sections: Game Theory for Wireless Networks; Games for Economy and Resource Allocation; and Game Theory for Social Networks.Table of ContentsGame Theory for Wireless Networks.- Games for Economy and Resource Allocation.- Game Theory for Social Networks.

Read more less

Book SynopsisThis book constitutes the thoroughly refereed proceedings of the 16th Italian Research Conference on Digital Libraries, IRCDL 2020, held in Bari, Italy, in January 2020.The 12 full papers and 6 short papers presented were carefully selected from 26 submissions. The papers are organized in topical sections on information retrieval, bid data and data science in DL; cultural heritage; open science. Table of ContentsInformation Retrieval.- Bid Data and Data Science in DL.- Cultural Heritage.- Open Science.

Read more less

Book SynopsisThis book constitutes the refereed proceedings of the 27th China Conference on Information Retrieval, CCIR 2021, held in Dalian, China, in October 2021.The 15 full papers presented were carefully reviewed and selected from 124 submissions. The papers are organized in topical sections: search and recommendation, NLP for IR, IR in Education, and IR in Biomedicine.Table of ContentsSearch and Recommendation.- NLP for IR.- IR in Education.- IR in Biomedicine.

Read more less

Book SynopsisThis SpringerBrief provides the first systematic review of the existing works of cohesive subgraph search (CSS) over large heterogeneous information networks (HINs). It also covers the research breakthroughs of this area, including models, algorithms and comparison studies in recent years. This SpringerBrief offers a list of promising future research directions of performing CSS over large HINs.The authors first classify the existing works of CSS over HINs according to the classic cohesiveness metrics such as core, truss, clique, connectivity, density, etc., and then extensively review the specific models and their corresponding search solutions in each group. Note that since the bipartite network is a special case of HINs, all the models developed for general HINs can be directly applied to bipartite networks, but the models customized for bipartite networks may not be easily extended for other general HINs due to their restricted settings. The authors also analyze and compare these cohesive subgraph models (CSMs) and solutions systematically. Specifically, the authors compare different groups of CSMs and analyze both their similarities and differences, from multiple perspectives such as cohesiveness constraints, shared properties, and computational efficiency. Then, for the CSMs in each group, the authors further analyze and compare their model properties and high-level algorithm ideas.This SpringerBrief targets researchers, professors, engineers and graduate students, who are working in the areas of graph data management and graph mining. Undergraduate students who are majoring in computer science, databases, data and knowledge engineering, and data science will also want to read this SpringerBrief.Table of Contents1. Introduction2. Preliminaries3. CSS on Bipartite Networks4. CSS on Other General HINs5. Comparison Analysis6. Related Work on CSMs and solutions7. Future Work and Conclusion

Read more less

Book SynopsisWith the proliferation of huge amounts of (heterogeneous) data on the Web, the importance of information retrieval (IR) has grown considerably over the last few years. Big players in the computer industry, such as Google, Microsoft and Yahoo!, are the primary contributors of technology for fast access to Web-based information; and searching capabilities are now integrated into most information systems, ranging from business management software and customer relationship systems to social networks and mobile phone applications.Ceri and his co-authors aim at taking their readers from the foundations of modern information retrieval to the most advanced challenges of Web IR. To this end, their book is divided into three parts. The first part addresses the principles of IR and provides a systematic and compact description of basic information retrieval techniques (including binary, vector space and probabilistic models as well as natural language search processing) before focusing on its application to the Web. Part two addresses the foundational aspects of Web IR by discussing the general architecture of search engines (with a focus on the crawling and indexing processes), describing link analysis methods (specifically Page Rank and HITS), addressing recommendation and diversification, and finally presenting advertising in search (the main source of revenues for search engines). The third and final part describes advanced aspects of Web search, each chapter providing a self-contained, up-to-date survey on current Web research directions. Topics in this part include meta-search and multi-domain search, semantic search, search in the context of multimedia data, and crowd search.The book is ideally suited to courses on information retrieval, as it covers all Web-independent foundational aspects. Its presentation is self-contained and does not require prior background knowledge. It can also be used in the context of classic courses on data management, allowing the instructor to cover both structured and unstructured data in various formats. Its classroom use is facilitated by a set of slides, which can be downloaded from www.search-computing.org.Trade ReviewFrom the reviews:“The book covers not only a wide range, but everything that is essential to the topic of Web information retrieval. … this book is an excellent starting point into the field of Web information retrieval, and can be recommended for classroom use.” (Gottfried Vossen, zbMATH, Vol. 1283, 2014)“... this book is a valuable resource for students and instructors in web IR, primarily as a reference to supplement course teaching. Researchers and practitioners should find the book a useful quick reference guide for key concepts, techniques, and recent trends in web IR.” (Wingyan Chung, ACM Computing Reviews, July 2014)Table of ContentsPart I Principles of Information Retrieval.- An Introduction to Information Retrieval.- The Information Retrieval Process.- Information Retrieval Models.- Classification and Clustering.- Natural Language Processing for Search.- Part II Information Retrieval for the Web.- Search Engines.- Link Analysis.- Recommendation and Diversification for the Web.- Advertising in Search.- Part III Advanced Aspects of Web Search.- Publishing Data on the Web.- Meta-Search and Multi-Domain Search.- Semantic Search.- Multimedia Search.- Search Process and Interfaces.- Human Computation and Crowd Search.

Read more less

Book SynopsisWith the proliferation of huge amounts of (heterogeneous) data on the Web, the importance of information retrieval (IR) has grown considerably over the last few years. Big players in the computer industry, such as Google, Microsoft and Yahoo!, are the primary contributors of technology for fast access to Web-based information; and searching capabilities are now integrated into most information systems, ranging from business management software and customer relationship systems to social networks and mobile phone applications.Ceri and his co-authors aim at taking their readers from the foundations of modern information retrieval to the most advanced challenges of Web IR. To this end, their book is divided into three parts. The first part addresses the principles of IR and provides a systematic and compact description of basic information retrieval techniques (including binary, vector space and probabilistic models as well as natural language search processing) before focusing on its application to the Web. Part two addresses the foundational aspects of Web IR by discussing the general architecture of search engines (with a focus on the crawling and indexing processes), describing link analysis methods (specifically Page Rank and HITS), addressing recommendation and diversification, and finally presenting advertising in search (the main source of revenues for search engines). The third and final part describes advanced aspects of Web search, each chapter providing a self-contained, up-to-date survey on current Web research directions. Topics in this part include meta-search and multi-domain search, semantic search, search in the context of multimedia data, and crowd search.The book is ideally suited to courses on information retrieval, as it covers all Web-independent foundational aspects. Its presentation is self-contained and does not require prior background knowledge. It can also be used in the context of classic courses on data management, allowing the instructor to cover both structured and unstructured data in various formats. Its classroom use is facilitated by a set of slides, which can be downloaded from www.search-computing.org.Trade ReviewFrom the reviews:“The book covers not only a wide range, but everything that is essential to the topic of Web information retrieval. … this book is an excellent starting point into the field of Web information retrieval, and can be recommended for classroom use.” (Gottfried Vossen, zbMATH, Vol. 1283, 2014)“... this book is a valuable resource for students and instructors in web IR, primarily as a reference to supplement course teaching. Researchers and practitioners should find the book a useful quick reference guide for key concepts, techniques, and recent trends in web IR.” (Wingyan Chung, ACM Computing Reviews, July 2014)Table of ContentsPart I Principles of Information Retrieval.- An Introduction to Information Retrieval.- The Information Retrieval Process.- Information Retrieval Models.- Classification and Clustering.- Natural Language Processing for Search.- Part II Information Retrieval for the Web.- Search Engines.- Link Analysis.- Recommendation and Diversification for the Web.- Advertising in Search.- Part III Advanced Aspects of Web Search.- Publishing Data on the Web.- Meta-Search and Multi-Domain Search.- Semantic Search.- Multimedia Search.- Search Process and Interfaces.- Human Computation and Crowd Search.

Read more less