Data mining Books

Book SynopsisIn the second edition of this practical guide, you'll learn how to conduct analytics on data where it lives, whether it's a data lake using Hive, a modern lakehouse with Iceberg or Delta Lake, a different system like Cassandra, Kafka, or SingleStore, or a relational database like PostgreSQL or Oracle.

Read more less

Book SynopsisA holistic and real-world approach to operationalizing artificial intelligence in your company InOperating AI, Director of Technology and Architecture at Ericsson AB, Ulrika Jägare, delivers an eye-opening new discussion of how to introduce your organization to artificial intelligence by balancing data engineering, model development, and AI operations. You'll learn the importance of embracing an AI operational mindset to successfully operate AI and lead AI initiatives through the entire lifecycle, including keyareas such as; data mesh, data fabric,aspects ofsecurity,data privacy,data rights and IPR related to data and AI models. In the book, you'll also discover: How to reduce the risk of entering bias in our artificial intelligence solutions and how to approach explainable AI (XAI)The importance of efficient and reproduceable data pipelines, including how to manage your company's dataAn operational perspective on the development of AI models using the MLOps (Machine Learning Operations) approach, including how to deploy, run and monitor models and ML pipelines in production using CI/CD/CT techniques, that generates value in the real worldKey competences and toolsets in AI development, deployment and operationsWhat to consider when operating different types of AI business models With a strong emphasis on deployment and operations of trustworthy and reliable AI solutions that operate well in the real worldand not just the labOperating AIis a must-read for business leaders looking for ways to operationalize an AI business model that actually makes money, from the concept phase to running in a live production environment.Table of ContentsForeword xii Introduction xv Chapter 1 Balancing the AI Investment 1 Defining AI and Related Concepts 3 Operational Readiness and Why It Matters 8 Applying an Operational Mind- set from the Start 12 The Operational Challenge 15 Strategy, People, and Technology Considerations 19 Strategic Success Factors in Operating AI 20 People and Mind- sets 23 The Technology Perspective 28 Chapter 2 Data Engineering Focused on AI 31 Know Your Data 32 Know the Data Structure 32 Know the Data Records 34 Know the Business Data Oddities 35 Know the Data Origin 36 Know the Data Collection Scope 37 The Data Pipeline 38 Types of Data Pipeline Solutions 41 Data Quality in Data Pipelines 44 The Data Quality Approach in AI/ML 45 Scaling Data for AI 49 Key Capabilities for Scaling Data 51 Introducing a Data Mesh 53 When You Have No Data 55 The Role of a Data Fabric 56 Why a Data Fabric Matters in AI/ML 58 Key Competences and Skillsets in Data Engineering 60 Chapter 3 Embracing MLOps 71 MLOps as a Concept 72 From ML Models to ML Pipelines 76 The ML Pipeline 78 Adopt a Continuous Learning Approach 84 The Maturity of Your AI/ML Capability 86 Level 0— Model Focus and No MLOps 88 Level 1— Pipelines Rather than Models 89 Level 2— Leveraging Continuous Learning 90 The Model Training Environment 91 Enabling ML Experimentation 92 Using a Simulator for Model Training 94 Environmental Impact of Training AI Models 96 Considering the AI/ML Functional Technology Stack 97 Key Competences and Toolsets in MLOps 103 Clarifying Similarities and Differences 106 MLOps Toolsets 107 Chapter 4 Deployment with AI Operations in Mind 115 Model Serving in Practice 117 Feature Stores 118 Deploying, Serving, and Inferencing Models at Scale 121 The ML Inference Pipeline 123 Model Serving Architecture Components 125 Considerations Regarding Toolsets for Model Serving 129 The Industrialization of AI 129 The Importance of a Cultural Shift 139 Chapter 5 Operating AI Is Different from Operating Software 143 Model Monitoring 144 Ensuring Efficient ML Model Monitoring 145 Model Scoring in Production 146 Retraining in Production Using Continuous Training 151 Data Aspects Related to Model Retraining 155 Understanding Different Retraining Techniques 156 Deployment after Retraining 159 Disadvantages of Retraining Models Frequently 159 Diagnosing and Managing Model Performance Issues in Operations 161 Issues with Data Processing 162 Issues with Data Schema Change 163 Data Loss at the Source 165 Models Are Broken Upstream 166 Monitoring Data Quality and Integrity 167 Monitoring the Model Calls 167 Monitoring the Data Schema 168 Detecting Any Missing Data 168 Validating the Feature Values 169 Monitor the Feature Processing 170 Model Monitoring for Stakeholders 171 Ensuring Stakeholder Collaboration for Model Success 173 Toolsets for Model Monitoring in Production 175 Chapter 6 AI Is All About Trust 181 Anonymizing Data 182 Data Anonymization Techniques 185 Pros and Cons of Data Anonymization 187 Explainable AI 189 Complex AI Models Are Harder to Understand 190 What Is Interpretability? 191 The Need for Interpretability in Different Phases 192 Reducing Bias in Practice 194 Rights to the Data and AI Models 199 Data Ownership 200 Who Owns What in a Trained AI Model? 202 Balancing the IP Approach for AI Models 205 The Role of AI Model Training 206 Addressing IP Ownership in AI Results 207 Legal Aspects of AI Techniques 208 Operational Governance of Data and AI 210 Chapter 7 Achieving Business Value from AI 215 The Challenge of Leveraging Value from AI 216 Productivity 216 Reliability 217 Risk 218 People 219 Top Management and AI Business Realization 219 Measuring AI Business Value 223 Measuring AI Value in Nonrevenue Terms 227 Operating Different AI Business Models 229 Operating Artificial Intelligence as a Service 230 Operating Embedded AI Solutions 236 Operating a Hybrid AI Business Model 239 Index 241

Read more less

Book Synopsis Human attention is in the highest demand it has ever been. The drastic increase in available information has compelled individuals to find a way to sift through the media that is literally at their fingertips. Content recommendation systems have emerged as the technological solution to this social and informational problem, but they''ve also created a bigger crisis in confirming our biases by showing us only, and exactly, what it predicts we want to see.   Data versus Democracy investigates and explores how, in the era of social media, human cognition, algorithmic recommendation systems, and human psychology are all working together to reinforce (and exaggerate) human bias. The dangerous confluence of these factors is driving media narratives, influencing opinions, and possibly changing election results.  In this book, algorithmic recommendations, clickbait, familiarity bias, propaganda, Trade Review“A very well written book that has an engaging style of writing, doesn’t become dry or bogged down in the details, but still showcases the depth of knowledge that Shaffer has on the subject. … It’s accessible and it provides a satisfying read to those looking for deep analysis of this emerging problem faced by the world.” (The Robotics Law Journal, Vol. 5 (2), September - October, 2019) Table of ContentsPart I: The Propaganda Problem.- Chapter 1: Pay Attention: How Information Abundance Affects the Way We Consume Media .- Chapter 2: Cog in the System: How the Limits of Our Brains Leave Us Vulnerable to Cognitive Hacking.- Chapter 3: Swimming Upstream: How Content Recommendation Engines Impact Information and Manipulate Our Attention.- Part II: Case Studies.- Chapter 4: Domestic Disturbance: Ferguson, GamerGate, and the Rise of the American Alt-Right.- Chapter 5: Democracy Hacked, Part 1: Russian Interference and the New Cold War .- Chapter 6: Democracy Hacked, Part 2: Rumors, Bots, and Genocide in the Global South .- Chapter 7: Conclusion: Where Do We Go from Here?.-

Read more less

a huge range and FREE tracked UK delivery on ALL orders.

Read more less

Book SynopsisWith this in-depth book, current and aspiring engineers will learn powerful, real-world best practices for managing data big and small. Contributors from notable companies including Twitter, Google, StitchFix, Microsoft and Capital One share their experiences and lessons learned for overcoming a variety of specific and often nagging challenges.

Read more less

Book SynopsisMost economists agree that AI is a general purpose technology (GPT) like the steam engine, electricity, and the computer. AI will drive innovation in all sectors of the economy for the foreseeable future. Practical AI for Business Leaders, Product Managers, and Entrepreneurs is a technical guidebook for the business leader or anyone responsible for leading AI-related initiatives in their organization. The book can also be used as a foundation to explore the ethical implications of AI. Authors Alfred Essa and Shirin Mojarad provide a gentle introduction to foundational topics in AI. Each topic is framed as a triad: concept, theory, and practice. The concept chapters develop the intuition, culminating in a practical case study. The theory chapters reveal the underlying technical machinery. The practice chapters provide code in Python to implement the models discussed in the case study. With this book, readers will learn: The technical foundations of machine learning and deep leaTable of Contents Introduction What is AI and why it is at the center of major business transformation? How is it related to machine learning? What is deep learning, and how is it related to ML? Why is it important? How the book is organized Who is the audience? Section 1: Machine Learning　Chapter 1.1, introduction, machine learning, different types of machine learning　 Chapter 1.2,　Machine Learning Technical Overview　 Chapter 1.3, Hands-On Machine Learning with Scikit Learn Chapter 1.4,　　Advanced Topics/flavors of Machine learning Appendix: mathematical interlude Section 2: Deep Learning　 Chapter 2.1, introduction (what is it, why is it important) Chapter 2.2,　Deep Learning Technical Overview　 Chapter 2.3, Hands-On Deep Learning with Keras Chapter 2.4,　　Advanced Topics/flavors of deep learning Appendix: mathematical interlude Section 3: Putting AI into Practice: Innovation Framework Chapter 3.1: Diffusion and Dynamics of Innovation Chapter 3.2: Managing an Innovation Portfolio

Read more less

Book SynopsisGraph Databases in Action teaches readers everything they need to know to begin building and running applications powered by graph databases. Right off the bat, seasoned graph database experts introduce readers to just enough graph theory, the graph database ecosystem, and a variety of datastores. They also explore modelling basics in action with real-world examples, then go hands-on with querying, coding traversals, parsing results, and other essential tasks as readers build their own graph-backed social network app complete with a recommendation engine! Key Features · Graph database fundamentals · An overview of the graph database ecosystem · Relational vs. graph database modelling · Querying graphs using Gremlin · Real-world common graph use cases For readers with basic Java and application development skills building in RDBMS systems such as Oracle, SQL Server, MySQL, and Postgres. No experience with graph databases is required. About the technology Graph databases store interconnected data in a more natural form, making them superior tools for representing data with rich relationships. Unlike in relational database management systems (RDBMS), where a more rigid view of data connections results in the loss of valuable insights, in graph databases, data connections are first priority. Dave Bechberger has extensive experience using graph databases as a product architect and a consultant. He’s spent his career leveraging cutting-edge technologies to build software in complex data domains such as bioinformatics, oil and gas, and supply chain management. He’s an active member of the graph community and has presented on a wide variety of graph-related topics at national and international conferences. Josh Perryman is technologist with over two decades of diverse experience building and maintaining complex systems, including high performance computing (HPC) environments. Since 2014 he has focused on graph databases, especially in distributed or big data environments, and he regularly blogs and speaks at conferences about graph databases.

Read more less

Book SynopsisThe vast amounts of ontologically unstructured information on the Web, including HTML, XML and JSON documents, natural language documents, tweets, blogs, markups, and even structured documents like CSV tables, all contain useful knowledge that can present a tremendous advantage to the Artificial Intelligence community if extracted robustly, efficiently and semi-automatically as knowledge graphs. Domain-specific Knowledge Graph Construction (KGC) is an active research area that has recently witnessed impressive advances due to machine learning techniques like deep neural networks and word embeddings. This book will synthesize Knowledge Graph Construction over Web Data in an engaging and accessible manner. The book describes a timely topic for both early -and mid-career researchers. Every year, more papers continue to be published on knowledge graph construction, especially for difficult Web domains. This book serves as a useful reference, as well as an accessible but rigorous overview of this body of work. The book presents interdisciplinary connections when possible to engage researchers looking for new ideas or synergies. The book also appeals to practitioners in industry and data scientists since it has chapters on both data collection, as well as a chapter on querying and off-the-shelf implementations.Table of Contents1. What is a knowledge graph?.- 2. Information Extraction.- 3. Entity Resolution.- 4. Advanced Topic: Knowledge Graph Completion.- 5. Ecosystems

Read more less

Book SynopsisThis book constitutes the refereed proceedings of the 8th International Conference on Evolutionary Computation in Combinatorial Optimization, EvoMUSART 2019, held in Leipzig, Germany, in April 2019, co-located with the Evo*2019 events EuroGP, EvoCOP and EvoApplications. The 16 revised full papers presented were carefully reviewed and selected from 24 submissions. The papers cover a wide range of topics and application areas, including: visual art and music generation, analysis, and interpretation; sound synthesis; architecture; video; poetry; design; and other creative tasks.Table of ContentsDeep Learning Concepts for Evolutionary Art.- Adversarial Evolution and Deep Learning – How Does An Artist Play with Our Visual System.- Autonomy, Authenticity, Authorship and Intention in Computer Generated Art.- Camera Obscurer: Generative Art for Design Inspiration.- Swarm-Based Identification of Animation Key Points from 2D-medialness Maps.- Paintings, Polygons and Plant Propagation.- Evolutionary Games for Audiovisual Works: Exploring the Demographic Prisoner's Dilemma.- Emojinating: Evolving Emoji Blends.- Automatically Generating Engaging Presentation Slide Decks.- Tired of choosing? Just Add Structure and Virtual Reality.- EvoChef: Show Me What to Cook! Artificial Evolution of Culinary Arts.- Comparing Models for Harmony Prediction in An Interactive Audio Looper.- Stochastic Synthesizer Patch Exploration in Edisyn.- Evolutionary Multi-Objective Training Set Selection of Data Instances and Augmentations for Vocal Detection.- Automatic Jazz Melody Composition Through a Learning-Based Genetic Algorithm.- Exploring Transfer Functions in Evolved CTRNNs for Music Generation.

Read more less

Book SynopsisThis book constitutes the workshop proceedings of the 24th International Conference on Database Systems for Advanced Applications, DASFAA 2019, held in Chiang Mai, Thailand, in April 2019. The 14 full papers presented were carefully selected and reviewed from 26 submissions to the three following workshops: the 6th International Workshop on Big Data Management and Service, BDMS 2019; the 4th International Workshop on Big Data Quality Management, BDQM 2019; and the Third International Workshop on Graph Data Management and Analysis, GDMA 2019. This volume also includes the short papers, demo papers, and tutorial papers of the main conference DASFAA 2019.Table of ContentsThe 6th International Workshop on Big Data Management and Service (BDSM 2019).- A Probabilistic Approach for Inferring Latent Entity Associations in Textual Web Contents.- UHRP Uncertainty-Based Pruning Method for Anonymized Data Linear Regression.- Meta-path based MiRNA-disease Association Prediction.- Medical Question Retrieval based on Siamese Neural Network and Transfer learning method.- An adaptive Kalman filter based Ocean Wave Prediction Model using Motion Reference Unit Data.- ASLM: Adaptive Single Layer Model for Learned Index.- SparseMAAC: Sparse Attention for Multi-Agent Reinforcement Learning.- The 4th International Workshop on Big Data Quality Management (BDQM 2019).- Identifying Reference Relationship of Desktop Files Based on Access Logs.- Visualization of Photo Album: Selecting a Representative Photo of a Specific Event.- Data Quality Management in Institutional Research Output Data Center.- Generalized Bayesian Structure Learning from Noisy Datasets.- The Third International Workshop on Graph Data Management and Analysis (GDMA 2019).- ANDMC: An Algorithm for Author Name Disambiguation Based on Molecular Cross Clustering.- Graph Based Aspect Extraction and Rating Classification of Customer Review Data.- Streaming Massive Electric Power Data Analysis Based on Spark Streaming.- Short Papers.- Deletion Robust k-Coverage Queries.- Episodic Memory Network with Self-Attention for Emotion Detection.- Detecting Suicidal Ideation with Data Protection in Online Communities.- Hierarchical Conceptual Labeling.- Anomaly Detection in Time-Evolving Attributed Networks.- A Multi-task Learning Framework for Automatic Early Detection of Alzheimer’s.- Top-k Spatial Keyword Query with Typicality and Semantics.- Align Reviews with Topics in Attention Network for Rating Prediction.- PSMSP: A Parallelized Sampling-based Approach for Mining Top-k Sequential Patterns in Database Graphs.- Value-Oriented Ranking of Online Reviews Based on Reviewer-influenced Graph.- Ancient Chinese Landscape Painting Composition Classification by Using Semantic Variational Autoencoder.- Learning Time-Aware Distributed Representations of Locations from Spatio-Temporal Trajectories.- Hyper2vec: Biased Random Walk for Hyper-Network Embedding.- Privacy-preserving and dynamic spatial range aggregation query processing in wireless sensor networks.- Adversarial Discriminative Denoising for Distant Supervision Relation Extraction.- Nonnegative Spectral Clustering for Large-Scale Semi-Supervised Learning.- Distributed PARAFAC Decomposition Method based on In-Memory Big Data System.- GPU-Accelerated Dynamic Graph Coloring.- Relevance-based Entity Embedding.- An Iterative Map-Trajectory Co-Optimisation Framework Based on Map-Matching and Map Update.- Exploring Regularity in Traditional Chinese Medicine Clinical Data Using Heterogeneous Weighted Networks Embedding.- AGREE: Attentive Tour Group Recommendation with Multi-Modal Data.- Random Decision DAG: An Entropy Based Compression Approach for Random Forest.- Generating Behavior Features for Cold-Start Spam Review Detection.- TCL: Tensor-CNN-LSTM for Travel Time Prediction with Sparse Trajectory Data.- A Semi-supervised Classification Approach for Multiple Time-varying Networks with Total Variation.- Multidimensional Skylines Over Streaming Data.- A domain adaptation approach for multistream classification.- Gradient Boosting Censored Regression for Winning Price Prediction in Real-Time Bidding.- Deep Sequential Multi-task Modeling for Next Check-in Time and Location Prediction.- SemiSync: Semi-supervised Clustering by Synchronization.- Neural Review Rating Prediction with Hierarchical Attentions and Latent Factors.- MVS-match: An Efficient Subsequence Matching Approach Based on the Series Synopsis.- Temporal-Spatial Recommendation for On-demand Cinemas.- Finding the key influences on the house price by Finite Mixture Model based on the real estate data in Changchun.- Semi-supervised Clustering with Deep Metric Learning.- Spatial Bottleneck Minimum Task Assignment with Time-delay.- A Mimic Learning Method for Disease Risk Prediction with Incomplete Initial Data.- Hospitalization Behavior Prediction Based on Attention and Time Adjustment Factors in Bidirectional LSTM.- Modeling Item Category for Effective Recommendation.- Distributed Reachability Queries on Massive Graphs.- Edge-Based Shortest Path Caching in Road Networks.- Extracting Definitions and Hypernyms with a Two-Phase Framework.- Tag Recommendation by Word-Level Tag Sequence Modeling.- A New Statistics Collecting Method with Adaptive Strategy.- Word Sense Disambiguation with Massive Contextual Texts.- Learning DMEs from Positive and Negative Examples.- Serial and Parallel Recurrent Convolutional Neural Networks for Biomedical Named Entity Recognition.- DRGAN: A GAN-based Framework for Doctor Recommendation in Chinese On-line QA Communities.- Attention-based Abnormal-Aware Fusion Network for Radiology Report Generation.- LearningTour: A Machine Learning Approach for Tour Recommendation based on Users’ Historical Travel Experience.- TF-Miner: Topic-specific Facet Mining by Label Propagation.- Fast Raft Replication for Transactional Database Systems over Unreliable Networks.- Parallelizing Big De Bruijn Graph Traversal for Genome Assembly on GPU Clusters.- GScan: Exploiting Sequential Scans for Subgraph Matching.- SIMD Accelerates the Probe Phase of Star Joins in Main Memory Databases.- A Deep Recommendation Model Incorporating Adaptive Knowledge-based Representations.- BLOMA: Explain Collaborative Filtering via Boosted Local Rank-One Matrix Approximation.- Spatiotemporal-Aware Region Recommendation with Deep Metric Learning.- On the Impact of the Length of Subword Vectors on Word Embeddings.- Using Dilated Residual Network to Model Distant Supervision Relation Extraction.- Modeling More Globally: A Hierarchical Attention Network via Multi-Task Learning for Aspect-Based Sentiment Analysis.- A Sparse Matrix-based Join for SPARQL Query Processing.- Change Point Detection for Streaming High-Dimensional time series.- Demo Papers.- Distributed Query Engine for Multiple-Query Optimization over Data Stream.- Adding Value by Combining Business and Sensor Data: An Industry 4.0 Use Case.- AgriKG: An Agricultural Knowledge Graph and Its Applications.- KGVis: An Interactive Visual Query Language for Knowledge Graphs.- OperaMiner: Extracting Character Relations from Opera Scripts using Deep Neural Networks.- GparMiner: A System to mine Graph Pattern Association Rules.- A Data Publishing System Based on Privacy Preservation.- Privacy as a Service: Publishing Data and Models.- Dynamic Bus Route Adjustment Based on Hot Bus Stop Pair Extraction.- DHDSearch: A Framework for Batch Time Series Searching on MapReduce.- Bus Stop Refinement based on Hot Spot Extraction.- Adaptive Transaction Scheduling for Highly Contended Workloads.- IMOptimizer: An Online Interactive Parameter Optimization System based on Big Data.- Tutorial Papers.- Cohesive Subgraphs with Hierarchical Decomposition on Big Graphs.- Tracking User Behaviours: Laboratory-Based and In-The-Wild User S.- Mining Knowledge Graphs for Vision Tasks.- Enterprise Knowledge Graph From Specific Business Task to Enterprise Knowledge Management.- Knowledge Graph Data Management.- Deep learning for Healthcare Data Processing.

Read more less

Book SynopsisThe two volumes LNCS 11935 and 11936 constitute the proceedings of the 9th International Conference on Intelligence Science and Big Data Engineering, IScIDE 2019, held in Nanjing, China, in October 2019. The 84 full papers presented were carefully reviewed and selected from 252 submissions.The papers are organized in two parts: visual data engineering; and big data and machine learning. They cover a large range of topics including information theoretic and Bayesian approaches, probabilistic graphical models, big data analysis, neural networks and neuro-informatics, bioinformatics, computational biology and brain-computer interfaces, as well as advances in fundamental pattern recognition techniques relevant to image processing, computer vision and machine learning.

Read more less

Book SynopsisMaking use of data is not anymore a niche project but central to almost every project. With access to massive compute resources and vast amounts of data, it seems at least in principle possible to solve any problem. However, successful data science projects result from the intelligent application of: human intuition in combination with computational power; sound background knowledge with computer-aided modelling; and critical reflection of the obtained insights and results.Substantially updating the previous edition, then entitled Guide to Intelligent Data Analysis, this core textbook continues to provide a hands-on instructional approach to many data science techniques, and explains how these are used to solve real world problems. The work balances the practical aspects of applying and using data science techniques with the theoretical and algorithmic underpinnings from mathematics and statistics. Major updates on techniques and subject coverage (including deep learning) are included.Topics and features: guides the reader through the process of data science, following the interdependent steps of project understanding, data understanding, data blending and transformation, modeling, as well as deployment and monitoring; includes numerous examples using the open source KNIME Analytics Platform, together with an introductory appendix; provides a review of the basics of classical statistics that support and justify many data analysis methods, and a glossary of statistical terms; integrates illustrations and case-study-style examples to support pedagogical exposition; supplies further tools and information at an associated website.This practical and systematic textbook/reference is a “need-to-have” tool for graduate and advanced undergraduate students and essential reading for all professionals who face data science problems. Moreover, it is a “need to use, need to keep” resource following one's exploration of the subject.Table of ContentsIntroduction Practical Data Analysis: An Example Project Understanding Data Understanding Principles of Modeling Data Preparation Finding Patterns Finding Explanations Finding Predictors Evaluation and DeploymentThe Labelling Problem Appendix A: Statistics Appendix B: KNIME

Read more less

Book SynopsisThis second edition textbook covers a coherently organized framework for text analytics, which integrates material drawn from the intersecting topics of information retrieval, machine learning, and natural language processing. Particular importance is placed on deep learning methods. The chapters of this book span three broad categories:1. Basic algorithms: Chapters 1 through 7 discuss the classical algorithms for text analytics such as preprocessing, similarity computation, topic modeling, matrix factorization, clustering, classification, regression, and ensemble analysis.2. Domain-sensitive learning and information retrieval: Chapters 8 and 9 discuss learning models in heterogeneous settings such as a combination of text with multimedia or Web links. The problem of information retrieval and Web search is also discussed in the context of its relationship with ranking and machine learning methods. 3. Natural language processing: Chapters 10 through 16 discuss various sequence-centric and natural language applications, such as feature engineering, neural language models, deep learning, transformers, pre-trained language models, text summarization, information extraction, knowledge graphs, question answering, opinion mining, text segmentation, and event detection. Compared to the first edition, this second edition textbook (which targets mostly advanced level students majoring in computer science and math) has substantially more material on deep learning and natural language processing. Significant focus is placed on topics like transformers, pre-trained language models, knowledge graphs, and question answering.Table of Contents1 An Introduction to Text Analytics.- 2 Text Preparation and Similarity Computation.- 3 Matrix Factorization and Topic Modeling.- 4 Text Clustering.- 5 Text Classification: Basic Models.- 6 Linear Models for Classification and Regression.- 7 Classifier Performance and Evaluation.- 8 Joint Text Mining with Heterogeneous Data.- 9 Information Retrieval and Search Engines.- 10 Language Modeling and Deep Learning.- 11 Attention Mechanisms and Transformers.- 12 Text Summarization.- 13 Information Extraction and Knowledge Graphs.- 14 Question Answering.- 15 Opinion Mining and Sentiment Analysis.- 16 Text Segmentation and Event Detection.

Read more less

Book SynopsisThis book provides a practical and fairly comprehensive review of Data Science through the lens of dimensionality reduction, as well as hands-on techniques to tackle problems with data collected in the real world. State-of-the-art results and solutions from statistics, computer science and mathematics are explained from the point of view of a practitioner in any domain science, such as biology, cyber security, chemistry, sports science and many others. Quantitative and qualitative assessment methods are described to implement and validate the solutions back in the real world where the problems originated.The ability to generate, gather and store volumes of data in the order of tera- and exo bytes daily has far outpaced our ability to derive useful information with available computational resources for many domains.This book focuses on data science and problem definition, data cleansing, feature selection and extraction, statistical, geometric, information-theoretic, biomolecular and machine learning methods for dimensionality reduction of big datasets and problem solving, as well as a comparative assessment of solutions in a real-world setting.This book targets professionals working within related fields with an undergraduate degree in any science area, particularly quantitative. Readers should be able to follow examples in this book that introduce each method or technique. These motivating examples are followed by precise definitions of the technical concepts required and presentation of the results in general situations. These concepts require a degree of abstraction that can be followed by re-interpreting concepts like in the original example(s). Finally, each section closes with solutions to the original problem(s) afforded by these techniques, perhaps in various ways to compare and contrast dis/advantages to other solutions.Table of Contents1. What is Data Science (DS)?1.1 Major Families of Data Science Problems1.1.1 Classification Problems1.1.2 Prediction Problems1.1.3 Clustering Problems1.2 Data, Big Data and Pre-processing1.2.1 What is Data?1.2.2 Big data1.2.3 Data Cleansing1.2.4 Data Visualization1.2.5 Data Understanding1.3 Populations and Data Sampling1.3.1 Sampling1.3.2 Training, Testing and Validation1.4 Overview and Scope1.4.1 Prerequisites and Layout1.4.2 Data Science Methodology1.4.3 Scope of the Book2. Solutions to Data Science Problems2.1 Conventional Statistical Solutions2.1.1 Linear Multiple Regression Model: Continuous Response2.1.2 Logistic Regression: Categorical Response2.1.3 Variable Selection and Model Building2.1.4 Generalized Linear Model (GLM)2.1.5 Decision Trees2.1.6 Bayesian Learning2.2 Machine Learning Solutions: Supervised2.2.1 k-Nearest Neighbors (kNN)2.2.2 Ensemble Methods2.2.3 Support Vector Machines (SVMs)2.2.4 Neural Networks (NNs)2.3 Machine Learning Solutions: Unsupervised2.3.1 Hard Clustering2.3.2 Soft Clustering2.4 Controls, Evaluation and Assessment2.4.1 Evaluation Methods2.4.2 Metrics for Assessment3. What is Dimensionality Reduction (DR)?3.1 Dimensionality Reduction3.2 Major Approaches to Dimensionality Reduction3.2.1 Conventional Statistical Approaches3.2.2 Geometric Approaches3.2.3 Information-theoretic Approaches3.2.4 Molecular Computing Approaches3.3 The Blessings of Dimensionality4. Conventional Statistical Approaches4.1 Principal Component Analysis (PCA)4.1.1 Obtaining the Principal Components4.1.2 Singular value decomposition (SVD)4.2 Nonlinear PCA 4.2.1 Kernel PCA4.2.2 Independent component analysis (ICA)4.3 Nonnegative Matrix Factorization (NMF)4.3.1 Approximate Solutions4.3.2 Clustering and Other Applications4.4 Discriminant Analysis4.4.1 Linear discriminant analysis (LDA)4.4.2 Quadratic discriminant analysis (QDA)4.5 Sliced Inverse Regression (SIR)5. Geometric Approaches5.1 Introduction to Manifolds5.2 Manifold Learning Methods5.2.1 Multi-Dimensional Scaling (MDS)5.2.2 Isometric Mapping (ISOMAP)5.2.3 t-Stochastic Neighbor Embedding ( t-SNE )5.3 Exploiting Randomness (RND)6. Information-theoretic Approaches6.1 Shannon Entropy (H)6.2 Reduction by Conditional Entropy6.3 Reduction by Iterated Conditional Entropy6.4 Reduction by Conditional Entropy on Targets6.5 Other Variations7. Molecular Computing Approaches7.1 Encoding Abiotic Data into DNA7.2 Deep Structure of DNA Spaces7.2.1 Structural Properties of DNA Spaces7.2.2 Noncrosshybridizing (nxh) Bases7.3 Reduction by Genomic Signatures7.3.1 Background7.3.2 Genomic Signatures7.4 Reduction by Pmeric Signatures8. Statistical Learning Approaches8.1 Reduction by Multiple Regression8.2 Reduction by Ridge Regression8.3 Reduction by Lasso Regression 8.4 Selection versus Shrinkage8.5 Further refinements9. Machine Learning Approaches9.1 Autoassociative Feature Encoders9.1.1 Undercomplete Autoencoders 9.1.2 Sparse Autoencoders9.1.3 Variational Autoencoders9.1.4 Dimensionality Reduction in MNIST Images9.2 Neural Feature Selection9.2.1 Facial Features, Expressions and Displays9.2.2 The Cohn-Kanade Dataset9.2.3 Primary and Derived Features9.3 Other Methods10. Metaheuristics of DR Methods10.1 Exploiting Feature Grouping10.2 Exploiting Domain Knowledge10.2.1 What is Domain Knowledge?10.2.2 Domain Knowledge for Dimensionality Reduction10.3 Heuristic Rules for Feature Selection, Extraction and Number10.4 About Explainability of Solutions10.4.1 What is Explainability?10.4.2 Explainability in Dimensionality Reduction10.5 Choosing Wisely10.6 About the Curse of Dimensionality10.7 About the No-Free-Lunch Theorem (NFL)11. Appendices11.1 Statistics and Probability Background11.1.1 Commonly Used Discrete Distributions11.1.2 Commonly Used Continuous Distributions11.1.3 Major Results In Probability and Statistics11.2 Linear Algebra Background11.2.1 Fields, Vector Spaces and Subspaces11.2.2 Linear independence, Bases and Dimension11.2.3 Linear Transformations and Matrices11.2.4 Eigenvalues and Spectral Decomposition11.3 Computer Science Background11.3.1 Computational Science and Complexity11.3.2 Machine Learning11.4 Typical Data Science Problems11.5 A Sample of Common and Big Datasets11.6 Computing Platforms11.6.1 The Environment R11.6.2 Python environmentsReferences

Read more less

Book SynopsisThis book constitutes the post-conference proceedings of the 23rd International Conference on Data Analytics and Management in Data Intensive Domains, DAMDID/RCDL 2021, held in Moscow, Russia, in October 2021*.The 16 revised full papers were carefully reviewed and selected from 61 submissions. The papers are organized in the following topical sections: problem solving infrastructures, experiment organization, and machine learning applications; data analysis in astronomy; data analysis in material and earth sciences; information extraction from text* The conference was held virtually due to the COVID-19 pandemic.Table of ContentsProblem Solving Infrastructures, Experiment Organization, and Machine Learning Applications.- MLDev: Data Science Experiment Automation and Reproducibility Software.- Response to Cybersecurity Threats of Informational Infrastructure Based on Conceptual Models.- Social Network Analysis of the Professional Community Interaction - Movie Industry Case.- Data Analysis in Astronomy.- Cross-Matching of Large Sky Surveys and Study of Astronomical Objects Apparent in Ultraviolet Band Only.- The Diversity of Light Curves of Supernovae Associated with Gamma-Ray Bursts.- Application of Machine Learning Methods for Cross-Matching Astronomical Catalogs.- Pipeline for Detection of Transient Objects in Optical Surveys.- VALD in Astrophysics.- Data Analysis in Material and Earth Sciences.- Machine Learning Application to Predict New Inorganic Compounds – Results and Perspectives.- Interoperability and Architecture Requirements Analysis and Metadata Standardization for a Research Data Infrastructure in Catalysis.- Fast Predictions of Lattice Energies by Continuous Isometry Invariants of Crystal Structures.- Image Recognition for Large Soil Maps Archive Overview: Metadata Extraction and Georeferencing Tool Development.- Information Extraction from Text.- Cross-lingual Plagiarism Detection Method.- Methods for Automatic Argumentation Structure Prediction.- A System for Information Extraction from Scientific Texts in Russian.- Improving Neural Abstractive Summarization with Reliable Sentence Sampling.

Read more less

Book SynopsisThis textbook on practical data analytics unites fundamental principles, algorithms, and data. Algorithms are the keystone of data analytics and the focal point of this textbook. Clear and intuitive explanations of the mathematical and statistical foundations make the algorithms transparent. But practical data analytics requires more than just the foundations. Problems and data are enormously variable and only the most elementary of algorithms can be used without modification. Programming fluency and experience with real and challenging data is indispensable and so the reader is immersed in Python and R and real data analysis. By the end of the book, the reader will have gained the ability to adapt algorithms to new problems and carry out innovative analyses.This book has three parts:(a) Data Reduction: Begins with the concepts of data reduction, data maps, and information extraction. The second chapter introduces associative statistics, the mathematical foundation of scalable algorithms and distributed computing. Practical aspects of distributed computing is the subject of the Hadoop and MapReduce chapter.(b) Extracting Information from Data: Linear regression and data visualization are the principal topics of Part II. The authors dedicate a chapter to the critical domain of Healthcare Analytics for an extended example of practical data analytics. The algorithms and analytics will be of much interest to practitioners interested in utilizing the large and unwieldly data sets of the Centers for Disease Control and Prevention's Behavioral Risk Factor Surveillance System.(c) Predictive Analytics Two foundational and widely used algorithms, k-nearest neighbors and naive Bayes, are developed in detail. A chapter is dedicated to forecasting. The last chapter focuses on streaming data and uses publicly accessible data streams originating from the Twitter API and the NASDAQ stock market in the tutorials.This book is intended for a one- or two-semester course in data analytics for upper-division undergraduate and graduate students in mathematics, statistics, and computer science. The prerequisites are kept low, and students with one or two courses in probability or statistics, an exposure to vectors and matrices, and a programming course will have no difficulty. The core material of every chapter is accessible to all with these prerequisites. The chapters often expand at the close with innovations of interest to practitioners of data science. Each chapter includes exercises of varying levels of difficulty. The text is eminently suitable for self-study and an exceptional resource for practitioners.Trade Review“This 430-page book contains an excellent collection of information on the subject of practical algorithms used in data science. The discussion of each algorithm starts with some basic concepts, followed by a tutorial with real datasets and detailed code examples in Python or R. Each chapter has a set of exercise problems so readers can practice the concepts learned in the chapter. … a good reference for practitioners, or a good textbook for graduate or upper-class undergraduate students.” (Xiannong Meng, Computing Reviews, September, 2017)“This textbook on practical data analytics unites fundamental principles, algorithms, and data. … this book is devoted to upper-division undergraduate and graduate students in mathematics, statistics, and computer science. It is intended for a one- or two-semester course in data analytics and reflects the authors’ research experience in data science concepts and the teaching skills in various areas. … The text is eminently suitable for self-study and an exceptional resource for practitioners.” (Krzysztof J. Szajowski, zbMATH 1367.62005, 2017) Table of ContentsIntroduction.- Data Mapping and Data Dictionaries.- Scalable Algorithms and Associative Statistics.- Hadoop and MapReduce.- Data Visualization.- Linear Regression Methods.- Healthcare Analytics.- Cluster Analysis.- k-Nearest Neighbor Prediction Functions.- The Multinomial Naive Bayes Prediction Function.- Forecasting.- Real-time Analytics.

Read more less

Book SynopsisMachine learning (ML) is the fastest growing field in computer science, and Health Informatics (HI) is amongst the greatest application challenges, providing future benefits in improved medical diagnoses, disease analyses, and pharmaceutical development. However, successful ML for HI needs a concerted effort, fostering integrative research between experts ranging from diverse disciplines from data science to visualization. Tackling complex challenges needs both disciplinary excellence and cross-disciplinary networking without any boundaries. Following the HCI-KDD approach, in combining the best of two worlds, it is aimed to support human intelligence with machine intelligence. This state-of-the-art survey is an output of the international HCI-KDD expert network and features 22 carefully selected and peer-reviewed chapters on hot topics in machine learning for health informatics; they discuss open problems and future challenges in order to stimulate further research and international progress in this field.Table of ContentsMachine Learning for Health Informatics.- Bagging Soft Decision Trees.- Grammars for Discrete Dynamics.- Empowering Bridging Term Discovery for Cross-domain Literature Mining in the TextFlows Platform.- Visualisation of Integrated Patient-Centric Data as Pathways: Enhancing Electronic Medical Records in Clinical Practice.- Deep learning trends for focal brain pathology segmentation in MRI.- Differentiation between Normal and Epileptic EEG using K-Nearest-Neighbors Technique.- Survey on Feature Extraction and Applications of Biosignals.- Argumentation for knowledge representation, conflict resolution, defeasible inference and its integration with machine learning.- Machine Learning and Data mining Methods for Managing Parkinson’s Disease.- Challenges of Medical Text and Image Processing: Machine Learning Approaches.- Visual Intelligent Decision Support Systems in the medical field: design and evaluation.

Read more less

Book SynopsisThis engaging and clearly written textbook/reference provides a must-have introduction to the rapidly emerging interdisciplinary field of data science. It focuses on the principles fundamental to becoming a good data scientist and the key skills needed to build systems for collecting, analyzing, and interpreting data. The Data Science Design Manual is a source of practical insights that highlights what really matters in analyzing data, and provides an intuitive understanding of how these core concepts can be used. The book does not emphasize any particular programming language or suite of data-analysis tools, focusing instead on high-level discussion of important design principles. This easy-to-read text ideally serves the needs of undergraduate and early graduate students embarking on an “Introduction to Data Science” course. It reveals how this discipline sits at the intersection of statistics, computer science, and machine learning, with a distinct heft and character of its own. Practitioners in these and related fields will find this book perfect for self-study as well. Additional learning tools: Contains “War Stories,” offering perspectives on how data science applies in the real world Includes “Homework Problems,” providing a wide range of exercises and projects for self-study Provides a complete set of lecture slides and online video lectures at www.data-manual.com Provides “Take-Home Lessons,” emphasizing the big-picture concepts to learn from each chapter Recommends exciting “Kaggle Challenges” from the online platform Kaggle Highlights “False Starts,” revealing the subtle reasons why certain approaches fail Offers examples taken from the data science television show “The Quant Shop” (www.quant-shop.com) Trade Review “The book is more than a typical manual. In fact, the author himself designates it as a textbook for an introductory course on data science. The chapters are richly equipped with exercises. The topics are always explained starting with a proper motivation and continuing with practical examples. This is perhaps the most outstanding feature of the book. It can serve as a regular textbook for an academic course. In fact, I should like to recommend it exactly for this purpose. On the other hand, it provides a wealth of material for people from industry, such as software engineers, and can serve as a manual for them to accomplish data science tasks. It should be noted that the book is not just a text, but a much more complex product, including a full set of lecture slides available online as well as a solutions wiki.” (P. Navrat, Computing Reviews, February, 23, 2018) ​Table of ContentsWhat is Data Science? Mathematical Preliminaries Data Munging Scores and Rankings Statistical Analysis Visualizing Data Mathematical Models Linear Algebra Linear and Logistic Regression Distance and Network Methods Machine Learning Big Data: Achieving Scale

Read more less

a huge range and FREE tracked UK delivery on ALL orders.

Read more less

Book SynopsisThis book is a comprehensive introduction to the methods and algorithms of modern data analytics. It provides a sound mathematical basis, discusses advantages and drawbacks of different approaches, and enables the reader to design and implement data analytics solutions for real-world applications.

Read more less

Book SynopsisA hands-on guide to making valuable decisions from data using advanced data mining methods and techniques This second installment in the Making Sense of Data series continues to explore a diverse range of commonly used approaches to making and communicating decisions from data. Delving into more technical topics, this book equips readers with advanced data mining methods that are needed to successfully translate raw data into smart decisions across various fields of research including business, engineering, finance, and the social sciences. Following a comprehensive introduction that details how to define a problem, perform an analysis, and deploy the results, Making Sense of Data II addresses the following key techniques for advanced data analysis: Data Visualization reviews principles and methods for understanding and communicating data through the use of visualization including single variables, the relationship between two or more vaTrade Review?Experts, researchers, practitioners, or readers who need a quick reference or who want to get up to speed quickly on data analysis will love having a copy of this work. Summing Up: Highly recommended.? (CHOICE, October 2009)Table of Contents PREFACE xi 1 INTRODUCTION 1 1.1 Overview 1 1.2 Definition 1 1.3 Preparation 2 1.3.1 Overview 2 1.3.2 Accessing Tabular Data 3 1.3.3 Accessing Unstructured Data 3 1.3.4 Understanding the Variables and Observations 3 1.3.5 Data Cleaning 6 1.3.6 Transformation 7 1.3.7 Variable Reduction 9 1.3.8 Segmentation 10 1.3.9 Preparing Data to Apply 10 1.4 Analysis 11 1.4.1 Data Mining Tasks 11 1.4.2 Optimization 12 1.4.3 Evaluation 12 1.4.4 Model Forensics 13 1.5 Deployment 13 1.6 Outline of Book 14 1.6.1 Overview 14 1.6.2 Data Visualization 14 1.6.3 Clustering 15 1.6.4 Predictive Analytics 15 1.6.5 Applications 16 1.6.6 Software 16 1.7 Summary 16 1.8 Further Reading 17 2 DATA VISUALIZATION 19 2.1 Overview 19 2.2 Visualization Design Principles 20 2.2.1 General Principles 20 2.2.2 Graphics Design 23 2.2.3 Anatomy of a Graph 28 2.3 Tables 32 2.3.1 Simple Tables 32 2.3.2 Summary Tables 33 2.3.3 Two-Way Contingency Tables 34 2.3.4 Supertables 34 2.4 Univariate Data Visualization 36 2.4.1 Bar Chart 36 2.4.2 Histograms 37 2.4.3 Frequency Polygram 41 2.4.4 Box Plots 41 2.4.5 Dot Plot 43 2.4.6 Stem-and-Leaf Plot 44 2.4.7 Quantile Plot 46 2.4.8 Quantile–Quantile Plot 48 2.5 Bivariate Data Visualization 49 2.5.1 Scatterplot 49 2.6 Multivariate Data Visualization 50 2.6.1 Histogram Matrix 52 2.6.2 Scatterplot Matrix 54 2.6.3 Multiple Box Plot 56 2.6.4 Trellis Plot 56 2.7 Visualizing Groups 59 2.7.1 Dendrograms 59 2.7.2 Decision Trees 60 2.7.3 Cluster Image Maps 60 2.8 Dynamic Techniques 63 2.8.1 Overview 63 2.8.2 Data Brushing 64 2.8.3 Nearness Selection 65 2.8.4 Sorting and Rearranging 65 2.8.5 Searching and Filtering 65 2.9 Summary 65 2.10 Further Reading 66 3 CLUSTERING 67 3.1 Overview 67 3.2 Distance Measures 75 3.2.1 Overview 75 3.2.2 Numeric Distance Measures 77 3.2.3 Binary Distance Measures 79 3.2.4 Mixed Variables 84 3.2.5 Other Measures 86 3.3 Agglomerative Hierarchical Clustering 87 3.3.1 Overview 87 3.3.2 Single Linkage 88 3.3.3 Complete Linkage 92 3.3.4 Average Linkage 93 3.3.5 Other Methods 96 3.3.6 Selecting Groups 96 3.4 Partitioned-Based Clustering 98 3.4.1 Overview 98 3.4.2 k-Means 98 3.4.3 Worked Example 100 3.4.4 Miscellaneous Partitioned-Based Clustering 101 3.5 Fuzzy Clustering 103 3.5.1 Overview 103 3.5.2 Fuzzy k-Means 103 3.5.3 Worked Examples 104 3.6 Summary 109 3.7 Further Reading 110 4 PREDICTIVE ANALYTICS 111 4.1 Overview 111 4.1.1 Predictive Modeling 111 4.1.2 Testing Model Accuracy 116 4.1.3 Evaluating Regression Models’ Predictive Accuracy 117 4.1.4 Evaluating Classification Models’ Predictive Accuracy 119 4.1.5 Evaluating Binary Models’ Predictive Accuracy 120 4.1.6 ROC Charts 122 4.1.7 Lift Chart 124 4.2 Principal Component Analysis 126 4.2.1 Overview 126 4.2.2 Principal Components 126 4.2.3 Generating Principal Components 127 4.2.4 Interpretation of Principal Components 128 4.3 Multiple Linear Regression 130 4.3.1 Overview 130 4.3.2 Generating Models 133 4.3.3 Prediction 136 4.3.4 Analysis of Residuals 136 4.3.5 Standard Error 139 4.3.6 Coefficient of Multiple Determination 140 4.3.7 Testing the Model Significance 142 4.3.8 Selecting and Transforming Variables 143 4.4 Discriminant Analysis 145 4.4.1 Overview 145 4.4.2 Discriminant Function 146 4.4.3 Discriminant Analysis Example 146 4.5 Logistic Regression 151 4.5.1 Overview 151 4.5.2 Logistic Regression Formula 151 4.5.3 Estimating Coefficients 153 4.5.4 Assessing and Optimizing Results 156 4.6 Naive Bayes Classifiers 157 4.6.1 Overview 157 4.6.2 Bayes Theorem and the Independence Assumption 158 4.6.3 Independence Assumption 158 4.6.4 Classification Process 159 4.7 Summary 161 4.8 Further Reading 163 5 APPLICATIONS 165 5.1 Overview 165 5.2 Sales and Marketing 166 5.3 Industry-Specific Data Mining 169 5.3.1 Finance 169 5.3.2 Insurance 171 5.3.3 Retail 172 5.3.4 Telecommunications 173 5.3.5 Manufacturing 174 5.3.6 Entertainment 175 5.3.7 Government 176 5.3.8 Pharmaceuticals 177 5.3.9 Healthcare 179 5.4 microRNA Data Analysis Case Study 181 5.4.1 Defining the Problem 181 5.4.2 Preparing the Data 181 5.4.3 Analysis 183 5.5 Credit Scoring Case Study 192 5.5.1 Defining the Problem 192 5.5.2 Preparing the Data 192 5.5.3 Analysis 199 5.5.4 Deployment 203 5.6 Data Mining Nontabular Data 203 5.6.1 Overview 203 5.6.2 Data Mining Chemical Data 203 5.6.3 Data Mining Text 210 5.7 Further Reading 213 APPENDIX A MATRICES 215 A.1 Overview of Matrices 215 A.2 Matrix Addition 215 A.3 Matrix Multiplication 216 A.4 Transpose of a Matrix 217 A.5 Inverse of a Matrix 217 APPENDIX B SOFTWARE 219 B.1 Software Overview 219 B.1.1 Software Objectives 219 B.1.2 Access and Installation 221 B.1.3 User Interface Overview 221 B.2 Data Preparation 223 B.2.1 Overview 223 B.2.2 Reading in Data 224 B.2.3 Searching the Data 225 viii CONTENTS B.2.4 Variable Characterization 227 B.2.5 Removing Observations and Variables 228 B.2.6 Cleaning the Data 228 B.2.7 Transforming the Data 230 B.2.8 Segmentation 235 B.2.9 Principal Component Analysis 236 B.3 Tables and Graphs 238 B.3.1 Overview 238 B.3.2 Contingency Tables 239 B.3.3 Summary Tables 240 B.3.4 Graphs 242 B.3.5 Graph Matrices 246 B.4 Statistics 246 B.4.1 Overview 246 B.4.2 Descriptive Statistics 248 B.4.3 Confidence Intervals 248 B.4.4 Hypothesis Tests 249 B.4.5 Chi-Square Test 250 B.4.6 ANOVA 251 B.4.7 Comparative Statistics 251 B.5 Grouping 253 B.5.1 Overview 253 B.5.2 Clustering 254 B.5.3 Associative Rules 257 B.5.4 Decision Trees 258 B.6 Prediction 261 B.6.1 Overview 261 B.6.2 Linear Regression 263 B.6.3 Discriminant Analysis 265 B.6.4 Logistic Regression 266 B.6.5 Naive Bayes 267 B.6.6 kNN 269 B.6.7 CART 269 B.6.8 Neural Networks 270 B.6.9 Apply Model 271 BIBLIOGRAPHY 273 INDEX 279

Read more less

Book SynopsisAs third in the series, this book focuses on a style of data analysis that makes graphics central to exploration. Making Sense of Data III explains how to implement decision support systems and provides an interactive approach to data analysis that allows users to see, manipulate, explore, mine data, and share results with colleagues.Trade Review“It is an essential book for understanding the principal role that graphics play in data visualization.” (Zentralblatt MATH, 1 April 2015) Table of ContentsPreface. 1. Introduction. 1.1 Overview. 1.2 Visual Perception. 1.3 Visualization. 1.4 Designing for High-throughput Data Exploration. 1.5 Summary. 1.6 Further reading. 2. The Cognitive and Visual Systems. 2.1 External Representation. 2.2 The Cognitive System. 2.3 Visual Perception. 2.4 Influencing Visual Perception. 2.5 Summary. 2.6 Further reading. 3. Graphic Representations. 3.1 Jacques Bertin: Semiology of Graphics. 3.2 Wilkinson: Grammar of Graphics. 3.3 Wickham: ggplot2. 3.4 Bostock and Heer: Protovis. 3.5 Summary. 3.6 Further reading. 4. Designing Visual Interactions. 4.1 Designing for Complexity. 4.2 The Process of Design. 4.3 Visual Interaction Design. 5. Hands-on: Creating Interactive Visualizations with Protovis. 5.1 Using Protovis. 5.2 Creating Code using the Protovis Graphical Framework. 5.3 Basic Protovis Marks. 5.4 Creating Customized Plots. 5.5 Creating Basic Plots. 5.6 Data Analysis Graphs. 5.7 Composite Plots. 5.8 Interactive Plots. 5.9 Protovis Summary. 5.10 Further Reading. Appendix. A Exercise Code Examples. Bibliography. Index.

Read more less

Book SynopsisThe leading introductory book on data mining, fully updated and revised! When Berry and Linoff wrote the first edition of Data Mining Techniques in the late 1990s, data mining was just starting to move out of the lab and into the office and has since grown to become an indispensable tool of modern business.Table of ContentsIntroduction xxxvii Chapter 1 What Is Data Mining and Why Do It? 1 What Is Data Mining? 2 Data Mining Is a Business Process 2 Large Amounts of Data 3 Meaningful Patterns and Rules 3 Data Mining and Customer Relationship Management 4 Why Now? 6 Data Is Being Produced 6 Data Is Being Warehoused 6 Computing Power Is Affordable 7 Interest in Customer Relationship Management Is Strong 7 Commercial Data Mining Software Products Have Become Available 8 Skills for the Data Miner 9 The Virtuous Cycle of Data Mining 9 A Case Study in Business Data Mining 11 Identifying BofA’s Business Challenge 12 Applying Data Mining 12 Acting on the Results 13 Measuring the Effects of Data Mining 14 Steps of the Virtuous Cycle 15 Identify Business Opportunities 16 Transform Data into Information 17 Act on the Information 19 Measure the Results 20 Data Mining in the Context of the Virtuous Cycle 23 Lessons Learned 26 Chapter 2 Data Mining Applications in Marketing and Customer Relationship Management 27 Two Customer Lifecycles 27 The Customer’s Lifecycle 28 The Customer Lifecycle 28 Subscription Relationships versus Event-Based Relationships 30 Organize Business Processes Around the Customer Lifecycle 32 Customer Acquisition 33 Customer Activation 36 Customer Relationship Management 37 Winback 38 Data Mining Applications for Customer Acquisition 38 Identifying Good Prospects 39 Choosing a Communication Channel 39 Picking Appropriate Messages 40 A Data Mining Example: Choosing the Right Place to Advertise 40 Who Fits the Profile? 41 Measuring Fitness for Groups of Readers 44 Data Mining to Improve Direct Marketing Campaigns 45 Response Modeling 46 Optimizing Response for a Fixed Budget 47 Optimizing Campaign Profitability 49 Reaching the People Most Influenced by the Message 53 Using Current Customers to Learn About Prospects 54 Start Tracking Customers Before They Become “Customers” 55 Gather Information from New Customers 55 Acquisition-Time Variables Can Predict Future Outcomes 56 Data Mining Applications for Customer Relationship Management 56 Matching Campaigns to Customers 56 Reducing Exposure to Credit Risk 58 Determining Customer Value 59 Cross-selling, Up-selling, and Making Recommendations 60 Retention 60 Recognizing Attrition 60 Why Attrition Matters 61 Different Kinds of Attrition 62 Different Kinds of Attrition Model 63 Beyond the Customer Lifecycle 64 Lessons Learned 65 Chapter 3 The Data Mining Process 67 What Can Go Wrong? 68 Learning Things That Aren’t True 68 Learning Things That Are True, but Not Useful 73 Data Mining Styles 74 Hypothesis Testing 75 Directed Data Mining 81 Undirected Data Mining 81 Goals, Tasks, and Techniques 82 Data Mining Business Goals 82 Data Mining Tasks 83 Data Mining Techniques 88 Formulating Data Mining Problems: From Goals to Tasks to Techniques 88 What Techniques for Which Tasks? 95 Is There a Target or Targets? 96 What Is the Target Data Like? 96 What Is the Input Data Like? 96 How Important Is Ease of Use? 97 How Important Is Model Explicability? 97 Lessons Learned 98 Chapter 4 Statistics 101: What You Should Know About Data 101 Occam’s Razor 103 Skepticism and Simpson’s Paradox 103 The Null Hypothesis 104 P-Values 105 Looking At and Measuring Data 106 Categorical Values 106 Numeric Variables 117 A Couple More Statistical Ideas 120 Measuring Response 120 Standard Error of a Proportion 121 Comparing Results Using Confidence Bounds 123 Comparing Results Using Difference of Proportions 124 Size of Sample 125 What the Confidence Interval Really Means 126 Size of Test and Control for an Experiment 127 Multiple Comparisons 129 The Confidence Level with Multiple Comparisons 129 Bonferroni’s Correction 129 Chi-Square Test 130 Expected Values 130 Chi-Square Value 132 Comparison of Chi-Square to Difference of Proportions 134 An Example: Chi-Square for Regions and Starts 134 Case Study: Comparing Two Recommendation Systems with an A/B Test 138 First Metric: Participating Sessions 140 Data Mining and Statistics 144 Lessons Learned 148 Chapter 5 Descriptions and Prediction: Profiling and Predictive Modeling 151 Directed Data Mining Models 152 Defining the Model Structure and Target 152 Incremental Response Modeling 154 Model Stability 156 Time-Frames in the Model Set 157 Directed Data Mining Methodology 159 Step 1: Translate the Business Problem into a Data Mining Problem 161 How Will Results Be Used? 163 How Will Results Be Delivered? 163 The Role of Domain Experts and Information Technology 164 Step 2: Select Appropriate Data 165 What Data Is Available? 166 How Much Data Is Enough? 167 How Much History Is Required? 167 How Many Variables? 168 What Must the Data Contain? 168 Step 3: Get to Know the Data 169 Examine Distributions 169 Compare Values with Descriptions 170 Validate Assumptions 170 Ask Lots of Questions 171 Step 4: Create a Model Set 172 Assembling Customer Signatures 172 Creating a Balanced Sample 172 Including Multiple Timeframes 174 Creating a Model Set for Prediction 174 Creating a Model Set for Profiling 176 Partitioning the Model Set 176 Step 5: Fix Problems with the Data 177 Categorical Variables with Too Many Values 177 Numeric Variables with Skewed Distributions and Outliers 178 Missing Values 178 Values with Meanings That Change over Time 179 Inconsistent Data Encoding 179 Step 6: Transform Data to Bring Information to the Surface 180 Step 7: Build Models 180 Step 8: Assess Models 180 Assessing Binary Response Models and Classifiers 181 Assessing Binary Response Models Using Lift 182 Assessing Binary Response Model Scores Using Lift Charts 184 Assessing Binary Response Model Scores Using Profitability Models 185 Assessing Binary Response Models Using ROC Charts 186 Assessing Estimators 188 Assessing Estimators Using Score Rankings 189 Step 9: Deploy Models 190 Practical Issues in Deploying Models 190 Optimizing Models for Deployment 191 Step 10: Assess Results 191 Step 11: Begin Again 193 Lessons Learned 193 Chapter 6 Data Mining Using Classic Statistical Techniques 195 Similarity Models 196 Similarity and Distance 196 Example: A Similarity Model for Product Penetration 197 Table Lookup Models 203 Choosing Dimensions 204 Partitioning the Dimensions 205 From Training Data to Scores 205 Handling Sparse and Missing Data by Removing Dimensions 205 RFM: A Widely Used Lookup Model 206 RFM Cell Migration 207 RFM and the Test-and-Measure Methodology 208 RFM and Incremental Response Modeling 209 Naïve Bayesian Models 210 Some Ideas from Probability 210 The Naïve Bayesian Calculation 212 Comparison with Table Lookup Models 213 Linear Regression 213 The Best-fit Line 215 Goodness of Fit 217 Multiple Regression 220 The Equation 220 The Range of the Target Variable 221 Interpreting Coefficients of Linear Regression Equations 221 Capturing Local Effects with Linear Regression 223 Additional Considerations with Multiple Regression 224 Variable Selection for Multiple Regression 225 Logistic Regression 227 Modeling Binary Outcomes 227 The Logistic Function 229 Fixed Effects and Hierarchical Effects 231 Hierarchical Effects 232 Within and Between Effects 232 Fixed Effects 233 Lessons Learned 234 Chapter 7 Decision Trees 237 What Is a Decision Tree and How Is It Used? 238 A Typical Decision Tree 238 Using the Tree to Learn About Churn 240 Using the Tree to Learn About Data and Select Variables 241 Using the Tree to Produce Rankings 243 Using the Tree to Estimate Class Probabilities 243 Using the Tree to Classify Records 244 Using the Tree to Estimate Numeric Values 244 Decision Trees Are Local Models 245 Growing Decision Trees 247 Finding the Initial Split 248 Growing the Full Tree 251 Finding the Best Split 252 Gini (Population Diversity) as a Splitting Criterion 253 Entropy Reduction or Information Gain as a Splitting Criterion 254 Information Gain Ratio 256 Chi-Square Test as a Splitting Criterion 256 Incremental Response as a Splitting Criterion 258 Reduction in Variance as a Splitting Criterion for Numeric Targets 259 F Test 262 Pruning 262 The CART Pruning Algorithm 263 Pessimistic Pruning: The C5.0 Pruning Algorithm 267 Stability-Based Pruning 268 Extracting Rules from Trees 269 Decision Tree Variations 270 Multiway Splits 270 Splitting on More Than One Field at a Time 271 Creating Nonrectangular Boxes 271 Assessing the Quality of a Decision Tree 275 When Are Decision Trees Appropriate? 276 Case Study: Process Control in a Coffee Roasting Plant 277 Goals for the Simulator 277 Building a Roaster Simulation 278 Evaluation of the Roaster Simulation 278 Lessons Learned 279 Chapter 8 Artificial Neural Networks 281 A Bit of History 282 The Biological Model 283 The Biological Neuron 285 The Biological Input Layer 286 The Biological Output Layer 287 Neural Networks and Artificial Intelligence 287 Artificial Neural Networks 288 The Artificial Neuron 288 The Multi-Layer Perceptron 291 A Network Example 292 Network Topologies 293 A Sample Application: Real Estate Appraisal 295 Training Neural Networks 299 How Does a Neural Network Learn Using Back Propagation? 299 Pruning a Neural Network 300 Radial Basis Function Networks 303 Overview of RBF Networks 303 Choosing the Locations of the Radial Basis Functions 305 Universal Approximators 305 Neural Networks in Practice 308 Choosing the Training Set 309 Coverage of Values for All Features 309 Number of Features 310 Size of Training Set 310 Number and Range of Outputs 310 Rules of Thumb for Using MLPs 310 Preparing the Data 311 Interpreting the Output from a Neural Network 313 Neural Networks for Time Series 315 Time Series Modeling 315 A Neural Network Time Series Example 316 Can Neural Network Models Be Explained? 317 Sensitivity Analysis 318 Using Rules to Describe the Scores 318 Lessons Learned 319 Chapter 9 Nearest Neighbor Approaches: Memory-Based Reasoning and Collaborative Filtering 321 Memory-Based Reasoning 322 Look-Alike Models 323 Example: Using MBR to Estimate Rents in Tuxedo, New York 324 Challenges of MBR 327 Choosing a Balanced Set of Historical Records 328 Representing the Training Data 328 Determining the Distance Function, Combination Function, and Number of Neighbors 331 Case Study: Using MBR for Classifying Anomalies in Mammograms 331 The Business Problem: Identifying Abnormal Mammograms 332 Applying MBR to the Problem 332 The Total Solution 334 Measuring Distance and Similarity 335 What Is a Distance Function? 335 Building a Distance Function One Field at a Time 337 Distance Functions for Other Data Types 340 When a Distance Metric Already Exists 341 The Combination Function: Asking the Neighbors for Advice 342 The Simplest Approach: One Neighbor 342 The Basic Approach for Categorical Targets: Democracy 342 Weighted Voting for Categorical Targets 344 Numeric Targets 344 Case Study: Shazam — Finding Nearest Neighbors for Audio Files 345 Why This Feat Is Challenging 346 The Audio Signature 347 Measuring Similarity 348 Collaborative Filtering: A Nearest-Neighbor Approach to Making Recommendations 351 Building Profiles 352 Comparing Profiles 352 Making Predictions 353 Lessons Learned 354 Chapter 10 Knowing When to Worry: Using Survival Analysis to Understand Customers 357 Customer Survival 360 What Survival Curves Reveal 360 Finding the Average Tenure from a Survival Curve 362 Customer Retention Using Survival 364 Looking at Survival as Decay 365 Hazard Probabilities 367 The Basic Idea 368 Examples of Hazard Functions 369 Censoring 371 The Hazard Calculation 372 Other Types of Censoring 375 From Hazards to Survival 376 Retention 376 Survival 378 Comparison of Retention and Survival 378 Proportional Hazards 380 Examples of Proportional Hazards 381 Stratification: Measuring Initial Effects on Survival 382 Cox Proportional Hazards 382 Survival Analysis in Practice 385 Handling Different Types of Attrition 385 When Will a Customer Come Back? 387 Understanding Customer Value 389 Forecasting 392 Hazards Changing over Time 393 Lessons Learned 394 Chapter 11 Genetic Algorithms and Swarm Intelligence 397 Optimization 398 What Is an Optimization Problem? 398 An Optimization Problem in Ant World 399 E Pluribus Unum 400 A Smarter Ant 401 Genetic Algorithms 403 A Bit of History 404 Genetics on Computers 404 Representing the Genome 413 Schemata: The Building Blocks of Genetic Algorithms 414 Beyond the Simple Algorithm 417 The Traveling Salesman Problem 418 Exhaustive Search 419 A Simple Greedy Algorithm 419 The Genetic Algorithms Approach 419 The Swarm Intelligence Approach 420 Case Study: Using Genetic Algorithms for Resource Optimization 421 Case Study: Evolving a Solution for Classifying Complaints 423 Business Context 424 Data 425 The Comment Signature 425 The Genomes 426 The Fitness Function 427 The Results 427 Lessons Learned 427 Chapter 12 Tell Me Something New: Pattern Discovery and Data Mining 429 Undirected Techniques, Undirected Data Mining 431 Undirected versus Directed Techniques 431 Undirected versus Directed Data Mining 431 Case Study: Undirected Data Mining Using Directed Techniques 432 What is Undirected Data Mining? 435 Data Exploration 435 Segmentation and Clustering 436 Target Variable Definition, When the Target Is Not Explicit 438 Simulation, Forecasting, and Agent-Based Modeling 443 Methodology for Undirected Data Mining 455 There Is No Methodology 456 Things to Keep in Mind 456 Lessons Learned 457 Chapter 13 Finding Islands of Similarity: Automatic Cluster Detection 459 Searching for Islands of Simplicity 461 Customer Segmentation and Clustering 461 Similarity Clusters 463 Tracking Campaigns by Cluster-Based Segments 464 Clustering Reveals an Overlooked Market Segment 466 Fitting the Troops 467 The K-Means Clustering Algorithm 468 Two Steps of the K-Means Algorithm 468 Voronoi Diagrams and K-Means Clusters 471 Choosing the Cluster Seeds 473Choosing K 473 Using K-Means to Detect Outliers 474 Semi-Directed Clustering 475 Interpreting Clusters 475 Characterizing Clusters by Their Centroids 476 Characterizing Clusters by What Differentiates Them 477 Using Decision Trees to Describe Clusters 478 Evaluating Clusters 479 Cluster Measurements and Terminology 480 Cluster Silhouettes 480 Limiting Cluster Diameter for Scoring 483 Case Study: Clustering Towns 484 Creating Town Signatures 484 Creating Clusters 486 Determining the Right Number of Clusters 486 Evaluating the Clusters 487 Using Demographic Clusters to Adjust Zone Boundaries 488 Business Success 490 Variations on K-Means 490 K-Medians, K-Medoids, and K-Modes 490 The Soft Side of K-Means 494 Data Preparation for Clustering 495 Scaling for Consistency 496 Use Weights to Encode Outside Information 496 Selecting Variables for Clustering 497 Lessons Learned 497 Chapter 14 Alternative Approaches to Cluster Detection 499 Shortcomings of K-Means 500 Reasonableness 500 An Intuitive Example 501 Fixing the Problem by Changing the Scales 503 What This Means in Practice 504 Gaussian Mixture Models 505 Adding “Gaussians” to K-Means 505 Back to Gaussian Mixture Models 508 Scoring GMMs 510 Applying GMMs 511 Divisive Clustering 513 A Decision Tree–Like Method for Clustering 513 Scoring Divisive Clusters 515 Clusters and Trees 515 Agglomerative (Hierarchical) Clustering 516 Overview of Agglomerative Clustering Methods 516 Clustering People by Age: An Example of An Agglomerative Clustering Algorithm 520 Scoring Agglomerative Clusters 522 Limitations of Agglomerative Clustering 523 Agglomerative Clustering in Practice 525 Combining Agglomerative Clustering and K-Means 526 Self-Organizing Maps 527 What Is a Self-Organizing Map? 527 Training an SOM 530 Scoring an SOM 531 The Search Continues for Islands of Simplicity 532 Lessons Learned 533 Chapter 15 Market Basket Analysis and Association Rules 535 Defining Market Basket Analysis 536 Four Levels of Market Basket Data 537 The Foundation of Market Basket Analysis: Basic Measures 539 Order Characteristics 540 Item (Product) Popularity 541 Tracking Marketing Interventions 542 Case Study: Spanish or English 543 The Business Problem 543 The Data 544 Defining “Hispanicity” Preference 545 The Solution 546 Association Analysis 547 Rules Are Not Always Useful 548 Item Sets to Association Rules 551 How Good Is an Association Rule? 553 Building Association Rules 555 Choosing the Right Set of Items 556 Anonymous Versus Identified 561 Generating Rules from All This Data 561 Overcoming Practical Limits 565 The Problem of Big Data 567 Extending the Ideas 569 Different Items on the Right- and Left-Hand Sides 569 Using Association Rules to Compare Stores 570 Association Rules and Cross-Selling 572 A Typical Cross-Sell Model 572 A More Confident Approach to Product Propensities 573 Results from Using Confidence 574 Sequential Pattern Analysis 574 Finding the Sequences 575 Sequential Association Rules 578 Sequential Analysis Using Other Data Mining Techniques 579 Lessons Learned 579 Chapter 16 Link Analysis 581 Basic Graph Theory 582 What Is a Graph? 582 Directed Graphs 584 Weighted Graphs 585 Seven Bridges of Königsberg 585 Detecting Cycles in a Graph 588 The Traveling Salesman Problem Revisited 589 Social Network Analysis 593 Six Degrees of Separation 593 What Your Friends Say About You 595 Finding Childcare Benefits Fraud 596 Who Responds to Whom on Dating Sites 597 Social Marketing 598 Mining Call Graphs 598 Case Study: Tracking Down the Leader of the Pack 601 The Business Goal 601 The Data Processing Challenge 601 Finding Social Networks in Call Data 602 How the Results Are Used for Marketing 602 Estimating Customer Age 603 Case Study: Who Is Using Fax Machines from Home? 604 Why Finding Fax Machines Is Useful 604 How Do Fax Machines Behave? 604 A Graph Coloring Algorithm 605 “Coloring” the Graph to Identify Fax Machines 606 How Google Came to Rule the World 607 Hubs and Authorities 608 The Details 609 Hubs and Authorities in Practice 611 Lessons Learned 612 Chapter 17 Data Warehousing, OLAP, Analytic Sandboxes, and Data Mining 613 The Architecture of Data 615 Transaction Data, the Base Level 616 Operational Summary Data 617 Decision-Support Summary Data 617 Database Schema/Data Models 618 Metadata 623 Business Rules 623 A General Architecture for Data Warehousing 624 Source Systems 624 Extraction, Transformation, and Load 626 Central Repository 627 Metadata Repository 630 Data Marts 630 Operational Feedback 631 Users and Desktop Tools 631 Analytic Sandboxes 633 Why Are Analytic Sandboxes Needed? 634 Technology to Support Analytic Sandboxes 636 Where Does OLAP Fit In? 639 What’s in a Cube? 641 Star Schema 646 OLAP and Data Mining 648 Where Data Mining Fits in with Data Warehousing 650 Lots of Data 651 Consistent, Clean Data 651 Hypothesis Testing and Measurement 652 Scalable Hardware and RDBMS Support 653 Lessons Learned 653 Chapter 18 Building Customer Signatures 655 Finding Customers in Data 656 What Is a Customer? 657 Accounts? Customers? Households? 658 Anonymous Transactions 658 Transactions Linked to a Card 659 Transactions Linked to a Cookie 659 Transactions Linked to an Account 660 Transactions Linked to a Customer 661 Designing Signatures 661 Is a Customer Signature Necessary? 666 What Does a Row Represent? 666 Will the Signature Be Used for Predictive Modeling? 671 Has a Target Been Defined? 672 Are There Constraints Imposed by the Particular Data Mining Techniques to be Employed? 672 Which Customers Will Be Included? 673 What Might Be Interesting to Know About Customers? 673 What a Signature Looks Like 674 Process for Creating Signatures 677 Some Data Is Already at the Right Level of Granularity 678 Pivoting a Regular Time Series 679 Aggregating Time-Stamped Transactions 680 Dealing with Missing Values 685 Missing Values in Source Data 685 Unknown or Non-Existent? 687 What Not to Do 687 Things to Consider 689 Lessons Learned 691 Chapter 19 Derived Variables: Making the Data Mean More 693 Handset Churn Rate as a Predictor of Churn 694 Single-Variable Transformations 696 Standardizing Numeric Variables 696 Turning Numeric Values into Percentiles 697 Turning Counts into Rates 698 Relative Measures 699 Replacing Categorical Variables with Numeric Ones 700 Combining Variables 707 Classic Combinations 707 Combining Highly Correlated Variables 710 Rent to Home Value 712 Extracting Features from Time Series 718 Trend 719 Seasonality 721 Extracting Features from Geography 722 Geocoding 722 Mapping 723 Using Geography to Create Relative Measures 724 Using Past Values of the Target Variable 725 Using Model Scores as Inputs 725 Handling Sparse Data 726 Account Set Patterns 726 Binning Sparse Values 727 Capturing Customer Behavior from Transactions 727 Widening Narrow Data 728 Sphere of Influence as a Predictor of Good Customers 728 An Example: Ratings to Rater Profile 730 Sample Fields from the Rater Signature 730 The Rating Signature and Derived Variables 732 Lessons Learned 733 Chapter 20 Too Much of a Good Thing? Techniques for Reducing the Number of Variables 735 Problems with Too Many Variables 736 Risk of Correlation Among Input Variables 736 Risk of Overfitting 738 The Sparse Data Problem 738 Visualizing Sparseness 739 Independence 740 Exhaustive Feature Selection 743 Flavors of Variable Reduction Techniques 744 Using the Target 744 Original versus New Variables 744 Sequential Selection of Features 745 The Traditional Forward Selection Methodology 745 Forward Selection Using a Validation Set 747 Stepwise Selection 748 Forward Selection Using Non-Regression Techniques 748 Backward Selection 748 Undirected Forward Selection 749 Other Directed Variable Selection Methods 749 Using Decision Trees to Select Variables 750 Variable Reduction Using Neural Networks 752 Principal Components 753 What Are Principal Components? 753 Principal Components Example 758 Principal Component Analysis 763 Factor Analysis 767 Variable Clustering 768 Example of Variable Clusters 768 Using Variable Clusters 770 Hierarchical Variable Clustering 770 Divisive Variable Clustering 773 Lessons Learned 774 Chapter 21 Listen Carefully to What Your Customers Say: Text Mining 775 What Is Text Mining? 776 Text Mining for Derived Columns 776 Beyond Derived Features 777 Text Analysis Applications 778 Working with Text Data 781 Sources of Text 781 Language Effects 782 Basic Approaches to Representing Documents 783 Representing Documents in Practice 784 Documents and the Corpus 786 Case Study: Ad Hoc Text Mining 786 The Boycott 787 Business as Usual 787 Combining Text Mining and Hypothesis Testing 787 The Results 788 Classifying News Stories Using MBR 789 What Are the Codes? 789 Applying MBR 790 The Results 793 From Text to Numbers 794 Starting with a “Bag of Words” 794 Term-Document Matrix 796 Corpus Effects 797 Singular Value Decomposition (SVD) 798 Text Mining and Naïve Bayesian Models 800 Naïve Bayesian in the Text World 801 Identifying Spam Using Naïve Bayesian 801 Sentiment Analysis 806 DIRECTV: A Case Study in Customer Service 809 Background 809 Applying Text Mining 811 Taking the Technical Approach 814 Not an Iterative Process 818 Continuing to Benefit 818 Lessons Learned 819 Index 821

Read more less

Book SynopsisThis is an applied handbook for the application of data mining techniques in the CRM framework. It combines a technical and a business perspective to cover the needs of business users who are looking for a practical guide on data mining.Trade Review"The book is written in a language that is easily accessible to business users who are not fluent in statistical methods and who have no prior exposure to the data mining or customer segmentation domain . . . This book is poised to become a standard reference, and I unconditionally recommend it to anyone working in this field." (Computing Reviews, 23 June 2011) "This is an excellent book for any data miner or anybody involved in CRM. The text is clear and pictures are well done and funny which is rare enough to be mentioned. From basic to advanced topics, the book is a very pleasant journey inside data mining with a clear focus on customer segmentation. Really advised if you're not a fan of formulas." (Data Mining Research, 18 March 2011)Table of ContentsAcknowledgements. 1. Data Mining in CRM. The CRM Strategy. What Can Data Mining Do? The Data Mining Methodology. Data Mining and Business Domain Expertise. Summary. 2. An Overview of Data Mining Techniques. Supervised Modeling. Unsupervised Modeling Techniques. Machine Learning/Artificial Intelligence vs. Statistical Techniques. Summary. 3. Data Mining Techniques for Segmentation. Segmenting Customers with Data Mining Techniques. Principal Components Analysis. Clustering Techniques. Examining and Evaluating the Cluster Solution. Understanding the Clusters through Profiling. Selecting the Optimal Cluster Solution. Cluster Profiling and Scoring with Supervised Models. An Introduction to Decision Tree Models. Summary. 4. The Mining Data Mart. Designing the Mining Data Mart. The Time Frame Covered by the Mining Data Mart. The Mining Data Mart for Retail Banking. The Mining Data Mart for Mobile Telephony Consumer (Residential) Customers. The Mining Data Mart for Retailers. Summary. 5. Customer Segmentation. An Introduction to Customer Segmentation. Segmentation Types in Consumer Markets. Segmentation in Business Markets. A Guide for Behavioral Segmentation. Segmentation Management Strategy. A Guide for Value-Based Segmentation. Designing Differentiated Strategies for the Value Segments. Summary. 6. Segmentation Applications in Banking. Segmentation for Credit Card Holders. Segmentation in Retail Banking. The Marketing Process. Segmentation in Retail Banking; A Summary. 7. Segmentation Applications in Telecommunications. Mobile Telephony. The Fixed Telephony Case. Summary. 8. Segmentation for Retailers. Segmentation in the Retail Industry. The RFM Analysis. Grouping Customers According to the Products They Buy. Summary. Further Reading. Index.

Read more less

Book SynopsisData Mining Algorithms is a practical, technically-oriented guide to data mining algorithms that covers the most important algorithms for building classification, regression, and clustering models, as well as techniques used for attribute selection and transformation, model quality evaluation, and creating model ensembles. The author presents many of the important topics and methodologies widely used in data mining, whilst demonstrating the internal operation and usage of data mining algorithms using examples in R.Table of ContentsAcknowledgements xix Preface xxi References xxxi Part I Preliminaries 1 1 Tasks 3 1.1 Introduction 3 1.2 Inductive learning tasks 5 1.3 Classification 9 1.4 Regression 14 1.5 Clustering 16 1.6 Practical issues 19 1.7 Conclusion 20 1.8 Further readings 21 References 22 2 Basic statistics 23 2.1 Introduction 23 2.2 Notational conventions 24 2.3 Basic statistics as modeling 24 2.4 Distribution description 25 2.5 Relationship detection 47 2.6 Visualization 62 2.7 Conclusion 65 2.8 Further readings 66 References 67 Part II Classification 69 3 Decision trees 71 3.1 Introduction 71 3.2 Decision tree model 72 3.3 Growing 76 3.4 Pruning 90 3.5 Prediction 103 3.6 Weighted instances 105 3.7 Missing value handling 106 3.8 Conclusion 114 3.9 Further readings 114 References 116 4 Naïve Bayes classifier 118 4.1 Introduction 118 4.2 Bayes rule 118 4.3 Classification by Bayesian inference 120 4.4 Practical issues 125 4.5 Conclusion 131 4.6 Further readings 131 References 132 5 Linear classification 134 5.1 Introduction 134 5.2 Linear representation 136 5.3 Parameter estimation 145 5.4 Discrete attributes 154 5.5 Conclusion 155 5.6 Further readings 156 References 157 6 Misclassification costs 159 6.1 Introduction 159 6.2 Cost representation 161 6.3 Incorporating misclassification costs 164 6.4 Effects of cost incorporation 176 6.5 Experimental procedure 180 6.6 Conclusion 184 6.7 Further readings 185 References 187 7 Classification model evaluation 189 7.1 Introduction 189 7.2 Performance measures 190 7.3 Evaluation procedures 213 7.4 Conclusion 231 7.5 Further readings 232 References 233 Part III Regression 235 8 Linear regression 237 8.1 Introduction 237 8.2 Linear representation 238 8.3 Parameter estimation 242 8.4 Discrete attributes 250 8.5 Advantages of linear models 251 8.6 Beyond linearity 252 8.7 Conclusion 258 8.8 Further readings 258 References 259 9 Regression trees 261 9.1 Introduction 261 9.2 Regression tree model 262 9.3 Growing 263 9.4 Pruning 274 9.5 Prediction 277 9.6 Weighted instances 278 9.7 Missing value handling 279 9.8 Piecewise linear regression 284 9.9 Conclusion 292 9.10 Further readings 292 References 293 10 Regression model evaluation 295 10.1 Introduction 295 10.2 Performance measures 296 10.3 Evaluation procedures 303 10.4 Conclusion 309 10.5 Further readings 309 References 310 Part IV Clustering 311 11 (Dis)similarity measures 313 11.1 Introduction 313 11.2 Measuring dissimilarity and similarity 313 11.3 Difference-based dissimilarity 314 11.4 Correlation-based similarity 321 11.5 Missing attribute values 324 11.6 Conclusion 325 11.7 Further readings 325 References 326 12 k-Centers clustering 328 12.1 Introduction 328 12.2 Algorithm scheme 330 12.3 k-Means 334 12.4 Beyond means 338 12.5 Beyond (fixed) k 342 12.6 Explicit cluster modeling 343 12.7 Conclusion 345 12.8 Further readings 345 References 347 13 Hierarchical clustering 349 13.1 Introduction 349 13.2 Cluster hierarchies 351 13.3 Agglomerative clustering 353 13.4 Divisive clustering 361 13.5 Hierarchical clustering visualization 364 13.6 Hierarchical clustering prediction 366 13.7 Conclusion 369 13.8 Further readings 370 References 371 14 Clustering model evaluation 373 14.1 Introduction 373 14.2 Per-cluster quality measures 376 14.3 Overall quality measures 385 14.4 External quality measures 393 14.5 Using quality measures 397 14.6 Conclusion 398 14.7 Further readings 398 References 399 Part V Getting Better Models 401 15 Model ensembles 403 15.1 Introduction 403 15.2 Model committees 404 15.3 Base models 406 15.4 Model aggregation 420 15.5 Specific ensemble modeling algorithms 431 15.6 Quality of ensemble predictions 448 15.7 Conclusion 449 15.8 Further readings 450 References 451 16 Kernel methods 454 16.1 Introduction 454 16.2 Support vector machines 457 16.3 Support vector regression 473 16.4 Kernel trick 482 16.5 Kernel functions 484 16.6 Kernel prediction 487 16.7 Kernel-based algorithms 489 16.8 Conclusion 494 16.9 Further readings 495 References 496 17 Attribute transformation 498 17.1 Introduction 498 17.2 Attribute transformation task 499 17.3 Simple transformations 504 17.4 Multiclass encoding 510 17.5 Conclusion 521 17.6 Further readings 521 References 522 18 Discretization 524 18.1 Introduction 524 18.2 Discretization task 525 18.3 Unsupervised discretization 530 18.4 Supervised discretization 533 18.5 Effects of discretization 551 18.6 Conclusion 553 18.7 Further readings 553 References 556 19 Attribute selection 558 19.1 Introduction 558 19.2 Attribute selection task 559 19.3 Attribute subset search 562 19.4 Attribute selection filters 568 19.5 Attribute selection wrappers 588 19.6 Effects of attribute selection 593 19.7 Conclusion 598 19.8 Further readings 599 References 600 20 Case studies 602 20.1 Introduction 602 20.2 Census income 605 20.3 Communities and crime 631 20.4 Cover type 640 20.5 Conclusion 654 20.6 Further readings 655 References 655 Closing 657 A Notation 659 A.1 Attribute values 659 A.2 Data subsets 659 A.3 Probabilities 660 B R packages 661 B.1 CRAN packages 661 B.2 DMR packages 662 B.3 Installing packages 663 References 664 C Datasets 666 Index 667

Read more less

Book SynopsisPraise for the First Edition . a well-written book on data analysis and data mining that provides an excellent foundation. CHOICE This is a must-read book for learning practical statistics and data analysis.Table of ContentsPREFACE ix 1 INTRODUCTION 1 1.1 Overview 1 1.2 Sources of Data 2 1.3 Process for Making Sense of Data 3 1.4 Overview of Book 13 1.5 Summary 16 Further Reading 16 2 DESCRIBING DATA 17 2.1 Overview 17 2.2 Observations and Variables 18 2.3 Types of Variables 20 2.4 Central Tendency 22 2.5 Distribution of the Data 24 2.6 Confidence Intervals 36 2.7 Hypothesis Tests 40 Exercises 42 Further Reading 45 3 PREPARING DATA TABLES 47 3.1 Overview 47 3.2 Cleaning the Data 48 3.3 Removing Observations and Variables 49 3.4 Generating Consistent Scales Across Variables 49 3.5 New Frequency Distribution 51 3.6 Converting Text to Numbers 52 3.7 Converting Continuous Data to Categories 53 3.8 Combining Variables 54 3.9 Generating Groups 54 3.10 Preparing Unstructured Data 55 Exercises 57 Further Reading 57 4 UNDERSTANDING RELATIONSHIPS 59 4.1 Overview 59 4.2 Visualizing Relationships Between Variables 60 4.3 Calculating Metrics About Relationships 69 Exercises 81 Further Reading 82 5 IDENTIFYING AND UNDERSTANDING GROUPS 83 5.1 Overview 83 5.2 Clustering 88 5.3 Association Rules 111 5.4 Learning Decision Trees from Data 122 Exercises 137 Further Reading 140 6 BUILDING MODELS FROM DATA 141 6.1 Overview 141 6.2 Linear Regression 149 6.3 Logistic Regression 161 6.4 k-Nearest Neighbors 167 6.5 Classification and Regression Trees 172 6.6 Other Approaches 178 Exercises 179 Further Reading 182 APPENDIX A ANSWERS TO EXERCISES 185 APPENDIX B HANDS-ON TUTORIALS 191 B.1 Tutorial Overview 191 B.2 Access and Installation 191 B.3 Software Overview 192 B.4 Reading in Data 193 B.5 Preparation Tools 195 B.6 Tables and Graph Tools 199 B.7 Statistics Tools 202 B.8 Grouping Tools 204 B.9 Models Tools 207 B.10 Apply Model 211 B.11 Exercises 211 BIBLIOGRAPHY 227 INDEX 231

Read more less

Book SynopsisCollecting, analyzing, and extracting valuable information from a large amount of data requires easily accessible, robust, computational and analytical tools. Data Mining and Business Analytics with R utilizes the open source software R for the analysis, exploration, and simplification of large high-dimensional data sets.Trade Review"I first taught a Ph.D. level course in business applications of data mining 10 years ago. I regularly search the web, looking for business-oriented data mining books, and this is the first one I have found that is suitable for an MS in business analytics. I plan to use it. Anyone who teaches such a class and is inclined toward R should consider this text." (Journal of the American Statistical Association, 1 January 2014)Table of ContentsPreface ix Acknowledgments xi 1. Introduction 1 Reference 6 2. Processing the Information and Getting to Know Your Data 7 2.1 Example 1: 2006 Birth Data 7 2.2 Example 2: Alumni Donations 17 2.3 Example 3: Orange Juice 31 References 39 3. Standard Linear Regression 40 3.1 Estimation in R 43 3.2 Example 1: Fuel Efficiency of Automobiles 43 3.3 Example 2: Toyota Used-Car Prices 47 Appendix 3.A The Effects of Model Overfitting on the Average Mean Square Error of the Regression Prediction 53 References 54 4. Local Polynomial Regression: a Nonparametric Regression Approach 55 4.1 Model Selection 56 4.2 Application to Density Estimation and the Smoothing of Histograms 58 4.3 Extension to the Multiple Regression Model 58 4.4 Examples and Software 58 References 65 5. Importance of Parsimony in Statistical Modeling 67 5.1 How Do We Guard Against False Discovery 67 References 70 6. Penalty-Based Variable Selection in Regression Models with Many Parameters (LASSO) 71 6.1 Example 1: Prostate Cancer 74 6.2 Example 2: Orange Juice 78 References 82 7. Logistic Regression 83 7.1 Building a Linear Model for Binary Response Data 83 7.2 Interpretation of the Regression Coefficients in a Logistic Regression Model 85 7.3 Statistical Inference 85 7.4 Classification of New Cases 86 7.5 Estimation in R 87 7.6 Example 1: Death Penalty Data 87 7.7 Example 2: Delayed Airplanes 92 7.8 Example 3: Loan Acceptance 100 7.9 Example 4: German Credit Data 103 References 107 8. Binary Classification, Probabilities, and Evaluating Classification Performance 108 8.1 Binary Classification 108 8.2 Using Probabilities to Make Decisions 108 8.3 Sensitivity and Specificity 109 8.4 Example: German Credit Data 109 9. Classification Using a Nearest Neighbor Analysis 115 9.1 The k-Nearest Neighbor Algorithm 116 9.2 Example 1: Forensic Glass 117 9.3 Example 2: German Credit Data 122 Reference 125 10. The Na¨ýve Bayesian Analysis: a Model for Predicting a Categorical Response from Mostly Categorical Predictor Variables 126 10.1 Example: Delayed Airplanes 127 Reference 131 11. Multinomial Logistic Regression 132 11.1 Computer Software 134 11.2 Example 1: Forensic Glass 134 11.3 Example 2: Forensic Glass Revisited 141 Appendix 11.A Specification of a Simple Triplet Matrix 147 References 149 12. More on Classification and a Discussion on Discriminant Analysis 150 12.1 Fisher’s Linear Discriminant Function 153 12.2 Example 1: German Credit Data 154 12.3 Example 2: Fisher Iris Data 156 12.4 Example 3: Forensic Glass Data 157 12.5 Example 4: MBA Admission Data 159 Reference 160 13. Decision Trees 161 13.1 Example 1: Prostate Cancer 167 13.2 Example 2: Motorcycle Acceleration 179 13.3 Example 3: Fisher Iris Data Revisited 182 14. Further Discussion on Regression and Classification Trees, Computer Software, and Other Useful Classification Methods 185 14.1 R Packages for Tree Construction 185 14.2 Chi-Square Automatic Interaction Detection (CHAID) 186 14.3 Ensemble Methods: Bagging, Boosting, and Random Forests 188 14.4 Support Vector Machines (SVM) 192 14.5 Neural Networks 192 14.6 The R Package Rattle: A Useful Graphical User Interface for Data Mining 193 References 195 15. Clustering 196 15.1 k-Means Clustering 196 15.2 Another Way to Look at Clustering: Applying the Expectation-Maximization (EM) Algorithm to Mixtures of Normal Distributions 204 15.3 Hierarchical Clustering Procedures 212 References 219 16. Market Basket Analysis: Association Rules and Lift 220 16.1 Example 1: Online Radio 222 16.2 Example 2: Predicting Income 227 References 234 17. Dimension Reduction: Factor Models and Principal Components 235 17.1 Example 1: European Protein Consumption 238 17.2 Example 2: Monthly US Unemployment Rates 243 18. Reducing the Dimension in Regressions with Multicollinear Inputs: Principal Components Regression and Partial Least Squares 247 18.1 Three Examples 249 References 257 19. Text as Data: Text Mining and Sentiment Analysis 258 19.1 Inverse Multinomial Logistic Regression 259 19.2 Example 1: Restaurant Reviews 261 19.3 Example 2: Political Sentiment 266 Appendix 19.A Relationship Between the Gentzkow Shapiro Estimate of “Slant” and Partial Least Squares 268 References 271 20. Network Data 272 20.1 Example 1: Marriage and Power in Fifteenth Century Florence 274 20.2 Example 2: Connections in a Friendship Network 278 References 292 Appendix A: Exercises 293 Exercise 1 294 Exercise 2 294 Exercise 3 296 Exercise 4 298 Exercise 5 299 Exercise 6 300 Exercise 7 301 Appendix B: References 338 Index 341

Read more less

Book SynopsisA comprehensive introduction to statistical methods for data mining and knowledge discovery.Table of ContentsPreface xiii Part I Background: Introductory Statistical Analytics 1 1 Data analytics and data mining 3 1.1 Knowledge discovery: finding structure in data 3 1.2 Data quality versus data quantity 5 1.3 Statistical modeling versus statistical description 7 2 Basic probability and statistical distributions 10 2.1 Concepts in probability 10 2.1.1 Probability rules 11 2.1.2 Random variables and probability functions 12 2.1.3 Means, variances, and expected values 17 2.1.4 Median, quartiles, and quantiles 18 2.1.5 Bivariate expected values, covariance, and correlation 20 2.2 Multiple random variables∗ 21 2.3 Univariate families of distributions 23 2.3.1 Binomial distribution 23 2.3.2 Poisson distribution 26 2.3.3 Geometric distribution 27 2.3.4 Negative binomial distribution 27 2.3.5 Discrete uniform distribution 28 2.3.6 Continuous uniform distribution 29 2.3.7 Exponential distribution 29 2.3.8 Gamma and chi-square distributions 30 2.3.9 Normal (Gaussian) distribution 32 2.3.10 Distributions derived from normal 37 2.3.11 The exponential family 41 3 Data manipulation 49 3.1 Random sampling 49 3.2 Data types 51 3.3 Data summarization 52 3.3.1 Means, medians, and central tendency 52 3.3.2 Summarizing variation 56 3.3.3 Summarizing (bivariate) correlation 59 3.4 Data diagnostics and data transformation 60 3.4.1 Outlier analysis 60 3.4.2 Entropy∗ 62 3.4.3 Data transformation 64 3.5 Simple smoothing techniques 65 3.5.1 Binning 66 3.5.2 Moving averages∗ 67 3.5.3 Exponential smoothing∗ 69 4 Data visualization and statistical graphics 76 4.1 Univariate visualization 77 4.1.1 Strip charts and dot plots 77 4.1.2 Boxplots 79 4.1.3 Stem-and-leaf plots 81 4.1.4 Histograms and density estimators 83 4.1.5 Quantile plots 87 4.2 Bivariate and multivariate visualization 89 4.2.1 Pie charts and bar charts 90 4.2.2 Multiple boxplots and QQ plots 95 4.2.3 Scatterplots and bubble plots 98 4.2.4 Heatmaps 102 4.2.5 Time series plots∗ 105 5 Statistical inference 115 5.1 Parameters and likelihood 115 5.2 Point estimation 117 5.2.1 Bias 118 5.2.2 The method of moments 118 5.2.3 Least squares/weighted least squares 119 5.2.4 Maximum likelihood∗ 120 5.3 Interval estimation 123 5.3.1 Confidence intervals 123 5.3.2 Single-sample intervals for normal (Gaussian) parameters 124 5.3.3 Two-sample intervals for normal (Gaussian) parameters 128 5.3.4 Wald intervals and likelihood intervals∗ 131 5.3.5 Delta method intervals∗ 135 5.3.6 Bootstrap intervals∗ 137 5.4 Testing hypotheses 138 5.4.1 Single-sample tests for normal (Gaussian) parameters 140 5.4.2 Two-sample tests for normal (Gaussian) parameters 142 5.4.3 Walds tests, likelihood ratio tests, and ‘exact’ tests∗ 145 5.5 Multiple inferences∗ 148 5.5.1 Bonferroni multiplicity adjustment 149 5.5.2 False discovery rate 151 Part II Statistical Learning and Data Analytics 161 6 Techniques for supervised learning: simple linear regression 163 6.1 What is “supervised learning?” 163 6.2 Simple linear regression 164 6.2.1 The simple linear model 164 6.2.2 Multiple inferences and simultaneous confidence bands 171 6.3 Regression diagnostics 175 6.4 Weighted least squares (WLS) regression 184 6.5 Correlation analysis 187 6.5.1 The correlation coefficient 187 6.5.2 Rank correlation 190 7 Techniques for supervised learning: multiple linear regression 198 7.1 Multiple linear regression 198 7.1.1 Matrix formulation 199 7.1.2 Weighted least squares for the MLR model 200 7.1.3 Inferences under the MLR model 201 7.1.4 Multicollinearity 208 7.2 Polynomial regression 210 7.3 Feature selection 211 7.3.1 R2p plots 212 7.3.2 Information criteria: AIC and BIC 215 7.3.3 Automated variable selection 216 7.4 Alternative regression methods∗ 223 7.4.1 Loess 224 7.4.2 Regularization: ridge regression 230 7.4.3 Regularization and variable selection: the Lasso 238 7.5 Qualitative predictors: ANOVA models 242 8 Supervised learning: generalized linear models 258 8.1 Extending the linear regression model 258 8.1.1 Nonnormal data and the exponential family 258 8.1.2 Link functions 259 8.2 Technical details for GLiMs∗ 259 8.2.1 Estimation 260 8.2.2 The deviance function 261 8.2.3 Residuals 262 8.2.4 Inference and model assessment 264 8.3 Selected forms of GLiMs 265 8.3.1 Logistic regression and binary-data GLiMs 265 8.3.2 Trend testing with proportion data 271 8.3.3 Contingency tables and log-linear models 273 8.3.4 Gamma regression models 281 9 Supervised learning: classification 291 9.1 Binary classification via logistic regression 292 9.1.1 Logistic discriminants 292 9.1.2 Discriminant rule accuracy 296 9.1.3 ROC curves 297 9.2 Linear discriminant analysis (LDA) 297 9.2.1 Linear discriminant functions 297 9.2.2 Bayes discriminant/classification rules 302 9.2.3 Bayesian classification with normal data 303 9.2.4 Naïve Bayes classifiers 308 9.3 k-Nearest neighbor classifiers 308 9.4 Tree-based methods 312 9.4.1 Classification trees 312 9.4.2 Pruning 314 9.4.3 Boosting 321 9.4.4 Regression trees 321 9.5 Support vector machines∗ 322 9.5.1 Separable data 322 9.5.2 Nonseparable data 325 9.5.3 Kernel transformations 326 10 Techniques for unsupervised learning: dimension reduction 341 10.1 Unsupervised versus supervised learning 341 10.2 Principal component analysis 342 10.2.1 Principal components 342 10.2.2 Implementing a PCA 344 10.3 Exploratory factor analysis 351 10.3.1 The factor analytic model 351 10.3.2 Principal factor estimation 353 10.3.3 Maximum likelihood estimation 354 10.3.4 Selecting the number of factors 355 10.3.5 Factor rotation 356 10.3.6 Implementing an EFA 357 10.4 Canonical correlation analysis∗ 361 11 Techniques for unsupervised learning: clustering and association 373 11.1 Cluster analysis 373 11.1.1 Hierarchical clustering 376 11.1.2 Partitioned clustering 384 11.2 Association rules/market basket analysis 395 11.2.1 Association rules for binary observations 396 11.2.2 Measures of rule quality 397 11.2.3 The Apriori algorithm 398 11.2.4 Statistical measures of association quality 402 A Matrix manipulation 411 A.1 Vectors and matrices 411 A.2 Matrix algebra 412 A.3 Matrix inversion 414 A.4 Quadratic forms 415 A.5 Eigenvalues and eigenvectors 415 A.6 Matrix factorizations 416 A.6.1 QR decomposition 417 A.6.2 Spectral decomposition 417 A.6.3 Matrix square root 417 A.6.4 Singular value decomposition 418 A.7 Statistics via matrix operations 419 B Brief introduction to R 421 B.1 Data entry and manipulation 422 B.2 A turbo-charged calculator 426 B.3 R functions 427 B.3.1 Inbuilt R functions 427 B.3.2 Flow control 429 B.3.3 User-defined functions 429 B.4 R packages 430 References 432 Index 453

Read more less

Book SynopsisAddresses the impacts of data mining on education and reviews applications in educational research teaching, and learning This book discusses the insights, challenges, issues, expectations, and practical implementation of data mining (DM) within educational mandates. Initial series of chapters offer a general overview of DM, Learning Analytics (LA), and data collection models in the context of educational research, while also defining and discussing data mining's four guiding principles prediction, clustering, rule association, and outlier detection. The next series of chapters showcase the pedagogical applications of Educational Data Mining (EDM) and feature case studies drawn from Business, Humanities, Health Sciences, Linguistics, and Physical Sciences education that serve to highlight the successes and some of the limitations of data mining research applications in educational settings. The remaining chapters focus exclusively on EDM's emerging role in helping to aTable of ContentsNotes on Contributors xi Introduction: Education At Computational Crossroads xxiiiSamira ElAtia, Donald Ipperciel, and Osmar R. Zaïane Part I At The Intersection of Two Fields: EDM 1 Chapter 1 Educational Process Mining: A Tutorial and Case Study Using Moodle Data Sets 3Cristóbal Romero, Rebeca Cerezo, Alejandro Bogarín, and Miguel Sanchez‐Santillán 1.1 Background 5 1.2 Data Description and Preparation 7 1.2.1 Preprocessing Log Data 7 1.2.2 Clustering Approach for Grouping Log Data 11 1.3 Working with ProM 16 1.3.1 Discovered Models 19 1.3.2 Analysis of the Models’ Performance 23 1.4 Conclusion 26 Acknowledgments 27 References 27 Chapter 2 On Big Data And Text Mining in the Humanities29Geoffrey Rockwell and Bettina Berendt 2.1 Busa and the Digital Text 30 2.2 Thesaurus Linguae Graecae and the Ibycus Computer as Infrastructure 32 2.2.1 Complete Data Sets 33 2.3 Cooking with Statistics 35 2.4 Conclusions 37 References 38 Chapter 3 Finding Predictors in Higher Education41David Eubanks, William Evers Jr., and Nancy Smith 3.1 Contrasting Traditional and Computational Methods 42 3.2 Predictors and Data Exploration 45 3.3 Data Mining Application: An Example 50 3.4 Conclusions 52 References 53 Chapter 4 Educational Data Mining: A MOOC Experience55Ryan S. Baker, Yuan Wang, Luc Paquette, Vincent Aleven, Octav Popescu, Jonathan Sewall, Carolyn Rosé, Gaurav Singh Tomar, Oliver Ferschke, Jing Zhang, Michael J. Cennamo, Stephanie Ogden, Therese Condit, José Diaz, Scott Crossley, Danielle S. McNamara, Denise K. Comer, Collin F. Lynch, Rebecca Brown, Tiffany Barnes, and Yoav Bergner 4.1 Big Data in Education: The Course 55 4.1.1 Iteration 1: Coursera 55 4.1.2 Iteration 2: edX 56 4.2 Cognitive Tutor Authoring Tools 57 4.3 Bazaar 58 4.4 Walkthrough 58 4.4.1 Course Content 58 4.4.2 Research on BDEMOOC 61 4.5 Conclusion 65 Acknowledgments 65 References 65 Chapter 5 Data Mining and Action Research 67Ellina Chernobilsky, Edith Ries, and Joanne Jasmine 5.1 Process 69 5.2 Design Methodology 71 5.3 Analysis and Interpretation of Data 72 5.3.1 Quantitative Data Analysis and Interpretation 73 5.3.2 Qualitative Data Analysis and Interpretation 74 5.4 Challenges 75 5.5 Ethics 76 5.6 Role of Administration in the Data Collection Process 76 5.7 Conclusion 77 References 77 Part II Pedagogical Applications of EDM79 Chapter 6 Design of an Adaptive Learning System and Educational Data Mining81Zhiyong Liu and Nick Cercone 6.1 Dimensionalities of the User Model in ALS 83 6.2 Collecting Data for ALS 85 6.3 Data Mining in ALS 86 6.3.1 Data Mining for User Modeling 87 6.3.2 Data Mining for Knowledge Discovery 88 6.4 ALS Model and Function Analyzing 90 6.4.1 Introduction of Module Functions 90 6.4.2 Analyzing the Workflow 93 6.5 Future Works 94 6.6 Conclusions 94 Acknowledgment 95 References 95 Chapter 7 The “Geometry” of Naive Bayes: Teaching Probabilities by “Drawing” Them99Giorgio Maria Di Nunzio 7.1 Introduction 99 7.1.1 Main Contribution 100 7.1.2 Related Works 101 7.2 The Geometry of NB Classification 102 7.2.1 Mathematical Notation 102 7.2.2 Bayesian Decision Theory 103 7.3 Two-Dimensional Probabilities 105 7.3.1 Working with Likelihoods and Priors Only 107 7.3.2 De‐normalizing Probabilities 108 7.3.3 NB Approach 109 7.3.4 Bernoulli Naïve Bayes 110 7.4 A New Decision Line: Far from the Origin 111 7.4.1 De‐normalization Makes (Some) Problems Linearly Separable 112 7.5 Likelihood Spaces, When Logarithms make a Difference (or a SUM) 114 7.5.1 De‐normalization Makes (Some) Problems Linearly Separable 115 7.5.2 A New Decision in Likelihood Spaces 116 7.5.3 A Real Case Scenario: Text Categorization 117 7.6 Final Remarks 118 References 119 Chapter 8 Examining the Learning Networks of a MOOC121Meaghan Brugha and Jean‐Paul Restoule 8.1 Review of Literature 122 8.2 Course Context 124 8.3 Results and Discussion 125 8.4 Recommendations for Future Research 133 8.5 Conclusions 134 References 135 Chapter 9 Exploring the Usefulness of Adaptive ELearning Laboratory Environments in Teaching Medical Science139Thuan Thai and Patsie Polly 9.1 Introduction 139 9.2 Software for Learning and Teaching 141 9.2.1 Reflective Practice: ePortfolio 141 9.2.2 Online Quizzes 143 9.2.3 Online Practical Lessons 144 9.2.4 Virtual Laboratories 145 9.2.5 The Gene Suite 147 9.3 Potential Limitations 152 9.4 Conclusion 153 Acknowledgments 153 References 154 Chapter 10 Investigating Co‐Occurrence Patterns of Learners’ Grammatical Errors across Proficiency Levels and Essay Topics Based on Association Analysis 157Yutaka Ishii 10.1 Introduction 157 10.1.1 The Relationship between Data Mining and Educational Research 157 10.1.2 English Writing Instruction in the Japanese Context 158 10.2 Literature Review 159 10.3 Method 160 10.3.1 Konan‐JIEM Learner Corpus 160 10.3.2 Association Analysis 162 10.4 Experiment 1 162 10.5 Experiment 2 163 10.6 Discussion and Conclusion 164 Appendix A: Example of Learner’s Essay (University Life) 164 Appendix B: Support Values of all Topics 165 Appendix C: Support Values of Advanced, Intermediate, and Beginner Levels of Learners 168 References 169 Part III EDM and Educational Research 173 Chapter 11 Mining Learning Sequences in MOOCs: Does Course Design Constrain Students’ Behaviors Or Do Students Shape Their Own Learning? 175Lorenzo Vigentini, Simon McIntyre, Negin Mirriahi, and Dennis Alonzo 11.1 Introduction 175 11.1.1 Perceptions and Challenges of MOOC Design 176 11.1.2 What Do We Know About Participants’ Navigation: Choice and Control 177 11.2 Data Mining in MOOCs: Related Work 178 11.2.1 Setting the Hypotheses 179 11.3 The Design and Intent of the LTTO MOOC 180 11.3.1 Course Grading and Certification 183 11.3.2 Delivering the Course 183 11.3.3 Operationalize Engagement, Personal Success, and Course Success in LTTO 184 11.4 Data Analysis 184 11.4.1 Approaches to Process the Data Sources 185 11.4.2 LTTO in Numbers 186 11.4.3 Characterizing Patterns of Completion and Achievement 186 11.4.4 Redefining Participation and Engagement 189 11.5 Mining Behaviors and Intents 191 11.5.1 Participants’ Intent and Behaviors: A Classification Model 191 11.5.2 Natural Clustering Based on Behaviors 194 11.5.3 Stated Intents and Behaviors: Are They Related? 198 11.6 Closing the Loop: Informing Pedagogy and Course Enhancement 198 11.6.1 Conclusions, Lessons Learnt, and Future Directions 200 References 201 Chapter 12 Understanding Communication Patterns in MOOCs: Combining Data Mining and Qualitative Methods 207Rebecca Eynon, Isis Hjorth, Taha Yasseri, and Nabeel Gillani 12.1 Introduction 207 12.2 Methodological Approaches to Understanding Communication Patterns in MOOCs 209 12.3 Description 210 12.3.1 Structural Connections 211 12.4 Examining Dialogue 213 12.5 Interpretative Models 214 12.6 Understanding Experience 215 12.7 Experimentation 216 12.8 Future Research 217 References 218 Chapter 13 An Example of Data Mining: Exploring The Relationship Between Applicant Attributes and Academic Measures of Success in a Pharmacy Program 223Dion Brocks and Ken Cor 13.1 Introduction 223 13.2 Methods 225 13.3 Results 228 13.4 Discussion 230 13.4.1 Prerequisite Predictors 230 13.4.2 Demographic Predictors 232 13.5 Conclusion 234 Appendix A 234 References 236 Chapter 14 A New Way of Seeing: Using a Data Mining Approach to Understand Children’s Views of Diversity and “Difference” in Picture Books237Robin A. Moeller and Hsin‐liang Chen 14.1 Introduction 237 14.2 Study 1: Using Data Mining to Better Understand Perceptions of Race 238 14.2.1 Background 238 14.2.2 Research Questions 239 14.2.3 Methods 240 14.2.4 Findings 240 14.2.5 Discussion 248 14.3 Study 2: Translating Data Mining Results to Picture Book Concepts of “Difference” 248 14.3.1 Background 248 14.3.2 Research Questions 249 14.3.3 Methodology 250 14.3.4 Findings 250 14.3.5 Discussion and Implications 252 14.4 Conclusions 252 References 252 Chapter 15 Data Mining with Natural Language Processing and Corpus Linguistics: Unlocking Access to School Children’s Language in Diverse Contexts to Improve Instructional and Assessment Practices255Alison L. Bailey, Anne Blackstock‐Bernstein, Eve Ryan, and Despina Pitsoulakis 15.1 Introduction 255 15.2 Identifying the Problem 256 15.3 Use of Corpora and Technology in Language Instruction and Assessment 261 15.3.1 Language Corpora in ESL and EFL Teaching and Learning 261 15.3.2 Previous Extensions of Corpus Linguistics to School‐Age Language 262 15.3.3 Corpus Linguistics in Language Assessment 263 15.3.4 Big Data Purposes, Techniques, and Technology 264 15.4 Creating a School‐Age Learner Corpus and Digital Data Analytics System 266 15.4.1 Language Measures Included in DRGON 267 15.4.2 The DLLP as a Promising Practice 268 15.5 Next Steps, “Modest Data,” and Closing Remarks 269 Acknowledgments 271 Appendix A: Examples of Oral and Written Explanation Elicitation Prompts 272 References 272 Index 277

Read more less

Book SynopsisA step-by-step guide to data mining applications in CRM. Following a handbook approach, this book bridges the gap between analytics and their use in everyday marketing, providing guidance on solving real business problems using data mining techniques. The book is organized into three parts.Table of ContentsPreface xiii Acknowledgments xv 1 An overview of data mining: The applications, the methodology, the algorithms, and the data 1 1.1 The applications 1 1.2 The methodology 4 1.3 The algorithms 6 1.3.1 Supervised models 6 1.3.1.1 Classification models 7 1.3.1.2 Estimation (regression) models 9 1.3.1.3 Feature selection (field screening) 10 1.3.2 Unsupervised models 10 1.3.2.1 Cluster models 11 1.3.2.2 Association (affinity) and sequence models 12 1.3.2.3 Dimensionality reduction models 14 1.3.2.4 Record screening models 14 1.4 The data 15 1.4.1 The mining datamart 16 1.4.2 The required data per industry 16 1.4.3 The customer “signature”: from the mining datamart to the enriched, marketing reference table 16 1.5 Summary 20 Part I The Methodology 21 2 Classification modeling methodology 23 2.1 An overview of the methodology for classification modeling 23 2.2 Business understanding and design of the process 24 2.2.1 Definition of the business objective 24 2.2.2 Definition of the mining approach and of the data model 26 2.2.3 Design of the modeling process 27 2.2.3.1 Defining the modeling population 27 2.2.3.2 Determining the modeling (analysis) level 28 2.2.3.3 Definition of the target event and population 28 2.2.3.4 Deciding on time frames 29 2.3 Data understanding, preparation, and enrichment 33 2.3.1 Investigation of data sources 34 2.3.2 Selecting the data sources to be used 34 2.3.3 Data integration and aggregation 35 2.3.4 Data exploration, validation, and cleaning 35 2.3.5 Data transformations and enrichment 38 2.3.6 Applying a validation technique 40 2.3.6.1 Split or Holdout validation 40 2.3.6.2 Cross or n‐fold validation 45 2.3.6.3 Bootstrap validation 47 2.3.7 Dealing with imbalanced and rare outcomes 48 2.3.7.1 Balancing 48 2.3.7.2 Applying class weights 53 2.4 Classification modeling 57 2.4.1 Trying different models and parameter settings 57 2.4.2 Combining models 60 2.4.2.1 Bagging 61 2.4.2.2 Boosting 62 2.4.2.3 Random Forests 63 2.5 Model evaluation 64 2.5.1 Thorough evaluation of the model accuracy 65 2.5.1.1 Accuracy measures and confusion matrices 66 2.5.1.2 Gains, Response, and Lift charts 70 2.5.1.3 ROC curve 78 2.5.1.4 Profit/ROI charts 81 2.5.2 Evaluating a deployed model with test–control groups 85 2.6 Model deployment 88 2.6.1 Scoring customers to roll the marketing campaign 88 2.6.1.1 Building propensity segments 93 2.6.2 Designing a deployment procedure and disseminating the results 94 2.7 Using classification models in direct marketing campaigns 94 2.8 Acquisition modeling 95 2.8.1.1 Pilot campaign 95 2.8.1.2 Profiling of high‐value customers 96 2.9 Cross‐selling modeling 97 2.9.1.1 Pilot campaign 98 2.9.1.2 Product uptake 98 2.9.1.3 Profiling of owners 99 2.10 Offer optimization with next best product campaigns 100 2.11 Deep‐selling modeling 102 2.11.1.1 Pilot campaign 102 2.11.1.2 Usage increase 103 2.11.1.3 Profiling of customers with heavy product usage 104 2.12 Up‐selling modeling 105 2.12.1.1 Pilot campaign 105 2.12.1.2 Product upgrade 107 2.12.1.3 Profiling of “premium” product owners 107 2.13 Voluntary churn modeling 108 2.14 Summary of what we’ve learned so far: it’s not about the tool or the modeling algorithm. It’s about the methodology and the design of the process 111 3 Behavioral segmentation methodology 112 3.1 An introduction to customer segmentation 112 3.2 An overview of the behavioral segmentation methodology 113 3.3 Business understanding and design of the segmentation process 115 3.3.1 Definition of the business objective 115 3.3.2 Design of the modeling process 115 3.3.2.1 Selecting the segmentation population 115 3.3.2.2 Selection of the appropriate segmentation criteria 116 3.3.2.3 Determining the segmentation level 116 3.3.2.4 Selecting the observation window 116 3.4 Data understanding, preparation, and enrichment 117 3.4.1 Investigation of data sources 117 3.4.2 Selecting the data to be used 117 3.4.3 Data integration and aggregation 118 3.4.4 Data exploration, validation, and cleaning 118 3.4.5 Data transformations and enrichment 122 3.4.6 Input set reduction 124 3.5 Identification of the segments with cluster modeling 126 3.6 Evaluation and profiling of the revealed segments 128 3.6.1 “Technical” evaluation of the clustering solution 128 3.6.2 Profiling of the revealed segments 132 3.6.3 Using marketing research information to evaluate the clusters and enrich their profiles 138 3.6.4 Selecting the optimal cluster solution and labeling the segments 139 3.7 Deployment of the segmentation solution, design and delivery of differentiated strategies 139 3.7.1 Building the customer scoring model for updating the segments 140 3.7.1.1 Building a Decision Tree for scoring: fine‐tuning the segments 141 3.7.2 Distribution of the segmentation information 141 3.7.3 Design and delivery of differentiated strategies 142 3.8 Summary 142 Part II The Algorithms 143 4 Classification algorithms 145 4.1 Data mining algorithms for classification 145 4.2 An overview of Decision Trees 146 4.3 The main steps of Decision Tree algorithms 146 4.3.1 Handling of predictors by Decision Tree models 148 4.3.2 Using terminating criteria to prevent trivial tree growing 149 4.3.3 Tree pruning 150 4.4 CART, C5.0/C4.5, and CHAID and their attribute selection measures 150 4.4.1 The Gini index used by CART 151 4.4.2 The Information Gain Ratio index used by C5.0/C4.5 155 4.4.3 The chi‐square test used by CHAID 158 4.5 Bayesian networks 170 4.6 Naive Bayesian networks 172 4.7 Bayesian belief networks 176 4.8 Support vector machines 184 4.8.1 Linearly separable data 184 4.8.2 Linearly inseparable data 187 4.9 Summary 191 5 Segmentation algorithms 192 5.1 Segmenting customers with data mining algorithms 192 5.2 Principal components analysis 192 5.2.1 How many components to extract? 194 5.2.1.1 The eigenvalue (or latent root) criterion 196 5.2.1.2 The percentage of variance criterion 197 5.2.1.3 The scree test criterion 198 5.2.1.4 The interpretability and business meaning of the components 198 5.2.2 What is the meaning of each component? 199 5.2.3 Moving along with the component scores 201 5.3 Clustering algorithms 203 5.3.1 Clustering with K‐means 204 5.3.2 Clustering with TwoStep 211 5.4 Summary 213 Part III The Case Studies 215 6 A voluntary churn propensity model for credit card holders 217 6.1 The business objective 217 6.2 The mining approach 218 6.2.1 Designing the churn propensity model process 218 6.2.1.1 Selecting the data sources and the predictors 218 6.2.1.2 Modeling population and level of data 218 6.2.1.3 Target population and churn definition 218 6.2.1.4 Time periods and historical information required 219 6.3 The data dictionary 219 6.4 The data preparation procedure 221 6.4.1 From cards to customers: aggregating card‐level data 221 6.4.2 Enriching customer data 225 6.4.3 Defining the modeling population and the target field 228 6.5 Derived fields: the final data dictionary 232 6.6 The modeling procedure 232 6.6.1 Applying a Split (Holdout) validation: splitting the modelling dataset for evaluation purposes 232 6.6.2 Balancing the distribution of the target field 232 6.6.3 Setting the role of the fields in the model 239 6.6.4 Training the churn model 239 6.7 Understanding and evaluating the models 241 6.8 Model deployment: using churn propensities to target the retention campaign 248 6.9 The voluntary churn model revisited using RapidMiner 251 6.9.1 Loading the data and setting the roles of the attributes 251 6.9.2 Applying a Split (Holdout) validation and adjusting the imbalance of the target field’s distribution 252 6.9.3 Developing a Naïve Bayes model for identifying potential churners 252 6.9.4 Evaluating the performance of the model and deploying it to calculate churn propensities 253 6.10 Developing the churn model with Data Mining for Excel 254 6.10.1 Building the model using the Classify Wizard 256 6.10.2 Selecting the classification algorithm and its parameters 257 6.10.3 Applying a Split (Holdout) validation 257 6.10.4 Browsing the Decision Tree model 259 6.10.5 Validation of the model performance 259 6.10.6 Model deployment 263 6.11 Summary 266 7 Value segmentation and cross‐selling in retail 267 7.1 The business background and objective 267 7.2 An outline of the data preparation procedure 268 7.3 The data dictionary 272 7.4 The data preparation procedure 272 7.4.1 Pivoting and aggregating transactional data at a customer level 272 7.4.2 Enriching customer data and building the customer signature 276 7.5 The data dictionary of the modeling file 279 7.6 Value segmentation 285 7.6.1 Grouping customers according to their value 285 7.6.2 Value segments: exploration and marketing usage 287 7.7 The recency, frequency, and monetary (RFM) analysis 290 7.7.1 RFM basics 290 7.8 The RFM cell segmentation procedure 293 7.9 Setting up a cross‐selling model 295 7.10 The mining approach 295 7.10.1 Designing the cross‐selling model process 296 7.10.1.1 The data and the predictors 296 7.10.1.2 Modeling population and level of data 296 7.10.1.3 Target population and definition of target attribute 296 7.10.1.4 Time periods and historical information required 296 7.11 The modeling procedure 296 7.11.1 Preparing the test campaign and loading the campaign responses for modeling 298 7.11.2 Applying a Split (Holdout) validation: splitting the modelling dataset for evaluation purposes 298 7.11.3 Setting the roles of the attributes 299 7.11.4 Training the cross‐sell model 300 7.12 Browsing the model results and assessing the predictive accuracy of the classifiers 301 7.13 Deploying the model and preparing the cross‐selling campaign list 308 7.14 The retail case study using RapidMiner 309 7.14.1 Value segmentation and RFM cells analysis 310 7.14.2 Developing the cross‐selling model 312 7.14.3 Applying a Split (Holdout) validation 313 7.14.4 Developing a Decision Tree model with Bagging 314 7.14.5 Evaluating the performance of the model 317 7.14.6 Deploying the model and scoring customers 317 7.15 Building the cross‐selling model with Data Mining for Excel 319 7.15.1 Using the Classify Wizard to develop the model 319 7.15.2 Selecting a classification algorithm and setting the parameters 320 7.15.3 Applying a Split (Holdout) validation 322 7.15.4 Browsing the Decision Tree model 322 7.15.5 Validation of the model performance 325 7.15.6 Model deployment 329 7.16 Summary 331 8 Segmentation application in telecommunications 332 8.1 Mobile telephony: the business background and objective 332 8.2 The segmentation procedure 333 8.2.1 Selecting the segmentation population: the mobile telephony core segments 333 8.2.2 Deciding the segmentation level 335 8.2.3 Selecting the segmentation dimensions 335 8.2.4 Time frames and historical information analyzed 335 8.3 The data preparation procedure 335 8.4 The data dictionary and the segmentation fields 336 8.5 The modeling procedure 336 8.5.1 Preparing data for clustering: combining fields into data components 340 8.5.2 Identifying the segments with a cluster model 342 8.5.3 Profiling and understanding the clusters 344 8.5.4 Segmentation deployment 354 8.6 Segmentation using RapidMiner and K‐means cluster 354 8.6.1 Clustering with the K‐means algorithm 354 8.7 Summary 359 Bibliography 360 Index 362

Read more less

Book SynopsisSolutions Manual to accompany Statistical Data Analytics: Foundations for Data Mining, Informatics, and Knowledge Discovery A comprehensive introduction to statistical methods for data mining and knowledge discovery.Extensivesolutions using actual data (with sample R programming code) are provided, illustrating diverse informatic sources in genomics, biomedicine, ecological remote sensing, astronomy, socioeconomics, marketing, advertising and finance, among many others.Table of ContentsPreface vii 1 Data analytics and data mining 1 2 Basic probability and statistical distributions 3 3 Data manipulation 14 4 Data visualization and statistical graphics 28 5 Statistical inference 45 6 Techniques for supervised learning: simple linear regression 65 7 Techniques for supervised learning: multiple linear regression 90 8 Supervised learning: generalized linear models 134 9 Supervised learning: classification 154 10 Techniques for unsupervised learning: dimension reduction 185 11 Techniques for unsupervised learning: clustering and association 200 References 216

Read more less

Book SynopsisA reliable, cost-effective approach to extracting priceless business information from all sources of text Excavating actionable business insights from data is a complex undertaking, and that complexity is magnified by an order of magnitude when the focus is on documents and other text information.Table of ContentsForeword 1 Chapter 1: What is Text Mining? 1 1.1 What is it? 1 1.1.1 What is text mining in practice? 1 1.1.2 Where does text mining fit? 1 1.2 Why we care about text mining? 1 1.2.1 What are the consequences of ignoring text? 1 1.2.2 What are the benefits of text mining? 1 1.2.3 Setting Expectations: When text mining should (and should not) be used. 1 1.3 A basic workflow. How the process works. 1 1.4 What tools do I need to get started with this? 1 1.5 A Simple Example 1 1.6 A Real World Use Case 1 1.7 Summary 1 Chapter 2: Basics of text mining 1 2.1 What is Text Mining in a practical sense? 1 2.2 Types of Text Mining: Bag of Words. 1 2.2.1 Types of Text Mining: Syntactic Parsing. 1 2.3 The text mining process in context 1 2.4 String Manipulation: Number of Characters & Substitutions 1 2.4.1 String Manipulations: Paste, Character Splits & Extractions 1 2.5 Keyword Scanning 1 2.6 String Packages stringr & stringi 1 2.7 Preprocessing Steps for Bag of Words Text Mining 1 2.8 Spell Check 1 2.9 Frequent Terms & Associations 1 2.9 Delta Assist Wrap Up 1 2.10 Summary 1 Chapter 3: Common Text Mining Visualizations 1 3.1 A tale of two (or three) cultures 1 3.2 Simple Exploration: Term Frequency, Associations & Word Networks 1 3.2.1 Term Frequency 1 3.2.2 Word Associations 1 3.2.3 Word Networks 1 3.3 Simple Word Clusters: Hierarchical Dendrograms 1 3.4 Word Clouds: Overused but Effective 1 3.4.1 One Corpus Word Clouds 1 3.4.2 Comparing and Contrasting Corpora in Word Clouds 1 3.4.3 Polarized Tag Plot 1 3.5 Summary 1 Chapter 4: Sentiment Scoring 1 4.1 What is Sentiment Analysis? 1 4.2 Sentiment Scoring: Parlor Trick or Insightful? 1 4.3 Polarity: Simple Sentiment Scoring 1 4.3.1 Subjectivity Lexicons 1 4.3.2 Qdap’s Scoring for positive and negative word choice 1 4.3.3 Revisiting Word Clouds…Sentiment Word Clouds 1 4.4 Emoticons :) Dealing with these perplexing clues 1 4.4.1 Symbol-Based Emoticons Native to R 1 4.4.2 Punctuation Based Emoticons 1 4.4.3 Emoji 1 4.5 R’s Archived Sentiment Scoring Library 1 4.5 Sentiment the tidytext way 1 4.6 Airbnb.com Boston Wrap Up 1 4.7 Summary 1 Chapter 5: Hidden Structures: Clustering, String Distance, Text Vectors & Topic Modeling 1 5.1 What is clustering? 1 5.1.1 K Means Clustering 1 5.1.2 Spherical K Means Clustering 1 5.1.3 K Mediod Clustering 1 5.1.4 Evaluating the cluster approaches 1 5.2 Calculating & Exploring String Distance 1 5.2.1 What is string distance? 1 5.2.2 Fuzzy Matching-amatch, ain 1 5.2.3 Similarity Distances- stringdist, stringdistmatrix 1 5.3 LDA Topic Modeling Explained 1 5.3.2 Topic Modeling Case Study 1 5.3.2 LDA &LDAvis 1 5.4 Text to Vectors using “text2vec” 1 5.4.1 text2vec 1 5.5 Summary 1 Chapter 6: Document Classification: Finding Clickbait from Headlines 1 6.1 What is document classification? 1 6.2 Clickbait Case Study 1 6.2.2 Session & Data Set Up 1 6.2.3 GLMNET Training 1 6.2.4 GLMNET Test Predictions 1 6.2.5 Test Set Evaluation 1 6.2.6 Finding the most impactful words 1 6.2.7 Case study Wrap Up: Model Accuracy & Improving Performance Recommendations 1 6.3 Summary 1 Chapter 7: Predictive Modeling: Using text for classifying & predicting outcomes. 1 7.1 Classification Vs Prediction 1 7.2 Case Study I: Will this patient come back to the hospital? 1 7.2.2 Patient Readmission in the Text Mining Workflow 1 7.2.3 Session & Data Set Up 1 7.2.4 Patient Modeling 1 7.2.5 More Model KPI: AUC, Recall, Precision & F1 1 7.2.5.1 Additional Evaluation Metrics 1 7.2.6 Apply the model to new patients 1 7.2.7 Patient Readmission Conclusion 1 7.3 Case Study II: Predicting Box Office Success 1 7.3.2 Opening Weekend Revenue in the Text Mining Workflow 1 7.3.3 Session & Data Set Up 1 7.3.4 Opening Weekend Modeling 1 7.3.5 Model Evaluation 1 7.3.6 Apply the Model to new Movie Reviews 1 7.3.7 Movie Revenue Conclusion 1 7.4 Summary 1 Chapter 8: The OpenNLP Project 1 8.1 What is the OpenNLP project? 1 8.2 R’s OpenNLP Package 1 8.3 Named Entities in Hillary Clinton’s Email 1 8.3.1 R Session Set-up 1 8.3.2 Minor Text Cleaning 1 8.3.3 Using OpenNLP on a single email 1 8.3.4 Using OpenNLP on multiple documents 1 8.3.5 Revisiting the Text Mining Workflow 1 8.4 Analyzing the Named Entities 1 8.4.1 Worldwide Map of Hillary Clinton’s Location Mentions 1 8.4.2 Mapping Only European Locations 1 8.4.3 Entities & Polarity: How does Hillary Clinton feel about an entity? 1 8.4.4 Stock Charts for Entities 1 8.4.5 Reach an Insight or Conclusion about Hillary Clinton’s Emails 1 8.5 Summary 1 Chapter 9: Text Sources 1 9.1 Sourcing Text 1 9.2 Web Sources 1 9.2.1 Web Scraping a Single Page with rvest 1 9.2.2 Web Scraping Multiple Pages with rvest 1 9.2.3 Application Program Interfaces (APIs) 1 9.2.4 Newspaper Articles from The Guardian Newspaper 1 9.2.5 Tweets using the “twitteR” Package 1 9.2.6 Calling an API without a dedicated R package 1 9.2.7 Using jsonlite to access the New York Times 1 9.2.8 Using RCurl & XML to Parse Google News Feeds 1 9.2.9 The tm library Web-Mining Plugin 1 9.3 Getting Text from File Sources 1 9.3.1 Individual CSV, TXT and Microsoft Office Files 1 9.3.2 Reading multiple files quickly 1 9.3.2 Extracting Text from PDFs 1 9.3.3 Optical Character Recognition: Extracting Text from Images 1 9.4 Summary 1

Read more less

Book SynopsisAll the answers to your data science questions Over half of all businesses are using data science to generate insights and value from big data. How are they doing it? Data Science Strategy For Dummies answers all your questions about how to build a data science capability from scratch, starting with the what and the why of data science and covering what it takes to lead and nurture a top-notch team of data scientists. With this book, you'll learn how to incorporate data science as a strategic function into any business, large or small. Find solutions to your real-life challenges as you uncover the stories and value hidden within data. Learn exactly what data science is and why it's importantAdopt a data-driven mindset as the foundation to successUnderstand the processes and common roadblocks behind data scienceKeep your data science program focused on generating business valueNurture a top-quality data science team In non-technical language, Data Science Strategy For Dummies outlTable of ContentsForeword xv Introduction 1 About This Book 2 Foolish Assumptions 3 How This Book is Organized 3 Icons Used In This Book 4 Beyond The Book 4 Where To Go From Here 5 Part 1: Optimizing Your Data Science Investment 7 Chapter 1: Framing Data Science Strategy 9 Establishing the Data Science Narrative 10 Capture 11 Maintain 12 Process 13 Analyze 14 Communicate 16 Actuate 17 Sorting Out the Concept of a Data-driven Organization 19 Approaching data-driven 20 Being data obsessed 21 Sorting Out the Concept of Machine Learning 22 Defining and Scoping a Data Science Strategy 26 Objectives 26 Approach 27 Choices 27 Data 27 Legal 28 Ethics 28 Competence 28 Infrastructure 29 Governance and security 29 Commercial/business models 30 Measurements 30 Chapter 2: Considering the Inherent Complexity in Data Science 31 Diagnosing Complexity in Data Science 32 Recognizing Complexity as a Potential 33 Enrolling in Data Science Pitfalls 101 34 Believing that all data is needed 34 Thinking that investing in a data lake will solve all your problems 35 Focusing on AI when analytics is enough 36 Believing in the 1-tool approach 37 Investing only in certain areas 37 Leveraging the infrastructure for reporting rather than exploration 38 Underestimating the need for skilled data scientists 39 `Navigating the Complexity 40 Chapter 3: Dealing with Difficult Challenges 41 Getting Data from There to Here 41 Handling dependencies on data owned by others 42 Managing data transfer and computation across-country borders 43 Managing Data Consistency Across the Data Science Environment 44 Securing Explainability in AI 45 Dealing with the Difference between Machine Learning and Traditional Software Programming 47 Managing the Rapid AI Technology Evolution and Lack of Standardization 50 Chapter 4: Managing Change in Data Science 51 Understanding Change Management in Data Science 52 Approaching Change in Data Science 53 Recognizing what to avoid when driving change in data science 56 Using Data Science Techniques to Drive Successful Change 59 Using digital engagement tools 59 Applying social media analytics to identify stakeholder sentiment 60 Capturing reference data in change projects 61 Using data to select people for change roles 61 Automating change metrics 62 Getting Started 62 Part 2: Making Strategic Choices for Your Data 65 Chapter 5: Understanding the Past, Present, and Future of Data 67 Sorting Out the Basics of Data 68 Explaining traditional data versus big data 69 Knowing the value of data 71 Exploring Current Trends in Data 73 Data monetization 73 Responsible AI 74 Cloud-based data architectures 75 Computation and intelligence in the edge 75 Digital twins 77 Blockchain 78 Conversational platforms 79 Elaborating on Some Future Scenarios 80 Standardization for data science productivity 80 From data monetization scenarios to a data economy 82 An explosion of human/machine hybrid systems 82 Quantum computing will solve the unsolvable problems 83 Chapter 6: Knowing Your Data 85 Selecting Your Data 85 Describing Data 87 Exploring Data 89 Assessing Data Quality 93 Improving Data Quality 95 Chapter 7: Considering the Ethical Aspects of Data Science 97 Explaining AI Ethics 98 Addressing trustworthy artificial intelligence 99 Introducing Ethics by Design 101 Chapter 8: Becoming Data-driven 103 Understanding Why Data-Driven is a Must 103 Transitioning to a Data-Driven Model 105 Securing management buy-in and assigning a chief data officer (CDO) 106 Identifying the key business value aligned with the business maturity 107 Developing a Data Strategy 108 Caring for your data 109 Democratizing the data 109 Driving data standardization 110 Structuring the data strategy 110 Establishing a Data-Driven Culture and Mindset 111 Chapter 9: Evolving from Data-driven to Machine-driven 113 Digitizing the Data 114 Applying a Data-driven Approach 115 Automating Workflows 116 Introducing AI/ML capabilities 116 Part 3: Building a Successful Data Science Organization 119 Chapter 10: Building Successful Data Science Teams 121 Starting with the Data Science Team Leader 121 Adopting different leadership approaches 122 Approaching data science leadership 124 Finding the right data science leader or manager 124 Defining the Prerequisites for a Successful Team 125 Developing a team structure 125 Establishing an infrastructure 126 Ensuring data availability 126 Insisting on interesting projects 127 Promoting continuous learning 127 Encouraging research studies 128 Building the Team 128 Developing smart hiring processes 129 Letting your teams evolve organically 130 Connecting the Team to the Business Purpose 131 Chapter 11: Approaching a Data Science Organizational Setup 133 Finding the Right Organizational Design 134 Designing the data science function 134 Evaluating the benefits of a center of excellence for data science 136 Identifying success factors for a data science center of excellence 137 Applying a Common Data Science Function 138 Selecting a location 138 Approaching ways of working 139 Managing expectations 141 Selecting an execution approach 142 Chapter 12: Positioning the Role of the Chief Data Officer (CDO) 145 Scoping the Role of the Chief Data Officer (CDO) 146 Explaining Why a Chief Data Officer is Needed 149 Establishing the CDO Role 150 The Future of the CDO Role 152 Chapter 13: Acquiring Resources and Competencies 155 Identifying the Roles in a Data Science Team 156 Data scientist 157 Data engineer 157 Machine learning engineer 158 Data architect 159 Business analyst 159 Software engineer 159 Domain expert 160 Seeing What Makes a Great Data Scientist 160 Structuring a Data Science Team 163 Hiring and evaluating the data science talent you need 165 Retaining Competence in Data Science 167 Understanding what makes a data scientist leave 169 Part 4: Investing in the Right Infrastructure 173 Chapter 14: Developing a Data Architecture 175 Defining What Makes Up a Data Architecture 176 Describing traditional architectural approaches 176 Elements of a data architecture 177 Exploring the Characteristics of a Modern Data Architecture 178 Explaining Data Architecture Layers 181 Listing the Essential Technologies for a Modern Data Architecture 184 NoSQL databases 184 Real-time streaming platforms 185 Docker and containers 185 Container repositories 186 Container orchestration 187 Microservices 187 Function as a service 188 Creating a Modern Data Architecture 189 Chapter 15: Focusing Data Governance on the Right Aspects 193 Sorting Out Data Governance 194 Data governance for defense or offense 195 Objectives for data governance 196 Explaining Why Data Governance is Needed 197 Data governance saves money 197 Bad data governance is dangerous 198 Good data governance provides clarity 198 Establishing Data Stewardship to Enforce Data Governance Rules 198 Implementing a Structured Approach to Data Governance 199 Chapter 16: Managing Models During Development and Production 203 Unfolding the Fundamentals of Model Management 203 Working with many models 204 Making the case for efficient model management 206 Implementing Model Management 207 Pinpointing implementation challenges 208 Managing model risk 210 Measuring the risk level 211 Identifying suitable control mechanisms 211 Chapter 17: Exploring the Importance of Open Source 213 Exploring the Role of Open Source 213 Understanding the importance of open source in smaller companies 214 Understanding the trend 215 Describing the Context of Data Science Programming Languages 215 Unfolding Open Source Frameworks for AI/ML Models 218 TensorFlow 219 Theano 219 Torch 219 Caffe and Caffe2 220 The Microsoft Cognitive Toolkit (previously known as Microsoft CNTK) 220 Keras 220 Scikit-learn 221 Spark MLlib 221 Azure ML Studio 221 Amazon Machine Learning 221 Choosing Open Source or Not? 222 Chapter 18: Realizing the Infrastructure 223 Approaching Infrastructure Realization 223 Listing Key Infrastructure Considerations for AI and ML Support 226 Location 226 Capacity 227 Data center setup 227 End-to-end management 227 Network infrastructure 228 Security and ethics 228 Advisory and supporting services 229 Ecosystem fit 229 Automating Workflows in Your Data Infrastructure 229 Enabling an Efficient Workspace for Data Engineers and Data Scientists 230 Part 5: Data as a Business 233 Chapter 19: Investing in Data as a Business 235 Exploring How to Monetize Data 236 Approaching data monetization is about treating data as an asset 237 Data monetization in a data economy 238 Looking to the Future of the Data Economy 240 Chapter 20: Using Data for Insights or Commercial Opportunities 243 Focusing Your Data Science Investment 243 Determining the Drivers for Internal Business Insights 244 Recognizing data science categories for practical implementation 245 Applying data-science-driven internal business insights 247 Using Data for Commercial Opportunities 248 Defining a data product 249 Distinguishing between categories of data products 250 Balancing Strategic Objectives 252 Chapter 21: Engaging Differently with Your Customers 255 Understanding Your Customers 255 Step 1: Engage your customers 256 Step 2: Identify what drives your customers 257 Step 3: Apply analytics and machine learning to customer actions 258 Step 4: Predict and prepare for the next step 259 Step 5: Imagine your customer’s future 260 Keeping Your Customers Happy 261 Serving Customers More Efficiently 263 Predicting demand 263 Automating tasks 264 Making company applications predictive 264 Chapter 22: Introducing Data-driven Business Models 265 Defining Business Models 265 Exploring Data-driven Business Models 267 Creating data-centric businesses 268 Investigating different types of data-driven business models 268 Using a Framework for Data-driven Business Models 275 Creating a data-driven business model using a framework 276 Key resources 277 Key activities 277 Offering/value proposition 278 Customer segment 278 Revenue model 279 Cost structure 280 Putting it all together 280 Chapter 23: Handling New Delivery Models 281 Defining Delivery Models for Data Products and Services 282 Understanding and Adapting to New Delivery Models 282 Introducing New Ways to Deliver Data Products 284 Self-service analytics environments as a delivery model 285 Applications, websites, and product/service interfaces as delivery models 287 Existing products and services 289 Downloadable files 290 APIs 290 Cloud services 291 Online market places 291 Downloadable licenses 292 Online services 293 Onsite services 293 Part 6: The Part of Tens 295 Chapter 24: Ten Reasons to Develop a Data Science Strategy 297 Expanding Your View on Data Science 297 Aligning the Company View 298 Creating a Solid Base for Execution 299 Realizing Priorities Early 299 Putting the Objective into Perspective 300 Creating an Excellent Base for Communication 300 Understanding Why Choices Matter 301 Identifying the Risks Early 301 Thoroughly Considering Your Data Need 302 Understanding the Change Impact 303 Chapter 25: Ten Mistakes to Avoid When Investing in Data Science 305 Don’t Tolerate Top Management’s Ignorance of Data Science 305 Don’t Believe That AI is Magic 306 Don’t Approach Data Science as a Race to the Death between Man and Machine 307 Don’t Underestimate the Potential of AI 308 Don’t Underestimate the Needed Data Science Skill Set 308 Don’t Think That a Dashboard is the End Objective 309 Don’t Forget about the Ethical Aspects of AI 310 Don’t Forget to Consider the Legal Rights to the Data 311 Don’t Ignore the Scale of Change Needed 312 Don’t Forget the Measurements Needed to Prove Value 313 Index 315

Read more less

Book SynopsisWith the advancements of semantic web, ontology has become the crucial mechanism for representing concepts in various domains. For research and dispersal of customized healthcare services, a major challenge is to efficiently retrieve and analyze individual patient data from a large volume of heterogeneous data over a long time span. This requirement demands effective ontology-based information retrieval approaches for clinical information systems so that the pertinent information can be mined from large amount of distributed data. This unique and groundbreaking book highlights the key advances in ontology-based information retrieval techniques being applied in the healthcare domain and covers the following areas: Semantic data integration in e-health care systems Keyword-based medical information retrieval Ontology-based query retrieval support for e-health implementation Ontologies as a database management system technology for medicalTable of ContentsPreface xix Acknowledgment xxiii 1 Role of Ontology in Health Care 1Sonia Singla 1.1 Introduction 2 1.2 Ontology in Diabetes 3 1.2.1 Ontology Process 4 1.2.2 Impediments of the Present Investigation 5 1.3 Role of Ontology in Cardiovascular Diseases 6 1.4 Role of Ontology in Parkinson Diseases 8 1.4.1 The Spread of Disease With Age and Onset of Disease 10 1.4.2 Cost of PD for Health Care, Household 11 1.4.3 Treatment and Medicines 11 1.5 Role of Ontology in Depression 13 1.6 Conclusion 15 1.7 Future Scope 15 References 15 2 A Study on Basal Ganglia Circuit and Its Relation With Movement Disorders 19Dinesh Bhatia 2.1 Introduction 19 2.2 Anatomy and Functioning of Basal Ganglia 21 2.2.1 The Striatum-Major Entrance to Basal Ganglia Circuitry 22 2.2.2 Direct and Indirect Striatofugal Projections 23 2.2.3 The STN: Another Entrance to Basal Ganglia Circuitry 25 2.3 Movement Disorders 26 2.3.1 Parkinson Disease 26 2.3.2 Dyskinetic Disorder 27 2.3.3 Dystonia 28 2.4 Effect of Basal Ganglia Dysfunctioning on Movement Disorders 29 2.5 Conclusion and Future Scope 31 References 31 3 Extraction of Significant Association Rules Using Pre- and Post-Mining Techniques—An Analysis 37M. Nandhini and S. N. Sivanandam 3.1 Introduction 38 3.2 Background 39 3.2.1 Interestingness Measures 39 3.2.2 Pre-Mining Techniques 40 3.2.2.1 Candidate Set Reduction Schemes 40 3.2.2.2 Optimal Threshold Computation Schemes 41 3.2.2.3 Weight-Based Mining Schemes 42 3.2.3 Post-Mining Techniques 42 3.2.3.1 Rule Pruning Schemes 43 3.2.3.2 Schemes Using Knowledge Base 43 3.3 Methodology 44 3.3.1 Data Preprocessing 44 3.3.2 Pre-Mining 46 3.3.2.1 Pre-Mining Technique 1: Optimal Support and Confidence Threshold Value Computation Using PSO 46 3.3.2.2 Pre-Mining Technique 2: Attribute Weight Computation Using IG Measure 48 3.3.3 Association Rule Generation 50 3.3.3.1 ARM Preliminaries 50 3.3.3.2 WARM Preliminaries 52 3.3.4 Post-Mining 56 3.3.4.1 Filters 56 3.3.4.2 Operators 58 3.3.4.3 Rule Schemas 58 3.4 Experiments and Results 59 3.4.1 Parameter Settings for PSO-Based Pre-Mining Technique 60 3.4.2 Parameter Settings for PAW-Based Pre-Mining Technique 60 3.5 Conclusions 63 References 65 4 Ontology in Medicine as a Database Management System 69Shobowale K. O. 4.1 Introduction 70 4.1.1 Ontology Engineering and Development Methodology 72 4.2 Literature Review on Medical Data Processing 72 4.3 Information on Medical Ontology 75 4.3.1 Types of Medical Ontology 75 4.3.2 Knowledge Representation 76 4.3.3 Methodology of Developing Medical Ontology 76 4.3.4 Medical Ontology Standards 77 4.4 Ontologies as a Knowledge-Based System 78 4.4.1 Domain Ontology in Medicine 79 4.4.2 Brief Introduction of Some Medical Standards 81 4.4.2.1 Medical Subject Headings (MeSH) 81 4.4.2.2 Medical Dictionary for Regulatory Activities (MedDRA) 81 4.4.2.3 Medical Entities Dictionary (MED) 81 4.4.3 Reusing Medical Ontology 82 4.4.4 Ontology Evaluation 85 4.5 Conclusion 86 4.6 Future Scope 86 References 87 5 Using IoT and Semantic Web Technologies for Healthcare and Medical Sector 91Nikita Malik and Sanjay Kumar Malik 5.1 Introduction 92 5.1.1 Significance of Healthcare and Medical Sector and Its Digitization 92 5.1.2 e-Health and m-Health 92 5.1.3 Internet of Things and Its Use 94 5.1.4 Semantic Web and Its Technologies 96 5.2 Use of IoT in Healthcare and Medical Domain 98 5.2.1 Scope of IoT in Healthcare and Medical Sector 98 5.2.2 Benefits of IoT in Healthcare and Medical Systems 100 5.2.3 IoT Healthcare Challenges and Open Issues 100 5.3 Role of SWTs in Healthcare Services 101 5.3.1 Scope and Benefits of Incorporating Semantics in Healthcare 101 5.3.2 Ontologies and Datasets for Healthcare and Medical Domain 103 5.3.3 Challenges in the Use of SWTs in Healthcare Sector 104 5.4 Incorporating IoT and/or SWTs in Healthcare and Medical Sector 106 5.4.1 Proposed Architecture or Framework or Model 106 5.4.2 Access Mechanisms or Approaches 108 5.4.3 Applications or Systems 109 5.5 Healthcare Data Analytics Using Data Mining and Machine Learning 110 5.6 Conclusion 112 5.7 Future Work 113 References 113 6 An Ontological Model, Design, and Implementation of CSPF for Healthcare 117Pooja Mohan 6.1 Introduction 117 6.2 Related Work 119 6.3 Mathematical Representation of CSPF Model 122 6.3.1 Basic Sets of CSPF Model 123 6.3.2 Conditional Contextual Security and Privacy Constraints 123 6.3.3 CSPF Model States CsetofStates 124 6.3.4 Permission Cpermission 124 6.3.5 Security Evaluation Function (SEFcontexts) 124 6.3.6 Secure State 125 6.3.7 CSPF Model Operations 125 6.3.7.1 Administrative Operations 125 6.3.7.2 Users’ Operations 127 6.4 Ontological Model 127 6.4.1 Development of Class Hierarchy 127 6.4.1.1 Object Properties of Sensor Class 129 6.4.1.2 Data Properties 129 6.4.1.3 The Individuals 129 6.5 The Design of Context-Aware Security and Privacy Model for Wireless Sensor Network 129 6.6 Implementation 133 6.7 Analysis and Results 135 6.7.1 Inference Time/Latency/Query Response Time vs. No. of Policies 135 6.7.2 Average Inference Time vs. Contexts 136 6.8 Conclusion and Future Scope 137 References 138 7 Ontology-Based Query Retrieval Support for E-Health Implementation 143Aatif Ahmad Khan and Sanjay Kumar Malik 7.1 Introduction 143 7.1.1 Health Care Record Management 144 7.1.1.1 Electronic Health Record 144 7.1.1.2 Electronic Medical Record 145 7.1.1.3 Picture Archiving and Communication System 145 7.1.1.4 Pharmacy Systems 145 7.1.2 Information Retrieval 145 7.1.3 Ontology 146 7.2 Ontology-Based Query Retrieval Support 146 7.3 E-Health 150 7.3.1 Objectives and Scope 150 7.3.2 Benefits of E-Health 151 7.3.3 E-Health Implementation 151 7.4 Ontology-Driven Information Retrieval for E-Health 154 7.4.1 Ontology for E-Heath Implementation 155 7.4.2 Frameworks for Information Retrieval Using Ontology for E-Health 157 7.4.3 Applications of Ontology-Driven Information Retrieval in Health Care 158 7.4.4 Benefits and Limitations 160 7.5 Discussion 160 7.6 Conclusion 164 References 164 8 Ontology-Based Case Retrieval in an E-Mental Health Intelligent Information System 167Georgia Kaoura, Konstantinos Kovas and Basilis Boutsinas 8.1 Introduction 167 8.2 Literature Survey 170 8.3 Problem Identified 173 8.4 Proposed Solution 174 8.4.1 The PAVEFS Ontology 174 8.4.2 Knowledge Base 179 8.4.3 Reasoning 180 8.4.4 User Interaction 182 8.5 Pros and Cons of Solution 183 8.5.1 Evaluation Methodology and Results 183 8.5.2 Evaluation Methodology 185 8.5.2.1 Evaluation Tools 186 8.5.2.2 Results 187 8.6 Conclusions 189 8.7 Future Scope 190 References 190 9 Ontology Engineering Applications in Medical Domain 193Mariam Gawich and Marco Alfonse 9.1 Introduction 193 9.2 Ontology Activities 195 9.2.1 Ontology Learning 195 9.2.2 Ontology Matching 195 9.2.3 Ontology Merging (Unification) 195 9.2.4 Ontology Validation 196 9.2.5 Ontology Verification 196 9.2.6 Ontology Alignment 196 9.2.7 Ontology Annotation 196 9.2.8 Ontology Evaluation 196 9.2.9 Ontology Evolution 196 9.3 Ontology Development Methodologies 197 9.3.1 TOVE 197 9.3.2 Methontology 198 9.3.3 Brusa et al. Methodology 198 9.3.4 UPON Methodology 199 9.3.5 Uschold and King Methodology 200 9.4 Ontology Languages 203 9.4.1 RDF-RDF Schema 203 9.4.2 OWL 205 9.4.3 OWL 2 205 9.5 Ontology Tools 208 9.5.1 Apollo 208 9.5.2 NeON 209 9.5.3 Protégé 210 9.6 Ontology Engineering Applications in Medical Domain 212 9.6.1 Ontology-Based Decision Support System (DSS) 213 9.6.1.1 OntoDiabetic 213 9.6.1.2 Ontology-Based CDSS for Diabetes Diagnosis 214 9.6.1.3 Ontology-Based Medical DSS within E-Care Telemonitoring Platform 215 9.6.2 Medical Ontology in the Dynamic Healthcare Environment 216 9.6.3 Knowledge Management Systems 217 9.6.3.1 Ontology-Based System for Cancer Diseases 217 9.6.3.2 Personalized Care System for Chronic Patients at Home 218 9.7 Ontology Engineering Applications in Other Domains 219 9.7.1 Ontology Engineering Applications in E-Commerce 219 9.7.1.1 Automated Approach to Product Taxonomy Mapping in E-Commerce 219 9.7.1.2 LexOnt Matching Approach 221 9.7.2 Ontology Engineering Applications in Social Media Domain 222 9.7.2.1 Emotive Ontology Approach 222 9.7.2.2 Ontology-Based Approach for Social Media Analysis 224 9.7.2.3 Methodological Framework for Semantic Comparison of Emotional Values 225 References 226 10 Ontologies on Biomedical Informatics 233Marco Alfonse and Mariam Gawich 10.1 Introduction 233 10.2 Defining Ontology 234 10.3 Biomedical Ontologies and Ontology-Based Systems 235 10.3.1 MetaMap 235 10.3.2 GALEN 236 10.3.3 NIH-CDE 236 10.3.4 LOINC 237 10.3.5 Current Procedural Terminology (CPT) 238 10.3.6 Medline Plus Connect 238 10.3.7 Gene Ontology 239 10.3.8 UMLS 240 10.3.9 SNOMED-CT 240 10.3.10 OBO Foundry 240 10.3.11 Textpresso 240 10.3.12 National Cancer Institute Thesaurus 241 References 241 11 Machine Learning Techniques Best for Large Data Prediction: A Case Study of Breast Cancer Categorical Data: k-Nearest Neighbors 245Yagyanath Rimal 11.1 Introduction 246 11.2 R Programming 250 11.3 Conclusion 255 References 255 12 Need of Ontology-Based Systems in Healthcare System 257Tshepiso Larona Mokgetse 12.1 Introduction 258 12.2 What is Ontology? 259 12.3 Need for Ontology in Healthcare Systems 260 12.3.1 Primary Healthcare 262 12.3.1.1 Semantic Web System 262 12.3.2 Emergency Services 263 12.3.2.1 Service-Oriented Architecture 263 12.3.2.2 IOT Ontology 264 12.3.3 Public Healthcare 265 12.3.3.1 IOT Data Model 265 12.3.4 Chronic Disease Healthcare 266 12.3.4.1 Clinical Reminder System 266 12.3.4.2 Chronic Care Model 267 12.3.5 Specialized Healthcare 268 12.3.5.1 E-Health Record System 268 12.3.5.2 Maternal and Child Health 269 12.3.6 Cardiovascular System 270 12.3.6.1 Distributed Healthcare System 270 12.3.6.2 Records Management System 270 12.3.7 Stroke Rehabilitation 271 12.3.7.1 Patient Information System 271 12.3.7.2 Toronto Virtual System 271 12.4 Conclusion 272 References 272 13 Exploration of Information Retrieval Approaches With Focus on Medical Information Retrieval 275Mamata Rath and Jyotir Moy Chatterjee 13.1 Introduction 276 13.1.1 Machine Learning-Based Medical Information System 278 13.1.2 Cognitive Information Retrieval 278 13.2 Review of Literature 279 13.3 Cognitive Methods of IR 281 13.4 Cognitive and Interactive IR Systems 286 13.5 Conclusion 288 References 289 14 Ontology as a Tool to Enable Health Internet of Things Viable 5G Communication Networks 293Nidhi Sharma and R. K. Aggarwal 14.1 Introduction 293 14.2 From Concept Representations to Medical Ontologies 295 14.2.1 Current Medical Research Trends 296 14.2.2 Ontology as a Paradigm Shift in Health Informatics 296 14.3 Primer Literature Review 297 14.3.1 Remote Health Monitoring 298 14.3.2 Collecting and Understanding Medical Data 298 14.3.3 Patient Monitoring 298 14.3.4 Tele-Health 299 14.3.5 Advanced Human Services Records Frameworks 299 14.3.6 Applied Autonomy and Healthcare Mechanization 300 14.3.7 IoT Powers the Preventive Healthcare 301 14.3.8 Hospital Statistics Control System (HSCS) 301 14.3.9 End-to-End Accessibility and Moderateness 301 14.3.10 Information Mixing and Assessment 302 14.3.11 Following and Alerts 302 14.3.12 Remote Remedial Assistance 302 14.4 Establishments of Health IoT 303 14.4.1 Technological Challenges 304 14.4.2 Probable Solutions 306 14.4.3 Bit-by-Bit Action Statements 307 14.5 Incubation of IoT in Health Industry 307 14.5.1 Hearables 308 14.5.2 Ingestible Sensors 308 14.5.3 Moodables 308 14.5.4 PC Vision Innovation 308 14.5.5 Social Insurance Outlining 308 14.6 Concluding Remarks 309 References 309 15 Tools and Techniques for Streaming Data: An Overview 313K. Saranya, S. Chellammal and Pethuru Raj Chelliah 15.1 Introduction 314 15.2 Traditional Techniques 315 15.2.1 Random Sampling 315 15.2.2 Histograms 316 15.2.3 Sliding Window 316 15.2.4 Sketches 317 15.2.4.1 Bloom Filters 317 15.2.4.2 Count-Min Sketch 317 15.3 Data Mining Techniques 317 15.3.1 Clustering 318 15.3.1.1 STREAM 318 15.3.1.2 BRICH 318 15.3.1.3 CLUSTREAM 319 15.3.2 Classification 319 15.3.2.1 Naïve Bayesian 319 15.3.2.2 Hoeffding 320 15.3.2.3 Very Fast Decision Tree 320 15.3.2.4 Concept Adaptive Very Fast Decision Tree 320 15.4 Big Data Platforms 320 15.4.1 Apache Storm 321 15.4.2 Apache Spark 321 15.4.2.1 Apache Spark Core 321 15.4.2.2 Spark SQL 322 15.4.2.3 Machine Learning Library 322 15.4.2.4 Streaming Data API 322 15.4.2.5 GraphX 323 15.4.3 Apache Flume 323 15.4.4 Apache Kafka 323 15.4.5 Apache Flink 326 15.5 Conclusion 327 References 328 16 An Ontology-Based IR for Health Care 331J. P. Patra, Gurudatta Verma and Sumitra Samal 16.1 Introduction 331 16.2 General Definition of Information Retrieval Model 333 16.3 Information Retrieval Model Based on Ontology 334 16.4 Literature Survey 336 16.5 Methodolgy for IR 339 References 344

Read more less

Book SynopsisCOMPUTATION IN BIOINFORMATICS Bioinformatics is a platform between the biology and information technology and this book provides readers with an understanding of the use of bioinformatics tools in new drug design. The discovery of new solutions to pandemics is facilitated through the use of promising bioinformatics techniques and integrated approaches. This book covers a broad spectrum of the bioinformatics field, starting with the basic principles, concepts, and application areas. Also covered is the role of bioinformatics in drug design and discovery, including aspects of molecular modeling. Some of the chapters provide detailed information on bioinformatics related topics, such as silicon design, protein modeling, DNA microarray analysis, DNA-RNA barcoding, and gene sequencing, all of which are currently needed in the industry. Also included are specialized topics, such as bioinformatics in cancer detection, genomics, and proteomics. Moreover, a few chapters explTable of ContentsPreface xiii 1 Bioinfomatics as a Tool in Drug Designing 1Rene Barbie Browne, Shiny C. Thomas and Jayanti Datta Roy 1.1 Introduction 1 1.2 Steps Involved in Drug Designing 3 1.2.1 Identification of the Target Protein/Enzyme 5 1.2.2 Detection of Molecular Site (Active Site) in the Target Protein 6 1.2.3 Molecular Modeling 6 1.2.4 Virtual Screening 9 1.2.5 Molecular Docking 10 1.2.6 QSAR (Quantitative Structure-Activity Relationship) 12 1.2.7 Pharmacophore Modeling 14 1.2.8 Solubility of Molecule 14 1.2.9 Molecular Dynamic Simulation 14 1.2.10 ADME Prediction 15 1.3 Various Softwares Used in the Steps of Drug Designing 16 1.4 Applications 18 1.5 Conclusion 20 References 20 2 New Strategies in Drug Discovery 25Vivek Chavda, Yogita Thalkari and Swati Marwadi 2.1 Introduction 26 2.2 Road Toward Advancement 27 2.3 Methodology 30 2.3.1 Target Identification 30 2.3.2 Docking-Based Virtual Screening 32 2.3.3 Conformation Sampling 33 2.3.4 Scoring Function 34 2.3.5 Molecular Similarity Methods 35 2.3.6 Virtual Library Construction 37 2.3.7 Sequence-Based Drug Design 37 2.4 Role of OMICS Technology 38 2.5 High-Throughput Screening and Its Tools 40 2.6 Chemoinformatic 44 2.6.1 Exploratory Data Analysis 45 2.6.2 Example Discovery 46 2.6.3 Pattern Explanation 46 2.6.4 New Technologies 46 2.7 Concluding Remarks and Future Prospects 46 References 48 3 Role of Bioinformatics in Early Drug Discovery: An Overview and Perspective 49Shasank S. Swain and Tahziba Hussain 3.1 Introduction 50 3.2 Bioinformatics and Drug Discovery 51 3.2.1 Structure-Based Drug Design (SBDD) 52 3.2.2 Ligand-Based Drug Design (LBDD) 53 3.3 Bioinformatics Tools in Early Drug Discovery 54 3.3.1 Possible Biological Activity Prediction Tools 55 3.3.2 Possible Physicochemical and Drug-Likeness Properties Verification Tools 58 3.3.3 Possible Toxicity and ADME/T Profile Prediction Tools 60 3.4 Future Directions With Bioinformatics Tool 61 3.5 Conclusion 63 Acknowledgements 64 References 64 4 Role of Data Mining in Bioinformatics 69Vivek P. Chavda, Amit Sorathiya, Disha Valu and Swati Marwadi 4.1 Introduction 70 4.2 Data Mining Methods/Techniques 71 4.2.1 Classification 71 4.2.1.1 Statistical Techniques 71 4.2.1.2 Clustering Technique 73 4.2.1.3 Visualization 74 4.2.1.4 Induction Decision Tree Technique 74 4.2.1.5 Neural Network 75 4.2.1.6 Association Rule Technique 75 4.2.1.7 Classification 75 4.3 DNA Data Analysis 77 4.4 RNA Data Analysis 79 4.5 Protein Data Analysis 79 4.6 Biomedical Data Analysis 80 4.7 Conclusion and Future Prospects 81 References 81 5 In Silico Protein Design and Virtual Screening 85Vivek P. Chavda, Zeel Patel, Yashti Parmar and Disha Chavda 5.1 Introduction 86 5.2 Virtual Screening Process 88 5.2.1 Before Virtual Screening 90 5.2.2 General Process of Virtual Screening 90 5.2.2.1 Step 1 (The Establishment of the Receptor Model) 91 5.2.2.2 Step 2 (The Generation of Small-Molecule Libraries) 92 5.2.2.3 Step 3 (Molecular Docking) 92 5.2.2.4 Step 4 (Selection of Lead Protein Compounds) 94 5.3 Machine Learning and Scoring Functions 94 5.4 Conclusion and Future Prospects 95 References 96 6 New Bioinformatics Platform-Based Approach for Drug Design 101Vivek Chavda, Soham Sheta, Divyesh Changani and Disha Chavda 6.1 Introduction 102 6.2 Platform-Based Approach and Regulatory Perspective 104 6.3 Bioinformatics Tools and Computer-Aided Drug Design 107 6.4 Target Identification 109 6.5 Target Validation 110 6.6 Lead Identification and Optimization 111 6.7 High-Throughput Methods (HTM) 112 6.8 Conclusion and Future Prospects 114 References 115 7 Bioinformatics and Its Application Areas 121Ragini Bhardwaj, Mohit Sharma and Nikhil Agrawal 7.1 Introduction 121 7.2 Review of Bioinformatics 124 7.3 Bioinformatics Applications in Different Areas 126 7.3.1 Microbial Genome Application 126 7.3.2 Molecular Medicine 129 7.3.3 Agriculture 130 7.4 Conclusion 131 References 131 8 DNA Microarray Analysis: From Affymetrix CEL Files to Comparative Gene Expression 139Sandeep Kumar, Shruti Shandilya, Suman Kapila, Mohit Sharma and Nikhil Agrawal 8.1 Introduction 140 8.2 Data Processing 140 8.2.1 Installation of Workflow 140 8.2.2 Importing the Raw Data for Processing 141 8.2.3 Retrieving Sample Annotation of the Data 142 8.2.4 Quality Control 143 8.2.4.1 Boxplot 144 8.2.4.2 Density Histogram 145 8.2.4.3 MA Plot 145 8.2.4.4 NUSE Plot 145 8.2.4.5 RLE Plot 145 8.2.4.6 RNA Degradation Plot 145 8.2.4.7 QCstat 148 8.3 Normalization of Microarray Data Using the RMA Method 148 8.3.1 Background Correction 148 8.3.2 Normalization 149 8.3.3 Summarization 149 8.4 Statistical Analysis for Differential Gene Expression 151 8.5 Conclusion 153 References 153 9 Machine Learning in Bioinformatics 155Rahul Yadav, Mohit Sharma and Nikhil Agrawal 9.1 Introduction and Background 156 9.1.1 Bioinformatics 158 9.1.2 Text Mining 159 9.1.3 IoT Devices 159 9.2 Machine Learning Applications in Bioinformatics 159 9.3 Machine Learning Approaches 161 9.4 Conclusion and Closing Remarks 162 References 162 10 DNA-RNA Barcoding and Gene Sequencing 165Gifty Sawhney, Mohit Sharma and Nikhil Agrawal 10.1 Introduction 166 10.2 RNA 169 10.3 DNA Barcoding 172 10.3.1 Introduction 172 10.3.2 DNA Barcoding and Molecular Phylogeny 177 10.3.3 Ribosomal DNA (rDNA) of the Nuclear Genome (nuDNA)—ITS 178 10.3.4 Chloroplast DNA 180 10.3.5 Mitochondrial DNA 181 10.3.6 Molecular Phylogenetic Analysis 181 10.3.7 Metabarcoding 189 10.3.8 Materials for DNA Barcoding 190 10.4 Main Reasons of DNA Barcoding 191 10.5 Limitations/Restrictions of DNA Barcoding 192 10.6 RNA Barcoding 192 10.6.1 Overview of the Method 193 10.7 Methodology 194 10.7.1 Materials Required 195 10.7.2 Barcoded RNA Sequencing High-Level Mapping of Single-Neuron Projections 196 10.7.3 Using RNA to Trace Neurons 196 10.7.4 A Life Conservation Barcoder 198 10.7.5 Gene Sequencing 199 10.7.5.1 DNA Sequencing Methods 200 10.7.5.2 First-Generation Sequencing Techniques 204 10.7.5.3 Maxam’s and Gilbert’s Chemical Method 204 10.7.5.4 Sanger Sequencing 205 10.7.5.5 Automation in DNA Sequencing 206 10.7.5.6 Use of Fluorescent-Marked Primers and ddNTPs 206 10.7.5.7 Dye Terminator Sequencing 207 10.7.5.8 Using Capillary Electrophoresis 207 10.7.6 Developments and High-Throughput Methods in DNA Sequencing 208 10.7.7 Pyrosequencing Method 209 10.7.8 The Genome Sequencer 454 FLX System 210 10.7.9 Illumina/Solexa Genome Analyzer 210 10.7.10 Transition Sequencing Techniques 211 10.7.11 Ion-Torrent’s Semiconductor Sequencing 211 10.7.12 Helico’s Genetic Analysis Platform 211 10.7.13 Third-Generation Sequencing Techniques 212 10.8 Conclusion 212 Abbreviations 213 Acknowledgement 214 References 214 11 Bioinformatics in Cancer Detection 229Mohit Sharma, Umme Abiha, Parul Chugh, Balakumar Chandrasekaran and Nikhil Agrawal 11.1 Introduction 230 11.2 The Era of Bioinformatics in Cancer 230 11.3 Aid in Cancer Research via NCI 232 11.4 Application of Big Data in Developing Precision Medicine 233 11.5 Historical Perspective and Development 235 11.6 Bioinformatics-Based Approaches in the Study of Cancer 237 11.6.1 SLAMS 237 11.6.2 Module Maps 238 11.6.3 COPA 239 11.7 Conclusion and Future Challenges 240 References 240 12 Genomic Association of Polycystic Ovarian Syndrome: Single-Nucleotide Polymorphisms and Their Role in Disease Progression 245Gowtham Kumar Subbaraj and Sindhu Varghese 12.1 Introduction 246 12.2 FSHR Gene 252 12.3 IL-10 Gene 252 12.4 IRS-1 Gene 253 12.5 PCR Primers Used 254 12.6 Statistical Analysis 255 12.7 Conclusion 258 References 259 13 An Insight of Protein Structure Predictions Using Homology Modeling 265S. Muthumanickam, P. Boomi, R. Subashkumar, S. Palanisamy, A. Sudha, K. Anand, C. Balakumar, M. Saravanan, G. Poorani, Yao Wang, K. Vijayakumar and M. Syed Ali 13.1 Introduction 266 13.2 Homology Modeling Approach 268 13.2.1 Strategies for Homology Modeling 269 13.2.2 Procedure 269 13.3 Steps Involved in Homology Modeling 270 13.3.1 Template Identification 270 13.3.2 Sequence Alignment 271 13.3.3 Backbone Generation 271 13.3.4 Loop Modeling 271 13.3.5 Side Chain Modeling 272 13.3.6 Model Optimization 272 13.3.6.1 Model Validation 272 13.4 Tools Used for Homology Modeling 273 13.4.1 Robetta 273 13.4.2 M4T (Multiple Templates) 273 13.4.3 I-Tasser (Iterative Implementation of the Threading Assembly Refinement) 273 13.4.4 ModBase 274 13.4.5 Swiss Model 274 13.4.6 PHYRE2 (Protein Homology/Analogy Recognition Engine 2) 274 13.4.7 Modeller 274 13.4.8 Conclusion 275 Acknowledgement 275 References 275 14 Basic Concepts in Proteomics and Applications 279Jesudass Joseph Sahayarayan, A.S. Enogochitra and Murugesan Chandrasekaran 14.1 Introduction 280 14.2 Challenges on Proteomics 281 14.3 Proteomics Based on Gel 283 14.4 Non-Gel–Based Electrophoresis Method 284 14.5 Chromatography 284 14.6 Proteomics Based on Peptides 285 14.7 Stable Isotopic Labeling 286 14.8 Data Mining and Informatics 287 14.9 Applications of Proteomics 289 14.10 Future Scope 290 14.11 Conclusion 291 References 292 15 Prospects of Covalent Approaches in Drug Discovery: An Overview 295Balajee Ramachandran, Saravanan Muthupandian and Jeyakanthan Jeyaraman 15.1 Introduction 296 15.2 Covalent Inhibitors Against the Biological Target 297 15.3 Application of Physical Chemistry Concepts in Drug Designing 299 15.4 Docking Methodologies—An Overview 301 15.5 Importance of Covalent Targets 302 15.6 Recent Framework on the Existing Docking Protocols 303 15.7 SN2 Reactions in the Computational Approaches 304 15.8 Other Crucial Factors to Consider in the Covalent Docking 305 15.8.1 Role of Ionizable Residues 305 15.8.2 Charge Regulation 306 15.8.3 Charge-Charge Interactions 306 15.9 QM/MM Approaches 309 15.10 Conclusion and Remarks 310 Acknowledgements 311 References 311 Index 321

Read more less

Book SynopsisLearn how to apply the principles of machine learning totime series modeling with thisindispensableresource Machine Learning for Time Series Forecasting with Pythonis an incisive and straightforward examination of one of the most crucial elements of decision-makingin finance,marketing,education, and healthcare:time series modeling. Despitethe centrality of time series forecasting, few business analysts are familiar with the power or utility of applying machine learning to time series modeling. Author Francesca Lazzeri, a distinguishedmachine learning scientistandeconomist,corrects that deficiency by providing readers withcomprehensiveand approachableexplanation andtreatment of the applicationof machine learning to time series forecasting. Written for readers who have little to no experience in time seriesforecastingor machine learning, the book comprehensively coversall the topics necessary to: Understand time series forecasting concepts, such asstationarity,horizon,trend,and seasonalityPrepare time series dataformodelingEvaluatetime series forecasting models'performance and accuracyUnderstand when to use neural networks instead of traditional time series models in time series forecasting Machine Learning for Time Series Forecasting with Pythonis fullreal-world examples, resourcesand concrete strategies to help readers explore and transform data and develop usable, practical time series forecasts. Perfect for entry-level data scientists, business analysts,developers, and researchers, this book is an invaluable and indispensable guide to the fundamental and advanced concepts of machine learning applied to time series modeling. Table of ContentsAcknowledgments vii Introduction xv Chapter 1 Overview of Time Series Forecasting 1 Flavors of Machine Learning for Time Series Forecasting 3 Supervised Learning for Time Series Forecasting 14 Python for Time Series Forecasting 21 Experimental Setup for Time Series Forecasting 24 Conclusion 26 Chapter 2 How to Design an End-to-End Time Series Forecasting Solution on the Cloud 29 Time Series Forecasting Template 31 Business Understanding and Performance Metrics 33 Data Ingestion 36 Data Exploration and Understanding 39 Data Pre-processing and Feature Engineering 40 Modeling Building and Selection 42 An Overview of Demand Forecasting Modeling Techniques 44 Model Evaluation 46 Model Deployment 48 Forecasting Solution Acceptance 53 Use Case: Demand Forecasting 54 Conclusion 58 Chapter 3 Time Series Data Preparation 61 Python for Time Series Data 62 Common Data Preparation Operations for Time Series 65 Time stamps vs. Periods 66 Converting to Timestamps 69 Providing a Format Argument 70 Indexing 71 Time/Date Components 76 Frequency Conversion 78 Time Series Exploration and Understanding 79 How to Get Started with Time Series Data Analysis 79 Data Cleaning of Missing Values in the Time Series 84 Time Series Data Normalization and Standardization 86 Time Series Feature Engineering 89 Date Time Features 90 Lag Features and Window Features 92 Rolling Window Statistics 95 Expanding Window Statistics 97 Conclusion 98 Chapter 4 Introduction to Autoregressive and Automated Methods for Time Series Forecasting 101 Autoregression 102 Moving Average 119 Autoregressive Moving Average 120 Autoregressive Integrated Moving Average 122 Automated Machine Learning 129 Conclusion 136 Chapter 5 Introduction to Neural Networks for Time Series Forecasting 137 Reasons to Add Deep Learning to Your Time Series Toolkit 138 Deep Learning Neural Networks Are Capable of Automatically Learning and Extracting Features from Raw and Imperfect Data 140 Deep Learning Supports Multiple Inputs and Outputs 142 Recurrent Neural Networks Are Good at Extracting Patterns from Input Data 143 Recurrent Neural Networks for Time Series Forecasting 144 Recurrent Neural Networks 145 Long Short-Term Memory 147 Gated Recurrent Unit 148 How to Prepare Time Series Data for LSTMs and GRUs 150 How to Develop GRUs and LSTMs for Time Series Forecasting 154 Keras 155 TensorFlow 156 Univariate Models 156 Multivariate Models 160 Conclusion 164 Chapter 6 Model Deployment for Time Series Forecasting 167 Experimental Set Up and Introduction to Azure Machine Learning SDK for Python 168 Workspace 169 Experiment 169 Run 169 Model 170 Compute Target, RunConfiguration, and ScriptRun Config 171 Image and Webservice 172 Machine Learning Model Deployment 173 How to Select the Right Tools to Succeed with Model Deployment 175 Solution Architecture for Time Series Forecasting with Deployment Examples 177 Train and Deploy an ARIMA Model 179 Configure the Workspace 182 Create an Experiment 183 Create or Attach a Compute Cluster 184 Upload the Data to Azure 184 Create an Estimator 188 Submit the Job to the Remote Cluster 188 Register the Model 189 Deployment 189 Define Your Entry Script and Dependencies 190 Automatic Schema Generation 191 Conclusion 196 References 197 Index 199

Read more less

Book SynopsisTable of ContentsForeword xix Preface to the Fourth Edition xxi Acknowledgments xxv PART I PRELIMINARIES CHAPTER 1 Introduction 3 CHAPTER 2 Overview of the Machine Learning Process 15 PART II DATA EXPLORATION AND DIMENSION REDUCTION CHAPTER 3 Data Visualization 59 CHAPTER 4 Dimension Reduction 91 PART III PERFORMANCE EVALUATION CHAPTER 5 Evaluating Predictive Performance 115 PART IV PREDICTION AND CLASSIFICATION METHODS CHAPTER 6 Multiple Linear Regression 151 CHAPTER 7 k-Nearest-Neighbors (k-NN) 169 CHAPTER 8 The Naive Bayes Classifier 181 CHAPTER 9 Classification and Regression Trees 197 CHAPTER 10 Logistic Regression 229 CHAPTER 11 Neural Nets 257 CHAPTER 12 Discriminant Analysis 283 CHAPTER 13 Generating, Comparing, and Combining Multiple Models 303 PART V INTERVENTION AND USER FEEDBACK CHAPTER 14 Experiments, Uplift Modeling, and Reinforcement Learning 319 PART VI MINING RELATIONSHIPS AMONG RECORDS CHAPTER 15 Association Rules and Collaborative Filtering 341 CHAPTER 16 Cluster Analysis 369 PART VII FORECASTING TIME SERIES CHAPTER 17 Handling Time Series 401 CHAPTER 18 Regression-Based Forecasting 415 CHAPTER 19 Smoothing Methods 445 PART VIII DATA ANALYTICS CHAPTER 20 Social Network Analytics 467 CHAPTER 21 Text Mining 487 CHAPTER 22 Responsible Data Science 507 PART IX CASES CHAPTER 23 Cases 537 References 575 Data Files Used in the Book 577 Index 579

Read more less

Book SynopsisFUZZY COMPUTING IN DATA SCIENCE This book comprehensively explains how to use various fuzzy-based models to solve real-time industrial challenges. The book provides information about fundamental aspects of the field and explores the myriad applications of fuzzy logic techniques and methods. It presents basic conceptual considerations and case studies of applications of fuzzy computation. It covers the fundamental concepts and techniques for system modeling, information processing, intelligent system design, decision analysis, statistical analysis, pattern recognition, automated learning, system control, and identification. The book also discusses the combination of fuzzy computation techniques with other computational intelligence approaches such as neural and evolutionary computation. Audience Researchers and students in computer science, artificial intelligence, machine learning, big data analytics, and information and communication technology.Table of ContentsPreface xvii Acknowledgement xxi 1 Band Reduction of HSI Segmentation Using FCM 1 V. Saravana Kumar, S. Anantha Sivaprakasam, E.R. Naganathan, Sunil Bhutada and M. Kavitha 1.1 Introduction 2 1.2 Existing Method 3 1.2.1 K-Means Clustering Method 3 1.2.2 Fuzzy C-Means 3 1.2.3 Davies Bouldin Index 4 1.2.4 Data Set Description of HSI 4 1.3 Proposed Method 5 1.3.1 Hyperspectral Image Segmentation Using Enhanced Estimation of Centroid 5 1.3.2 Band Reduction Using K-Means Algorithm 6 1.3.3 Band Reduction Using Fuzzy C-Means 7 1.4 Experimental Results 8 1.4.1 DB Index Graph 8 1.4.2 K-Means–Based PSC (EEOC) 9 1.4.3 Fuzzy C-Means–Based PSC (EEOC) 10 1.5 Analysis of Results 12 1.6 Conclusions 16 References 17 2 A Fuzzy Approach to Face Mask Detection 21 Vatsal Mishra, Tavish Awasthi, Subham Kashyap, Minerva Brahma, Monideepa Roy and Sujoy Datta 2.1 Introduction 22 2.2 Existing Work 23 2.3 The Proposed Framework 26 2.4 Set-Up and Libraries Used 26 2.5 Implementation 27 2.6 Results and Analysis 29 2.7 Conclusion and Future Work 33 References 34 3 Application of Fuzzy Logic to the Healthcare Industry 37 Biswajeet Sahu, Lokanath Sarangi, Abhinadita Ghosh and Hemanta Kumar Palo 3.1 Introduction 38 3.2 Background 41 3.3 Fuzzy Logic 42 3.4 Fuzzy Logic in Healthcare 45 3.5 Conclusions 49 References 50 4 A Bibliometric Approach and Systematic Exploration of Global Research Activity on Fuzzy Logic in Scopus Database 55 Sugyanta Priyadarshini and Nisrutha Dulla 4.1 Introduction 56 4.2 Data Extraction and Interpretation 58 4.3 Results and Discussion 59 4.3.1 Per Year Publication and Citation Count 59 4.3.2 Prominent Affiliations Contributing Toward Fuzzy Logic 60 4.3.3 Top Journals Emerging in Fuzzy Logic in Major Subject Areas 61 4.3.4 Major Contributing Countries Toward Fuzzy Research Articles 63 4.3.5 Prominent Authors Contribution Toward the Fuzzy Logic Analysis 66 4.3.6 Coauthorship of Authors 67 4.3.7 Cocitation Analysis of Cited Authors 68 4.3.8 Cooccurrence of Author Keywords 68 4.4 Bibliographic Coupling of Documents, Sources, Authors, and Countries 70 4.4.1 Bibliographic Coupling of Documents 70 4.4.2 Bibliographic Coupling of Sources 71 4.4.3 Bibliographic Coupling of Authors 72 4.4.4 Bibliographic Coupling of Countries 73 4.5 Conclusion 74 References 76 5 Fuzzy Decision Making in Predictive Analytics and Resource Scheduling 79 Rekha A. Kulkarni, Suhas H. Patil and Bithika Bishesh 5.1 Introduction 80 5.2 History of Fuzzy Logic and Its Applications 81 5.3 Approximate Reasoning 82 5.4 Fuzzy Sets vs Classical Sets 83 5.5 Fuzzy Inference System 84 5.5.1 Characteristics of FIS 85 5.5.2 Working of FIS 85 5.5.3 Methods of FIS 86 5.6 Fuzzy Decision Trees 86 5.6.1 Characteristics of Decision Trees 87 5.6.2 Construction of Fuzzy Decision Trees 87 5.7 Fuzzy Logic as Applied to Resource Scheduling in a Cloud Environment 88 5.8 Conclusion 90 References 91 6 Application of Fuzzy Logic and Machine Learning Concept in Sales Data Forecasting Decision Analytics Using ARIMA Model 93 S. Mala and V. Umadevi 6.1 Introduction 94 6.1.1 Aim and Scope 94 6.1.2 R-Tool 94 6.1.3 Application of Fuzzy Logic 94 6.1.4 Dataset 95 6.2 Model Study 96 6.2.1 Introduction to Machine Learning Method 96 6.2.2 Time Series Analysis 96 6.2.3 Components of a Time Series 97 6.2.4 Concepts of Stationary 99 6.2.5 Model Parsimony 100 6.3 Methodology 100 6.3.1 Exploratory Data Analysis 100 6.3.1.1 Seed Types—Analysis 101 6.3.1.2 Comparison of Location and Seeds 101 6.3.1.3 Comparison of Season (Month) and Seeds 103 6.3.2 Forecasting 103 6.3.2.1 Auto Regressive Integrated Moving Average (ARIMA) 103 6.3.2.2 Data Visualization 106 6.3.2.3 Implementation Model 108 6.4 Result Analysis 108 6.5 Conclusion 110 References 110 7 Modified m-Polar Fuzzy Set ELECTRE-I Approach 113 Madan Jagtap, Prasad Karande and Pravin Patil 7.1 Introduction 114 7.1.1 Objectives 114 7.2 Implementation of m-Polar Fuzzy ELECTRE-I Integrated Shannon’s Entropy Weight Calculations 115 7.2.1 The m-Polar Fuzzy ELECTRE-I Integrated Shannon’s Entropy Weight Calculation Method 115 7.3 Application to Industrial Problems 118 7.3.1 Cutting Fluid Selection Problem 118 7.3.2 Results Obtained From m-Polar Fuzzy ELECTRE-I for Cutting Fluid Selection Problem 122 7.3.3 FMS Selection Problem 125 7.3.4 Results Obtained From m-Polar Fuzzy ELECTRE-I for FMS Selection 130 7.4 Conclusions 143 References 143 8 Fuzzy Decision Making: Concept and Models 147 Bithika Bishesh 8.1 Introduction 148 8.2 Classical Set 149 8.3 Fuzzy Set 150 8.4 Properties of Fuzzy Set 151 8.5 Types of Decision Making 153 8.5.1 Individual Decision Making 153 8.5.2 Multiperson Decision Making 157 8.5.3 Multistage Decision Making 158 8.5.4 Multicriteria Decision Making 160 8.6 Methods of Multiattribute Decision Making (MADM) 162 8.6.1 Weighted Sum Method (WSM) 162 8.6.2 Weighted Product Method (WPM) 162 8.6.3 Weighted Aggregates Sum Product Assessment (WASPAS) 163 8.6.4 Technique for Order Preference by Similarity to Ideal Solutions (TOPSIS) 166 8.7 Applications of Fuzzy Logic 167 8.8 Conclusion 169 References 169 9 Use of Fuzzy Logic for Psychological Support to Migrant Workers of Southern Odisha (India) 173 Sanjaya Kumar Sahoo and Sukanta Chandra Swain 9.1 Introduction 174 9.2 Objectives and Methodology 175 9.2.1 Objectives 175 9.2.2 Methodology 176 9.3 Effect of COVID-19 on the Psychology and Emotion of Repatriated Migrants 176 9.3.1 Psychological Variables Identified 176 9.3.2 Fuzzy Logic for Solace to Migrants 176 9.4 Findings 178 9.5 Way Out for Strengthening the Psychological Strength of the Migrant Workers through Technological Aid 178 9.6 Conclusion 179 References 180 10 Fuzzy-Based Edge AI Approach: Smart Transformation of Healthcare for a Better Tomorrow 181 B. RaviKrishna, Sirisha Potluri, J. Rethna Virgil Jeny, Guna Sekhar Sajja and Katta Subba Rao 10.1 Significance of Machine Learning in Healthcare 182 10.2 Cloud-Based Artificial Intelligent Secure Models 183 10.3 Applications and Usage of Machine Learning in Healthcare 183 10.3.1 Detecting Diseases and Diagnosis 183 10.3.2 Drug Detection and Manufacturing 183 10.3.3 Medical Imaging Analysis and Diagnosis 184 10.3.4 Personalized/Adapted Medicine 185 10.3.5 Behavioral Modification 185 10.3.6 Maintenance of Smart Health Data 185 10.3.7 Clinical Trial and Study 185 10.3.8 Crowdsourced Information Discovery 185 10.3.9 Enhanced Radiotherapy 186 10.3.10 Outbreak/Epidemic Prediction 186 10.4 Edge AI: For Smart Transformation of Healthcare 186 10.4.1 Role of Edge in Reshaping Healthcare 186 10.4.2 How AI Powers the Edge 187 10.5 Edge AI-Modernizing Human Machine Interface 188 10.5.1 Rural Medicine 188 10.5.2 Autonomous Monitoring of Hospital Rooms—A Case Study 188 10.6 Significance of Fuzzy in Healthcare 189 10.6.1 Fuzzy Logic—Outline 189 10.6.2 Fuzzy Logic-Based Smart Healthcare 190 10.6.3 Medical Diagnosis Using Fuzzy Logic for Decision Support Systems 191 10.6.4 Applications of Fuzzy Logic in Healthcare 193 10.7 Conclusion and Discussions 193 References 194 11 Video Conferencing (VC) Software Selection Using Fuzzy TOPSIS 197 Rekha Gupta 11.1 Introduction 197 11.2 Video Conferencing Software and Its Major Features 199 11.2.1 Video Conferencing/Meeting Software (VC/MS) for Higher Education Institutes 199 11.3 Fuzzy TOPSIS 203 11.3.1 Extension of TOPSIS Algorithm: Fuzzy TOPSIS 203 11.4 Sample Numerical Illustration 207 11.5 Conclusions 213 References 213 12 Estimation of Nonperforming Assets of Indian Commercial Banks Using Fuzzy AHP and Goal Programming 215 Kandarp Vidyasagar and Rajiv Kr. Dwivedi 12.1 Introduction 216 12.1.1 Basic Concepts of Fuzzy AHP and Goal Programming 217 12.2 Research Model 221 12.2.1 Average Growth Rate Calculation 227 12.3 Result and Discussion 233 12.4 Conclusion 234 References 234 13 Evaluation of Ergonomic Design for the Visual Display Terminal Operator at Static Work Under FMCDM Environment 237 Bipradas Bairagi 13.1 Introduction 238 13.2 Proposed Algorithm 240 13.3 An Illustrative Example on Ergonomic Design Evaluation 245 13.4 Conclusions 249 References 249 14 Optimization of Energy Generated from Ocean Wave Energy Using Fuzzy Logic 253 S. B. Goyal, Pradeep Bedi, Jugnesh Kumar and Prasenjit Chatterjee 14.1 Introduction 254 14.2 Control Approach in Wave Energy Systems 255 14.3 Related Work 257 14.4 Mathematical Modeling for Energy Conversion from Ocean Waves 259 14.5 Proposed Methodology 260 14.5.1 Wave Parameters 261 14.5.2 Fuzzy-Optimizer 262 14.6 Conclusion 264 References 264 15 The m-Polar Fuzzy TOPSIS Method for NTM Selection 267 Madan Jagtap and Prasad Karande 15.1 Introduction 268 15.2 Literature Review 268 15.3 Methodology 270 15.3.1 Steps of the mFS TOPSIS 270 15.4 Case Study 272 15.4.1 Effect of Analytical Hierarchy Process (AHP) Weight Calculation on the mFS TOPSIS Method 273 15.4.2 Effect of Shannon’s Entropy Weight Calculation on the m-Polar Fuzzy Set TOPSIS Method 277 15.5 Results and Discussions 281 15.5.1 Result Validation 281 15.6 Conclusions and Future Scope 283 References 284 16 Comparative Analysis on Material Handling Device Selection Using Hybrid FMCDM Methodology 287 Bipradas Bairagi 16.1 Introduction 288 16.2 MCDM Techniques 289 16.2.1 Fahp 289 16.2.2 Entropy Method as Weights (Influence) Evaluation Technique 290 16.3 The Proposed Hybrid and Super Hybrid FMCDM Approaches 291 16.3.1 Topsis 291 16.3.2 FMOORA Method 292 16.3.3 FVIKOR 292 16.3.4 Fuzzy Grey Theory (FGT) 293 16.3.5 COPRAS –G 293 16.3.6 Super Hybrid Algorithm 294 16.4 Illustrative Example 295 16.5 Results and Discussions 298 16.5.1 FTOPSIS 298 16.5.2 FMOORA 298 16.5.3 FVIKRA 298 16.5.4 Fuzzy Grey Theory (FGT) 299 16.5.5 COPRAS-G 299 16.5.6 Super Hybrid Approach (SHA) 299 16.6 Conclusions 302 References 302 17 Fuzzy MCDM on CCPM for Decision Making: A Case Study 305 Bimal K. Jena, Biswajit Das, Amarendra Baral and Sushanta Tripathy 17.1 Introduction 306 17.2 Literature Review 307 17.3 Objective of Research 308 17.4 Cluster Analysis 308 17.4.1 Hierarchical Clustering 309 17.4.2 Partitional Clustering 309 17.5 Clustering 310 17.6 Methodology 314 17.7 TOPSIS Method 316 17.8 Fuzzy TOPSIS Method 318 17.9 Conclusion 325 17.10 Scope of Future Study 326 References 326 Index 329

Read more less

Book SynopsisThis book will provide a comprehensive overview of business analytics, for those who have either a technical background (quantitative methods) or a practitioner business background. Business analytics, in the context of the 4th Industrial Revolution, is the "new normal" for businesses that operate in this digital age. This book provides a comprehensive primer and overview of the field (and related fields such as Business Intelligence and Data Science). It will discuss the field as it applies to financial institutions, with some minor departures to other industries. Readers will gain understanding and insight into the field of data science, including traditional as well as emerging techniques. Further, many chapters are dedicated to the establishment of a data-driven team – from executive buy-in and corporate governance to managing and quantifying the return of data-driven projects.

Read more less

Book SynopsisThis book will provide a comprehensive overview of business analytics, for those who have either a technical background (quantitative methods) or a practitioner business background. Business analytics, in the context of the 4th Industrial Revolution, is the "new normal" for businesses that operate in this digital age. This book provides a comprehensive primer and overview of the field (and related fields such as Business Intelligence and Data Science). It will discuss the field as it applies to financial institutions, with some minor departures to other industries. Readers will gain understanding and insight into the field of data science, including traditional as well as emerging techniques. Further, many chapters are dedicated to the establishment of a data-driven team – from executive buy-in and corporate governance to managing and quantifying the return of data-driven projects.

Read more less

Book SynopsisThe series, Contemporary Perspectives on Data Mining, is composed of blind refereed scholarly research methods and applications of data mining. This series will be targeted both at the academic community, as well as the business practitioner.Data mining seeks to discover knowledge from vast amounts of data with the use of statistical and mathematical techniques. The knowledge is extracted from this data by examining the patterns of the data, whether they be associations of groups or things, predictions, sequential relationships between time order events or natural groups.Data mining applications are in marketing (customer loyalty, identifying profitable customers, instore promotions, e-commerce populations); in business (teaching data mining, efficiency of the Chinese automobile industry, moderate asset allocation funds); and techniques (veterinary predictive models, data integrity in the cloud, irregular pattern detection in a mobility network and road safety modeling.)

Read more less

Book SynopsisThe series, Contemporary Perspectives on Data Mining, is composed of blind refereed scholarly research methods and applications of data mining. This series will be targeted both at the academic community, as well as the business practitioner.Data mining seeks to discover knowledge from vast amounts of data with the use of statistical and mathematical techniques. The knowledge is extracted from this data by examining the patterns of the data, whether they be associations of groups or things, predictions, sequential relationships between time order events or natural groups.Data mining applications are in marketing (customer loyalty, identifying profitable customers, instore promotions, e-commerce populations); in business (teaching data mining, efficiency of the Chinese automobile industry, moderate asset allocation funds); and techniques (veterinary predictive models, data integrity in the cloud, irregular pattern detection in a mobility network and road safety modeling.)

Read more less

Book SynopsisDigital technologies are reshaping every field of social and economic lives, so do they in the world of scientific knowledge. “The New Challenges of Knowledge” aims at understanding how the new digital technologies alter the production, diffusion and valorization of knowledge. We propose to give an insight into the economical, geopolitical and political stakes of numeric in knowledge in different countries. Law is at the center of this evolution, especially in the case of national and international confusion about Internet, Science and knowledge.Trade Review“Sharing economy models are rippling through the world of scientific knowledge and research; open access brings challenges for developers, researchers, and policy makers – all treated here in the context of law-making” The Magpi, issue 60, Aug 2017Table of ContentsIntroduction . xiii Part 1. Production: Global Knowledge and Science in the Digital Era 1 Chapter 1. Current Knowledge Dynamics 3 1.1. Transparency of scientific data 4 1.2. Transparency of experimental protocol 6 1.3. A necessary form of research engineering 7 1.4. Confusion between data and scientific results: avoiding manipulation of research results 8 Chapter 2. Digital Conditions for Knowledge Production 11 2.1. An economic system oriented toward innovation 11 2.2. What of knowledge and indeed the concept of the commons? 13 2.3. From analog to digital 14 2.4. User–producer: civil society enters the knowledge production system 16 2.5. The interactions between the various spheres of knowledge production 18 2.6. Collaboration between society and knowledge: producing authorities should be put into perspective 20 Chapter 3. The Dual Relationship between the User and the Developer 23 3.1. Legal arrangements for knowledge-sharing using development platforms 23 3.2. The user contributes to the creation and development of content process 25 Chapter 4. Researchers’ Uses and Needs for Scientific and Technical Information 29 4.1. The CNRS survey 29 4.2. Diverse uses and dual needs 31 4.3. An explanation through differentiated scientific analysis 33 Chapter 5. New Tools for Knowledge Capture 37 5.1. The growth of metadata exploitation 37 5.2. Are we moving toward a semantic Web? 38 5.3. Tools and limits for metadata processing 39 5.4. The challenges of the semantic Web 40 Chapter 6. Modes of Knowledge Sharing and Technologies 43 6.1. Data storage technologies and access allowing knowledge sharing 43 6.2. Exchange platforms and catalogs 44 6.3. Knowledge-processing and digital editions 45 Part 2. Sharing Mechanisms: Knowledge Sharing and the Knowledge-based Economy 47 Chapter 7. Business Model for Scientific Publication 49 7.1. The current economic model is changing so as to adapt to new conditions for knowledge sharing 49 7.2. Creation of a new model 51 7.3. The issues raised by the creation of a new economic model 52 7.4. A new economic model struggling to fine its niche 54 Chapter 8. Actor Strategy: International Scientific Publishing, Services with High Added Value and Research Communities 57 8.1. Publishing, editing and existing: live issues within the publication of Scientific and Technical Information (STI) 58 8.2. Who is subject to it? The other players in scientific publishing 59 8.3. The characteristics of SMS (Science of Man and Society) 60 8.4. Existing without publishing? New STI directions 62 8.5. Alternatives to scientific publishing 63 Chapter 9. New Approaches to Scientific Production 67 9.1. New means of access to scientific production: innovative models 67 9.2. Two main objectives: accelerating knowledge sharing and promoting scientific collaboration 71 9.3. The need for new analytical tools and the risk of reprivatization of scientific knowledge. 72 9.4. The absence of the usage doctrine and the risk of reprivatization of science: the case of social networks 74 Chapter 10. The Geopolitics of Science 77 10.1. National convergent research models 78 10.2. Science is a source of international cooperation 81 10.3. International scientific cooperation is accelerating 84 Chapter 11. Copyright Serving the Market 85 Part 3. Enhancement Knowledge Rights and Public Policies in the Wake of Digital Technology 89 Chapter 12. Legal Protection of Scientific Research Results in the Humanities and Social Sciences 91 12.1.Different legal protections for different kinds of science 91 12.2. Why protect? 92 12.3. How to protect 93 12.4. Protect against whom? 98 12.5. Changing the challenges of Internet protection 99 12.6. Legal obstacles related to the author’s right 100 Chapter 13. Development of Knowledge and Public Policies 103 13.1. Knowledge enhancement concerns everyone 104 13.2. What are the public policies for enhancing knowledge? 105 13.3. State establishment of connections between actors: a key tool in knowledge enhancement 107 13.4. Comparing the United States and the European Union 109 Chapter 14. From Author to Enhancer 111 14.1. Enhancing scientific research is a complex process 112 14.2. Scientific research enhancement follows a legislative framework intended to promote innovation 114 Chapter 15. The Right to Knowledge: Moving Toward a Universal Law? 117 15.1. Unclear regulatory frameworks 118 15.2. Developing legal frameworks related to the Internet is complicated 121 15.3. Proposals for developing legal frameworks for the Internet 123 Chapter 16. Governing by Algorithm 127 16.1. Statistics that foreshadow algorithms 128 16.2. Algorithmic governance and democratic opportunities 130 Chapter 17. Public Data and Science in e-Government 133 17.1. Disseminating data and disseminating science: a new requirement 134 17.2. Public data in the e-government 137 17.3. Science within e-government 139 Chapter 18. Surveillance, Sousveillance, Improper Capturing 141 18.1. The traditional legal framework for information capture 142 18.2. The clear need for a specific law 145 Chapter 19. Public Knowledge Policies in the Digital Age 149 19.1. GAFA domination and the oligopolization of the market 150 19.2. Isolated digital ecosystems 152 19.3. Regulation through competition law 153 19.4. Data protection: moving toward a law for the digital community 154 Chapter 20. The Politics of Creating Artificial Intelligence 157 20.1. History 158 20.2. Artificial intelligence has become a priority for public and private actors 160 20.4. The appearance of legal problems 162 Chapter 21. Security Policies in Artificial Intelligence 165 21.1. Security as a comment on machines and data 166 21.2. From the security of machines to the security of humans 169 Conclusion 175 Postscript 177 Glossary 179 Bibliography 185 Index 201

Read more less

Book SynopsisThis book is an introduction to the linguistic concepts of argumentation relevant for argument mining, an important research and development activity which can be viewed as a highly complex form of information retrieval, requiring high-level natural language processing technology. While the first four chapters develop the linguistic and conceptual aspects of argument expression, the last four are devoted to their application to argument mining. These chapters investigate the facets of argument annotation, as well as argument mining system architectures and evaluation. How annotations may be used to develop linguistic data and how to train learning algorithms is outlined. A simple implementation is then proposed. The book ends with an analysis of non-verbal argumentative discourse. Argument Mining is an introductory book for engineers or students of linguistics, artificial intelligence and natural language processing. Most, if not all, the concepts of argumentation crucial for argument mining are carefully introduced and illustrated in a simple manner.Table of ContentsPreface xi Chapter 1. Introduction and Challenges 1 1.1. What is argumentation? 1 1.2. Argumentation and argument mining 4 1.3. The origins of argumentation 7 1.4. The argumentative discourse 8 1.5. Contemporary trends 10 Chapter 2. The Structure of Argumentation 13 2.1. The argument–conclusion pair 13 2.2. The elementary argumentative schema 14 2.2.1. Toulmin’s argumentative model 14 2.2.2. Some elaborations and refinements of Toulmin’s model 17 2.2.3. The geometry of arguments 18 2.3. Modeling agreement and disagreement 20 2.3.1. Agreeing versus disagreeing 20 2.3.2. The art of resolving divergences 23 2.4. The structure of an argumentation: argumentation graphs 25 2.5. The role of argument schemes in argumentation 27 2.5.1. Argument schemes: main concepts 27 2.5.2. A few simple illustrations 28 2.5.3. Argument schemes based on analogy 29 2.5.4. Argument schemes based on causality 30 2.6. Relations between Toulmin’s model and argumentation schemes 31 2.6.1. Warrants as a popular opinion 32 2.6.2. Argument schemes based on rules, explanations or hypothesis 34 2.6.3. Argument schemes based on multiple supports or attacks 35 2.6.4. Causality and warrants 37 Chapter 3. The Linguistics of Argumentation 39 3.1. The structure of claims 40 3.2. The linguistics of justifications 45 3.3. Evaluating the strength of claims, justifications and arguments 47 3.3.1. Strength factors within a proposition 49 3.3.2. Structuring expressions of strength by semantic category 51 3.3.3. A simple representation of strength when combining several factors 52 3.3.4. Pragmatic factors of strength expression 53 3.4. Rhetoric and argumentation 59 3.4.1. Rhetoric and communication 60 3.4.2. Logos: the art of reasoning and of constructing demonstrations 61 3.4.3. Ethos: the orator profile 62 3.4.4. Pathos: how to persuade an audience 63 Chapter 4. Advanced Features of Argumentation for Argument Mining 65 4.1. Managing incoherent claims and justifications 65 4.1.1. The case of justifications supporting opposite claims 66 4.1.2. The case of opposite justifications justifying the same claim 67 4.2. Relating claims and justifications: the need for knowledge and reasoning 67 4.2.1. Investigating relatedness via corpus analysis 68 4.2.2. A corpus analysis of the knowledge involved 69 4.2.3. Observation synthesis 72 4.3. Argument synthesis in natural language 74 4.3.1. Features of a synthesis 75 4.3.2. Structure of an argumentation synthesis 76 Chapter 5. From Argumentation to Argument Mining 79 5.1. Some facets of argument mining 79 5.2. Designing annotation guidelines: some methodological elements 81 5.3. What results can be expected from an argument mining system? 82 5.4. Architecture of an argument mining system 83 5.5. The next chapters 84 Chapter 6. Annotation Frameworks and Principles of Argument Analysis 85 6.1. Principles of argument analysis 86 6.1.1. Argumentative discourse units 86 6.1.2. Conclusions and premises 88 6.1.3. Warrants and backings 89 6.1.4. Qualifiers 89 6.1.5. Argument schemes 90 6.1.6. Attack relations: rebuttals, refutations, undercutters 90 6.1.7. Illocutionary forces, speech acts 92 6.1.8. Argument relations 93 6.1.9. Implicit argument components and tailored annotation frameworks 95 6.2. Examples of argument analysis frameworks 97 6.2.1. Rhetorical Structure Theory 97 6.2.2. Toulmin’s model 98 6.2.3. Inference Anchoring Theory 99 6.2.4. Summary 102 6.3. Guidelines for argument analysis 103 6.3.1. Principles of annotation guidelines 103 6.3.2. Inter-annotator agreements 104 6.3.3. Interpretation of IAA measures 105 6.3.4. Some examples of IAAs 106 6.3.5. Summary 107 6.4. Annotation tools 108 6.4.1. Brat 108 6.4.2. RST tool 109 6.4.3. AGORA-net 110 6.4.4. Araucaria 110 6.4.5. Rationale 111 6.4.6. OVA+ 112 6.4.7. Summary 113 6.5. Argument corpora 114 6.5.1. COMARG 115 6.5.2. A news editorial corpus 115 6.5.3. THF Airport ArgMining corpus 115 6.5.4. A Wikipedia articles corpus 115 6.5.5. AraucariaDB 115 6.5.6. An annotated essays corpus 116 6.5.7. A written dialogs corpus 116 6.5.8. A web discourse corpus 116 6.5.9. Argument Interchange Format Database 116 6.5.10. Summary 117 6.6. Conclusion 118 Chapter 7. Argument Mining Applications and Systems 119 7.1. Application domains for argument mining 119 7.1.1. Opinion analysis augmented by argument mining 120 7.1.2. Summarization 120 7.1.3. Essays 120 7.1.4. Dialogues 120 7.1.5. Scientific and news articles 120 7.1.6. The web 121 7.1.7. Legal field 121 7.1.8. Medical field 121 7.1.9. Education 121 7.2. Principles of argument mining systems 122 7.2.1. Argumentative discourse units detection 123 7.2.2. Units labeling 123 7.2.3. Argument structure detection 124 7.2.4. Argument completion 125 7.2.5. Argument structure representation 125 7.3. Some existing systems for argument mining 126 7.3.1. Automatic detection of rhetorical relations 126 7.3.2. Argument zoning 126 7.3.3. Stance detection 127 7.3.4. Argument mining for persuasive essays 127 7.3.5. Argument mining for web discourse 127 7.3.6. Argument mining for social media 128 7.3.7. Argument scheme classification and enthymemes reconstruction 128 7.3.8. Argument classes and argument strength classification 128 7.3.9. Textcoop 129 7.3.10. IBM debating technologies 129 7.3.11. Argument mining for legal texts 129 7.4. Efficiency and limitations of existing argument mining systems 130 7.5. Conclusion 131 Chapter 8. A Computational Model and a Simple Grammar-Based Implementation 133 8.1. Identification of argumentative units 134 8.1.1. Challenges raised by the identification of argumentative units 134 8.1.2. Some linguistic techniques to identify ADUs 135 8.2. Mining for claims 139 8.2.1. The grammar formalisms 140 8.2.2. Lexical issues 142 8.2.3. Grammatical issues 145 8.2.4. Templates for claim analysis 148 8.3. Mining for supports and attacks 150 8.3.1. Structures introduced by connectors 150 8.3.2. Structures introduced by propositional attitudes 151 8.3.3. Other linguistic forms to express supports or attacks 152 8.4. Evaluating strength 153 8.5. Epilogue 154 Chapter 9. Non-Verbal Dimensions of Argumentation: a Challenge for Argument Mining 155 9.1. The text and its additions 156 9.1.1. Text, pictures and icons 156 9.1.2. Transcriptions of oral debates 156 9.2. Argumentation and visual aspects 157 9.3. Argumentation and sound aspects 158 9.3.1. Music and rationality 159 9.3.2. Main features of musical structure: musical knowledge representation 160 9.4. Impact of non-verbal aspects on argument strength and on argument schemes 161 9.5. Ethical aspects 162 Bibliography 163 Index 175

Read more less

Book SynopsisThe different facets of the sharing economy offer numerous opportunities for businesses ? particularly those that can be distinguished by their creative ideas and their ability to easily connect buyers and senders of goods and services via digital platforms. At the beginning of the growth of this economy, the advanced digital technologies generated billions of bytes of data that constitute what we call Big Data. This book underlines the facilitating role of Big Data analytics, explaining why and how data analysis algorithms can be integrated operationally, in order to extract value and to improve the practices of the sharing economy. It examines the reasons why these new techniques are necessary for businesses of this economy and proposes a series of useful applications that illustrate the use of data in the sharing ecosystem.Table of ContentsPreface xi Introduction xiii Part 1. The Sharing Economy or the Emergence of a New Business Model 1 Chapter 1. The Sharing Economy: A Concept Under Construction 3 1.1. Introduction 3 1.2. From simple sharing to the sharing economy 5 1.2.1. The genesis of the sharing economy and the break with “consumer” society 5 1.2.2. The sharing economy: which economy? 8 1.3. The foundations of the sharing economy 10 1.3.1. Peer-to-peer (P2P): a revolution in computer networks 10 1.3.2. The gift: the abstract aspect of the sharing economy 13 1.3.3. The service economy and the offer of use 18 1.4. Conclusion 24 Chapter 2. An Opportunity for the Business World 25 2.1. Introduction 25 2.2. Prosumption: a new sharing economy trend for the consumer 27 2.3. Poverty: a target in the spotlight of the shared economy 29 2.4. Controversies on economic opportunities of the sharing economy 31 2.5. Conclusion 37 Chapter 3. Risks and Issues of the Sharing Economy 39 3.1. Introduction 39 3.2. Uberization: a white grain or just a summer breeze? 40 3.3. The sharing economy: a disruptive model 43 3.4. Major issues of the sharing economy 47 3.5. Conclusion 50 Chapter 4. Digital Platforms and the Sharing Mechanism 51 4.1. Introduction 51 4.2. Digital platforms: “What growth!” 52 4.3. Digital platforms or technology at the service of the economy 54 4.4. From the sharing economy to the sharing platform economy 57 4.5. Conclusion 59 Part 2. Big Data Analytics at the Service of the Sharing Economy 61 Chapter 5. Beyond the Word “Big”: The Changes 63 5.1. Introduction 63 5.2. The 3 Vs and much more: volume, variety, velocity 64 5.2.1. Volume 65 5.2.2. The variety 66 5.2.3. Velocity 67 5.2.4. What else? 68 5.3. The growth of computing and storage capacities 69 5.3.1. Big Data versus Big Computing 70 5.3.2. Big Data storage 71 5.3.3. Updating Moore’s Law 73 5.4. Business context change in the era of Big Data 74 5.4.1. The decision-making process and the dynamics of value creation 75 5.4.2. The emergence of new data-driven business models 77 5.5. Conclusion 78 Chapter 6. The Art of Analytics 81 6.1. Introduction 81 6.2. From simple analysis to Big Data analytics 82 6.2.1. Descriptive analysis: learning from past behavior to influence future outcomes 84 6.2.2. Predictive analysis: analyzing data to predict future outcomes 84 6.2.3. Prescriptive analysis: recommending one or more action plan(s) 85 6.2.4. From descriptive analysis to prescriptive analysis: an example 87 6.3. The process of Big Data analytics: from the data source to its analysis 88 6.3.1. Definition of objectives and requirements 90 6.3.2. Data collection 91 6.3.3. Data preparation 92 6.3.4. Exploration and interpretation 94 6.3.5. Modeling 95 6.3.6. Deployment 97 6.4. Conclusion 97 Chapter 7. Data and Platforms in the Sharing Context 99 7.1. Introduction 99 7.2. Pioneers in Big Data 101 7.2.1. Big Data on Walmart’s shelves 101 7.2.2. The Big Data behind Netflix’s success story 102 7.2.3. The Amazon version of Big Data 103 7.2.4. Big data and social networks: the case of Facebook 104 7.2.5. IBM and data analysis in the health sector 105 7.3. Data, essential for sharing 106 7.3.1. Data and platforms at the heart of the sharing economy 108 7.3.2. The data of sharing economy companies 110 7.3.3. Privacy and data security in a sharing economy 111 7.3.4. Open Data and platform data sharing 114 7.4. Conclusion 116 Chapter 8. Big Data Analytics Applied to the Sharing Economy 119 8.1. Introduction 119 8.2. Big Data and Machine Learning algorithms serving the sharing economy 121 8.2.1. Machine Learning algorithms 122 8.2.2. Algorithmic applications in the sharing economy context 124 8.3. Big Data technologies: the sharing economy companies’ toolbox 125 8.3.1. The appearance of a new concept and the creation of new technologies 127 8.4. Big Data on the agenda of sharing economy companies 130 8.4.1. Uber 131 8.4.2. Airbnb 132 8.4.3. BlaBlaCar 133 8.4.4. Lyft 134 8.4.5. Yelp 135 8.4.6. Other cases 137 8.5. Conclusion 139 Part 3. The Sharing Economy? Not Without Big Data Algorithms 141 Chapter 9. Linear Regression 143 9.1. Introduction 143 9.2. Linear regression: an advanced analysis algorithm 144 9.2.1. How are regression problems identified? 145 9.2.2. The linear regression model 146 9.2.3. Minimizing modeling error 148 9.3. Other regression methods 149 9.3.1. Logistic regression 150 9.3.2. Additional regression models: regularized regression 151 9.4. Building your first predictive model: a use case 152 9.4.1. What variables help set a rental price on Airbnb? 152 9.5. Conclusion 169 Chapter 10. Classification Algorithms 171 10.1. Introduction 171 10.2. A tour of classification algorithms 172 10.2.1. Decision trees 172 10.2.2. Naïve Bayes 175 10.2.3. Support Vector Machine (SVM) 177 10.2.4. Other classification algorithms 179 10.3. Modeling Airbnb prices with classification algorithms 183 10.3.1. The work that’s already been done: overview 184 10.3.2. Models based on trees: decision tree versus Random Forest 185 10.3.3. Price prediction with kNN 190 10.4. Conclusion 193 Chapter 11. Cluster Analysis 195 11.1. Introduction 195 11.2. Cluster analysis: general framework 196 11.2.1. Cluster analysis applications 197 11.2.2. The clustering algorithm and the similarity measure 198 11.3. Grouping similar objects using k-means 200 11.3.1. The k-means algorithm 201 11.3.2. Determine the number of clusters 203 11.4. Hierarchical classification 205 11.4.1. The hierarchical model approach 206 11.4.2. Dendrograms 207 11.5. Discovering hidden structures with clustering algorithms 208 11.5.1. Illustration of the classification of prices based on different characteristics using the k-means algorithm 209 11.5.2. Identify the number of clusters k 210 11.6. Conclusion 213 Conclusion 215 References 217 Index 233

Read more less

Book SynopsisThe concept of digital risk, which has become ubiquitous in the media, sustains a number of myths and beliefs about the digital world. This book explores the opposite view of these ideologies by focusing on digital risks as perceived by actors in their respective contexts.Perceptions and Analysis of Digital Risks identifies the different types of risks that concern actors and actually impact their daily lives, within education or various socio-professional environments. It provides an analysis of the strategies used by the latter to deal with these risks as they conduct their activities; thus making it possible to characterize the digital cultures and, more broadly, the informational cultures at work.This book offers many avenues for action in terms of educating the younger generations, training teachers and leaders, and mediating risks.Table of ContentsForeword xiFranc MORANDI Introduction xviiCamille CAPELLE Part 1. Risk Perceptions, Education and Learning 1 Chapter 1. Digital Risks: An Obstacle or a Lever for Education? 3Camille CAPELLE 1.1. Introduction 3 1.2. Digital risks and education: what are we talking about? 4 1.2.1. Digital risks 4 1.2.2. What are the risks in education? 8 1.3. Questioning perceptions of digital risks among new teachers 9 1.3.1. Why was this target audience chosen? 9 1.3.2. Methodology and data collection 10 1.4. Teachers’ perceptions of digital risks 11 1.4.1. When perceptions of risk inhibit any practice 11 1.4.2. When perceptions of risk freeze practices 14 1.4.3. When risk perceptions lead us to consider them in order to overcome them 18 1.5. Reflection on the role of digital risk representations in education 21 1.6. Conclusion 24 1.7. References 25 Chapter 2. Teenagers Faced with “Fake News”: Perceptions and the Evaluation of an Epistemic Risk 27Gilles SAHUT and Sylvie FRANCISCO 2.1. Introduction 27 2.2. Fake news: From production to reception 28 2.2.1. Characterizing the fake news phenomenon 29 2.2.2. The potential risks associated with fake news 31 2.2.3. The credibility of fake news 32 2.3. Methodological framework of the study 34 2.4. Results of the study 36 2.4.1. A heterogeneous understanding of the concept 37 2.4.2. A blurred perception of the goals of fake news 39 2.4.3. The diversity of fake news sources 40 2.4.4. Identifying fake news: heuristic processing and analytical strategies 42 2.4.5. A remote and controlled phenomenon? 45 2.5. Discussion of the results and reflections on media and information literacy 46 2.6. Conclusion 49 2.7. References 50 Chapter 3. “A Big Nebula that is a Bit Scary” (Louise, Trainee Schoolteacher): Training through/in Digital Technology, in School and in Professional Training 55Anne CORDIER 3.1. Social beings, above all else 57 3.1.1. A “fluid identity” to be grasped 57 3.1.2. Digital technology in the actors’ personal ecosystem 61 3.2. Understanding of digital technology in the classroom 62 3.2.1. Crystallization and awareness of issues 62 3.2.2. When the socio-technical framework hinders the entry of digital technology into the classroom 64 3.2.3. Rather modest and low-risk experiments 66 3.3. Teaching with and through digital technology: Constant risks 68 3.3.1. Tensions in the classroom 68 3.3.2. Tensions in training 71 3.3.3. Desires on both sides 73 3.4. Potential courses of action 76 3.5. References 78 Part 2. Risks in the Light of Socio-Economic Issues 81 Chapter 4. Top Managers Confronted with Information Risks: An Exploratory Study within the Telecommunications Sector 83Dijana LEKIC, Anna LEZON-RIVIÈRE and Madjid IHADJADENE 4.1. Introduction 83 4.2. Information risk: The conceptual field 84 4.3. Controlling information risks: Security policy 89 4.4. Information risk and management 91 4.5. Study methodology and the stakeholder group 93 4.6. Information risk: The perspective of top telecoms managers 94 4.6.1. Top managers as responsible for information risk management 94 4.6.2. Information risk management 97 4.6.3. Operational challenges related to the information risk management approach 100 4.7. Conclusion 104 4.8 Acknowledgments 106 4.9. References 106 Chapter 5. Cell Phones and Scamming Risks in Cameroon: Users’ Experiences and Socio-Institutional Responses 111Freddy TSOPFACK FOFACK and Abdel Bernazi RENGOU 5.1. Introduction 111 5.2. Mechanisms behind cell phone scamming in Cameroon: Exhibiting credulity 115 5.2.1. Setting the scene 116 5.2.2. Enticing but misleading proposals 117 5.2.3. Disguised telephone number confusion 119 5.3. The dynamics of cell phone use in Cameroon 121 5.3.1. The Ministry of Posts and Telecommunications 121 5.3.2. Agence Nationale des Technologies de l’Information et de la Communication 122 5.3.3. Agence de Régulation des Télécommunications 122 5.3.4. Cell phone operators 123 5.3.5. The judicial system and cell phone scams 124 5.3.6. Cell phone users and consumer associations 125 5.4. Socio-institutional governance of cell phone use in Cameroon: Optimal or approximate mediations? 126 5.4.1. Information deficit of the users 126 5.4.2. Insufficient means of action 127 5.4.3. Mis-selling of SIM cards by mobile operators: An “ingredient” of mobile scammers 128 5.4.4. The ease of monetary transactions 129 5.4.5. Technological constraints and border porosity 129 5.5. Conclusion 130 5.6. References 131 Part 3. Digital Risks: Practices and Mediation 135 Chapter 6. Towards a Normative Prescription of Information Practices on Digital Social Networks: A Study of Documentary Pedagogical Projects in Middle School 137Adeline ENTRAYGUES 6.1. Introduction 137 6.2. Contextualization of risk 138 6.3. Issues to consider 138 6.4. Research objects 139 6.5. Research protocol 142 6.6. Risk regarding DSNs in the pedagogical approach 144 6.6.1. Raising awareness of risks: An obvious approach for teacher librarians 144 6.6.2. Considering the views of learners and teachers 145 6.6.3. Considering the risks: Learners aware of digital dangers 148 6.7. Discovering DSNs in a school context: Dealing with risks 151 6.7.1. Pedagogical projects on DSNs to prevent risks: Teachers’ perspectives 151 6.7.2. Overcoming risks: Learners’ perspectives 152 6.8. Perspectives for an information culture 153 6.8.1. Risks, standards and education 153 6.8.2. A culture of information in training 154 6.9. Conclusion 155 6.10. References 155 Chapter 7. MIL as a Tool for Teachers to Prevent Risk and Transmit Digital Culture 159Julie PASCAU 7.1. Studying digital technology in schools from the perspective of teachers’ representations 159 7.1.1. Why be interested in representations? 161 7.1.2. The social representation of digital risks through the analysis of institutional discourses 163 7.2. What do digital and media literacy evoke in teachers? 164 7.2.1. The weak presence of digital technology and MIL in elementary school 165 7.2.2. Risks in the representations of MIL among primary school teachers 166 7.2.3. A positive perception of the role of digital technology in the classroom 169 7.3. The contours of media and information literacy according to teachers 171 7.3.1. The objects of MIL from the discourse of primary school teachers 172 7.3.2. What does digital technology mean for teachers? 173 7.4. What does the requirement to transmit digital culture mean for teachers? 178 7.4.1. Digital culture: A very vague concept 178 7.4.2. What primary school teachers think digital literacy means 180 7.5. Conclusion 187 7.6. References 189 Conclusion 193Camille CAPELLE Postface 197Vincent LIQUÈTE List of Authors 201 Index 203

Read more less

Book SynopsisTechnology is not just limited to technology companies, it impacts sectors such as healthcare, agriculture, and security. In the last few decades, countries, too, have started developing technologies or integrating technologies into their systems. As a result, all countries, regardless of size, need to understand the management of engineering and technology concepts. Digital Transformations reviews fundamentals and applications through existing and emerging technologies all around the world.Big data availability and the emergence of new tools provide opportunities to detect the emergence of new technologies. Some of the major elements of such analyses include bibliometrics, patent analysis and social network analysis. The authors focus on these three tools and demonstrate their use through applications such as Blockchain, Artificial Intelligence, Robotics, 3D printing, Wireless Power, Autonomous and Electric Driving, and Smart Homes.Through the examination of cases based on emerging technologies, the book provides a spectrum of these recent applications and serves as a reference for professionals, researchers and students on fundamentals of technology utilization tools.Trade Review‘Dr. Tugrul Daim has championed another masterpiece with this manuscript. This book helps transform vital information from the academic world into a blueprint that can be used by government and industry commercial leaders. A true first of its kind, as there is no other manuscript available to help engineering and technology managers navigate through the challenges presented by Digital Transformations.’ -- Matthew L. Tompkins, TC Defense, US‘The authors introduce the use of statistical methods such as bibliometrics, patent analysis and network analysis to understand trends, connections and leadership in technology innovation and also to identify key issues. Real-world case studies explore an array of innovations in the medical, power, transportation and home appliance fields. This approach illustrates how the techniques are useful while telling the story of some of today’s pivotal innovations.’ -- Fred Gordon, Energy Trust of Oregon, USTable of ContentsContents: Introduction to Digital Transformations 1. Bibliometric-based analyses 2. Patent-based analyses 3. Network-based analyses 4. Integrated analyses 5. Conclusion References Index

Read more less

Book SynopsisMachine Learning under Resource Constraints addresses novel machine learning algorithms that are challenged by high-throughput data, by high dimensions, or by complex structures of the data in three volumes. Resource constraints are given by the relation between the demands for processing the data and the capacity of the computing machinery. The resources are runtime, memory, communication, and energy. Hence, modern computer architectures play a significant role. Novel machine learning algorithms are optimized with regard to minimal resource consumption. Moreover, learned predictions are executed on diverse architectures to save resources. It provides a comprehensive overview of the novel approaches to machine learning research that consider resource constraints, as well as the application of the described methods in various domains of science and engineering. Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to the different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Several machine learning methods are inspected with respect to their resource requirements and how to enhance their scalability on diverse computing architectures ranging from embedded systems to large computing clusters.

Read more less