Computing Books

Book SynopsisThis book fills a gap between high-level overview texts that are often too general and low-level detail oriented technical handbooks that lose sight the big picture.This book discusses SOA from the low-level perspective of middleware, various XML-based technologies, and basic service design.It also examines broader implications of SOA, particularly where it intersects with business process management and process modeling.Concrete overviews will be provided of the methodologies in those fields, so that students will have a hands-on grasp of how they may be used in the context of SOA.Table of ContentsList of Figures xv Acknowledgements xxiii 1. Introduction 1 References / 6 2. Middleware 7 2.1 Enterprise Information Systems / 7 2.2 Communication / 12 2.3 System and Failure Models / 21 2.4 Remote Procedure Call / 34 2.5 Message-Oriented Middleware / 42 2.6 Web Services and Service-Oriented Architecture (SOA) / 46 2.7 Cloud Computing / 52 2.8 Naming and Discovery / 55 2.9 Further Reading / 56 References / 57 3. Data Modeling 59 3.1 Entities and Relationships / 60 3.1.1 Concepts and Entities / 60 3.1.2 Attributes and Relationships / 61 3.1.3 Properties of Relationship Types / 65 3.1.4 Special Relationship Types / 69 3.2 XML Schemas / 74 3.3 Defining New Types / 79 3.3.1 Defining Simple Types / 79 3.3.2 Defining Complex Types / 82 3.4 Derived Types / 85 3.4.1 Derived Simple Types / 86 3.4.2 Derived Complex Types / 87 3.5 Document Hierarchies / 94 3.6 Relationship Types in XML Schemas / 98 3.7 Metaschemas and Metamodels / 100 3.8 Further Reading / 102 References / 102 4. Data Processing 104 4.1 Processing XML Data / 104 4.1.1 Tree Processing / 105 4.1.2 Schema Binding / 109 4.1.3 Stream Processing / 114 4.1.4 External Processing / 119 4.2 Query Languages and XQuery / 122 4.3 XML Databases / 134 4.3.1 Storage as Relational Tables / 135 4.3.2 Storage as Large Strings / 137 4.3.3 Native XML Storage / 137 4.4 Web Services / 138 4.4.1 SOAP: (not so) Simple Object Access Protocol / 139 4.4.2 WSDL: Web Services Description Language / 145 4.4.3 Web Service Policy / 155 4.5 Presentation Layer: JSON and JQUERY / 159 References / 166 5. Domain-Driven Architecture 167 5.1 Software Architecture / 167 5.2 Domain-Driven Design / 168 5.3 Application Frameworks / 175 5.4 Domain-Specific Languages (DSLs) / 180 5.5 An Example API for Persistent Domain Objects / 188 5.6 Domain-Driven Architecture / 197 5.7 Further Reading / 205 References / 205 6. Service-Oriented Architecture 207 6.1 Services and Procedures / 207 6.2 Service-Oriented Architecture (SOA) / 211 6.3 Service Design Principles / 216 6.4 Service-Oriented Architecture (SOA) Governance / 218 6.5 Standardized Service Contract / 221 6.5.1 Operations Contract / 222 6.5.2 Data Contract / 223 6.5.3 Policy Contract / 224 6.5.4 Binding Contract / 226 6.5.5 Contract Versioning / 231 6.6 Service Loose Coupling / 237 6.6.1 Motivation for Loose Coupling / 237 6.6.2 Contract Development / 239 6.6.3 Loose Coupling Patterns / 242 6.6.4 Cost of Loose Coupling / 246 6.7 Service Abstraction / 248 6.7.1 Platform Abstraction / 248 6.7.2 Protocol Abstraction / 249 6.7.3 Procedural Abstraction / 261 6.7.4 State Abstraction / 264 6.7.5 Data Abstraction / 269 6.7.6 Endpoint Abstraction / 278 6.8 Service Reusability / 278 6.8.1 Parameterization and Bounded Polymorphism / 279 6.8.2 Subtyping, Inheritance, and Contracts / 284 6.8.3 Does Service-Oriented Architecture Require Subtyping? / 289 6.8.4 Patterns for Service Reusability / 292 6.9 Service Autonomy / 299 6.9.1 Replicating Computation / 300 6.9.2 Replicating State / 303 6.9.3 Sources of Errors and Rejuvenation / 308 6.9.4 Caching / 313 6.10 Service Statelessness / 323 6.10.1 Contexts and Dependency Injection / 331 6.11 Service Discoverability / 336 6.11.1 Global Discovery / 336 6.11.2 Local Discovery / 337 6.11.3 Layered Naming / 347 6.12 Further Patterns / 351 6.13 Further Reading / 352 References / 352 7. Resource-Oriented Architecture 359 7.1 Representational State Transfer / 359 7.2 RESTful Web Services / 369 7.3 Resource-Oriented Architecture (ROA) / 379 7.4 Interface Description Languages / 387 7.4.1 Web Services Description Language (WSDL) / 387 7.4.2 Web Application Description Language (WADL) / 390 7.5 An Example Application Program Interface (API) for Resource-Oriented Web Services / 396 7.6 Hypermedia Control and Contract Conformance / 406 7.7 Concluding Remarks / 412 7.8 Further Reading / 414 References / 414 Appendix A: Introduction to Haskell 416 A.1 Types and Functions / 416 A.2 Type Classes and Functors / 425 A.3 Monads / 431 A.4 Further Reading / 436 References / 436 Appendix B: Time in Distributed Systems 437 B.1 What Time Is It? / 437 B.2 Time and Causality / 443 B.3 Applications of Logical and Vector Time / 450 B.3.1 Mutual Exclusion / 450 B.3.2 Quorum Consensus / 451 B.3.3 Distributed Logging / 456 B.3.4 Causal Message Delivery / 458 B.3.5 Distributed Snapshots / 463 B.4 Virtual Time / 468 B.5 Further Reading / 470 References / 470 Index 473

Read more less

Book SynopsisThe ultimate handbook for Excel analysts who need reporting solutions using Access Excel and Access are intended to work together. This book offers a comprehensive review of the extensive analytical and reporting functionality that Access provides and how it enhances Excel reporting functions. Sales managers, operations analysts, administrative assistants, office managers, and many others who rely heavily on data can benefit from learning to integrate Excel and Access, and this book shows you how. Coverage includes: Data Analysis in Access & the Basics of Access Beyond Select Queries Transforming Your Data with Access Working with Calculations and Dates Performing Conditional Analysis Adding Dimension with Subqueries and Domain Aggregate Functions Running Descriptive Statistics in Access Scheduling and Running Batch Analysis Leveraging VBA to Enhance Data Analysis<Table of ContentsIntroduction xxix Part I Fundamentals of Data Analysis in Access 1 Chapter 1 The Case for Data Analysis in Access 3 Chapter 2 Access Basics 13 Chapter 3 Beyond Select Queries 47 Part II Basic Analysis Techniques 87 Chapter 4 Transforming Your Data with Access 89 Chapter 5 Working with Calculations and Dates 113 Chapter 6 Performing Conditional Analysis 141 Part III Advanced Analysis Techniques 161 Chapter 7 Adding Dimension with Subqueries and Domain Aggregate Functions 163 Chapter 8 Running Descriptive Statistics in Access 189 Chapter 9 Scheduling and Running Batch Analysis 209 Chapter 10 Leveraging VBA to Enhance Data Analysis 243 Part IV Reports, Dashboards, and Visualization in Access 267 Chapter 11 Presenting Data with Access Reports 269 Chapter 12 Using Pivot Tables and Pivot Charts in Access 291 Chapter 13 Enhancing Queries and Reports with Visualizations 323 Part V Advanced Excel and Access Integration Techniques 345 Chapter 14 Getting Access Data into Excel 347 Chapter 15 Using VBA to Move Data between Excel and Access 365 Chapter 16 Exploring Excel and Access Automation 389 Chapter 17 Integrating Excel and Access with XML 423 Chapter 18 Integrating Excel and Other Office Applications 441 Part VI Appendixes 475 Appendix A Access VBA Fundamentals 477 Appendix B Understanding and Using SQL 489 Appendix C Query Performance, Database Corruption, and Other Thoughts 509 Appendix D Data Analyst’s Function Reference 521 Index 563

Read more less

Book SynopsisReady or not, it's high time to make BIM a part of your practice, or at least your vocabulary, and this book has as much to offer beginners as it does seasoned users of building information modeling software.Chicago Architect The first book devoted to the subject of how BIM affects individuals and organizations working within the ever-changing construction industry, BIM and Integrated Design discusses the implementation of building information modeling software as a cultural process with a focus on the technology's impact and transformative effectboth potentially disruptive and liberatingon the social, psychological, and practical aspects of the workplace. BIM and Integrated Design answers the questions that BIM poses to the firm that adopts it. Through thorough research and a series of case study interviews with industry leadersand leaders in the making out from behind the monitorBIM and Integrated Design helps you learn: Table of ContentsIntroductory Statement by The American Institute of Architects vii Preface ix Acknowledgments xiii Introduction xv PART I BIM As Though People Mattered 1 Chapter 1 WHAT YOU ADOPT WHEN ADOPTING BIM 3 Chapter 2 THE SOCIAL IMPLICATIONS OF IMPLEMENTING BIM 29 Case Study Interview with Paul Durand, AIA, and Allison Scott, Winter Street Architects 46 Case Study Interview with Aaron Greven, BIM Consultant 53 Chapter 3 WHO WORKS IN BIM AND WHO DOESN’T 63 Case Study Interview with Jack Hungerford, PhD 69 Case Study Interview with Kristine K. Fallon, FAIA, Kristine Fallon Associates 76 PART II Leading Integrated Design 89 Chapter 4 WORKING WITH OTHERS IN BIM 91 Case Study Interview with Rich Nitzsche, CIO, Perkins + Will 111 Chapter 5 BIM AND INTEGRATED DESIGN 127 Case Study Interview with Andy Stapleton and Peter Rumpf, Mortenson Construction 140 Case Study Interview with Jonathan Cohen, FAIA, Architect and Author 148 PART III Leading and Learning 157 Chapter 6 LEADING FROM THE MODEL 159 Case Study Interview with Bradley Beck, Architect and BIM Manager 171 Case Study Interview with Charles Hardy, director, Office of Project Delivery at U.S. General Services Administration (GSA) Public Buildings Service National Capital Region 191 Chapter 7 LEARNING BIM AND INTEGRATED DESIGN 201 Case Study Interview with Yanni Loukissas, PhD, Postdoctoral associate, Massachusetts Institute of Technology 209 Case Study Interview with Phil Bernstein, FAIA, vice president, Autodesk 218 Epilogue 235 Index 237

Read more less

Book SynopsisAn invaluable tool in Bioinformatics, this unique volume provides both theoretical and experimental results, and describes basic principles of computational intelligence and pattern analysis while deepening the reader''s understanding of the ways in which these principles can be used for analyzing biological data in an efficient manner. This book synthesizes current research in the integration of computational intelligence and pattern analysis techniques, either individually or in a hybridized manner. The purpose is to analyze biological data and enable extraction of more meaningful information and insight from it. Biological data for analysis include sequence data, secondary and tertiary structure data, and microarray data. These data types are complex and advanced methods are required, including the use of domain-specific knowledge for reducing search space, dealing with uncertainty, partial truth and imprecision, efficient linear and/or sub-linear scalability, incremental approaTrade Review"This collection of 16 papers, edited by Maulik (computer science and engineering, Jadavpur U., India), Bandyopadhyay (Indian Statistical Institute, India), and Wang (data and knowledge engineering, New Jersey Institute of Technology, US), brings together contributions from practitioners integrating computational intelligence and pattern analysis techniques for analyzing biological data, including sequence, structure, and microarray data. The material is organized into five sections that explore basic principles and methodologies of computational techniques, applications of computational intelligence and pattern analysis for biological sequence analysis, structural analysis; microarray data analysis, and systems biology." (Reference and Research Book News, February 2011)Table of ContentsPreface. Contributors. PART 1 INTRODUCTION. 1 Computational Intelligence: Foundations, Perspectives, and Recent Trends (Swagatam Das, Ajith Abraham, and B. K. Panigrahi). 2 Fundamentals of Pattern Analysis: A Brief Overview (Basabi Chakraborty). 3 Biological Informatics: Data, Tools, and Applications (Kevin Byron, Miguel Cervantes-Cervantes, and Jason T. L. Wang). PART II SEQUENCE ANALYSIS. 4 Promoter Recognition Using Neural Network Approaches (T. Sobha Rani, S. Durga Bhavani, and S. Bapi Raju). 5 Predicting microRNA Prostate Cancer Target Genes (Francesco Masulli, Stefano Rovetta, and Giuseppe Russo). PART III STRUCTURE ANALYSIS. 6 Structural Search in RNA Motif Databases (Dongrong Wen and Jason T. L. Wang). 7 Kernels on Protein Structures (Sourangshu Bhattacharya, Chiranjib Bhattacharyya, and Nagasuma R. Chandra). 8 Characterization of Conformational Patterns in Active and Inactive Forms of Kinases using Protein Blocks Approach (G. Agarwal, D. C. Dinesh, N. Srinivasan, and Alexandre G. de Brevern). 9 Kernel Function Applications in Cheminformatics (Aaron Smalter and Jun Huan). 10 In Silico Drug Design Using a Computational Intelligence Technique (Soumi Sengupta and Sanghamitra Bandyopadhyay). PART IV MICROARRAY DATA ANALYSIS. 11 Integrated Differential Fuzzy Clustering for Analysis of Microarray Data (Indrajit Saha and Ujjwal Maulik). 12 Identifying Potential Gene Markers Using SVM Classifier Ensemble (Anirban Mukhopadhyay, Ujjwal Maulik, and Sanghamitra Bandyopadhyay). 13 Gene Microarray Data Analysis Using Parallel Point Symmetry-Based Clustering (Ujjwal Maulik and Anasua Sarkar). PART V SYSTEMS BIOLOGY. 14 Techniques for Prioritization of Candidate Disease Genes (Jieun Jeong and Jake Y. Chen). 15 Prediction of Protein–Protein Interactions (Angshuman Bagchi). 16 Analyzing Topological Properties of Protein–Protein Interaction Networks: A Perspective Toward Systems Biology (Malay Bhattacharyya and Sanghamitra Bandyopadhyay). Index.

Read more less

Book SynopsisYour all-inclusive guide to all the latest technologies Providing you with a better understanding of the latest technologies, including Cloud Computing, Software as a Service, Service-Oriented Architecture (SOA), Open Source, Mobile Computing, Social Networking, and Business Intelligence, The Next Wave of Technologies: Opportunities in Chaos helps you know which questions to ask when considering if a specific technology is right for your organization. Demystifies powerful but largely misunderstood technologies Explains how each technology works Provides key guidance on determining if a particular technology is right for your organization Contains contributions from experts on Cloud Computing, Service-Oriented Architecture (SOA), Software as a Service (SaaS), Open Source, Mobile Technologies, Enterprise Risk Management, Social Media, Business Intelligence, and more More of a management text than a technical guide, thTrade Review"…is a must-read for IT professionals who are scrambling to keep up with the implications of new technologies and a book for their colleagues who need to interface with them." (impressionsthroughmedia.com, March 28, 2010)Table of ContentsForeword. Preface. Acknowledgments. About the Contributors. Part I Introduction, Background, and Definitions. Chapter 1 The Changing Landscapes of Business and Technology. Introduction. Enterprise 2.0: What's in a Name, Anyway? Understanding the Caution. Electronic Health Records (EHR): A Case in Point. Summary. Chapter 2 How the Game Has Changed. Introduction. The Typical Enterprise 1.0 Project. Comparing Enterprise 1.0 and 2.0 Projects. Three Requirements to Successful Enterprise 2.0 Projects. Scopes, Scales, and Reaches. Unstructured Development and Alternatives to the Waterfall. Summary. Next Steps. Chapter 3 The Role of IT in an Enterprise 2.0 World. Introduction. Collins' Model. IT's Traditional Charter. Considerations. Three Viewpoints. The Changing Role of the CIO. Summary. Next Steps. Part II Architecture, Software Development, and Frameworks. Chapter 4 Cloud Computing. A Brief History of Cloud Computing. Consumers and Small Office Home Office (SOHO) versus the Enterprise. A Cloud of One's Own. What is a Cloud? Definitions of Cloud Computing. Cloud Manifesto. Cloud Architecture. The Private Cloud: When Is a Cloud Not a Cloud? Cloud Economics. Vendor Strategies. Customer Strategies. Standards and Interoperability. Security. The Future of Clouds. Summary. Next Steps. Chapter 5 Open Source: The War That Both Sides Won. Introduction. A New Geography for Software. When UNIX Walked the Earth. Co-Opted by the System. What Is Open Source? The Costs of Free Software. Is it a Virus? Legal Risks to Using Open Source. The New Green. The Future of Open Source. Summary. Next Steps. Chapter 6 Software as a Service (SaaS). Introduction. Nothing's New: SaaS' Historical Precedents. What's Different This Time? Customer Expectations. Challenges and Choices. Summary. Next Steps. Chapter 7 Service-Oriented Architecture. Introduction. What is Service-Oriented Architecture? Business Benefits of SOA. Technical Benefits of SOA. Essentials of SOA. SOA in Practice. Lessons Learned. Best Practices. Summary. Next Steps. Chapter 8 Managing Mobile Business. Introduction. An Introduction to Mobility. The Mobile Enterprise. Risks and Considerations. Business Expectations from Mobile Technologies. The Mobile Enterprise Transition Framework. Phases of Mobile Enterprise Transition. Business Value of Mobile Technologies. Metrics. Mobile Organizational Structures. Mobile Content and Services. Mobile Business Application Considerations. Balancing Personal Services with Privacy. Summary. Next Steps. Chapter 9 Social Networking. Introduction. Why Social Networking and Why Not Just Use the Public Forums? Benefits of Social Networking. Impediments, Solutions, and Resolutions to Progress. Examples of Social Networking Tools. Summary. Next Steps. Part III Data, Information, and Knowledge. Chapter 10 Enterprise Search and Retrieval. Introduction. What Is ESR? Search and Information Architecture. The Business Case for ESR. Total Cost of Ownership. Forms of ESR Deployment. ESR in Action. Best Practices. Summary. Next Steps. Chapter 11 Enterprise 2.0 Business Intelligence. What Is Business Intelligence and Why Do We Need It? BI 2.0. Measuring BI Maturity. BI Challenges. The Data Warehouse. Key Factors. Recent BI Trends. Too Much BI? Summary. Next Steps. Chapter 12 Master Data Management. Introduction. The State of the Union. The Business Case for Master Data Management. MDM Approaches and Architectures. Selecting the Right MDM Approach. MDM Services and Components. Summary. Next Steps. Chapter 13 Procurement in Chaos. Introduction. Does Procure-to-Pay Matter? What Is Procure-to-Pay Automation? Benefiting from P2P Automation. Procure-to-Pay Leadership. Automation Risks and Challenges. Leveraging your ERP. Technology Overview. Vendor Portals. Summary: Rethinking Procurement. Next Steps. Part IV Management and Deployment. Chapter 14 Agile Software Development. Introduction. Limitations of the Waterfall Method of Software Development. Benefits of Agile Methods. Alternative Engineering Models. Agile Process in a Nutshell. The Agile Landscape. The Benefits of Simplicity. The Manager's Perspective. Limitations of Agile. Achieving Enterprise Agility. Summary. Next Steps. Chapter 15 Enterprise Risk Management. The High-Risk Society. Information Technology and the High-Risk Society. Enterprise Risk Management Overview. ERM and IT. ERM, Innovation, and Entrepreneurship. Who Owns ERM? Who Is Doing ERM? The Limits of ERM. Summary. Next Steps: Are You Ready for ERM? Chapter 16 Implementing Open Source Software. Introduction. A Different Software Model. Getting into Open Source. OS and Digital Presence. OS and Managing Your Business. Appearances Can Be Deceiving. Product Development Agility. Support. Product Types. Crowdsourcing. Niche Markets. Summary. Next Steps. Chapter 17 Global Engineering. Introduction. Distributed Teams: An Integral Part of Enterprise 2.0. Room for Improvement. Preconditions and Reasons for Distributing Technology Projects. Drivers of Global IT. Why International Distributed Projects Sometimes Fail. Global Engineering 2.0. Summary. Next Steps. Chapter 18 Enterprise 2.0 IT Project Failure. Introduction. Enterprise 2.0: An Evolving Concept. Understanding Traditional IT Failure. Enterprise 2.0 Failure. Reasons Enterprise 2.0 Projects Fail. Case Study: Social CRM. Preventing Failure through Measurement. Summary. Next Steps. Chapter 19 Readying the Troops for Battle. Introduction. Know Your Organization's Limitations from the Get-Go. Insist on Maximum Collaboration with External Parties. Bridge the Gap between Functional and Technical End Users. Prepare for Sudden, Unexpected Growth. Recommit to Employee Training. Embrace Uncertainty to Foster Innovation. Flip the Switch: Reverse Mentoring. Summary. Next Steps. Chapter 20 Sustainability and Green IT. Introduction. Growing Impact on Organizations. The Green Fad. Organizational Response to Sustainability. The Future of Green IT. Summary. Next Steps. Part V Conclusion. Chapter 21 Finding Opportunity in Chaos. Introduction. Summary of Book. Where Are We Now? Enterprise 2.0 Drivers Where Are We Going? Bibliography. About the Author. Index.

Read more less

Book SynopsisIn-depth guidance on Word 2010 from a Microsoft MVP Microsoft Word 2010 arrives with many changes and improvements, and this comprehensive guide from Microsoft MVP Herb Tyson is your expert, one-stop resource for it all.Table of ContentsIntroduction xxxvii Part I: My Word, and Welcome to It 1 Chapter 1: Brave New Word 3 Chapter 2: Quick Start 29 Chapter 3: Where in the Word Is . . .? 61 Chapter 4: Making Word Work for You 73 Chapter 5: The X Files: Understanding and Using Word’s New File Format 91 Chapter 6: Make It Stop! Cures and Treatments for Word’s Top Annoyances 101 Part II: Word on the Street 115 Chapter 7: Formatting 101: Font/Character Formatting 117 Chapter 8: Paragraph Formatting 135 Chapter 9: In Style! 155 Chapter 10: The Clipboard 173 Chapter 11: Find, Replace, and Go To 185 Part III: Writing Tools 217 Chapter 12: Language Tools 219 Chapter 13: Building Blocks and Quick Parts 241 Chapter 14: AutoCorrect 255 Chapter 15: AutoFormat 265 Chapter 16: Action Options (What Happened to Smart Tags?) 279 Part IV: More than Mere Words 283 Chapter 17: Tables 285 Chapter 18: Pictures and SmartArt 315 Chapter 19: Headers and Footers 339 Chapter 20: Symbols and Equations 351 Chapter 21: Field Guide 367 Chapter 22:WordArt 395 Chapter 23: Charts 409 Chapter 24: Inserting Objects and Files 425 Part V: Document Design 437 Chapter 25: Page Setup and Sections 439 Chapter 26: Textboxes and Other Shapes 455 Chapter 27: Columns 467 Chapter 28: On Background 477 Chapter 29: Publishing as PDF and XPS 489 Chapter 30: Blogging and Publishing as HTML 497 Chapter 31: Templates and Themes 509 Part VI: With All Due Reference 533 Chapter 32: Bookmarks 535 Chapter 33: Tables of Contents 545 Chapter 34: Master Documents 559 Chapter 35: Footnotes and Endnotes 573 Chapter 36: Citations and Bibliography 581 Chapter 37: Captions and Tables of Captioned Items 595 Chapter 38: Indexing 603 Chapter 39: Tables of Authorities 613 Chapter 40: Hyperlinks and Cross-References 621 Part VII: Getting Out the Word 637 Chapter 41: Data Sources 639 Chapter 42: Envelopes and Labels 649 Chapter 43: Data Documents and Mail Merge 661 Chapter 44: Forms 687 Part VIII: Power and Customization 713 Chapter 45: Keyboard Customization 715 Chapter 46: TheQuick Access Toolbar 727 Chapter 47: The Ribbon 737 Chapter 48: Options and Settings 749 Chapter 49: Macros: Recording, Editing, and Using 795 Part IX: Collaboration—Getting Along with Others 817 Chapter 50: Security, Tracking, and Comments 819 Chapter 51: Comparing and Combining Collaborative Documents 845 Chapter 52: SharePoint and SkyDrive 853 Chapter 53: SharePoint Workspace 865 Chapter 54: Integration with Other Office Applications 881 Index 895

Read more less

Book SynopsisMusic is much more than listening to audio encoded in some unreadable binary format. It is, instead, an adventure similar to reading a book and entering its world, complete with a story, plot, sound, images, texts, and plenty of related data with, for instance, historical, scientific, literary, and musicological contents. Navigation of this world, such as that of an opera, a jazz suite and jam session, a symphony, a piece from non-Western culture, is possible thanks to the specifications of new standard IEEE 1599, IEEE Recommended Practice for Defining a Commonly Acceptable Musical Application Using XML, which uses symbols in language XML and music layers to express all its multimedia characteristics. Because of its encompassing features, this standard allows the use of existing audio and video standards, as well as recuperation of material in some old format, the events of which are managed by a single XML file, which is human and machine readable - musical symbols have been reTable of ContentsPreface xi A Brief Introduction to the IEEE 1599 Standard xv Denis L. Baggi and Goffredo M. Haus List of Contributors xvii 1 THE IEEE 1599 STANDARD 1 Denis L. Baggi and Goffredo M. Haus 1.1 Introduction 1 Important Features of IEEE 1599 2 Examples of Applications of IEEE 1599 to Increase Music Enjoyment 3 Example I: A Score with Different Versions: “King Porter Stomp,” by Jelly Roll Morton 6 Example II: A Jazz Piece with No Score: “Crazy Rhythm” 6 Example III: An Opera Using the Composer’s Manuscript: Tosca, by Giacomo Puccini 9 Example IV: “Peaches en Regalia,” by Frank Zappa 9 Example V: “Il mio ben quando verrà,” by Giovanni Paisiello 12 Example VI: Brandenburg Concerto No. 3, by J.S. Bach 14 Example VII: Blues, a Didactical Tool to Learn Jazz Improvisation 14 Example VIII: “La caccia,” from Antonio Vivaldi’s Four Seasons (“Autumn”) 16 Example IX: A Musicological Fantasy: Tauhid, a Piece of Free Jazz 17 Conclusions 19 Acknowledgments 19 References 19 2 ENCODING MUSIC INFORMATION 21 Luca A. Ludovico 2.1 Introduction 21 2.2 Heterogeneous Descriptions of Music 22 2.3 Available File Formats 23 2.4 Key Features of IEEE 1599 24 2.5 Multi-Layer Structure 25 2.6 The Logic Layer 27 2.7 The Spine 29 2.7.1 Inter-layer and Intra-layer Synchronization 31 2.7.2 Virtual Timing and Position of Events 32 2.7.3 How to Build the Spine 33 References 36 3 STRUCTURING MUSIC INFORMATION 37 Adriano Baratè and Goffredo M. Haus 3.1 Introduction 37 3.2 Music Objects and Music Algorithms 38 3.2.1 Music Objects 38 3.2.2 Music Algorithms 38 3.2.3 Music Objects and Music Algorithms in IEEE 1599 39 3.3 Petri Nets 39 3.3.1 Petri Nets Extension: Hierarchy 40 3.3.2 Petri Nets Extension: Probabilistic Arc Weights 43 3.4 Music Petri Nets 44 3.4.1 Music Petri Nets in IEEE 1599 47 3.5 Music Analysis Using Music Petri Nets 47 3.6 Real-Time Interaction with Music Petri Nets 50 3.7 Conclusions 55 References 55 4 MODELING AND SEARCHING MUSIC COLLECTIONS 57 Alberto Pinto 4.1 Introduction 57 4.2 Describing Music Content 58 4.2.1 Music Search Engines 59 4.3 Music Description in IEEE 1599 60 4.3.1 Chord Grid Objects 64 4.3.2 Petri Net Objects 65 4.3.3 Analysis Objects 65 4.3.4 MIR Objects 66 4.4 The Theoretical Framework 66 4.4.1 The Model Perspective 66 4.4.2 Categories 67 4.5 Music Modeling and Retrieval in IEEE 1599 67 4.5.1 MIR Model 68 4.5.2 MIR Object 69 4.5.3 MIR Subobject 70 4.5.4 MIR Morphisms 70 4.5.5 MIR Features 70 4.5.6 GraphXML Encoding 71 4.6 Case Study: Graph-Categorial Modeling 72 4.6.1 Content Description 72 4.6.2 Content Retrieval 72 4.6.3 MIR Model 73 4.6.4 MIR Object and Subobject 74 4.6.5 MIR Morphism 75 References 75 5 FEATURE EXTRACTION AND SYNCHRONIZATION AMONG LAYERS 77 Antonello D’Aguanno, Goffredo M. Haus, and Davide A. Mauro 5.1 Introduction 77 5.2 Encoding Synchronization Information 78 5.2.1 Extraction of Synchronization Data 82 5.2.2 Case Study 84 5.3 Overview of Synchronization Algorithms 84 5.4 VarSi: An Automatic Score-to-Audio Synchronization Algorithm Based on the IEEE 1599 Format 88 5.4.1 Score Analysis 89 5.4.2 Audio Analysis 90 5.4.3 Decisional Phase 91 References 94 6 IEEE 1599 AND SOUND SYNTHESIS 97 Luca A. Ludovico 6.1 Introduction 97 6.2 From Music Symbols to Sound Synthesis 98 6.2.1 Translating Symbols into a Performance Language 99 6.2.2 Interpretative Models 105 6.2.3 Audio Rendering and Synchronization 106 6.3 From Sound Synthesis to Music Symbols 108 6.4 An Example of Encoding 110 6.5 Conclusions 113 References 114 7 IEEE 1599 APPLICATIONS FOR ENTERTAINMENT AND EDUCATION 115 Adriano Baratè and Luca A. Ludovico 7.1 Introduction 115 7.2 IEEE 1599 for Entertainment 116 7.3 IEEE 1599 for Music Education 117 7.4 IEEE 1599-Based Music Viewers 118 7.5 Case Studies 120 7.5.1 Navigating and Interacting with Music Notation and Audio 120 7.5.2 Musicological Analysis 121 7.5.3 Instrumental and Ear Training 126 7.5.4 IEEE 1599 Beyond Music 132 References 132 8 PAST PROJECTS USING SYMBOLS FOR MUSIC 133 Denis L. Baggi 8.1 Brief History 133 8.2 Bass Computerized Harmonization (BA-C-H) 134 8.3 Harmony Machine 135 8.4 NeurSwing, an Automatic Jazz Rhythm Section Built with Neural Nets 141 8.5 The Paul Glass System 145 8.6 A Program That Finds Notes and Type of a Chord and Plays It 147 8.7 Summary of Projects 149 8.8 Conclusions 150 References 150 Appendix A. Brief History of IEEE 1599 Standard, and Acknowledgments 151 Appendix B. IEEE Document-Type Defi nitions (DTDs) 153 Appendix C. IEEE 1599 Demonstration Videos 177 Index 179

Read more less

Book SynopsisTake your hobby to the next level and turn your blog into real income Anyone who blogs knows that it is a fun, creative way for sharing thoughts and opinions.Table of ContentsForeword xix Introduction 1 Part I: Getting Started with the Business of Blogging 7 Chapter 1: Examining Blogging at the Professional Level 9 Chapter 2: Finding Your Niche in the Blogosphere 33 Chapter 3: Protecting Your Blog with Appropriate Business Policies and Practices 55 Part II: Making Money with Your Blog 79 Chapter 4: Monetizing Your Blog Strategy 81 Chapter 5: Selling Products or Services on Your Blog 97 Chapter 6: Making Money from Advertising 117 Chapter 7: Getting Paid for Your Words 151 Part III: Building Your Blog, Step by Step 169 Chapter 8: Choosing Your Blog Name, Platform, and Web Hosting 171 Chapter 9: Designing Your Blog 189 Chapter 10: Developing Your Blog Content 211 Part IV: Maximizing Your Blog’s Success 237 Chapter 11: Getting the Word Out about Your Blog 239 Chapter 12: Responding When Companies Come Calling 267 Chapter 13: Monitoring and Measuring: Why They Matter 283 Chapter 14: Keeping Your Blog Fresh 303 Part V: The Part of Tens 317 Chapter 15: Ten Common Mistakes and How to Avoid Them 319 Chapter 16: Ten (Or More) Blogs You Can Learn from Simply by Reading 327 Chapter 17: Ten Tips for Jump-Starting Your Creativity 335 Index 343

Read more less

Book SynopsisA fully updated guide to making your landing pages profitable Effective Internet marketing requires that you test and optimize your landing pages to maximize exposure and conversion rate. This second edition of a bestselling guide to landing page optimization includes case studies with before-and-after results as well as new information on web site usability. It covers how to prepare all types of content for testing, how to interpret results, recognize the seven common design mistakes, and much more. Included is a gift card for Google AdWords. Features fully updated information and case studies on landing page optimization Shows how to use Google''s Website Optimizer tool, what to test and how to prepare your site for testing, the pros and cons of different test strategies, how to interpret results, and common site design mistakes Provides a step-by-step implementation plan and advice on getting support and resources Landing PagTable of ContentsIntroduction xv Part I Understanding Landing Page Optimization 1 Chapter 1 Setting the Stage 3 What Is a Landing Page? 4 A Few Precious Moments Online 4 Your Baby Is Ugly 6 Your Website Visitors: The Real Landing Page Experts 6 Understanding the Bigger Online Marketing Picture 8 The Myth of Perfect Conversion 17 Chapter 2 Understanding Your Landing Pages 19 Landing Page Types 20 What Parts of Your Site Are Mission Critical? 22 What Is Your Business Model? 28 The Types of Conversion Actions 30 Chapter 3 The Matrix—Moving People to Act 35 The Matrix Overview 36 Roles 36 Tasks 38 The Decision-Making Process 39 Awareness 40 Interest 43 Desire 45 Action 53 Part II Finding Opportunities for Site Improvement 63 Chapter 4 Common Problems—The Seven Deadly Sins of Landing Page Design 65 A Sober Look 66 Unclear Call-to-Action 66 Too Many Choices 73 Visual Distractions 76 Not Keeping Your Promises 83 Too Much Text 86 Asking for Too Much Information 87 Lack of Trust and Credibility 93 Real-World Case Study: CREDO Mobile 106 Chapter 5 Conversion Ninja Toolbox—Diagnosing Site Problems 111 You Are Not as Good as You Would Like to Believe 112 Focus on the Negative 113 Web Analytics Tools 114 Visual Analysis Tools 125 Feedback and Survey Tools 131 Website Performance Tools 133 Competitive Analysis Tools 135 Usability Testing Tools 136 E-mail Enhancement Tools 139 Chapter 6 Misunderstanding Your Visitors—Looking for Psychological Mismatches 141 Empathy: The Key Ingredient 142 Researching the Whole Story 143 Demographics and Segmentation 144 Welcome to Your Brain 148 Cognitive Styles 152 Persuasion Frameworks 157 Cultural Differences 165 Part III Fixing Your Site Problems 169 Chapter 7 Conversion Improvement Basics 171 Web Usability Overview 172 Visual Presentation 173 Writing for the Web 192 Usability Checks 197 Chapter 8 Best Practices for Common Situations 201 Homepages 202 Information Architecture and Navigation 205 E-commerce Catalogs 211 Registration and Multiple-Step Flows 234 Direct Response Pages 243 Mobile Websites 246 Chapter 9 The Strategy of What to Test 251 How to Think About Test Elements 252 Selecting Elements to Test 261 Testing Multiple-Page Flows 264 Timeless Testing Themes 267 Price Testing 273 Part IV The Mechanics of Testing 279 Chapter 10 Common Testing Questions 281 Lies, Damn Lies, and Statistics 282 Crash Course in Probability and Statistics 286 Have I Found Something Better? 293 How Sure Do I Need to Be? 295 How Much Better Is It? 298 How Long Should My Test Run? 300 Chapter 11 Preparing for Testing 305 Overview of Content Management and Testing 306 Content Management Configurations 308 Common Testing Issues 313 Chapter 12 Testing Methods 325 Introduction to Testing Terminology 326 Overview of Testing Methods 331 A-B Split Testing 332 Multivariate Testing 335 Variable Interactions 350 Part V Organization and Planning 359 Chapter 13 Assembling Your Team and Getting Buy-in 361 The Usual Suspects 362 Little Company, Big Company 372 The Company Politics of Tuning 375 Strategies for Getting Started 378 Insource or Outsource? 380 Chapter 14 Developing Your Action Plan 387 Before You Begin 388 Understand Your Business Objectives 389 What Is the Lifetime Value of the Conversion Action? 390 Assemble Your Team 401 Determine Your Landing Pages and Traffic Sources 403 Decide What Constitutes Success 405 Uncover Problems and Decide What to Test 407 Select an Appropriate Tuning Method 410 Implement and Conduct QA 412 Collect the Data 416 Analyze the Results and Verify Improvement 418 Chapter 15 Avoiding Real-World Pitfalls 421 Ignoring Your Baseline 422 Collecting Insufficient Data 422 Not Accounting for Seasonality 424 Assuming That Testing Has No Costs 424 Not Factoring In Delayed Conversions 426 Becoming Paralyzed by Search Engine Considerations 433 Failing to Act 436 Appendix Landing Page Testing Tools 437 Enterprise Tools 438 Free or Inexpensive Tools 440 Glossary 443 Index 451

Read more less

Book SynopsisA computer forensics how-to for fighting malicious code and analyzing incidents With our ever-increasing reliance on computers comes an ever-growing risk of malware. Security professionals will find plenty of solutions in this book to the problems posed by viruses, Trojan horses, worms, spyware, rootkits, adware, and other invasive software. Written by well-known malware experts, this guide reveals solutions to numerous problems and includes a DVD of custom programs and tools that illustrate the concepts, enhancing your skills. Security professionals face a constant battle against malicious software; this practical manual will improve your analytical capabilities and provide dozens of valuable and innovative solutions Covers classifying malware, packing and unpacking, dynamic malware analysis, decoding and decrypting, rootkit detection, memory forensics, open source malware research, and much more Includes generous amounts of source code in C,

Read more less

Book SynopsisGet up to speed on the latest features and enhancements to InDesign CS5 As the industry standard in professional layout and design, InDesign delivers powerful publishing solutions for magazine, newspaper, and other publishing fields.Table of ContentsIntroduction 1 Part I: Before You Begin 7 Chapter 1: Understanding InDesign Ingredients 9 Chapter 2: Making InDesign Work Your Way 33 Part II: Document Essentials 47 Chapter 3: Opening and Saving Your Work 49 Chapter 4: Discovering How Pages and Layers Work 59 Chapter 5: The Joys of Reuse 85 Chapter 6: Working with Color 99 Part III: Object Essentials 117 Chapter 7: Adding Essential Elements 119 Chapter 8: Manipulating Objects 137 Chapter 9: Organizing Objects 157 Chapter 10: Aligning and Arranging Objects 173 Part IV: Text Essentials 197 Chapter 11: Putting Words on the Page 199 Chapter 12: The Ins and Outs of Text Editing 219 Chapter 13: The Styles of Text 237 Chapter 14: Fine-Tuning Paragraph Details 247 Chapter 15: Finessing Character Details 263 Part V: Graphics Essentials 277 Chapter 16: Importing Graphics 279 Chapter 17: Fitting Graphics and Setting Paths 291 Part VI: Getting Down to Business 301 Chapter 18: Working with Tabs and Tables 303 Chapter 19: Working with Footnotes, Indexes, and TOCs 315 Chapter 20: Working with Automatic Text 327 Chapter 21: Publishing Books 337 Part VII: Printing, Presentation, and Web Essentials 345 Chapter 22: Printing and PDF’ing Your Work 347 Chapter 23: Web Project Basics 371 Chapter 24: Presentation Project Basics 383 Part VIII: The Part of Tens 405 Chapter 25: Top Ten New Features in InDesign CS5 407 Chapter 26: Top Ten Resources for InDesign Users 411 Index 415

Read more less

Book SynopsisNEW LANGUAGE VISUALIZES PROGRAM ABSTRACTIONS CLEARLY AND PRECISELY Popular software modelling notations visualize implementation minutiae but fail to scale, to capture design abstractions, and to deliver effective tool support. Tailored to overcome these limitations, Codecharts can elegantly model roadmaps and blueprints for Java, C++, and C# programs of any size clearly, precisely, and at any level of abstraction. More practically, significant productivity gains for programmers using tools supporting Codecharts have been demonstrated in controlled experiments. Hundreds of figures and examples in this book illustrate how Codecharts are used to: Visualize the building-blocks of object-oriented design Create bird''s-eye roadmaps of large programs with minimal symbols and no clutter Model blueprints of patterns, frameworks, and other design decisions Be exactly sure what diagrams claim about programs Table of ContentsPreface. Acknowledgements. Guide to the Reader. Codecharts. Propositions. Prologue. 1. Motivation. 2. Design Description Languages. 2.1 Theory Versus Practice. 2.2 Decidability. 2.3 Abstraction. 2.4 Elegance. 3. An Overview of Codecharts. 3.1 Object-Orientation. 3.2 Visualization. 3.3 Rigour. 3.4 Automated Verifiability. 3.5 Scalability. 3.6 Genericity. 3.7 Minimality. 3.8 Information Neglect. 4. UML Versus Codecharts. 5. Historical Notes. PART I: Practice. 6. Modelling Small Programs. 6.1 Modelling Individual Classes. 6.2 Modelling Individual Methods. 6.3 Modelling Properties. 6.4 Modelling Implementation Minutia. 6.5 Modelling Simple Relations. 6.6 Modelling Indirect Relations. 6.7 Subtyping. 7. Modelling Large Programs. 7.1 Modelling Sets of Classes. 7.2 Modelling Total Relations Between Sets. 7.3 Modelling Sets of Methods (Clans). 7.4 Modelling Isomorphic Relations. 7.5 Modelling Sets of Methods (Tribes). 7.6 Modelling Class Hierarchies. 7.7 Modelling Methods in Hierarchies. 7.8 Modelling Properties of Sets. 7.9 Case Study: Total Versus. Isomorphic. 7.10 Case Study: JDOM. 7.11 Case Study: Java 3D. 8. Modelling Industry-Scale Programs. 8.1 Modelling Sets of Hierarchies. 8.2 Modelling Sets of Sets of Methods (Clans). 8.3 Modelling Sets of Sets of Methods (Tribes). 8.4 Modelling Total Relations Revisited. 8.5 Modelling Isomorphic Relations Revisited. 9. Modelling Design Motifs. 10. Modelling Application Frameworks. 10.1 Case Study: Enterprise JavaBeans. 10.2 Case Study: JUnit. 11. Modelling Design Patterns. 11.1 Case Study: The Composite Pattern. 11.2 Case Study: The Iterator Pattern. 11.3 Case Study: The Factory Method Pattern. 11.4 Case Study: The Abstract Factory Pattern. 11.5 Concluding Remarks. 12. Modelling Early Design Revisited. 13. Advanced Modelling Techniques. 13.1 Ad Hoc Symbols. 13.2 Modelling Information Hiding. PART II: Theory. 14. Abstract Semantics. 14.1 Finite Structures. 14.2 Abstract Semantics Functions. 14.3 Design Models. 14.4 Program Modelling Revisited. 15. Verification. 15.1 Verifying Closed Specifications. 15.2 Verifying Open Specifications. 15.3 Verifying Pattern Implementations. 15.4 Tool Support for Automated Verification. 16. Schemas. 17. LePUS3 in Classical Logic. 17.1 LePUS3 and Class-Z as First-Order Languages. 17.2 Specifications in the Predicate Logic. 17.3 The Axioms of Class-Based Programs. 18. Reasoning about Charts. Appendix I: The Gang of Four Companion. Appendix II: Formal Definitions. Appendix III: UML Quick Reference. References. Index.

Read more less

Book SynopsisThe third book in the trilogy that explores the popular missional movement From Reggie McNeal, the bestselling author of The Present Future and Missional Renaissance, comes the third book in the series that helps to define and illuminate the popular missional movement. This newest book in the trilogy examines a natural outgrowth of the move toward a missional orientation: the deconstruction of congregations into very small Christian communities. For all those thousands of churches and leaders who have followed Reggie McNeal''s bold lead, this book details the rise of a new life form in churches. Discusses how to move a church from an internal to an external ministry focus Reggie McNeal is a recognized leader in the missional movement Outlines an alternative to the program church model that is focused on the projects and passions of the congregants This book draws on McNeal''s twenty years of leadership roles in localTable of ContentsAbout the Jossey-Bass Leadership Network Series xi Foreword by Hugh Halter xiii Acknowledgments xvii Introduction xix 1 ‘‘Let There Be . . . Missional Communities’’ 1 2 The Missional Church Conversation 15 3 Missional Communities—European Style 39 4 Soma Communities: Missional Communities as Organizing Architecture 65 5 Campus Renewal UT: Missional Communities as Campus Evangelism Strategy 85 6 Future Travelers: Missional Communities as Megachurch Strategy 103 7 Mission Houston: Missional Communities for Spiritual Formation and Community Transformation 125 8 Looking Ahead 145 About the Author 155 Index 157

Read more less

Book SynopsisA complete guide to Pentaho Kettle, the Pentaho Data lntegration toolset for ETL This practical book is a complete guide to installing, configuring, and managing Pentaho Kettle. If you're a database administrator or developer, you'll first get up to speed on Kettle basics and how to apply Kettle to create ETL solutionsbefore progressing to specialized concepts such as clustering, extensibility, and data vault models. Learn how to design and build every phase of an ETL solution. Shows developers and database administrators how to use the open-source Pentaho Kettle for enterprise-level ETL processes (Extracting, Transforming, and Loading data) Assumes no prior knowledge of Kettle or ETL, and brings beginners thoroughly up to speed at their own pace Explains how to get Kettle solutions up and running, then follows the 34 ETL subsystems model, as created by the Kimball Group, to explore the entire ETL lifecycle, including all aspects of data warehoTable of ContentsIntroduction xxxi Part I Getting Started 1 Chapter 1 ETL Primer 3 OLTP versus Data Warehousing 3 What Is ETL? 5 The Evolution of ETL Solutions 5 ETL Building Blocks 7 ETL, ELT, and EII 8 ELT 9 EII: Virtual Data Integration 10 Data Integration Challenges 11 Methodology: Agile BI 12 ETL Design 14 Data Acquisition 14 Beware of Spreadsheets 15 Design for Failure 15 Change Data Capture 16 Data Quality 16 Data Profiling 16 Data Validation 17 ETL Tool Requirements 17 Connectivity 17 Platform Independence 18 Scalability 18 Design Flexibility 19 Reuse 19 Extensibility 19 Data Transformations 20 Testing and Debugging 21 Lineage and Impact Analysis 21 Logging and Auditing 22 Summary 22 Chapter 2 Kettle Concepts 23 Design Principles 23 The Building Blocks of Kettle Design 25 Transformations 25 Steps 26 Transformation Hops 26 Parallelism 27 Rows of Data 27 Data Conversion 29 Jobs 30 Job Entries 31 Job Hops 31 Multiple Paths and Backtracking 32 Parallel Execution 33 Job Entry Results 34 Transformation or Job Metadata 36 Database Connections 37 Special Options 38 The Power of the Relational Database 39 Connections and Transactions 39 Database Clustering 40 Tools and Utilities 41 Repositories 41 Virtual File Systems 42 Parameters and Variables 43 Defining Variables 43 Named Parameters 44 Using Variables 44 Visual Programming 45 Getting Started 46 Creating New Steps 47 Putting It All Together 49 Summary 51 Chapter 3 Installation and Configuration 53 Kettle Software Overview 53 Integrated Development Environment: Spoon 55 Command-Line Launchers: Kitchen and Pan 57 Job Server: Carte 57 Encr.bat and encr.sh 58 Installation 58 Java Environment 58 Installing Java Manually 58 Using Your Linux Package Management System 59 Installing Kettle 59 Versions and Releases 59 Archive Names and Formats 60 Downloading and Uncompressing 60 Running Kettle Programs 61 Creating a Shortcut Icon or Launcher for Spoon 62 Configuration 63 Configuration Files and the .kettle Directory 63 The Kettle Shell Scripts 69 General Structure of the Startup Scripts 70 Adding an Entry to the Classpath 70 Changing the Maximum Heap Size 71 Managing JDBC Drivers 72 Summary 72 Chapter 4 An Example ETL Solution—Sakila 73 Sakila 73 The Sakila Sample Database 74 DVD Rental Business Process 74 Sakila Database Schema Diagram 75 Sakila Database Subject Areas 75 General Design Considerations 77 Installing the Sakila Sample Database 77 The Rental Star Schema 78 Rental Star Schema Diagram 78 Rental Fact Table 79 Dimension Tables 79 Keys and Change Data Capture 80 Installing the Rental Star Schema 81 Prerequisites and Some Basic Spoon Skills 81 Setting Up the ETL Solution 82 Creating Database Accounts 82 Working with Spoon 82 Opening Transformation and Job Files 82 Opening the Step’s Configuration Dialog 83 Examining Streams 83 Running Jobs and Transformations 83 The Sample ETL Solution 84 Static, Generated Dimensions 84 Loading the dim_date Dimension Table 84 Loading the dim_time Dimension Table 86 Recurring Load 87 The load_rentals Job 88 The load_dim_staff Transformation 91 Database Connections 91 The load_dim_customer Transformation 95 The load_dim_store Transformation 98 The fetch_address Subtransformation 99 The load_dim_actor Transformation 101 The load_dim_film Transformation 102 The load_fact_rental Transformation 107 Summary 109 Part II ETL 111 Chapter 5 ETL Subsystems 113 Introduction to the 34 Subsystems 114 Extraction 114 Subsystems 1–3: Data Profiling, Change Data Capture, and Extraction 115 Cleaning and Conforming Data 116 Subsystem 4: Data Cleaning and Quality Screen Handler System 116 Subsystem 5: Error Event Handler 117 Subsystem 6: Audit Dimension Assembler 117 Subsystem 7: Deduplication System 117 Subsystem 8: Data Conformer 118 Data Delivery 118 Subsystem 9: Slowly Changing Dimension Processor 118 Subsystem 10: Surrogate Key Creation System 119 Subsystem 11: Hierarchy Dimension Builder 119 Subsystem 12: Special Dimension Builder 120 Subsystem 13: Fact Table Loader 121 Subsystem 14: Surrogate Key Pipeline 121 Subsystem 15: Multi-Valued Dimension Bridge Table Builder 121 Subsystem 16: Late-Arriving Data Handler 122 Subsystem 17: Dimension Manager System 122 Subsystem 18: Fact Table Provider System 122 Subsystem 19: Aggregate Builder 123 Subsystem 20: Multidimensional (OLAP) Cube Builder 123 Subsystem 21: Data Integration Manager 123 Managing the ETL Environment 123 Summary 126 Chapter 6 Data Extraction 127 Kettle Data Extraction Overview 128 File-Based Extraction 128 Working with Text Files 128 Working with XML files 133 Special File Types 134 Database-Based Extraction 134 Web-Based Extraction 137 Text-Based Web Extraction 137 HTTP Client 137 Using SOAP 138 Stream-Based and Real-Time Extraction 138 Working with ERP and CRM Systems 138 ERP Challenges 139 Kettle ERP Plugins 140 Working with SAP Data 140 ERP and CDC Issues 146 Data Profiling 146 Using eobjects.org DataCleaner 147 Adding Profile Tasks 149 Adding Database Connections 149 Doing an Initial Profile 151 Working with Regular Expressions 151 Profiling and Exploring Results 152 Validating and Comparing Data 153 Using a Dictionary for Column Dependency Checks 153 Alternative Solutions 154 Text Profiling with Kettle 154 CDC: Change Data Capture 154 Source Data–Based CDC 155 Trigger-Based CDC 157 Snapshot-Based CDC 158 Log-Based CDC 162 Which CDC Alternative Should You Choose? 163 Delivering Data 164 Summary 164 Chapter 7 Cleansing and Conforming 167 Data Cleansing 168 Data-Cleansing Steps 169 Using Reference Tables 172 Conforming Data Using Lookup Tables 172 Conforming Data Using Reference Tables 175 Data Validation 179 Applying Validation Rules 180 Validating Dependency Constraints 183 Error Handling 183 Handling Process Errors 184 Transformation Errors 186 Handling Data (Validation) Errors 187 Auditing Data and Process Quality 191 Deduplicating Data 192 Handling Exact Duplicates 193 The Problem of Non-Exact Duplicates 194 Building Deduplication Transforms 195 Step 1: Fuzzy Match 197 Step 2: Select Suspects 198 Step 3: Lookup Validation Value 198 Step 4: Filter Duplicates 199 Scripting 200 Formula 201 JavaScript 202 User-Defined Java Expressions 202 Regular Expressions 203 Summary 205 Chapter 8 Handling Dimension Tables 207 Managing Keys 208 Managing Business Keys 209 Keys in the Source System 209 Keys in the Data Warehouse 209 Business Keys 209 Storing Business Keys 210 Looking Up Keys with Kettle 210 Generating Surrogate Keys 210 The “Add sequence” Step 211 Working with auto_increment or IDENTITY Columns 217 Keys for Slowly Changing Dimensions 217 Loading Dimension Tables 218 Snowflaked Dimension Tables 218 Top-Down Level-Wise Loading 219 Sakila Snowflake Example 219 Sample Transformation 221 Database Lookup Configuration 222 Sample Job 225 Star Schema Dimension Tables 226 Denormalization 226 Denormalizing to 1NF with the “Database lookup” Step 226 Change Data Capture 227 Slowly Changing Dimensions 228 Types of Slowly Changing Dimensions 228 Type 1 Slowly Changing Dimensions 229 The Insert / Update Step 229 Type 2 Slowly Changing Dimensions 232 The “Dimension lookup / update” Step 232 Other Types of Slowly Changing Dimensions 237 Type 3 Slowly Changing Dimensions 237 Hybrid Slowly Changing Dimensions 238 More Dimensions 239 Generated Dimensions 239 Date and Time Dimensions 239 Generated Mini-Dimensions 239 Junk Dimensions 241 Recursive Hierarchies 242 Summary 243 Chapter 9 Loading Fact Tables 245 Loading in Bulk 246 STDIN and FIFO 247 Kettle Bulk Loaders 248 MySQL Bulk Loading 249 LucidDB Bulk Loader 249 Oracle Bulk Loader 249 PostgreSQL Bulk Loader 250 Table Output Step 250 General Bulk Load Considerations 250 Dimension Lookups 251 Maintaining Referential Integrity 251 The Surrogate Key Pipeline 252 Using In-Memory Lookups 253 Stream Lookups 253 Late-Arriving Data 255 Late-Arriving Facts 256 Late-Arriving Dimensions 256 Fact Table Handling 260 Periodic and Accumulating Snapshots 260 Introducing State-Oriented Fact Tables 261 Loading Periodic Snapshots 263 Loading Accumulating Snapshots 264 Loading State-Oriented Fact Tables 265 Loading Aggregate Tables 266 Summary 267 Chapter 10 Working with OLAP Data 269 OLAP Benefits and Challenges 270 OLAP Storage Types 272 Positioning OLAP 272 Kettle OLAP Options 273 Working with Mondrian 274 Working with XML/A Servers 277 Working with Palo 282 Setting Up the Palo Connection 283 Palo Architecture 284 Reading Palo Data 285 Writing Palo Data 289 Summary 291 Part III Management and Deployment 293 Chapter 11 ETL Development Lifecycle 295 Solution Design 295 Best and Bad Practices 296 Data Mapping 297 Naming and Commentary Conventions 298 Common Pitfalls 299 ETL Flow Design 300 Reusability and Maintainability 300 Agile Development 301 Testing and Debugging 306 Test Activities 307 ETL Testing 308 Test Data Requirements 308 Testing for Completeness 309 Testing Data Transformations 311 Test Automation and Continuous Integration 311 Upgrade Tests 312 Debugging 312 Documenting the Solution 315 Why Isn’t There Any Documentation? 316 Myth 1: My Software Is Self-Explanatory 316 Myth 2: Documentation Is Always Outdated 316 Myth 3: Who Reads Documentation Anyway? 317 Kettle Documentation Features 317 Generating Documentation 319 Summary 320 Chapter 12 Scheduling and Monitoring 321 Scheduling 321 Operating System–Level Scheduling 322 Executing Kettle Jobs and Transformations from the Command Line 322 UNIX-Based Systems: cron 326 Windows: The at utility and the Task Scheduler 327 Using Pentaho’s Built-in Scheduler 327 Creating an Action Sequence to Run Kettle Jobs and Transformations 328 Kettle Transformations in Action Sequences 329 Creating and Maintaining Schedules with the Administration Console 330 Attaching an Action Sequence to a Schedule 333 Monitoring 333 Logging 333 Inspecting the Log 333 Logging Levels 335 Writing Custom Messages to the Log 336 E‑mail Notifications 336 Configuring the Mail Job Entry 337 Summary 340 Chapter 13 Versioning and Migration 341 Version Control Systems 341 File-Based Version Control Systems 342 Organization 342 Leading File-Based VCSs 343 Content Management Systems 344 Kettle Metadata 344 Kettle XML Metadata 345 Transformation XML 345 Job XML 346 Global Replace 347 Kettle Repository Metadata 348 The Kettle Database Repository Type 348 The Kettle File Repository Type 349 The Kettle Enterprise Repository Type 350 Managing Repositories 350 Exporting and Importing Repositories 350 Upgrading Your Repository 351 Version Migration System 352 Managing XML Files 352 Managing Repositories 352 Parameterizing Your Solution 353 Summary 356 Chapter 14 Lineage and Auditing 357 Batch-Level Lineage Extraction 358 Lineage 359 Lineage Information 359 Impact Analysis Information 361 Logging and Operational Metadata 363 Logging Basics 363 Logging Architecture 364 Setting a Maximum Buffer Size 365 Setting a Maximum Log Line Age 365 Log Channels 366 Log Text Capturing in a Job 366 Logging Tables 367 Transformation Logging Tables 367 Job Logging Tables 373 Summary 374 Part IV Performance and Scalability 375 Chapter 15 Performance Tuning 377 Transformation Performance: Finding the Weakest Link 377 Finding Bottlenecks by Simplifying 379 Finding Bottlenecks by Measuring 380 Copying Rows of Data 382 Improving Transformation Performance 384 Improving Performance in Reading Text Files 384 Using Lazy Conversion for Reading Text Files 385 Single-File Parallel Reading 385 Multi-File Parallel Reading 386 Configuring the NIO Block Size 386 Changing Disks and Reading Text Files 386 Improving Performance in Writing Text Files 387 Using Lazy Conversion for Writing Text Files 387 Parallel Files Writing 387 Changing Disks and Writing Text Files 387 Improving Database Performance 388 Avoiding Dynamic SQL 388 Handling Roundtrips 388 Handling Relational Databases 390 Sorting Data 392 Sorting on the Database 393 Sorting in Parallel 393 Reducing CPU Usage 394 Optimizing the Use of JavaScript 394 Launching Multiple Copies of a Step 396 Selecting and Removing Values 397 Managing Thread Priorities 397 Adding Static Data to Rows of Data 397 Limiting the Number of Step Copies 398 Avoiding Excessive Logging 398 Improving Job Performance 399 Loops in Jobs 399 Database Connection Pools 400 Summary 401 Chapter 16 Parallelization, Clustering, and Partitioning 403 Multi-Threading 403 Row Distribution 404 Row Merging 405 Row Redistribution 406 Data Pipelining 407 Consequences of Multi-Threading 408 Database Connections 408 Order of Execution 409 Parallel Execution in a Job 411 Using Carte as a Slave Server 411 The Configuration File 411 Defining Slave Servers 412 Remote Execution 413 Monitoring Slave Servers 413 Carte Security 414 Services 414 Clustering Transformations 417 Defining a Cluster Schema 417 Designing Clustered Transformations 418 Execution and Monitoring 420 Metadata Transformations 421 Rules 422 Data Pipelining 425 Partitioning 425 Defining a Partitioning Schema 425 Objectives of Partitioning 427 Implementing Partitioning 428 Internal Variables 428 Database Partitions 429 Partitioning in a Clustered Transformation 430 Summary 430 Chapter 17 Dynamic Clustering in the Cloud 433 Dynamic Clustering 433 Setting Up a Dynamic Cluster 434 Using the Dynamic Cluster 436 Cloud Computing 437 EC2 438 Getting Started with EC2 438 Costs 438 Customizing an AMI 439 Packaging a New AMI 442 Terminating an AMI 442 Running a Master 442 Running the Slaves 443 Using the EC2 Cluster 444 Monitoring 445 The Lightweight Principle and Persistence Options 446 Summary 447 Chapter 18 Real-Time Data Integration 449 Introduction to Real-Time ETL 449 Real-Time Challenges 450 Requirements 451 Transformation Streaming 452 A Practical Example of Transformation Streaming 454 Debugging 457 Third-Party Software and Real-Time Integration 458 Java Message Service 459 Creating a JMS Connection and Session 459 Consuming Messages 460 Producing Messages 460 Closing Shop 460 Summary 461 Part V Advanced Topics 463 Chapter 19 Data Vault Management 465 Introduction to Data Vault Modeling 466 Do You Need a Data Vault? 466 Data Vault Building Blocks 467 Hubs 467 Links 468 Satellites 469 Data Vault Characteristics 471 Building a Data Vault 471 Transforming Sakila to the Data Vault Model 472 Sakila Hubs 472 Sakila Links 473 Sakila Satellites 474 Loading the Data Vault: A Sample ETL Solution 477 Installing the Sakila Data Vault 477 Setting Up the ETL Solution 477 Creating a Database Account 477 The Sample ETL Data Vault Solution 478 Sample Hub: hub_actor 478 Sample Link: link_customer_store 480 Sample Satellite: sat_actor 483 Loading the Data Vault Tables 485 Updating a Data Mart from a Data Vault 486 The Sample ETL Solution 486 The dim_actor Transformation 486 The dim_customer Transformation 488 The dim_film Transformation 492 The dim_film_actor_bridge Transformation 492 The fact_rental Transformation 493 Loading the Star Schema Tables 495 Summary 495 Chapter 20 Handling Complex Data Formats 497 Non-Relational and Non-Tabular Data Formats 498 Non-Relational Tabular Formats 498 Handling Multi-Valued Attributes 498 Using the Split Field to Rows Step 499 Handling Repeating Groups 500 Using the Row Normaliser Step 500 Semi- and Unstructured Data 501 Kettle Regular Expression Example 503 Configuring the Regex Evaluation Step 504 Verifying the Match 507 Key/Value Pairs 508 Kettle Key/Value Pairs Example 509 Text File Input 509 Regex Evaluation 510 Grouping Lines into Records 511 Denormaliser: Turning Rows into Columns 512 Summary 513 Chapter 21 Web Services 515 Web Pages and Web Services 515 Kettle Web Features 516 General HTTP Steps 516 Simple Object Access Protocol 517 Really Simple Syndication 517 Apache Virtual File System Integration 517 Data Formats 517 XML 518 Kettle Steps for Working with XML 518 Kettle Job Entries for XML 519 HTML 520 JavaScript Object Notation 520 Syntax 521 JSON, Kettle, and ETL/DI 522 XML Examples 523 Example XML Document 523 XML Document Structure 523 Mapping to the Sakila Sample Database 524 Extracting Data from XML 525 Overall Design: The import_xml_into_db Transformation 526 Using the XSD Validator Step 528 Using the “Get Data from XML” Step 530 Generating XML Documents 537 Overall Design: The export_xml_from_db Transformation 537 Generating XML with the Add XML Step 538 Using the XML Join Step 541 SOAP Examples 544 Using the “Web services lookup” Step 544 Configuring the “Web services lookup” Step 544 Accessing SOAP Services Directly 546 JSON Example 549 The Freebase Project 549 Freebase Versus Wikipedia 549 Freebase Web Services 550 The Freebase Read Service 550 The Metaweb Query Language 551 Extracting Freebase Data with Kettle 553 Generate Rows 554 Issuing a Freebase Read Request 555 Processing the Freebase Result Envelope 556 Filtering Out the Original Row 557 Storing to File 558 RSS 558 RSS Structure 558 Channel 558 Item 559 RSS Support in Kettle 560 RSS Input 561 RSS Output 562 Summary 567 Chapter 22 Kettle Integration 569 The Kettle API 569 The LGPL License 569 The Kettle Java API 570 Source Code 570 Building Kettle 571 Building javadoc 571 Libraries and the Class Path 571 Executing Existing Transformations and Jobs 571 Executing a Transformation 572 Executing a Job 573 Embedding Kettle 574 Pentaho Reporting 574 Putting Data into a Transformation 576 Dynamic Transformations 580 Dynamic Template 583 Dynamic Jobs 584 Executing Dynamic ETL in Kettle 586 Result 587 Replacing Metadata 588 Direct Changes with the API 589 Using a Shared Objects File 589 OEM Versions and Forks 590 Creating an OEM Version of PDI 590 Forking Kettle 591 Summary 592 Chapter 23 Extending Kettle 593 Plugin Architecture Overview 593 Plugin Types 594 Architecture 595 Prerequisites 596 Kettle API Documentation 596 Libraries 596 Integrated Development Environment 596 Eclipse Project Setup 597 Examples 598 Transformation Step Plugins 599 StepMetaInterface 599 Value Metadata 605 Row Metadata 606 StepDataInterface 607 StepDialogInterface 607 Eclipse SWT 607 Form Layout 607 Kettle UI Elements 609 Hello World Example Dialog 609 StepInterface 614 Reading Rows from Specific Steps 616 Writing Rows to Specific Steps 616 Writing Rows to Error Handling 617 Identifying a Step Copy 617 Result Feedback 618 Variable Substitution 618 Apache VFS 619 Step Plugin Deployment 619 The User-Defined Java Class Step 620 Passing Metadata 620 Accessing Input and Fields 620 Snippets 620 Example 620 Job Entry Plugins 621 JobEntryInterface 622 JobEntryDialogInterface 624 Partitioning Method Plugins 624 Partitioner 625 Repository Type Plugins 626 Database Type Plugins 627 Summary 628 Appendix A The Kettle Ecosystem 629 Kettle Development and Versions 629 The Pentaho Community Wiki 631 Using the Forums 631 Jira 632 ##pentaho 633 Appendix B Kettle Enterprise Edition Features 635 Appendix C Built-in Variables and Properties Reference 637 Internal Variables 637 Kettle Variables 640 Variables for Configuring VFS 641 Noteworthy JRE Variables 642 Index 643

Read more less

Book SynopsisAn accessible introduction to the theoretical and computational aspects of linear algebra using MapleTM Many topics in linear algebra can be computationally intensive, and software programs often serve as important tools for understanding challenging concepts and visualizing the geometric aspects of the subject.Trade Review Table of ContentsPreface. Conventions and Notations. 1 An Introduction To Maple. 1.1 The Commands . 1.2 Programming. 2 Linear Systems of Equations and Matrices. 2.1 Linear Systems of Equations. 2.2 Augmented Matrix of a Linear System and Row Operations. 2.3 Some Matrix Arithmetic. 3 Gauss-Jordan Elimination and Reduced Row Echelon Form. 3.1 Gauss-Jordan Elimination and rref. 3.2 Elementary Matrices. 3.3 Sensitivity of Solutions to Error in the Linear System. 4 Applications of Linear Systems and Matrices. 4.1 Applications of Linear Systems to Geometry. 4.2 Applications of Linear Systems to Curve Fitting. 4.3 Applications of Linear Systems to Economics. 4.4 Applications of Matrix Multiplication to Geometry. 4.5 An Application of Matrix Multiplication to Economics. 5 Determinants, Inverses and Cramer’s Rule. 5.1 Determinants and Inverses from the Adjoint Formula. 5.2 Determinants by Expanding Along Any Row or Column . 5.3 Determinants Found by Triangularizing Matrices. 5.4 LU Factorization. 5.5 Inverses from rref. 5.6 Cramer’s Rule. 6 Basic Linear Algebra Topics. 6.1 Vectors. 6.2 Dot Product. 6.3 Cross Product. 6.4 Vector Projection. 7 A Few Advanced Linear Algebra Topics. 7.1 Rotations in Space. 7.2 ‘Rolling’ a Circle Along a Curve. 7.3 The TNB Frame. 8 Independence, Basis and Dimension for Subspaces of Rn. 8.1 Subspaces of Rn. 8.2 Independent and Dependent Sets of Vectors in Rn. 8.3 Basis and Dimension for Subspaces of Rn. 8.4 Vector Projection onto a Subspace of Rn. 8.5 The Gram-Schmidt Orthonormalization Process. 9 Linear Maps from Rn to Rm. 9.1 Basics About Linear Maps. 9.2 The Kernel and Image Subspaces of a Linear Map. 9.3 Composites of Two Linear maps and Inverses. 9.4 Change of Bases for the Matrix Representation of a Linear Map. 10 The Geometry of Linear and Affine Maps. 10.1 The Effect of a Linear Map on Area and Arclength in Two Dimensions. 10.2 The Decomposition of Linear Maps into Rotations, Reflections and Rescalings in R2. 10.3 The Effect of Linear Maps on Volume, Area and Arclength in R3. 10.4 Rotations, Reflections and Rescalings in Three Dimensions. 10.5 Affine Maps. 11 Least Squares Fits and Pseudoinverses. 11.1 Pseudoinverse to a Non-Square Matrix and Almost Solving an Overdetermined Linear System. 11.2 Fits and Pseudoinverses. 11.3 Least Squares Fits and Pseudoinverses. 12 Eigenvalues and Eigenvectors. 12.1 What Are Eigenvalues and Eigenvectors, and Why Do We Need Them? 12.2 Summary of Definitions and Methods for Computing Eigenvalues and Eigenvectors as well as the Exponential of a Matrix. 12.3 Applications of the Diagonalizability of Square Matrices. 12.4 Solving a Square First Order Linear. System of Differential Equations . . . . . . . . . . . . . . . . . . 12.5 Basic Facts About Eigenvalues and Eigenvectors, and Diagonalizability. 12.6 The Geometry of the Ellipse Using Eigenvalues and Eigenvectors. 12.7 A Maple Eigen-Procedure. Bibliography. Indices. Keyword Index. Index of Maple Commands and Packages.

Read more less

Book SynopsisThis handbook focuses on biopolymers for both environmental and biomedical applications. It shows recent advances in technology in all areas from chemical synthesis or biosynthesis to end use applications. These areas have not been covered in a single book before and they include biopolymers for chemical and biotechnological modifications, material structures, characterization, processing, properties, and applications. After the introduction which summarizes the importance of biopolymer in the market, the book covers almost all the topics related to polysaccharides, biofibers, bioplastics, biocomposites, natural rubber, gums, bacterial and blood compatible polymers, and applications of biopolymers in various fields.Table of ContentsIntroductory Preface. About the Editors. Part I. Polysaccharides. 1. Hyaluronic Acid: A Natural Biopolymer (Juergen Schiller, Nicola Volpi, Eva Hrabárova, and Ladislav Soltes). 2. Polysaccharide Graft Copolymers Synthesis, Properties and Applications (B. S. Kaith, Hemant Mittal, Jaspreet Kaur Bhatia, and Susheel Kalia). 3. Natural Polysaccharides: From Membranes to Active Food Packaging (Keith J. Fahnestock, Marjorie S. Austero, and Caroline L. Schauer). 4. Starch as Source of Polymeric Materials (Antonio A. J. Carvalho). 5. Grafted Polysaccharides: Smart Materials of Future, Synthesis and Applications (Gautam Sen, Ashoke Sharon, and Sagar Pal). 6. Chitosan: The Marine based Biopolymer for Applications (Debasish Sahoo, and P. L. Nayak). Part II. Bioplastics and Biocomposites. 7. Biopolymers Based-on Carboxylic Acids Derived from Renewable Resources (Sushil Kumar, Nikhil Prakash, and Dipaloy Datta). 8. Characteristics and Applications of PLA (Sandra Domenek, Cecile Courgneau, and Violette Ducruet). 9. Biobased Composites & Applications (Smita Mohanty, and Sanjay K. Nayak). Part III. Miscellaneous Biopolymers. 10. Cassia Seed Gums: A Renewable Reservoir for Synthesizing High Performance Materials for Water Remediation (Vandana Singh, and Pramendra Kumar). 11. Bacterial Polymers: Resources, Synthesis and Applications (GVN Rathna, and Sutapa Gosh). 12. Gum Arabica: A Natural Biopolymer (A. Sarkar). 13. Gluten: A Natural Biopolymer (S. Georgiev, and Tereza Dekova). 14. Natural Rubber: Production, Properties, and Applications (Thomas Kurian, and N. M. Mathew). 15. Electronic Structures and Conduction Properties of Biopolymers (Mohsineen Wazir, Vinita Arora, and A. K. Bakhshi). Part IV. Biopolymers for Specific Applications. 16. Applications of Biopolymers in Agriculture with Special Reference to Role of Plant Derived Biopolymers in Crop Protection (S. Niranjan Raj, S. N. Lavanya, J, Sudisha, and H. Shekar Shetty). 17. Modified Cellulose Fibers as a Biosorbent for the Organic Pollutants (Sami Boufi, and Sabrine Alila). 18. Polymers and Biopolymers in Pharmaceutical Technology (István Erös). 19. Biopolymers Employed in Drug Delivery (Betina Giehl Zanetti Ramos). 20. Natural Polymeric Vectors in Gene Therapy (Patit P. Kundu, and Kishor Sarkar).

Read more less

Book SynopsisThe techniques pioneered by the Kimball Group have become the industry standard for data warehouse design, development, and management. In this new edition of the Microsoft Data Warehouse Toolkit, the authors share best practices for using these techniques in SQL Server 2008 R2 and Office 2010.Table of ContentsForeword xxvii Introduction xxix Part 1 Requirements, Realities, and Architecture 1 Chapter 1 Defining Business Requirements 3 The Most Important Determinant of Long-Term Success 5 Adventure Works Cycles Introduction 6 Uncovering Business Value 6 Obtaining Sponsorship 7 Defining Enterprise-Level Business Requirements 8 Prioritizing the Business Requirements 22 Revisiting the Project Planning 25 Gathering Project-Level Requirements 26 Summary 28 Chapter 2 Designing the Business Process Dimensional Model 29 Dimensional Modeling Concepts and Terminology 30 Facts 31 Dimensions 33 Bringing Facts and Dimensions Together 34 The Bus Matrix, Conformed Dimensions, and Drill Across 36 Additional Design Concepts and Techniques 38 Surrogate Keys 38 Slowly Changing Dimensions 39 Dates 42 Degenerate Dimensions 43 Snowflaking 43 Many-to-Many or Multivalued Dimensions 44 Hierarchies 47 Aggregate Dimensions 49 Junk Dimensions 51 The Three Fact Table Types 52 Aggregates 53 The Dimensional Modeling Process 54 Preparation 55 Data Profiling and Research 60 Building Dimensional Models 63 Developing the Detailed Dimensional Model 66 Testing and Refining the Model 68 Reviewing and Validating the Model 68 Case Study: The Adventure Works Cycles Orders Dimensional Model 69 The Orders Fact Table 69 The Dimensions 69 Identifying Dimension Attributes and Facts for the Orders Business Process 72 The Final Draft of the Initial Orders Model 74 Detailed Orders Dimensional Model Development 75 Final Dimensional Model 77 Summary 77 Chapter 3 The Toolset 79 The Microsoft DW/BI Toolset 80 Why Use the Microsoft Toolset? 82 Architecture of a Microsoft DW/BI System 83 Why Analysis Services? 84 Why a Relational Store? 86 ETL Is Not Optional 86 The Role of Master Data Services 88 Delivering BI Applications 88 Overview of the Microsoft Tools 89 Which Products Do You Need? 90 SQL Server Development and Management Tools 92 Summary 97 Chapter 4 System Setup 99 System Sizing Considerations 100 Calculating Data Volumes 101 Determining Usage Complexity 102 Estimating Simultaneous Users 104 Assessing System Availability Requirements 105 How Big Will It Be? 105 System Configuration Considerations 105 Memory 106 Monolithic or Distributed? 106 Storage System Considerations 110 Processors 113 Setting Up for High Availability 114 Software Installation and Configuration 115 Development Environment Software Requirements 116 Test and Production Software Requirements 120 Operating Systems 122 SQL Server Relational Database Setup 122 Analysis Services Setup 126 Integration Services Setup 129 Reporting Services Setup 130 Summary 131 Part 2 Building and Populating the Databases 133 Chapter 5 Creating the Relational Data Warehouse 135 Getting Started 136 Complete the Physical Design 137 Surrogate Keys 138 String Columns 138 To Null, or Not to Null? 140 Housekeeping Columns 140 Table and Column Extended Properties 142 Define Storage and Create Constraints and Supporting Objects 142 Create Files and Filegroups 142 Data Compression 144 Entity and Referential Integrity Constraints 145 Initial Indexing and Database Statistics 147 Aggregate Tables 150 Create Table Views 151 Insert an Unknown Member Row 152 Example CREATE TABLE Statement 152 Partitioned Tables 153 Finishing Up 163 Staging Tables 163 Metadata Setup 163 Summary 164 Chapter 6 Master Data Management 165 Managing Master Reference Data 166 Incomplete Attributes 167 Data Integration 168 Systems Integration 170 Master Data Management Systems and the Data Warehouse 171 Introducing SQL Server Master Data Services 171 Model Definition Features 172 Data Management Features 174 User Interface: Exploring and Managing the Master Data 174 Importing and Updating Data 176 Exporting Data 177 Full Versioning of All Attributes 179 Creating a Simple Application 179 Summary 186 Chapter 7 Designing and Developing the ETL System 187 Round Up the Requirements 188 Develop the ETL Plan 191 Introducing SQL Server Integration Services 192 Control Flow and Data Flow 194 SSIS Package Architecture 197 The Major Subsystems of ETL 198 Extracting Data 199 Subsystem 1: Data Profiling 199 Subsystem 2: Change Data Capture System 200 Subsystem 3: Extract System 202 Cleaning and Conforming Data 206 Subsystem 4: Data Cleaning System 206 Subsystem 5: Error Event Schema 214 Subsystem 6: Audit Dimension Assembler 215 Subsystem 7: Deduplication System 216 Subsystem 8: Conforming System 217 Delivering Data for Presentation 218 Subsystem 9: Slowly Changing Dimension Manager 218 Subsystem 10: Surrogate Key Generator 223 Subsystem 11: Hierarchy Manager 223 Subsystem 12: Special Dimensions Manager 224 Subsystem 13: Fact Table Builders 225 Subsystem 14: Surrogate Key Pipeline 229 Subsystem 15: Multi-Valued Dimension Bridge Table Builder 235 Subsystem 16: Late Arriving Data Handler 235 Subsystem 17: Dimension Manager 238 Subsystem 18: Fact Provider System 238 Subsystem 19: Aggregate Builder 239 Subsystem 20: OLAP Cube Builder 239 Subsystem 21: Data Propagation Manager 240 Managing the ETL Environment 240 Summary 243 Chapter 8 The Core Analysis Services OLAP Database 245 Overview of Analysis Services OLAP 247 Why Use Analysis Services? 247 Why Not Analysis Services? 249 Designing the OLAP Structure 250 Planning 251 Getting Started 253 Create a Project and a Data Source View 255 Dimension Designs 257 Creating and Editing Dimensions 261 Creating and Editing the Cube 274 Physical Design Considerations 291 Understanding Storage Modes 293 Developing the Partitioning Plan 294 Designing Performance Aggregations 296 Planning for Deployment 298 Processing the Full Cube 299 Developing the Incremental Processing Plan 299 Summary 304 Chapter 9 Design Requirements for Real-Time BI 305 Real-Time Triage 306 What Does Real-Time Mean? 306 Who Needs Real Time? 307 Real-Time Tradeoffs 308 Scenarios and Solutions 311 Executing Reports in Real Time 313 Serving Reports from a Cache 313 Creating an ODS with Mirrors and Snapshots 314 Creating an ODS with Replication 314 Building a BizTalk Application 315 Building a Real-Time Relational Partition 315 Querying Real-Time Data in the Relational Database 317 Using Analysis Services to Query Real-Time Data 318 Summary 319 Part 3 Developing the BI Applications 321 Chapter 10 Building BI Applications in Reporting Services 323 A Brief Overview of BI Applications 324 Types of BI Applications 325 The Value of Business Intelligence Applications 326 A High-Level Architecture for Reporting 328 Reviewing Business Requirements for Reporting 328 Examining the Reporting Services Architecture 330 Using Reporting Services as a Standard Reporting Tool 332 Reporting Services Assessment 339 The Reporting System Design and Development Process 340 Reporting System Design 341 Reporting System Development 348 Building and Delivering Reports 351 Planning and Preparation 351 Creating Reports 354 Reporting Operations 368 Ad Hoc Reporting Options 369 The Report Model 370 Shared Datasets 371 Report Parts 371 Summary 372 Chapter 11 PowerPivot and Excel 375 Using Excel for Analysis and Reporting 376 The PowerPivot Architecture: Excel on Steroids 378 Creating and Using PowerPivot Databases 380 Getting Started 381 PowerPivot Table Design 381 Creating Analytics with PowerPivot 385 Observations and Guidelines on PowerPivot for Excel 392 PowerPivot for SharePoint 394 The PowerPivot SharePoint User Experience 394 Server-Level Resources 397 PowerPivot Monitoring and Management 397 PowerPivot’s Role in a Managed DW/BI Environment 400 Summary 401 Chapter 12 The BI Portal and SharePoint 403 The BI Portal 404 Planning the BI Portal 405 Impact on Design 406 Business Process Categories 407 Additional Functions 408 Building the BI Portal 409 Using SharePoint as the BI Portal 411 Architecture and Concepts 412 Setting Up SharePoint 417 Summary 426 Chapter 13 Incorporating Data Mining 429 Defining Data Mining 430 Basic Data Mining Terminology 432 Business Uses of Data Mining 433 Roles and Responsibilities 440 SQL Server Data Mining Architecture Overview 440 The Data Mining Design Environment 442 Build, Deploy, and Process 442 Accessing the Mining Models 443 Integration Services and Data Mining 443 Additional Features 444 Architecture Summary 445 Microsoft Data Mining Algorithms 445 Decision Trees 446 Naïve Bayes 447 Clustering 448 Sequence Clustering 448 Time Series 449 Association 449 Neural Network 449 The Data Mining Process 450 The Business Phase 451 The Data Mining Phase 453 The Operations Phase 460 Metadata 462 Data Mining Examples 463 Case Study: Categorizing Cities 463 Case Study: Product Recommendations 472 Summary 488 Part 4 Deploying and Managing the DW/BI System 491 Chapter 14 Designing and Implementing Security 493 Identifying the Security Manager 494 Securing the Hardware and Operating System 495 Securing the Operating System 495 Using Windows Integrated Security 496 Securing the Development Environment 497 Securing the Data 498 Providing Open Access for Internal Users 498 Itemizing Sensitive Data 500 Securing Various Types of Data Access 500 Securing the Components of the DW/BI System 502 Reporting Services Security 502 Analysis Services Security 505 Relational DW Security 514 Integration Services Security 520 Usage Monitoring 521 Summary 521 Chapter 15 Metadata Plan 523 Metadata Basics 524 The Purpose of Metadata 524 Metadata Categories 525 The Metadata Repository 526 Metadata Standards 526 SQL Server 2008 R2 Metadata 527 Cross-Tool Components 528 Relational Engine Metadata 532 Analysis Services 532 Integration Services 533 Reporting Services 533 Master Data Services 534 SharePoint 534 External Metadata Sources 534 Looking to the Future 535 A Practical Metadata Approach 535 Creating the Metadata Strategy 536 Business Metadata Reporting 538 Process Metadata Reporting 541 Technical Metadata Reporting 542 Ongoing Metadata Management 543 Summary 543 Chapter 16 Deployment 545 Setting Up the Environments 546 Testing 550 Development Testing 551 System Testing 555 Data Quality Assurance Testing 557 Performance Testing 559 Usability Testing 562 Testing Summary 563 Deploying to Production 564 Relational Database Deployment 565 Integration Services Package Deployment 567 Analysis Services Database Deployment 568 Reporting Services Report Deployment 571 Master Data Services Deployment 572 Data Warehouse and BI Documentation 573 Core Descriptions 573 Additional Documentation 575 User Training 576 User Support 579 Desktop Readiness and Configuration 580 Summary 581 Chapter 17 Operations and Maintenance 583 Providing User Support 584 Maintaining the BI Portal 585 Extending the BI Applications 586 System Management 587 Governing the DW/BI System 588 Performance Monitoring 593 Usage Monitoring 600 Managing Disk Space 602 Service and Availability Management 603 Performance Tuning the DW/BI System 604 Backup and Recovery 606 Executing the ETL Packages 611 Summary 611 Chapter 18 Present Imperatives and Future Outlook 613 Growing the DW/BI System 613 Lifecycle Review with Common Problems 615 Phase I — ​Requirements, Realities, Plans, and Designs 616 Phase II — ​Developing the Databases 616 Phase III — ​Developing the BI Applications and Portal Environment 617 Phase IV — ​Deploying and Managing the DW/BI System 618 Iteration and Growth 618 What We Like in the Microsoft BI Toolset 619 Future Directions: Room for Improvement 620 Conclusion 623 Index 625

Read more less

Book SynopsisThe sure way for design professionals to learn SketchUp modeling and rendering techniques Rendering In SketchUp provides instructions for creating 3D photoreal graphics for SketchUp models using integrated rendering programs. The book serves as a beginner rendering manual and reference guide to further develop rendering skills. With an emphasis on step-by-step process, SketchUp users learn a universal approach to rendering varied SketchUp projects, including architecture, interiors, and site design models. The book focuses on tasks and principles at the core of photorealistic rendering, including: Rendering process: Learn a step-by-step process focused on workflow within SketchUp''s familiar workspace. Universal method: Understand how the process can be used to work with a variety of different integrated rendering programs, including Shaderlight, SU Podium and Twilight Render**. These programs are easy to learn and funcTable of ContentsAcknowledgments ix Part 1 Overview and Concepts Chapter 1: Introduction to Rendering in SketchUp 2 Integrated Rendering Programs 3 Studio Rendering Programs 4 Digital Rendering and Photorealism 5 Using This Book 8 The Software 10 Chapter 2: Contents and Extended Features 14 Companion IRP Chapters 14 Method and Reference Guide 18 Chapter 3: The Rendering Process 23 Create the SketchUp Model 24 The Iterative Rendering Process 28 The Post-Rendering Process 33 Chapter 4: How Rendering Works 35 IRP Render Processing 35 Computer Hardware and Rendering 37 Other Rendering Options 40 Computer Specifications 42 Chapter 5: Learning to Look 45 Rendering as an Art Form 45 Becoming a Student of Light and Color 53 Part 2 Textures Chapter 6: Textures Overview 56 Textures in SketchUp 58 The Texturing Process 63 General Considerations 65 Texture Image Formats 68 Chapter 7: The Texture Library 69 SketchUp Native Textures 70 Web Sources 70 Choosing and Downloading Textures 72 Saving a Texture Library 76 Searching CG Textures 77 Linking the Texture Library 79 Chapter 8: The SketchUp Texture Tools 81 Macintosh Texture Tools 81 PC Texture Tools 82 The Paint Bucket Tool 83 The Styles Menu 95 The Right-Click Texture Menu 98 Chapter 9: Apply, Assess, and Adjust 109 The Three As 109 Apply 111 Assess and Adjust 121 Texture Tips 141 Chapter 10: Editing Textures in an External Photo Editor 146 Linking an Editor to SketchUp 146 Launching, Editing, and Saving 147 Typical Alterations 150 Part 3 Modeling Detail Chapter 11: An Overview of Modeling Detail 168 What Is Detail Modeling? 170 Methods 173 Chapter 12: The Detailing Tools 176 The Component Library 176 The Component Browser 178 Using Layers 181 SketchUp Scenes 186 The Camera Tools 188 Chapter 13: Component Details 193 What Is Component Detail? 193 Premade Components and Textures 197 Premade Component Websites 201 Chapter 14: Organizing the Model 219 What Is a Large Model? 220 Layering Strategy 223 Layer Conventions by Model Typology 226 Cleaning Up Layers 231 Controlling Layers with Scenes 234 Toggling Layers 237 Warning! 241 Chapter 15: Camera Scenes, Composition, and Backdrops 242 Camera Scenes 242 Composition 244 Backdrops 251 Chapter 16: Advanced Detailing 261 Texture Modeling 262 Ruby Scripts for Detailing 276 Part 4 Setting Light with Shadows Chapter 17: The Shadow Menu 290 The SketchUp Shadow Menu 290 Solar North 296 Working with Shadows 297 Troubleshooting Shadows 298 Chapter 18: Composing Light 300 Composing Light Tools 303 Composing Light Strategies 308 Composing the Light 314 Part 5 The Iterative Rendering Process Chapter 19: A Rendering Overview 320 IRP Universal Features 321 Custom Features 325 Chapter 20: Steps of the Iterative Rendering Process 331 Add Initial Values 332 Draft to Final Render 334 Simulated Light Drafts-to-Final Process 343 Chapter 21: Texture Values 351 IRPs and Texture Values 351 Bump Values 354 Surface Condition and Surface Reflection 358 Transparency 364 Texture Categories 365 Troubleshooting Textures 366 Chapter 22: Image Resolution 368 What Is Resolution? 368 Determining DPI 374 Large Resolutions 374 Chapter 23: Exterior Light 375 First Lighting Steps 375 SketchUp Shadows 376 Image-Based Lighting 376 Exposure/Gamma/Intensity 381 Chapter 24: Simulated Lighting 383 Types of Lighting 385 Placing and Editing Lights 394 Render Times 398 General Simulated Light Strategies 399 Part 6 Shaderlight by ArtVPS Chapter 25: Introduction to Shaderlight 412 Menu Overview 414 Secondary Menu 415 Special Features 415 Chapter 26: Shaderlight Iterative Rendering Settings 418 The Render Settings Menu 418 Dynamic Preview and Saving 422 Draft-to-Final Settings 424 Chapter 27: Shaderlight Texture Settings 430 Apply Texture Values 430 Texture Value Descriptions 432 Texture Settings Categories 437 Glass and Water Material Values 442 Chapter 28: Shaderlight Exterior Lighting and Backdrops 449 SketchUp Dark Slider 450 Physical Sky 451 HDRI Lighting 452 Background and Backdrops 458 Chapter 29: Shaderlight Simulated Lighting 461 Shaderlight Lighting Options 462 Light Editor 468 Shaderlight Render Settings 469 Quality Settings 470 Lighting Settings 470 Postproduction 478 Chapter 30: Shaderlight Special Features 481 Batch Rendering 481 ReplaceMe 487 Chalk Rendering 490 Part 7 The Photoshop Postproduction Process Chapter 31: Postproduction Effects 496 Methods 498 Light and Color 500 Effects 511 Chapter 32: Detailed Postproduction 519 Realistic Vegetation 519 Architecture Photo Placement 524 Backgrounds/Backdrops 526 Part 8 Anatomy of a Rendering Chapter 33: Building the Base Model 536 Chapter Relationships 537 The Base Model 538 Solid Color to Surfaces 539 Base Model Extrusion 541 Solid Colors Swapped with Textures 542 Chapter 34: Building Detail 545 Chapter 35: Interior Detail 552 Interior Base Model 552 Interior Detailing 557 Chapter 36: Site Detail 565 Chapter 37: Scenes 573 Cleaning Up the Layer List 573 Off/On Scenes 574 Specific Control Scenes 575 Camera View Scenes 578 Chapter 38: Setting Light with Shadows 581 Chapter 39: The Iterative Rendering Process for Exterior Scenes 587 Chapter 40: The Iterative Rendering Process for Interior Scenes 605 Chapter 41: Postproduction of Exterior Scene 620 Index 624

Read more less

Book SynopsisHands-on guidance to creating great test-driven development practice Test-driven development (TDD) practice helps developers recognize a well-designed application, and encourages writing a test before writing the functionality that needs to be implemented. This hands-on guide provides invaluable insight for creating successful test-driven development processes. With source code and examples featured in both C# and .NET, the book walks you through the TDD methodology and shows how it is applied to a real-world application. You'll witness the application built from scratch and details each step that is involved in the development, as well as any problems that were encountered and the solutions that were applied. Clarifies the motivation behind test-driven development (TDD), what it is, and how it works Reviews the various steps involved in developing an application and the testing that is involved prior to implementing the functionality DiscussesTable of ContentsINTRODUCTION xxv PART I: GETTING STARTED CHAPTER 1: THE ROAD TO TEST-DRIVEN DEVELOPMENT 3 The Classical Approach to Software Development 4 A Brief History of Software Engineering 4 From Waterfall to Iterative and Incremental 5 A Quick Introduction to Agile Methodologies 6 A Brief History of Agile Methodologies 6 The Principles and Practices of Test-Driven Development 7 The Concepts Behind TDD 8 TDD as a Design Methodology 8 TDD as a Development Practice 8 The Benefi ts of TDD 9 A Quick Example of the TDD Approach 10 Summary 17 CHAPTER 2: AN INTRODUCTION TO UNIT TESTING 19 What Is a Unit Test? 19 Unit Test Definition 20 What Is Not a Unit Test? 20 Other Types of Tests 22 A Brief Look at NUnit 24 What Is a Unit Test Framework? 24 The Basics of NUnit 25 Decoupling with Mock Objects 28 Why Mocking Is Important 28 Dummy, Fake, Stub, and Mock 29 Best and Worst Practices 35 A Brief Look at Moq 36 What Does a Mocking Framework Do? 36 A Bit About Moq 36 Moq Basics 36 Summary 40 CHAPTER 3: A QUICK REVIEW OF REFACTORING 41 Why Refactor? 42 A Project’s Lifecycle 42 Maintainability 43 Code Metrics 43 Clean Code Principles 45 OOP Principles 45 Encapsulation 45 Inheritance 46 Polymorphism 48 The SOLID Principles 49 The Single Responsibility Principle 50 The Open/Close Principle 50 The Liskov Substitution Principle 51 The Interface Segregation Principle 51 The Dependency Inversion Principle 52 Code Smells 52 What Is a Code Smell? 52 Duplicate Code and Similar Classes 53 Big Classes and Big Methods 54 Comments 55 Bad Names 56 Feature Envy 57 Too Much If/Switch 58 Try/Catch Bloat 59 Typical Refactoring 60 Extract Classes or Interfaces 60 Extract Methods 62 Rename Variables, Fields, Methods, and Classes 66 Encapsulate Fields 67 Replace Conditional with Polymorphism 68 Allow Type Inference 71 Summary 71 CHAPTER 4: TEST-DRIVEN DEVELOPMENT: LET THE TESTS BE YOUR GUIDE 73 It Starts with the Test 74 Red, Green, Refactor 76 The Three Phases of TDD 77 The Red Phase 77 The Green Phase 78 The Refactoring Phase 79 Starting Again 79 A Refactoring Example 79 The First Feature 80 Making the First Test Pass 83 The Second Feature 83 Refactoring the Unit Tests 85 The Third Feature 87 Refactoring the Business Code 88 Correcting Refactoring Defects 91 The Fourth Feature 93 Summary 94 CHAPTER 5: MOCKING EXTERNAL RESOURCES 97 The Dependency Injection Pattern 98 Working with a Dependency Injection Framework 99 Abstracting the Data Access Layer 108 Moving the Database Concerns Out of the Business Code 108 Isolating Data with the Repository Pattern 108 Injecting the Repository 109 Mocking the Repository 112 Summary 113 PART II: PUTTING BASICS INTO ACTION CHAPTER 6: STARTING THE SAMPLE APPLICATION 117 Defi ning the Project 118 Developing the Project Overview 118 Defi ning the Target Environment 119 Choosing the Application Technology 120 Defi ning the User Stories 120 Collecting the Stories 120 Defi ning the Product Backlog 122 The Agile Development Process 123 Estimating 124 Working in Iterations 124 Communication Within Your Team 126 Iteration Zero: Your First Iteration 127 Testing in Iteration Zero 127 Ending an Iteration 128 Creating the Project 129 Choosing the Frameworks 129 Defi ning the Project Structure 131 Organizing Project Folders 131 Creating the Visual Studio Solution 132 Summary 134 CHAPTER 7: IMPLEMENTING THE FIRST USER STORY 137 The First Test 138 Choosing the First Test 138 Naming the Test 139 Writing the Test 140 Implementing the Functionality 148 Writing the Simplest Thing That Could Possibly Work 148 Running the Passing Test 157 Writing the Next Test 158 Improving the Code by Refactoring 165 Triangulation of Tests 166 Summary 166 CHAPTER 8: INTEGRATION TESTING 169 Integrate Early; Integrate Often 170 Writing Integration Tests 171 How to Manage the Database 171 How to Write Integration Tests 172 Reviewing the ItemTypeRepository 173 Adding Ninject for Dependency Injection 174 Creating the Fluent NHibernate Confi guration 177 Creating the Fluent NHibernate Mapping 179 Creating the Integration Test 183 End-to-End Integration Tests 191 Keeping Various Types of Tests Apart 191 When and How to Run Integration Tests 191 Summary 192 PART III: TDD SCENARIOS CHAPTER 9: TDD ON THE WEB 197 ASP.NET Web Forms 197 Web Form Organization 198 ASPX Files 198 Code-Behind Files 198 Implementing Test-Driven Development with MVP and Web Forms 199 Working with the ASP.NET MVC 210 MVC 101 211 Microsoft ASP.NET MVC 3.0 212 Creating an ASP.NET MVC Project 212 Creating Your First Test 213 Making Your First Test Pass 215 Creating Your First View 216 Gluing Everything Together 217 Using the MVC Contrib Project 220 ASP.NET MVC Summarized 220 Working with JavaScript 220 JavaScript Testing Frameworks 221 Summary 226 CHAPTER 10: TESTING WINDOWS COMMUNICATION FOUNDATION SERVICES 227 WCF Services in Your Application 228 Services Are Code Too 228 Testing WCF Services 228 Refactoring for Testability 229 Introducing Dependency Injection to Your Service 231 Writing the Test 236 Stubbing the Dependencies 239 Verifying the Results 243 Trouble Spots to Watch 244 Summary 244 CHAPTER 11: TESTING WPF AND SILVERLIGHT APPLICATIONS 245 The Problem with Testing the User Interface 246 The MVVM Pattern 246 How MVVM Makes WPF/Silverlight Applications Testable 248 Bringing It All Together 261 Summary 263 PART IV: REQUIREMENTS AND TOOLS CHAPTER 12: DEALING WITH DEFECTS AND NEW REQUIREMENTS 267 Handling Change 268 Change Happens 268 Adding New Features 268 Addressing Defects 269 Starting with a Test 270 Changing the Code 272 Keeping the Tests Passing 276 Summary 276 CHAPTER 13: THE GREAT TOOL DEBATE 279 Test Runners 279 TestDriven.NET 280 Developer Express Test Runner 280 Gallio 281 Unit Testing Frameworks 282 MSTest 282 MbUnit 283 xUnit 284 Mocking Frameworks 285 Rhino Mocks 285 Type Mock 287 Dependency Injection Frameworks 289 Structure Map 289 Unity 291 Windsor 293 Autofac 294 Miscellaneous Useful Tools 295 nCover 295 PEX 295 How to Introduce TDD to Your Team 296 Working in Environments That Are Resistant to Change 297 Working in Environments That Are Accepting of Change 297 Summary 297 CHAPTER 14: CONCLUSIONS 299 What You Have Learned 299 You Are the Client of Your Code 300 Find the Solutions Step by Step 300 Use the Debugger as a Surgical Instrument 300 TDD Best Practices 301 Use Signifi cant Names 301 Write at Least One Test for One Unit of Functionality 301 Keep Your Mocks Simple 302 The Benefi ts of TDD 302 How to Introduce TDD in Your Team 303 Summary 304 APPENDIX: TDD KATAS 307 Working with TDD Katas 307 Share Your Work 308 OSIM User Stories 308 INDEX 311

Read more less

Book SynopsisProgrammers, scientists, and engineers are always in need of newer techniques and algorithms to manipulate and interpret images. Algorithms for Image Processing and Computer Vision is an accessible collection of algorithms for common image processing applications that simplifies complicated mathematical calculations.Table of ContentsPreface xxi Chapter 1 Practical Aspects of a Vision System — Image Display, Input/Output, and Library Calls 1 OpenCV 2 The Basic OpenCV Code 2 The IplImage Data Structure 3 Reading and Writing Images 6 Image Display 7 An Example 7 Image Capture 10 Interfacing with the AIPCV Library 14 Website Files 18 References 18 Chapter 2 Edge-Detection Techniques 21 The Purpose of Edge Detection 21 Traditional Approaches and Theory 23 Models of Edges 24 Noise 26 Derivative Operators 30 Template-Based Edge Detection 36 Edge Models: The Marr-Hildreth Edge Detector 39 The Canny Edge Detector 42 The Shen-Castan (ISEF) Edge Detector 48 A Comparison of Two Optimal Edge Detectors 51 Color Edges 53 Source Code for the Marr-Hildreth Edge Detector 58 Source Code for the Canny Edge Detector 62 Source Code for the Shen-Castan Edge Detector 70 Website Files 80 References 82 Chapter 3 Digital Morphology 85 Morphology Defined 85 Connectedness 86 Elements of Digital Morphology — Binary Operations 87 Binary Dilation 88 Implementing Binary Dilation 92 Binary Erosion 94 Implementation of Binary Erosion 100 Opening and Closing 101 MAX — A High-Level Programming Language for Morphology 107 The ‘‘Hit-and-Miss’’ Transform 113 Identifying Region Boundaries 116 Conditional Dilation 116 Counting Regions 119 Grey-Level Morphology 121 Opening and Closing 123 Smoothing 126 Gradient 128 Segmentation of Textures 129 Size Distribution of Objects 130 Color Morphology 131 Website Files 132 References 135 Chapter 4 Grey-Level Segmentation 137 Basics of Grey-Level Segmentation 137 Using Edge Pixels 139 Iterative Selection 140 The Method of Grey-Level Histograms 141 Using Entropy 142 Fuzzy Sets 146 Minimum Error Thresholding 148 Sample Results From Single Threshold Selection 149 The Use of Regional Thresholds 151 Chow and Kaneko 152 Modeling Illumination Using Edges 156 Implementation and Results 159 Comparisons 160 Relaxation Methods 161 Moving Averages 167 Cluster-Based Thresholds 170 Multiple Thresholds 171 Website Files 172 References 173 Chapter 5 Texture and Color 177 Texture and Segmentation 177 A Simple Analysis of Texture in Grey-Level Images 179 Grey-Level Co-Occurrence 182 Maximum Probability 185 Moments 185 Contrast 185 Homogeneity 185 Entropy 186 Results from the GLCM Descriptors 186 Speeding Up the Texture Operators 186 Edges and Texture 188 Energy and Texture 191 Surfaces and Texture 193 Vector Dispersion 193 Surface Curvature 195 Fractal Dimension 198 Color Segmentation 201 Color Textures 205 Website Files 205 References 206 Chapter 6 Thinning 209 What Is a Skeleton? 209 The Medial Axis Transform 210 Iterative Morphological Methods 212 The Use of Contours 221 Choi/Lam/Siu Algorithm 224 Treating the Object as a Polygon 226 Triangulation Methods 227 Force-Based Thinning 228 Definitions 229 Use of a Force Field 230 Subpixel Skeletons 234 Source Code for Zhang-Suen/Stentiford/Holt Combined Algorithm 235 Website Files 246 References 247 Chapter 7 Image Restoration 251 Image Degradations — The Real World 251 The Frequency Domain 253 The Fourier Transform 254 The Fast Fourier Transform 256 The Inverse Fourier Transform 260 Two-Dimensional Fourier Transforms 260 Fourier Transforms in OpenCV 262 Creating Artificial Blur 264 The Inverse Filter 270 The Wiener Filter 271 Structured Noise 273 Motion Blur — A Special Case 276 The Homomorphic Filter — Illumination 277 Frequency Filters in General 278 Isolating Illumination Effects 280 Website Files 281 References 283 Chapter 8 Classification 285 Objects, Patterns, and Statistics 285 Features and Regions 288 Training and Testing 292 Variation: In-Class and Out-Class 295 Minimum Distance Classifiers 299 Distance Metrics 300 Distances Between Features 302 Cross Validation 304 Support Vector Machines 306 Multiple Classifiers — Ensembles 309 Merging Multiple Methods 309 Merging Type 1 Responses 310 Evaluation 311 Converting Between Response Types 312 Merging Type 2 Responses 313 Merging Type 3 Responses 315 Bagging and Boosting 315 Bagging 315 Boosting 316 Website Files 317 References 318 Chapter 9 Symbol Recognition 321 The Problem 321 OCR on Simple Perfect Images 322 OCR on Scanned Images — Segmentation 326 Noise 327 Isolating Individual Glyphs 329 Matching Templates 333 Statistical Recognition 337 OCR on Fax Images — Printed Characters 339 Orientation — Skew Detection 340 The Use of Edges 345 Handprinted Characters 348 Properties of the Character Outline 349 Convex Deficiencies 353 Vector Templates 357 Neural Nets 363 A Simple Neural Net 364 A Backpropagation Net for Digit Recognition 368 The Use of Multiple Classifiers 372 Merging Multiple Methods 372 Results From the Multiple Classifier 375 Printed Music Recognition — A Study 375 Staff Lines 376 Segmentation 378 Music Symbol Recognition 381 Source Code for Neural Net Recognition System 383 Website Files 390 References 392 Chapter 10 Content-Based Search — Finding Images by Example 395 Searching Images 395 Maintaining Collections of Images 396 Features for Query by Example 399 Color Image Features 399 Mean Color 400 Color Quad Tree 400 Hue and Intensity Histograms 401 Comparing Histograms 402 Requantization 403 Results from Simple Color Features 404 Other Color-Based Methods 407 Grey-Level Image Features 408 Grey Histograms 409 Grey Sigma — Moments 409 Edge Density — Boundaries Between Objects 409 Edge Direction 410 Boolean Edge Density 410 Spatial Considerations 411 Overall Regions 411 Rectangular Regions 412 Angular Regions 412 Circular Regions 414 Hybrid Regions 414 Test of Spatial Sampling 414 Additional Considerations 417 Texture 418 Objects, Contours, Boundaries 418 Data Sets 418 Website Files 419 References 420 Systems 424 Chapter 11 High-Performance Computing for Vision and Image Processing 425 Paradigms for Multiple-Processor Computation 426 Shared Memory 426 Message Passing 427 Execution Timing 427 Using clock() 428 Using QueryPerformanceCounter 430 The Message-Passing Interface System 432 Installing MPI 432 Using MPI 433 Inter-Process Communication 434 Running MPI Programs 436 Real Image Computations 437 Using a Computer Network — Cluster Computing 440 A Shared Memory System — Using the PC Graphics Processor 444 GLSL 444 OpenGL Fundamentals 445 Practical Textures in OpenGL 448 Shader Programming Basics 451 Vertex and Fragment Shaders 452 Required GLSL Initializations 453 Reading and Converting the Image 454 Passing Parameters to Shader Programs 456 Putting It All Together 457 Speedup Using the GPU 459 Developing and Testing Shader Code 459 Finding the Needed Software 460 Website Files 461 References 461 Index 465

Read more less

Book SynopsisFocusing just on the essentials of command-line interface (CLI), this title shows how to quickly perform day-to-day tasks of Windows administration without ever touching the graphical user interface (GUI). It replaces many tedious GUI steps with just one command at the command-line, while easy to access answers provide solutions on the spot.Table of ContentsIntroduction xix Part I: Command Line Basics 1 Chapter 1: Configuring the Local Machine 3 Configure the Command Window 4 Set the Window Options 4 Change the Font 7 Choose a Window Layout 8 Define the Text Colors 9 Set the Environment 10 Manage Environment Variables with the Set Command 10 Manage Environment Variables with the SetX Utility 13 Perform Common Tasks 16 Clear the Display 16 Determine the Operating System Version 16 Start an Application 16 Work with Services 18 Shut Down the System 19 Obtain Command Line Help 20 Chapter 2: Making Remote Connections 23 Configure the Remote System 24 Change Security and Basic Setup 25 Setup Remote Administrator 28 Use the Remote Desktop Connection Application 30 Create a Connection 30 Use a Saved Connection 35 Set the Display 35 Access Local Resources 36 Run a Configuration Program 38 Optimize Performance 38 Terminate a Session 39 Use the Start Menu 40 Use the Logoff Utility 40 Chapter 3: Automating Tasks 41 View and Manage Tasks Using Scheduled Tasks 42 Configure the Task Scheduler 43 View Tasks 44 Create New Tasks 46 Delete Existing Tasks 50 Manage Tasks Using the SchTasks Command 51 Use the /Create Switch 51 Use the /Delete Switch 53 Use the /Query Switch 53 Use the /Change Switch 54 Use the /Run Switch 54 Use the /End Switch 54 Part II: Managing Data 55 Chapter 4: Working with File and Directory Objects 57 Manage Directory Objects 58 Find Directories 58 Find Directories Using Patterns 59 View the Current Directory 60 Change the Current Directory 60 Create Directories 61 Move Directories 61 Rename Directories 61 Remove a Directory 62 Display a Directory Structure 62 Manage File Objects 63 Find Files 63 Find Files in Sorted Order 64 Find Files by Attribute 65 Find Files Using Patterns 66 Copy Files 68 Perform Bulk File Transfers 69 Remove Files 70 Move Files 71 Rename a File 71 Set File Attributes 71 Work with File Associations and Types 72 Determine File Associations 72 Create File Associations 73 Determine File Types 73 Create File Types 73 Make Data Links 74 Create Simple Hard Links 75 View Simple Hard Links 75 Delete Simple Hard Links 75 Create Hard Links Using the New Technique 76 Create Symbolic Links 76 Create Junctions 77 Chapter 5: Administering File and Directory Content 79 Execute Applications Anywhere 80 View Application Paths 80 Set Application Paths 80 Locate Information in Files 81 Find Simple Strings 81 Find Complex Strings 82 Display Files Containing Strings 82 Perform Case Insensitive Searches 83 Monitor the File System with the FSUtil Command 83 Control File System Behavior 83 Manage the Volume Dirty Bit 86 Obtain the File System Information Using FSInfo 87 Manage Quotas 88 Repair File System Errors 90 Display Data Files 92 Display a Data File on Screen 92 Employ Data Redirection 92 Display Data One Page at a Time 95 Chapter 6: Managing the Hard Drive 99 Save Hard Drive Space 101 Compress Data 101 Uncompress Data 102 View Compression Status 102 Manage the Volume 102 Get Volume Information 103 Manage Volume Labels 103 Format a Disk 103 Mount a Volume 104 Maintain the Volume 106 Determine File and Directory Status 106 Locate Bad Sectors 107 Perform Boot-Time Disk Checks 108 Improve Disk Access Performance 109 Manage Partitions 110 Start DiskPart 110 List the Objects 111 See Object Details 112 Select an Object 113 Rescan a Computer for Objects 114 Create a Partition 114 Create a Volume 116 Clean a Drive 117 Mark a Partition as Active 117 Mark a Partition as Inactive 118 Assign a Drive Letter 118 Remove a Drive Letter 119 Extend a Volume 119 Delete an Object 120 Exit DiskPart 120 Chapter 7: Securing the Data 121 Protect Data 122 Encrypt a File or Directory 122 View Encrypted Files and Directories 123 Encrypt Hidden Files 124 Back Up Recovery Keys and Certificates 125 Add a User to a File or Directory 125 Remove a User from a File or Directory 126 Decrypt a File or Directory 127 Change File and Directory Access 127 Obtain the DACL 127 Find an SID 130 Grant Permission 130 Deny Permission 131 Remove Permission 131 Set the Owner 132 Verify Security 132 Detect Shared Open Files 132 Use the Query Option 132 Use the Disconnect Option 133 Use the Local Option 134 Take Ownership of Files 134 Set Administrator Ownership 134 Set Other User Ownership 135 Part III: Managing the Network 137 Chapter 8: Managing the Network 139 Get the Media Access Control Information 140 Interact with the Network Using the Net Utility 141 Manage Users 141 Manage Accounts 143 Manage Domains and Local Groups 146 Manage Computers 149 View and Close Sessions 149 Perform Server Configuration 151 View Workstation Configuration 153 Manage Services 153 Manage Files 155 Obtain Help for the Net Utility 156 Manage Print Jobs 157 Manage Resources 158 Obtain Statistics 161 Configure Time Synchronization 163 Chapter 9: Working with TCP/IP 165 Manage the Internet Protocol 166 Display the IP Information 166 Renew Addresses for an Adapter 168 Clear the DNS Resolver Cache 168 Renew DHCP Addresses and Register DNS Names 169 Release a Connection 169 Use Basic Diagnostics 170 Check Connections 170 Trace Transmission Paths 171 Track the Network Path 172 Perform Detailed Network Diagnostics 173 Obtain a Copy of NetDiag 174 Perform a Test 175 Understand Diagnostics 175 Locate and Fix Minor Problems 177 Get Network Statistics 177 Display All Connections and Ports 177 Display Application Statistics 178 Display Ethernet Statistics 179 Display Protocol Information 179 Set a Refresh Interval 180 Manipulate the Network Routing Tables 180 Print the Routing Tables 180 Add a New Route 181 Change a Route 182 Delete a Route 183 Chapter 10: Creating System Connections 185 Perform Remote System Management 186 Create Remote Connections 186 Set Up a Telephony Client 190 Perform Remote Windows Management 191 Execute Commands on a Remote System 206 Work with Terminal Server 206 Obtain Process Information 206 Get Session Information 207 Terminate a Session 208 Disconnect an Active Session 208 End Processes 208 Shut Down the Terminal Server 209 Part IV: Interacting with Active Directory 211 Chapter 11: Configuring Directory Services 213 Manage Directory Services Using the WMIC NTDomain Alias 214 List the Objects 215 List Object Properties 215 Get an Object Property 216 Set an Object Property 217 Query an Association 217 Manage Active Directory with the DSQuery Utility 218 Interact with Servers 218 Interact with Users 220 Interact with Computers 223 Interact with Contacts 223 Interact with Groups 223 Interact with Organizational Units 224 Manage the Active Directory Database 225 Issue a Command 225 Use a Stream 226 Chapter 12: Working with Directory Objects 229 Create New Objects 230 Add a Computer 230 Add a Contact 231 Add a Group 233 Add an Organizational Unit 234 Add a User 234 Get Objects 238 List Computers 238 List Contacts 239 List Groups 240 List Organizational Units 241 List Servers 241 List Users 242 Edit Existing Objects 243 Modify Computer Data 244 Modify Contact Data 245 Modify Group Data 246 Modify Organizational Unit Data 246 Modify User Data 246 Move Existing Objects 248 Delete Existing Objects 248 Part V: Performing Diagnostics 251 Chapter 13: Monitoring System Events 253 Create Simple System Events 254 Trigger System Events 257 Create an Event 258 Delete an Event 261 Query an Event 261 Manage Event Information 262 Display a List of Publishers 262 Get a Publisher 262 Enumerate the Logs 263 Query Log Events 264 Get a Log 267 Get Log Status Information 267 Set a Log 267 Export a Log 269 Archive a Log 270 Clear a Log 270 Chapter 14: Monitoring System Performance 271 Add Performance Counters 272 Load a Performance Counter 272 Save Performance Counter Settings 273 Restore Performance Counter Settings 273 Manage Performance Logs and Alerts 274 Create a Performance Log 274 Start Collecting Data 276 Stop Collecting Data 276 Query a Collection 277 Update a Collection 277 Delete a Collection 278 Create New Performance Logs from Existing Logs 278 Remove Performance Counters 281 Convert Event Trace Logs 281 Part VI: Performing Maintenance 283 Chapter 15: Performing Basic Maintenance 285 Configure the Server 286 Understand the SQL Syntax of WMIC 287 Use Aliases in WMIC 290 Get Help in WMIC 302 Format Data in WMIC 306 Translate Data in WMIC 310 Activate Windows 311 Perform an Activation 311 Display the Activation Information 312 Change the Product Key 312 Manage the System Time 312 Update the Time 313 Configure a Time Source 313 Obtain Time Settings Information 314 Manage the Boot Configuration 315 Enumerate the Configurations 315 Get BCDEdit Help 316 Edit an Existing Boot Setting 317 Change the Boot Sequence 318 Set the Default Boot Item 318 Chapter 16: Managing System Users 319 Audit User Access 320 List the Policies 321 Get a Policy 322 Set a Policy 326 Perform a Backup 327 Perform a Restore 327 Clear an Audit Policy 328 Remove an Audit Policy 328 Work with Group Policies 328 Obtain Group Policy Results 328 Manage Group Policies 330 Obtain Session Status Information 331 Get Process Information 331 Get Session Information 331 Get User Information 332 Get Terminal Server Information 332 Get the User’s Identity 332 Obtain User Logon Information 333 Discover User Identity 333 Chapter 17: Securing the System 335 Add Virus and External Intrusion Protection 337 Remove Viruses 337 Detect and Remove Malicious Software 338 Verify System Files 339 Verify Drivers 340 Change the Verifier Settings 342 Configure Local Security Policies 344 Perform an Analysis 344 Configure Security Policies 345 Export Policies 345 Import Policies 346 Validate a Policy File 346 Work with General Applications 347 Use TaskKill and TaskList Filters 347 Terminate Tasks 350 List Applications 350 List Services 351 Chapter 18: Interacting with the Registry 353 Perform Basic Registry Tasks 354 Export a Registry Key 356 Import a Registry Key 357 Delete a Registry Key 357 Save the Registry 358 Restore the Registry 358 Use the SCRegEdit Script 358 Set Automatic Updates 359 Enable Terminal Services 359 Configure the IP Security (IPSec) Monitor 360 Manage the DNS Service Priority and Weight 361 Use the Command Line Reference 361 Manage the Registry 362 Understand the Registry Settings 362 Query a Registry Entry 363 Add a Registry Entry 364 Delete a Registry Entry 365 Copy a Registry Entry 366 Compare Registry Entries 366 Export Registry Entries 367 Import Registry Entries 367 Restore Registry Entries 368 Part VII: Creating Batch Files 369 Chapter 19: Changing the Batch File Environment 371 Use the CMD Switches 372 Configure the Command Interpreter in the Registry 375 Use Command Extensions 377 Modify Config.NT 380 Use ANSI.SYS to Control the Environment 382 Set the Command Interpreter Location 382 Run Character Mode Applications Only 383 Display the Config.NT Commands 383 Control the Expanded Memory EMM Entry 383 Set the Number of Accessible Files 384 Control Extended Memory with HIMEM.SYS 385 Modify AutoExec.NT 387 Set the Code Page Number with the CHCP Utility 387 Add DPMI Support Using the DosX Utility 388 Enable Graphics Character Support with the GrafTabl Utility 389 Save Memory Using the LH Command 389 Install the Network Redirector Using the ReDir Utility 389 Chapter 20: Working at the Command Prompt 391 Redirect Command Line Output to the Clipboard 392 Manage Usernames and Passwords 393 Display Usernames 393 Create Users 394 Delete Users 394 Change Screen Colors 395 Configure the System Date 396 Configure the System Time 396 Change the Command Window Title 397 Chapter 21: Creating and Testing Batch Files 399 Use Batch File Commands 400 Employ the Call Command 401 Employ the Choice Command 403 Employ the Echo Command 406 Employ the Exit Command 406 Employ the ForFiles Utility 407 Employ the For Command 409 Employ the GoTo Command 414 Employ the If Command 414 Employ the Pause Command 418 Employ the Prompt Command 418 Employ the Rem Command 419 Employ the TimeOut Utility 420 Test Batch Files 420 Add Debug Information to Batch Files 421 Identify Batch Files and Their Actions 425 Use a Centralized Data Store 428 Store and Retrieve Directories with the PushD and PopD Commands 430 Part VIII: Creating Scripts 433 Chapter 22: Discovering Scripting Basics 435 Use Scripting Languages 436 Learn the Basics of JavaScript 436 Learn the Basics of VBScript 438 Use the Windows Scripting File 439 Execute Scripts 442 Run Scripts with the CScript and WScript Utilities 442 Configure the Host and Property Page Options 444 Chapter 23: Using the Scripting Objects 447 Use the WScript Object 449 Use the WScript Properties 449 Use the WScript Methods 451 Use the WScript.WshArguments Object 454 Use the WScript.WshShell Object 455 Use the WScript.WshNetwork Object 457 Use the WScript.WshNetwork Properties 457 Use the WScript.WshNetwork Methods 458 Create a Basic Script 462 Script the Command Line and System Environment 463 Script the Registry 466 Create .LNK Files 468 Chapter 24: Creating Advanced Scripting Examples 471 Script Registry Entries 472 Script Networking Solutions 473 Discover the NetSH Helper List 473 Get NetSH Help 474 Execute NetSH Commands 476 Understand the Basic NetSH Contexts 476 Use the Root Context Commands 478 Impersonate a User 480 Change the Environment 481 Change Logons, Ports, and Users 481 Enable or Disable Session Logons Directly 482 List COM Port Mappings 482 Modify the Install Mode 482 Map a Network Drive 483 Create a .CSV File 487 Appendix A: Alphabetical Command List 493 Appendix B: Topical Command List 505 Appendix C: Listing of Best Practices 521 Always Verify the Data 522 Real Administrators Use Help 523 Test Your Theories on a Test System 524 Use Batch Files, Scripts, and Written Procedures 525 Make Backups 526 Perform User-Specific Changes during Downtime 526 Index 529

Read more less

Book SynopsisThe leading introductory book on data mining, fully updated and revised! When Berry and Linoff wrote the first edition of Data Mining Techniques in the late 1990s, data mining was just starting to move out of the lab and into the office and has since grown to become an indispensable tool of modern business.Table of ContentsIntroduction xxxvii Chapter 1 What Is Data Mining and Why Do It? 1 What Is Data Mining? 2 Data Mining Is a Business Process 2 Large Amounts of Data 3 Meaningful Patterns and Rules 3 Data Mining and Customer Relationship Management 4 Why Now? 6 Data Is Being Produced 6 Data Is Being Warehoused 6 Computing Power Is Affordable 7 Interest in Customer Relationship Management Is Strong 7 Commercial Data Mining Software Products Have Become Available 8 Skills for the Data Miner 9 The Virtuous Cycle of Data Mining 9 A Case Study in Business Data Mining 11 Identifying BofA’s Business Challenge 12 Applying Data Mining 12 Acting on the Results 13 Measuring the Effects of Data Mining 14 Steps of the Virtuous Cycle 15 Identify Business Opportunities 16 Transform Data into Information 17 Act on the Information 19 Measure the Results 20 Data Mining in the Context of the Virtuous Cycle 23 Lessons Learned 26 Chapter 2 Data Mining Applications in Marketing and Customer Relationship Management 27 Two Customer Lifecycles 27 The Customer’s Lifecycle 28 The Customer Lifecycle 28 Subscription Relationships versus Event-Based Relationships 30 Organize Business Processes Around the Customer Lifecycle 32 Customer Acquisition 33 Customer Activation 36 Customer Relationship Management 37 Winback 38 Data Mining Applications for Customer Acquisition 38 Identifying Good Prospects 39 Choosing a Communication Channel 39 Picking Appropriate Messages 40 A Data Mining Example: Choosing the Right Place to Advertise 40 Who Fits the Profile? 41 Measuring Fitness for Groups of Readers 44 Data Mining to Improve Direct Marketing Campaigns 45 Response Modeling 46 Optimizing Response for a Fixed Budget 47 Optimizing Campaign Profitability 49 Reaching the People Most Influenced by the Message 53 Using Current Customers to Learn About Prospects 54 Start Tracking Customers Before They Become “Customers” 55 Gather Information from New Customers 55 Acquisition-Time Variables Can Predict Future Outcomes 56 Data Mining Applications for Customer Relationship Management 56 Matching Campaigns to Customers 56 Reducing Exposure to Credit Risk 58 Determining Customer Value 59 Cross-selling, Up-selling, and Making Recommendations 60 Retention 60 Recognizing Attrition 60 Why Attrition Matters 61 Different Kinds of Attrition 62 Different Kinds of Attrition Model 63 Beyond the Customer Lifecycle 64 Lessons Learned 65 Chapter 3 The Data Mining Process 67 What Can Go Wrong? 68 Learning Things That Aren’t True 68 Learning Things That Are True, but Not Useful 73 Data Mining Styles 74 Hypothesis Testing 75 Directed Data Mining 81 Undirected Data Mining 81 Goals, Tasks, and Techniques 82 Data Mining Business Goals 82 Data Mining Tasks 83 Data Mining Techniques 88 Formulating Data Mining Problems: From Goals to Tasks to Techniques 88 What Techniques for Which Tasks? 95 Is There a Target or Targets? 96 What Is the Target Data Like? 96 What Is the Input Data Like? 96 How Important Is Ease of Use? 97 How Important Is Model Explicability? 97 Lessons Learned 98 Chapter 4 Statistics 101: What You Should Know About Data 101 Occam’s Razor 103 Skepticism and Simpson’s Paradox 103 The Null Hypothesis 104 P-Values 105 Looking At and Measuring Data 106 Categorical Values 106 Numeric Variables 117 A Couple More Statistical Ideas 120 Measuring Response 120 Standard Error of a Proportion 121 Comparing Results Using Confidence Bounds 123 Comparing Results Using Difference of Proportions 124 Size of Sample 125 What the Confidence Interval Really Means 126 Size of Test and Control for an Experiment 127 Multiple Comparisons 129 The Confidence Level with Multiple Comparisons 129 Bonferroni’s Correction 129 Chi-Square Test 130 Expected Values 130 Chi-Square Value 132 Comparison of Chi-Square to Difference of Proportions 134 An Example: Chi-Square for Regions and Starts 134 Case Study: Comparing Two Recommendation Systems with an A/B Test 138 First Metric: Participating Sessions 140 Data Mining and Statistics 144 Lessons Learned 148 Chapter 5 Descriptions and Prediction: Profiling and Predictive Modeling 151 Directed Data Mining Models 152 Defining the Model Structure and Target 152 Incremental Response Modeling 154 Model Stability 156 Time-Frames in the Model Set 157 Directed Data Mining Methodology 159 Step 1: Translate the Business Problem into a Data Mining Problem 161 How Will Results Be Used? 163 How Will Results Be Delivered? 163 The Role of Domain Experts and Information Technology 164 Step 2: Select Appropriate Data 165 What Data Is Available? 166 How Much Data Is Enough? 167 How Much History Is Required? 167 How Many Variables? 168 What Must the Data Contain? 168 Step 3: Get to Know the Data 169 Examine Distributions 169 Compare Values with Descriptions 170 Validate Assumptions 170 Ask Lots of Questions 171 Step 4: Create a Model Set 172 Assembling Customer Signatures 172 Creating a Balanced Sample 172 Including Multiple Timeframes 174 Creating a Model Set for Prediction 174 Creating a Model Set for Profiling 176 Partitioning the Model Set 176 Step 5: Fix Problems with the Data 177 Categorical Variables with Too Many Values 177 Numeric Variables with Skewed Distributions and Outliers 178 Missing Values 178 Values with Meanings That Change over Time 179 Inconsistent Data Encoding 179 Step 6: Transform Data to Bring Information to the Surface 180 Step 7: Build Models 180 Step 8: Assess Models 180 Assessing Binary Response Models and Classifiers 181 Assessing Binary Response Models Using Lift 182 Assessing Binary Response Model Scores Using Lift Charts 184 Assessing Binary Response Model Scores Using Profitability Models 185 Assessing Binary Response Models Using ROC Charts 186 Assessing Estimators 188 Assessing Estimators Using Score Rankings 189 Step 9: Deploy Models 190 Practical Issues in Deploying Models 190 Optimizing Models for Deployment 191 Step 10: Assess Results 191 Step 11: Begin Again 193 Lessons Learned 193 Chapter 6 Data Mining Using Classic Statistical Techniques 195 Similarity Models 196 Similarity and Distance 196 Example: A Similarity Model for Product Penetration 197 Table Lookup Models 203 Choosing Dimensions 204 Partitioning the Dimensions 205 From Training Data to Scores 205 Handling Sparse and Missing Data by Removing Dimensions 205 RFM: A Widely Used Lookup Model 206 RFM Cell Migration 207 RFM and the Test-and-Measure Methodology 208 RFM and Incremental Response Modeling 209 Naïve Bayesian Models 210 Some Ideas from Probability 210 The Naïve Bayesian Calculation 212 Comparison with Table Lookup Models 213 Linear Regression 213 The Best-fit Line 215 Goodness of Fit 217 Multiple Regression 220 The Equation 220 The Range of the Target Variable 221 Interpreting Coefficients of Linear Regression Equations 221 Capturing Local Effects with Linear Regression 223 Additional Considerations with Multiple Regression 224 Variable Selection for Multiple Regression 225 Logistic Regression 227 Modeling Binary Outcomes 227 The Logistic Function 229 Fixed Effects and Hierarchical Effects 231 Hierarchical Effects 232 Within and Between Effects 232 Fixed Effects 233 Lessons Learned 234 Chapter 7 Decision Trees 237 What Is a Decision Tree and How Is It Used? 238 A Typical Decision Tree 238 Using the Tree to Learn About Churn 240 Using the Tree to Learn About Data and Select Variables 241 Using the Tree to Produce Rankings 243 Using the Tree to Estimate Class Probabilities 243 Using the Tree to Classify Records 244 Using the Tree to Estimate Numeric Values 244 Decision Trees Are Local Models 245 Growing Decision Trees 247 Finding the Initial Split 248 Growing the Full Tree 251 Finding the Best Split 252 Gini (Population Diversity) as a Splitting Criterion 253 Entropy Reduction or Information Gain as a Splitting Criterion 254 Information Gain Ratio 256 Chi-Square Test as a Splitting Criterion 256 Incremental Response as a Splitting Criterion 258 Reduction in Variance as a Splitting Criterion for Numeric Targets 259 F Test 262 Pruning 262 The CART Pruning Algorithm 263 Pessimistic Pruning: The C5.0 Pruning Algorithm 267 Stability-Based Pruning 268 Extracting Rules from Trees 269 Decision Tree Variations 270 Multiway Splits 270 Splitting on More Than One Field at a Time 271 Creating Nonrectangular Boxes 271 Assessing the Quality of a Decision Tree 275 When Are Decision Trees Appropriate? 276 Case Study: Process Control in a Coffee Roasting Plant 277 Goals for the Simulator 277 Building a Roaster Simulation 278 Evaluation of the Roaster Simulation 278 Lessons Learned 279 Chapter 8 Artificial Neural Networks 281 A Bit of History 282 The Biological Model 283 The Biological Neuron 285 The Biological Input Layer 286 The Biological Output Layer 287 Neural Networks and Artificial Intelligence 287 Artificial Neural Networks 288 The Artificial Neuron 288 The Multi-Layer Perceptron 291 A Network Example 292 Network Topologies 293 A Sample Application: Real Estate Appraisal 295 Training Neural Networks 299 How Does a Neural Network Learn Using Back Propagation? 299 Pruning a Neural Network 300 Radial Basis Function Networks 303 Overview of RBF Networks 303 Choosing the Locations of the Radial Basis Functions 305 Universal Approximators 305 Neural Networks in Practice 308 Choosing the Training Set 309 Coverage of Values for All Features 309 Number of Features 310 Size of Training Set 310 Number and Range of Outputs 310 Rules of Thumb for Using MLPs 310 Preparing the Data 311 Interpreting the Output from a Neural Network 313 Neural Networks for Time Series 315 Time Series Modeling 315 A Neural Network Time Series Example 316 Can Neural Network Models Be Explained? 317 Sensitivity Analysis 318 Using Rules to Describe the Scores 318 Lessons Learned 319 Chapter 9 Nearest Neighbor Approaches: Memory-Based Reasoning and Collaborative Filtering 321 Memory-Based Reasoning 322 Look-Alike Models 323 Example: Using MBR to Estimate Rents in Tuxedo, New York 324 Challenges of MBR 327 Choosing a Balanced Set of Historical Records 328 Representing the Training Data 328 Determining the Distance Function, Combination Function, and Number of Neighbors 331 Case Study: Using MBR for Classifying Anomalies in Mammograms 331 The Business Problem: Identifying Abnormal Mammograms 332 Applying MBR to the Problem 332 The Total Solution 334 Measuring Distance and Similarity 335 What Is a Distance Function? 335 Building a Distance Function One Field at a Time 337 Distance Functions for Other Data Types 340 When a Distance Metric Already Exists 341 The Combination Function: Asking the Neighbors for Advice 342 The Simplest Approach: One Neighbor 342 The Basic Approach for Categorical Targets: Democracy 342 Weighted Voting for Categorical Targets 344 Numeric Targets 344 Case Study: Shazam — Finding Nearest Neighbors for Audio Files 345 Why This Feat Is Challenging 346 The Audio Signature 347 Measuring Similarity 348 Collaborative Filtering: A Nearest-Neighbor Approach to Making Recommendations 351 Building Profiles 352 Comparing Profiles 352 Making Predictions 353 Lessons Learned 354 Chapter 10 Knowing When to Worry: Using Survival Analysis to Understand Customers 357 Customer Survival 360 What Survival Curves Reveal 360 Finding the Average Tenure from a Survival Curve 362 Customer Retention Using Survival 364 Looking at Survival as Decay 365 Hazard Probabilities 367 The Basic Idea 368 Examples of Hazard Functions 369 Censoring 371 The Hazard Calculation 372 Other Types of Censoring 375 From Hazards to Survival 376 Retention 376 Survival 378 Comparison of Retention and Survival 378 Proportional Hazards 380 Examples of Proportional Hazards 381 Stratification: Measuring Initial Effects on Survival 382 Cox Proportional Hazards 382 Survival Analysis in Practice 385 Handling Different Types of Attrition 385 When Will a Customer Come Back? 387 Understanding Customer Value 389 Forecasting 392 Hazards Changing over Time 393 Lessons Learned 394 Chapter 11 Genetic Algorithms and Swarm Intelligence 397 Optimization 398 What Is an Optimization Problem? 398 An Optimization Problem in Ant World 399 E Pluribus Unum 400 A Smarter Ant 401 Genetic Algorithms 403 A Bit of History 404 Genetics on Computers 404 Representing the Genome 413 Schemata: The Building Blocks of Genetic Algorithms 414 Beyond the Simple Algorithm 417 The Traveling Salesman Problem 418 Exhaustive Search 419 A Simple Greedy Algorithm 419 The Genetic Algorithms Approach 419 The Swarm Intelligence Approach 420 Case Study: Using Genetic Algorithms for Resource Optimization 421 Case Study: Evolving a Solution for Classifying Complaints 423 Business Context 424 Data 425 The Comment Signature 425 The Genomes 426 The Fitness Function 427 The Results 427 Lessons Learned 427 Chapter 12 Tell Me Something New: Pattern Discovery and Data Mining 429 Undirected Techniques, Undirected Data Mining 431 Undirected versus Directed Techniques 431 Undirected versus Directed Data Mining 431 Case Study: Undirected Data Mining Using Directed Techniques 432 What is Undirected Data Mining? 435 Data Exploration 435 Segmentation and Clustering 436 Target Variable Definition, When the Target Is Not Explicit 438 Simulation, Forecasting, and Agent-Based Modeling 443 Methodology for Undirected Data Mining 455 There Is No Methodology 456 Things to Keep in Mind 456 Lessons Learned 457 Chapter 13 Finding Islands of Similarity: Automatic Cluster Detection 459 Searching for Islands of Simplicity 461 Customer Segmentation and Clustering 461 Similarity Clusters 463 Tracking Campaigns by Cluster-Based Segments 464 Clustering Reveals an Overlooked Market Segment 466 Fitting the Troops 467 The K-Means Clustering Algorithm 468 Two Steps of the K-Means Algorithm 468 Voronoi Diagrams and K-Means Clusters 471 Choosing the Cluster Seeds 473Choosing K 473 Using K-Means to Detect Outliers 474 Semi-Directed Clustering 475 Interpreting Clusters 475 Characterizing Clusters by Their Centroids 476 Characterizing Clusters by What Differentiates Them 477 Using Decision Trees to Describe Clusters 478 Evaluating Clusters 479 Cluster Measurements and Terminology 480 Cluster Silhouettes 480 Limiting Cluster Diameter for Scoring 483 Case Study: Clustering Towns 484 Creating Town Signatures 484 Creating Clusters 486 Determining the Right Number of Clusters 486 Evaluating the Clusters 487 Using Demographic Clusters to Adjust Zone Boundaries 488 Business Success 490 Variations on K-Means 490 K-Medians, K-Medoids, and K-Modes 490 The Soft Side of K-Means 494 Data Preparation for Clustering 495 Scaling for Consistency 496 Use Weights to Encode Outside Information 496 Selecting Variables for Clustering 497 Lessons Learned 497 Chapter 14 Alternative Approaches to Cluster Detection 499 Shortcomings of K-Means 500 Reasonableness 500 An Intuitive Example 501 Fixing the Problem by Changing the Scales 503 What This Means in Practice 504 Gaussian Mixture Models 505 Adding “Gaussians” to K-Means 505 Back to Gaussian Mixture Models 508 Scoring GMMs 510 Applying GMMs 511 Divisive Clustering 513 A Decision Tree–Like Method for Clustering 513 Scoring Divisive Clusters 515 Clusters and Trees 515 Agglomerative (Hierarchical) Clustering 516 Overview of Agglomerative Clustering Methods 516 Clustering People by Age: An Example of An Agglomerative Clustering Algorithm 520 Scoring Agglomerative Clusters 522 Limitations of Agglomerative Clustering 523 Agglomerative Clustering in Practice 525 Combining Agglomerative Clustering and K-Means 526 Self-Organizing Maps 527 What Is a Self-Organizing Map? 527 Training an SOM 530 Scoring an SOM 531 The Search Continues for Islands of Simplicity 532 Lessons Learned 533 Chapter 15 Market Basket Analysis and Association Rules 535 Defining Market Basket Analysis 536 Four Levels of Market Basket Data 537 The Foundation of Market Basket Analysis: Basic Measures 539 Order Characteristics 540 Item (Product) Popularity 541 Tracking Marketing Interventions 542 Case Study: Spanish or English 543 The Business Problem 543 The Data 544 Defining “Hispanicity” Preference 545 The Solution 546 Association Analysis 547 Rules Are Not Always Useful 548 Item Sets to Association Rules 551 How Good Is an Association Rule? 553 Building Association Rules 555 Choosing the Right Set of Items 556 Anonymous Versus Identified 561 Generating Rules from All This Data 561 Overcoming Practical Limits 565 The Problem of Big Data 567 Extending the Ideas 569 Different Items on the Right- and Left-Hand Sides 569 Using Association Rules to Compare Stores 570 Association Rules and Cross-Selling 572 A Typical Cross-Sell Model 572 A More Confident Approach to Product Propensities 573 Results from Using Confidence 574 Sequential Pattern Analysis 574 Finding the Sequences 575 Sequential Association Rules 578 Sequential Analysis Using Other Data Mining Techniques 579 Lessons Learned 579 Chapter 16 Link Analysis 581 Basic Graph Theory 582 What Is a Graph? 582 Directed Graphs 584 Weighted Graphs 585 Seven Bridges of Königsberg 585 Detecting Cycles in a Graph 588 The Traveling Salesman Problem Revisited 589 Social Network Analysis 593 Six Degrees of Separation 593 What Your Friends Say About You 595 Finding Childcare Benefits Fraud 596 Who Responds to Whom on Dating Sites 597 Social Marketing 598 Mining Call Graphs 598 Case Study: Tracking Down the Leader of the Pack 601 The Business Goal 601 The Data Processing Challenge 601 Finding Social Networks in Call Data 602 How the Results Are Used for Marketing 602 Estimating Customer Age 603 Case Study: Who Is Using Fax Machines from Home? 604 Why Finding Fax Machines Is Useful 604 How Do Fax Machines Behave? 604 A Graph Coloring Algorithm 605 “Coloring” the Graph to Identify Fax Machines 606 How Google Came to Rule the World 607 Hubs and Authorities 608 The Details 609 Hubs and Authorities in Practice 611 Lessons Learned 612 Chapter 17 Data Warehousing, OLAP, Analytic Sandboxes, and Data Mining 613 The Architecture of Data 615 Transaction Data, the Base Level 616 Operational Summary Data 617 Decision-Support Summary Data 617 Database Schema/Data Models 618 Metadata 623 Business Rules 623 A General Architecture for Data Warehousing 624 Source Systems 624 Extraction, Transformation, and Load 626 Central Repository 627 Metadata Repository 630 Data Marts 630 Operational Feedback 631 Users and Desktop Tools 631 Analytic Sandboxes 633 Why Are Analytic Sandboxes Needed? 634 Technology to Support Analytic Sandboxes 636 Where Does OLAP Fit In? 639 What’s in a Cube? 641 Star Schema 646 OLAP and Data Mining 648 Where Data Mining Fits in with Data Warehousing 650 Lots of Data 651 Consistent, Clean Data 651 Hypothesis Testing and Measurement 652 Scalable Hardware and RDBMS Support 653 Lessons Learned 653 Chapter 18 Building Customer Signatures 655 Finding Customers in Data 656 What Is a Customer? 657 Accounts? Customers? Households? 658 Anonymous Transactions 658 Transactions Linked to a Card 659 Transactions Linked to a Cookie 659 Transactions Linked to an Account 660 Transactions Linked to a Customer 661 Designing Signatures 661 Is a Customer Signature Necessary? 666 What Does a Row Represent? 666 Will the Signature Be Used for Predictive Modeling? 671 Has a Target Been Defined? 672 Are There Constraints Imposed by the Particular Data Mining Techniques to be Employed? 672 Which Customers Will Be Included? 673 What Might Be Interesting to Know About Customers? 673 What a Signature Looks Like 674 Process for Creating Signatures 677 Some Data Is Already at the Right Level of Granularity 678 Pivoting a Regular Time Series 679 Aggregating Time-Stamped Transactions 680 Dealing with Missing Values 685 Missing Values in Source Data 685 Unknown or Non-Existent? 687 What Not to Do 687 Things to Consider 689 Lessons Learned 691 Chapter 19 Derived Variables: Making the Data Mean More 693 Handset Churn Rate as a Predictor of Churn 694 Single-Variable Transformations 696 Standardizing Numeric Variables 696 Turning Numeric Values into Percentiles 697 Turning Counts into Rates 698 Relative Measures 699 Replacing Categorical Variables with Numeric Ones 700 Combining Variables 707 Classic Combinations 707 Combining Highly Correlated Variables 710 Rent to Home Value 712 Extracting Features from Time Series 718 Trend 719 Seasonality 721 Extracting Features from Geography 722 Geocoding 722 Mapping 723 Using Geography to Create Relative Measures 724 Using Past Values of the Target Variable 725 Using Model Scores as Inputs 725 Handling Sparse Data 726 Account Set Patterns 726 Binning Sparse Values 727 Capturing Customer Behavior from Transactions 727 Widening Narrow Data 728 Sphere of Influence as a Predictor of Good Customers 728 An Example: Ratings to Rater Profile 730 Sample Fields from the Rater Signature 730 The Rating Signature and Derived Variables 732 Lessons Learned 733 Chapter 20 Too Much of a Good Thing? Techniques for Reducing the Number of Variables 735 Problems with Too Many Variables 736 Risk of Correlation Among Input Variables 736 Risk of Overfitting 738 The Sparse Data Problem 738 Visualizing Sparseness 739 Independence 740 Exhaustive Feature Selection 743 Flavors of Variable Reduction Techniques 744 Using the Target 744 Original versus New Variables 744 Sequential Selection of Features 745 The Traditional Forward Selection Methodology 745 Forward Selection Using a Validation Set 747 Stepwise Selection 748 Forward Selection Using Non-Regression Techniques 748 Backward Selection 748 Undirected Forward Selection 749 Other Directed Variable Selection Methods 749 Using Decision Trees to Select Variables 750 Variable Reduction Using Neural Networks 752 Principal Components 753 What Are Principal Components? 753 Principal Components Example 758 Principal Component Analysis 763 Factor Analysis 767 Variable Clustering 768 Example of Variable Clusters 768 Using Variable Clusters 770 Hierarchical Variable Clustering 770 Divisive Variable Clustering 773 Lessons Learned 774 Chapter 21 Listen Carefully to What Your Customers Say: Text Mining 775 What Is Text Mining? 776 Text Mining for Derived Columns 776 Beyond Derived Features 777 Text Analysis Applications 778 Working with Text Data 781 Sources of Text 781 Language Effects 782 Basic Approaches to Representing Documents 783 Representing Documents in Practice 784 Documents and the Corpus 786 Case Study: Ad Hoc Text Mining 786 The Boycott 787 Business as Usual 787 Combining Text Mining and Hypothesis Testing 787 The Results 788 Classifying News Stories Using MBR 789 What Are the Codes? 789 Applying MBR 790 The Results 793 From Text to Numbers 794 Starting with a “Bag of Words” 794 Term-Document Matrix 796 Corpus Effects 797 Singular Value Decomposition (SVD) 798 Text Mining and Naïve Bayesian Models 800 Naïve Bayesian in the Text World 801 Identifying Spam Using Naïve Bayesian 801 Sentiment Analysis 806 DIRECTV: A Case Study in Customer Service 809 Background 809 Applying Text Mining 811 Taking the Technical Approach 814 Not an Iterative Process 818 Continuing to Benefit 818 Lessons Learned 819 Index 821

Read more less

Book SynopsisThis book offers a comprehensive reference volume to the use of mobile and pervasive computing in construction.Table of ContentsContributors ix Preface xiii Acknowledgments xvii 1 Mobile and Pervasive Computing in Construction: an Introduction 1Chimay J. Anumba and Xiangyu Wang 1.1 Background 1 1.2 Fundamental Characteristics of Mobile Computing 2 1.3 Pervasive Computing 7 1.4 Summary 9 References 9 2 Mobile and Semantic Web-Based Delivery of Context-Aware Information and Services in Construction 11Chimay J. Anumba, Zeeshan Aziz and Darshan Ruikar 2.1 Introduction 11 2.2 Limitations of Current Processes and Technologies 12 2.3 Integrated Service Delivery Architecture 15 2.4 Prototype System Implementation 17 2.5 Development of the Project Repository 18 2.6 OntoWise 19 2.7 Deployment Case Studies 21 2.8 Summary and Conclusions 24 References 25 3 Communication Technology in Mobile and Pervasive Computing 26Jerker Delsing 3.1 Introduction 26 3.2 Mobile and Pervasive Devices 26 3.3 Communication Basics 27 3.4 Communication Protocols 31 3.5 Service Protocols 33 3.6 Proprietary Buses and Protocols 34 3.7 Summary 35 References 35 4 A Framework for Designing Mobile Virtual Training Systems through Virtual Modeling Technology 37Xiangyu Wang and Phillip S. Dunston 4.1 Introduction 37 4.2 Taxonomy for Defining Virtual Training Systems 39 4.3 Relating Virtual Technologies to Training Skills 47 4.4 Conclusions and Future Work 51 References 52 5 Mobile and Pervasive Construction Visualization Using Outdoor Augmented Reality 54 Amir H. Behzadan, Suyang Dong and Vineet R. Kamat 5.1 Introduction 54 5.2 Prior Related Work in Construction Visualization 56 5.3 Main Contributions 57 5.4 Technical Approach to Create AR Animations 58 5.5 ARVISCOPE Animation Authoring Language 60 5.6 Creating an AR Animation Trace File from a DES Model 63 5.7 ARVISCOPE Language Design Issues 66 5.8 Examples of Pervasive Outdoor AR Visualization 76 5.9 Summary and Conclusions 82 Acknowledgments 83 References 83 6 Ubiquitous User Localization for Pervasive Context-Aware Construction Applications 86Hiam M. Khoury, Manu Akula and Vineet R. Kamat 6.1 Introduction 86 6.2 Current State of Knowledge 88 6.3 User Tracking in Construction Environments 92 6.4 Validation of Accuracy in 3D Spatial User Tracking 106 6.5 Integration of GPS and Inertial Navigation 116 6.6 Summary and Conclusions 124 Acknowledgments 124 References 125 7 Person-oriented Mobile Information System Enhancing Engineering Communication in Construction Processes 128Danijel Rebolj and Ales Magdic 7.1 Introduction 128 7.2 Considering People in Processes 131 7.3 Dynamic Communication Environment (DyCE) 134 7.4 On-site Evaluation 139 7.5 Conclusions 144 7.6 Future Work 144 References 146 8 The iHelmet: An AR-enhanced Wearable Display for BIM Information 149Kai-Chen Yeh, Meng-Han Tsai and Shih-Chung Kang 8.1 Introduction 149 8.2 Design and Implementation of the iHelmet 153 8.3 Module Implementations 157 8.4 Discussion 163 8.5 Summary 164 References 165 9 Mobile and Pervasive Computing: The Future for Design Collaboration 169mi Jeong Kim, Mary Lou Maher and Ning Gu 9.1 Introduction 169 9.2 Analytical Frameworks for Understanding Collaborative Technologies in Design 170 9.3 Characterizing Early Collaborative Design Technologies 172 9.4 Understanding Mobile and Pervasive Computing in Design Collaboration 177 9.5 Towards the Future 182 9.6 Conclusion 184 References 185 10 Computer Vision and Pattern Recognition Technologies for Construction 189Ioannis Brilakis 10.1 Structural Element Recognition 189 10.2 Construction Equipment and Personnel Recognition 193 10.3 Damage and Defects Recognition 196 10.4 Videogrammetric Surveying 199 10.5 Summary 203 References 204 11 Structural Health Monitoring using Wireless Sensor Networks 210Jiannong Cao and Xuefeng Liu 11.1 Introduction 210 11.2 How to Realize Long-Term Monitoring with WSNs using Battery-Powered Wireless Sensor Nodes 219 11.3 How to Implement Simple and Effective SHM Algorithms 224 11.4 How to Realize Fast and Reliable Delivery of a Large Amount of Data 228 11.5 How to Deploy Sensor Nodes in WSN-based SHM System 229 11.6 How to Develop Middleware Framework for WSN-based SHM 230 11.7 Conclusion 233 Acknowledgments 233 References 233 12 Cloud Computing Support for Construction Collaboration 237Jack C.P. Cheng and Bimal Kumar 12.1 Introduction 237 12.2 What is Cloud Computing? 239 12.3 Cloud Computing as a Construction Collaboration Enabling Technology 243 12.4 Potential Benefits of Cloud Computing in the Construction Industry 244 12.5 Challenges of Cloud Computing Adoption in the Construction Industry 247 12.6 Proposed Collaboration Framework 250 12.7 Summary 252 References 252 13 Concluding Notes 255Chimay J. Anumba and Xiangyu Wang 13.1 Introduction 255 13.2 Summary 255 13.3 Benefits of Mobile and Pervasive Computing to Construction Sector Organizations 256 13.4 Considerations in the Effective Deployment of Mobile and Pervasive Computing in Construction 257 13.5 Future Directions 257 References 259 Index 261

Read more less

Book SynopsisIt is common to blame the inadequacy of credit risk models for the fact that the financial crisis has caught many market participants by surprise. On closer inspection, though, it often appears that market participants failed to understand or to use the models correctly.Table of ContentsPreface to the 2nd edition xi Preface to the 1st edition xiii Some Hints for Troubleshooting xv 1 Estimating Credit Scores with Logit 1 Linking scores, default probabilities and observed default behavior 1 Estimating logit coefficients in Excel 4 Computing statistics after model estimation 8 Interpreting regression statistics 10 Prediction and scenario analysis 12 Treating outliers in input variables 16 Choosing the functional relationship between the score and explanatory variables 20 Concluding remarks 25 Appendix 25 Logit and probit 25 Marginal effects 25 Notes and literature 26 2 The Structural Approach to Default Prediction and Valuation 27 Default and valuation in a structural model 27 Implementing the Merton model with a one-year horizon 30 The iterative approach 30 A solution using equity values and equity volatilities 35 Implementing the Merton model with a T -year horizon 39 Credit spreads 43 CreditGrades 44 Appendix 50 Notes and literature 52 Assumptions 52 Literature 53 3 Transition Matrices 55 Cohort approach 56 Multi-period transitions 61 Hazard rate approach 63 Obtaining a generator matrix from a given transition matrix 69 Confidence intervals with the binomial distribution 71 Bootstrapped confidence intervals for the hazard approach 74 Notes and literature 78 Appendix 78 Matrix functions 78 4 Prediction of Default and Transition Rates 83 Candidate variables for prediction 83 Predicting investment-grade default rates with linear regression 85 Predicting investment-grade default rates with Poisson regression 88 Backtesting the prediction models 94 Predicting transition matrices 99 Adjusting transition matrices 100 Representing transition matrices with a single parameter 101 Shifting the transition matrix 103 Backtesting the transition forecasts 108 Scope of application 108 Notes and literature 110 Appendix 110 5 Prediction of Loss Given Default 115 Candidate variables for prediction 115 Instrument-related variables 116 Firm-specific variables 117 Macroeconomic variables 118 Industry variables 118 Creating a data set 119 Regression analysis of LGD 120 Backtesting predictions 123 Notes and literature 126 Appendix 126 6 Modeling and Estimating Default Correlations with the Asset Value Approach 131 Default correlation, joint default probabilities and the asset value approach 131 Calibrating the asset value approach to default experience: the method of moments 133 Estimating asset correlation with maximum likelihood 136 Exploring the reliability of estimators with a Monte Carlo study 144 Concluding remarks 147 Notes and literature 147 7 Measuring Credit Portfolio Risk with the Asset Value Approach 149 A default-mode model implemented in the spreadsheet 149 VBA implementation of a default-mode model 152 Importance sampling 156 Quasi Monte Carlo 160 Assessing Simulation Error 162 Exploiting portfolio structure in the VBA program 165 Dealing with parameter uncertainty 168 Extensions 170 First extension: Multi-factor model 170 Second extension: t-distributed asset values 171 Third extension: Random LGDs 173 Fourth extension: Other risk measures 175 Fifth extension: Multi-state modeling 177 Notes and literature 179 8 Validation of Rating Systems 181 Cumulative accuracy profile and accuracy ratios 182 Receiver operating characteristic (ROC) 185 Bootstrapping confidence intervals for the accuracy ratio 187 Interpreting caps and ROCs 190 Brier score 191 Testing the calibration of rating-specific default probabilities 192 Validation strategies 195 Testing for missing information 198 Notes and literature 201 9 Validation of Credit Portfolio Models 203 Testing distributions with the Berkowitz test 203 Example implementation of the Berkowitz test 206 Representing the loss distribution 207 Simulating the critical chi-square value 209 Testing modeling details: Berkowitz on subportfolios 211 Assessing power 214 Scope and limits of the test 216 Notes and literature 217 10 Credit Default Swaps and Risk-Neutral Default Probabilities 219 Describing the term structure of default: PDs cumulative, marginal and seen from today 220 From bond prices to risk-neutral default probabilities 221 Concepts and formulae 221 Implementation 225 Pricing a CDS 232 Refining the PD estimation 234 Market values for a CDS 237 Example 239 Estimating upfront CDS and the ‘Big Bang’ protocol 240 Pricing of a pro-rata basket 241 Forward CDS spreads 242 Example 243 Pricing of swaptions 243 Notes and literature 247 Appendix 247 Deriving the hazard rate for a CDS 247 11 Risk Analysis and Pricing of Structured Credit: CDOs and First-to-Default Swaps 249 Estimating CDO risk with Monte Carlo simulation 249 The large homogeneous portfolio (LHP) approximation 253 Systemic risk of CDO tranches 256 Default times for first-to-default swaps 259 CDO pricing in the LHP framework 263 Simulation-based CDO pricing 272 Notes and literature 281 Appendix 282 Closed-form solution for the LHP model 282 Cholesky decomposition 283 Estimating PD structure from a CDS 284 12 Basel II and Internal Ratings 285 Calculating capital requirements in the Internal Ratings-Based (IRB) approach 285 Assessing a given grading structure 288 Towards an optimal grading structure 294 Notes and literature 297 Appendix A1 Visual Basics for Applications (VBA) 299 Appendix A2 Solver 307 Appendix A3 Maximum Likelihood Estimation and Newton’s Method 313 Appendix A4 Testing and Goodness of Fit 319 Appendix A5 User-defined Functions 325 Index 333

Read more less

Book SynopsisSteganography is the art of communicating a secret message, hiding the very existence of a secret message. This book is an introduction to steganalysis as part of the wider trend of multimedia forensics, as well as a practical tutorial on machine learning in this context.Table of ContentsPreface xi PART I OVERVIEW 1 Introduction 3 1.1 Real Threat or Hype? 3 1.2 Artificial Intelligence and Learning 4 1.3 How to Read this Book 5 2 Steganography and Steganalysis 7 2.1 Cryptography versus Steganography 7 2.2 Steganography 8 2.3 Steganalysis 17 2.4 Summary and Notes 23 3 Getting Started with a Classifier 25 3.1 Classification 25 3.2 Estimation and Confidence 28 3.3 Using libSVM 30 3.4 Using Python 33 3.5 Images for Testing 38 3.6 Further Reading 39 PART II FEATURES 4 Histogram Analysis 43 4.1 Early Histogram Analysis 43 4.2 Notation 44 4.3 Additive Independent Noise 44 4.4 Multi-dimensional Histograms 54 4.5 Experiment and Comparison 63 5 Bit-plane Analysis 65 5.1 Visual Steganalysis 65 5.2 Autocorrelation Features 67 5.3 Binary Similarity Measures 69 5.4 Evaluation and Comparison 72 6 More Spatial Domain Features 75 6.1 The Difference Matrix 75 6.2 Image Quality Measures 82 6.3 Colour Images 86 6.4 Experiment and Comparison 86 7 The Wavelets Domain 89 7.1 A Visual View 89 7.2 The Wavelet Domain 90 7.3 Farid’s Features 96 7.4 HCF in the Wavelet Domain 98 7.5 Denoising and the WAM Features 101 7.6 Experiment and Comparison 106 8 Steganalysis in the JPEG Domain 107 8.1 JPEG Compression 107 8.2 Histogram Analysis 114 8.3 Blockiness 122 8.4 Markov Model-based Features 124 8.5 Conditional Probabilities 126 8.6 Experiment and Comparison 128 9 Calibration Techniques 131 9.1 Calibrated Features 131 9.2 JPEG Calibration 133 9.3 Calibration by Downsampling 137 9.4 Calibration in General 146 9.5 Progressive Randomisation 148 PART III CLASSIFIERS 10 Simulation and Evaluation 153 10.1 Estimation and Simulation 153 10.2 Scalar Measures 158 10.3 The Receiver Operating Curve 161 10.4 Experimental Methodology 170 10.5 Comparison and Hypothesis Testing 173 10.6 Summary 176 11 Support Vector Machines 179 11.1 Linear Classifiers 179 11.2 The Kernel Function 186 11.3 ν-SVM 189 11.4 Multi-class Methods 191 11.5 One-class Methods 192 11.6 Summary 196 12 Other Classification Algorithms 197 12.1 Bayesian Classifiers 198 12.2 Estimating Probability Distributions 203 12.3 Multivariate Regression Analysis 209 12.4 Unsupervised Learning 212 12.5 Summary 215 13 Feature Selection and Evaluation 217 13.1 Overfitting and Underfitting 217 13.2 Scalar Feature Selection 220 13.3 Feature Subset Selection 222 13.4 Selection Using Information Theory 225 13.5 Boosting Feature Selection 238 13.6 Applications in Steganalysis 239 14 The Steganalysis Problem 245 14.1 Different Use Cases 245 14.2 Images and Training Sets 250 14.3 Composite Classifier Systems 258 14.4 Summary 262 15 Future of the Field 263 15.1 Image Forensics 263 15.2 Conclusions and Notes 265 Bibliography 267 Index 279

Read more less

Book SynopsisThis book focuses heavily on the principles, analysis and applications of code-division multiple-access (CDMA) techniques in optical communication systems and networks. In this book, the authors intimately discuss modern optical networks and their applications in current and emerging communication technologies, evaluating the quality, speed and number of supported services. In particular, principles and fundamentals of optical CDMA techniques from beginner to advanced levels are heavily covered. Furthermore, the authors concentrate on methods and techniques of various encoding and decoding schemes and their structures, as well as analysis of optical CDMA systems with various transceiver models including advanced multi-level incoherent and coherent modulations with the architecture of access/aggregation networks in mind. Moreover, authors examine intriguing topics of optical CDMA networking, compatibility with IP networks, and implementation of optical multi-rate multiTable of ContentsList of Figures xiii List of Tables xxv Preface xxvii Acknowledgements xxxiii 1 Introduction to Optical Communications 1 1.1 Evolution of Lightwave Technology 1 1.2 Laser Technologies 3 1.3 Optical Fibre Communication Systems 4 1.4 Lightwave Technology in Future 7 1.5 Optical Lightwave Spectrum 7 1.6 Optical Fibre Transmission 9 1.7 Multiple Access Techniques 10 1.8 Spread Spectrum Communications Techniques 14 1.9 Motivations for Optical CDMA Communications 21 1.10 Access Networks Challenges 22 1.11 Summary 23 References 24 2 Optical Spreading Codes 29 2.1 Introduction 29 2.2 Bipolar Codes 30 2.3 Unipolar Codes: Optical Orthogonal Codes 37 2.4 Unipolar Codes: Prime Code Families 41 2.5 Codes with Ideal In-Phase Cross-Correlation 62 2.6 Multidimensional Optical Codes 76 2.7 Channel Encoding in OCDMA Systems 84 2.8 Turbo-Coded Optical CDMA 100 2.9 Summary 110 References 111 3 Optical CDMA Review 115 3.1 Introduction 115 3.2 Optical Coding Principles 115 3.3 OCDMA Networking: Users Are Codes 117 3.4 Optical CDMA Techniques 119 3.5 Free-Space and Atmospheric Optical CDMA 126 3.6 Summary 128 References 128 4 Spectrally Encoded OCDMA Networks 133 4.1 Introduction 133 4.2 Spectral-Amplitude-Coding Schemes 134 4.3 System Considerations 141 4.4 Gaussian Approach Analysis 144 4.5 Negative Binomial Approach Analysis 153 4.6 Spectral-Phase-Coding Schemes 164 4.7 Summary 167 References 167 5 Incoherent Temporal OCDMA Networks 171 5.1 Introduction 171 5.2 PPM-OCDMA Signalling 172 5.3 PPM-OCDMA Transceiver Architecture 173 5.4 PPM-OCDMA Performance Analysis 180 5.5 Discussion of Results 183 5.6 Overlapping PPM-OCDMA Signalling 187 5.7 OPPM-OCDMA Transceiver Architecture 188 5.8 OPPM-OCDMA Performance Analysis 196 .9 Discussion of Results 203 5.10 Analysis of Throughput 209 5.11 Summary 211 References 211 6 Coherent Temporal OCDMA Networks 213 6.1 Introduction 213 6.2 Coherent Homodyne BPSK-OCDMA Architecture 214 6.3 Coherent Heterodyne BPSK-OCDMA Architecture 222 6.4 Summary 229 References 230 7 Hybrid Temporal Coherent and Incoherent OCDMA Networks 231 7.1 Introduction 231 7.2 Coherent Transmitter with Incoherent Receiver 232 7.3 Analysis of Transceivers with MAI Cancellation 235 7.4 Results and Throughput Analysis 239 7.5 Summary 244 References 244 8 Optical CDMA with Polarization Modulations 245 8.1 Introduction 245 8.2 Optical Polarization Shift Keying (PolSK) 247 8.3 PolSK-OCDMA Transceiver Architecture 254 8.4 Evaluation of PolSK-OCDMA Transceiver Performance 263 8.5 Transceiver Architecture for Hybrid F-PolSK-OCDMA 265 8.6 Performance of F-PolSK-OCDMA Transceiver 273 8.7 Long-Haul PolSK Transmission 273 8.8 Summary 278 References 278 9 Optical CDMA Networking 281 9.1 Introduction 281 9.2 OCDMA-PON 289 9.3 OCDMA-PON Architecture 290 9.4 IP Traffic over OCDMA Networks 299 9.5 Random Access Protocols 308 9.6 Multi-Protocol Label Switching 330 9.7 Summary 342 References 344 10 Services Differentiation and Quality of Services in Optical CDMA Networks 347 10.1 Introduction 347 10.2 Differentiated Services in Optical CDMA 351 10.3 Variable-Weight Optical Spreading Codes 354 10.4 Variable-Length Optical Spreading Codes 364 10.5 Multirate Differentiated Services in OCDMA Networks 376 10.6 Summary 383 References 384 Index 387

Read more less

Book SynopsisThe goal of this book is to teach undergraduate students how to use Scientific Notebook (SNB) to solve physics problems. SNB software combines word processing and mathematics in standard notation with the power of symbolic computation. As its name implies, SNB can be used as a notebook in which students set up a math or science problem, write and solve equations, and analyze and discuss their results. Written by a physics teacher with over 20 years experience, this text includes topics that have educational value, fit within the typical physics curriculum, and show the benefits of using SNB. This easy-to-read text: Provides step-by-step instructions for using Scientific Notebook (SNB) to solve physics problems Features examples in almost every section to enhance the reader''s understanding of the relevant physics and to provide detailed instructions on using SNB Follows the traditional physics cuTable of ContentsPreface xv So we’re all on the same page... xvii What is science? xviii To the Student xix To the Teacher xx Contact Information xx Acknowledgments xxi 1 Introduction to SNB 1 Why SNB? 1 The Basics 2 Physics à la mode: Math or Text 8 Creating Mathematical Expressions 8 Evaluate and Evaluate Numerically 11 Scientific Notation 13 Substitution and Endpoint Evaluation 14 Solving Equations 17 Solve Exact 18 Solve Numeric 21 Systems of Equations 24 The Compute Menu 25 Simplify and Expand 25 Factor 26 Rewrite and Combine 28 Check Equality 29 Polynomials 31 Power Series 32 Definitions 35 Other Good Stuff 37 Computing In-place 37 Making Assumptions About Variables 37 Limits 40 A Few Words About Calculus 42 Units 46 Converting Units 47 User-Defined Units 51 Plotting 52 Plot 2D Rectangular 54 Other 2-Dimensional Plots 55 Plot 3D Rectangular 58 Cylindrical and Spherical Plots 60 Plotting Data 63 Fitting a Curve to Data 63 Differential Equations 67 Solve ODE Exact and Laplace 68 Solve ODE Numeric 70 Problems 75 2 One-Dimensional Kinematics 83 Constant Acceleration 83 Displacement and Position 83 Velocity and Acceleration 84 Equations of Motion 86 Signs of the Times 88 Free Fall 89 Varying Acceleration 91 Displacement, Velocity, and Acceleration 91 Equations of Motion 93 Gravity and Air Resistance 96 Resisting Air Resistance is Futile 97 Long-Distance Free Fall 99 Problems 102 3 Vectors 105 Components of a Vector 107 Magnitude and Direction 108 Adding Vectors 111 The Component Method 112 The SNB Method 113 The Graphing Method 115 Unit Vectors 119 Multiplying Vectors 120 Dot Product 121 Cross Product 122 Problems 125 4 Projectile Motion 127 No Air Resistance 127 Trajectory 132 Time of Flight 134 Maximum Height 135 Linear Air Resistance 137 Trajectory 141 Time of Flight and Range 143 Maximum Height 145 Turn Off the Air! 146 Turn Down the Air! 147 Quadratic Air Resistance 151 Height-Dependent Air Resistance 152 Problems 154 5 Newton’s Laws of Motion 157 Newton’s First Law 157 Newton’s Second Law for Constant Forces 158 Newton’s Second Law for Varying Forces 165 Time-Dependent Forces 165 Velocity-Dependent Forces 167 Position-Dependent Forces 170 Newton’s Third Law 173 Problems 175 6 Conservation Laws 179 Definitions 179 Conservation of Energy 181 Work 181 The Work-Energy Theorem 185 Potential Energy 186 Mechanical Energy is Conserved 188 A Complete Bookkeeping 191 Conservation of Momentum 193 Collisions in 1-Dimension 193 Collisions in 2-Dimensions 196 Rockets 199 Deep Space 199 Launch 202 Air Resistance 207 Varying Gravity and Air Resistance 213 Problems 216 7 Circular Motion 221 Uniform Circular Motion 222 The Rotating Umbrella 224 Rotational Kinematics 227 The Compact Disk 229 Newton’s Second Law and Circular Motion 233 Uniform Circular Motion and the 2nd Law 233 Non-Uniform Circular Motion and the 2nd Law 235 Sliding on a Sphere 236 Problems 248 8 Harmonic Motion 251 Simple Harmonic Motion, Simply 251 Energy and SHM 254 Not-Quite-as-Simple Harmonic Motion 255 Energy and SHM, Again 257 Damped Harmonic Motion 259 Underdamped (β2 < ω20) 259 Critically Damped (β2 = ω20) 261 Overdamped (β2 > ω20) 262 Driven Harmonic Motion 263 Constant Driving Force, no Damping 263 Sinusoidal Driving Force, no Damping 264 Constant Driving Force with Damping 265 Sinusoidal Driving Force with Damping 267 Small Oscillations 270 Not-so-Simple Harmonic Motion 272 Problems 275 9 Central Forces 279 Equations of Motion 279 Newtonian Gravitation 285 Kepler’s Laws 286 The Effective Potential 292 Two Special Forces 296 The 3-d Harmonic Oscillator 296 The Inverse-Square Force 299 Numerical Stuff 303 Problems 305 10 Fluids 309 Density and Pressure 309 Static Fluids 311 Buoyancy 312 Fluids in Motion 314 Bernoulli’s Equation 316 Applications of Bernoulli’s Equation 318 A More Realistic Approach 320 Flow in a Pipe 321 Stokes’ Law 330 Problems 331 11 Temperature and Heat 335 Temperature Scales 335 Absolute Temperature 337 Heat and Work 338 Heat Flow 339 Change in Temperature: Specific Heat 339 Change in State: Latent Heat 340 Calorimetry 341 Varying Specific Heat 344 The Specific Heat of Solids 345 Problems 353 12 Special Relativity 359 The Two Postulates 360 The Consequences 361 Time Dilation 363 Length Contraction 364 Addition of Velocities 365 Simultaneity 367 The Lorentz Transformation 367 Space-Time 370 Relativistic Momentum and Energy 375 Relativistic Collisions 378 Relativistic Dynamics 382 Four-Vectors 387 Problems 392 A Topics in Classical Physics 397 Newton’s Nose-Cone Problem 397 Simple Shapes 398 Frusta and Fudges 403 Newton’s Minimizer 409 Indented Tips and the Minimizer 411 The Shape of the Eiffel Tower 414 An Interesting Classical Orbit 417 Fisher’s Crystal 421 Problems 428 B Topics in Modern Physics 435 The Tale of the Traveling Triplets 435 Trip 1: Constance goes to Vega 435 Relativistic Interlude: Constant Acceleration 437 Trip 2: Axel goes to Vega 441 What happens on the way to Vega... 443 Orbits in General Relativity 445 Angular Momentum 447 Precessing Ellipses and Periodic Orbits 451 Be the Ball: Embedding Diagrams 456 Classical Lifetime of a Hydrogen Atom 460 Missed It By That Much 460 Can Special Relativity Save the Day? 462 Quantum Mechanical Bound States 465 Infinite Square Well (“Particle in a Box”) 467 Finite Square Well 470 V-shaped Linear Well 477 Problems 483 References and Suggested Reading 491 Index 495

Read more less

Book SynopsisRapid development in different fields of Digital Audio Effects (DAFX) has led to new algorithms. The Second Edition of DAFX - Digital Audio Effects investigates digital signal processing, its application to sound, and how its effects on sound can be used within music.Table of ContentsPreface. List of Contributors. 1 Introduction (V. Verfaille, M. Holters, U. Zölzer). 1.1 Digital Audio Effects DAFX with MATLAB. 1.2 Classifications of DAFX. 1.3 Fundamentals of Digital Signal Processing. 1.4 Conclusion. Bibliography. 2 Filters and Delays (P. Dutilleux, M. Holters, S. Disch, U. Zölzer). 2.1 Introduction. 2.2 Basic Filters. 2.3 Equalizers. 2.4 Time-varying Filters. 2.5 Basic Delay Structures. 2.6 Delay-based Audio Effects. 2.7 Conclusion. Sound and Music. Bibliography. 3 Modulators and Demodulators (P. Dutilleux, M. Holters, S. Disch, U. Zölzer). 3.1 Introduction. 3.2 Modulators. 3.3 Demodulators. 3.4 Applications. 3.5 Conclusion. Sound and Music. Bibliography. 4 Nonlinear Processing (P. Dutilleux, K. Dempwolf, M. Holters, U. Zölzer). 4.1 Introduction. 4.2 Dynamic Range Control. 4.3 Musical Distortion and Saturation Effects. 4.4 Exciters and Enhancers. 4.5 Conclusion. Sound and Music. Bibliography. 5 Spatial Effects (V. Pulkki, T. Lokki, D. Rocchesso). 5.1 Introduction. 5.2 Concepts of spatial hearing. 5.3 Basic spatial effects for stereophonic loudspeaker and headphone playback. 5.4 Binaural techniques in spatial audio. 5.5 Spatial audio effects for multichannel loudspeaker layouts. 5.6 Reverberation. 5.7 Modeling of room acoustics. 5.8 Other spatial effects. 5.9 Conclusion. 5.10 Acknowledgements. References. 6 Time-Segment Processing (P. Dutilleux, G. De Poli, A. von dem Knesebeck, U. Zölzer). 6.1 Introduction. 6.2 Variable Speed Replay. 6.3 Time Stretching. 6.4 Pitch Shifting. 6.5 Time Shuffling and Granulation. 6.6 Conclusion. Sound and Music. References. 7 Time-Frequency Processing (D. Arfib, F. Keiler, U. Zölzer, V. Verfaille, J. Bonada). 7.1 Introduction. 7.2 Phase Vocoder Basics. 7.3 Phase Vocoder Implementations. 7.4 Phase Vocoder Effects. 7.5 Conclusion. References. 8 Source-Filter Processing (D. Arfib, F. Keiler, U. Zölzer, V. Verfaille). 8.1 Introduction. 8.2 Source-Filter Separation. 8.3 Source-Filter Transformations. 8.4 Conclusion. References. 9 Adaptive Digital Audio Effects (V. Verfaille, D. Arfib, F. Keiler, A. von dem Knesebeck, U. Zölzer). 9.1 Introduction. 9.2 Sound-Feature Extraction. 9.3 Mapping Sound Features to Control Parameters. 9.4 Examples of Adaptive DAFX. 9.5 Conclusions. References. 10 Spectral Processing (J. Bonada, X. Serra, X. Amatriain, A. Loscos). 10.1 Introduction. 10.2 Spectral Models. 10.3 Techniques. 10.4 Effects. 10.5 Conclusions. References. 11 Time and Frequency Warping-Musical Signals (G. Evangelista). 11.1 Introduction. 11.2 Warping. 11.3 Musical Uses of Warping. 11.4 Conclusion. References. 12 Virtual Analog Effects (V. Välimäki, S. Bilbao, J. O. Smith, J. S. Abel, J. Pakarinen, D. Berners). 12.1 Introduction. 12.2 Virtual Analog Filters. 12.3 Circuit-Based Valve Emulation. 12.4 Electromechanical Effects. 12.5 Tape-Based Echo Simulation. 12.6 Antiquing of Audio Files. 12.7 Conclusion. References. 13 Automatic Mixing (E. Perez-Gonzalez, J. D. Reiss). 13.1 Introduction. 13.2 AM-DAFX. 13.3 Cross-adaptive AM-DAFX. 13.4 AM-DAFX Implementations. 13.5 Conclusion. References. 14 Sound Source Separation (G. Evangelista, S. Marchand, M. D. Plumbley, E. Vincent). 14.1 Introduction. 14.2 Binaural Source Separation. 14.3 Source Separation from Single-Channel Signals. 14.4 Applications. 14.5 Conclusions. Acknowledgments. References. Glossary. Index.

Read more less

Book SynopsisThis new addition to the popular Essentials series provides a broad, general introduction to the topic of simulation within clinical education.Table of ContentsContributors vii Foreword x Glossary and abbreviations xii Features contained within your textbook xvi 1 Essential simulation in clinical education 1 Judy McKimm and Kirsty Forrest 2 Medical simulation: the journey so far 11 Aidan Byrne 3 The evidence: what works, why and how? 26 Doris Østergaard and Jacob Rosenberg 4 Pedagogy in simulation-based training in healthcare 43 Peter Dieckmann and Charlotte Ringsted 5 Assessment 59 Thomas Gale and Martin Roberts 6 The roles of faculty and simulated patients in simulation 87 Bryn Baxendale, Frank Coffey and Andrew Buttery 7 Surgical technical skills 111 Rajesh Aggarwal and Amit Mishra 8 The non-technical skills 131 Nikki Maran, Simon Edgar and Alistair May 9 Teamwork 146 Jennifer M. Weller 10 Designing effective simulation activities 168 Joanne Barrott, Ann B. Sunderland, Jane P. Nicklin and Michelle McKenzie Smith 11 Distributed simulation 196 Jessica Janice Tang, Jimmy Kyaw Tun, Roger L Kneebone and Fernando Bello 12 Providing effective simulation activities 213 Walter J. Eppich, Lanty O’Connor and Mark Adler 13 Simulation in practice 235 Jean Ker Simulation for learning cardiology 236 Ross J. Scalese Assessing leadership skills in medical undergraduates 238 Helen O’Sullivan, Arpan Guha and Michael Moneypenny Simulation for interprofessional learning 240 Stuart Marshall Use of in situ simulations to identify barriers to patient care for multidisciplinary teams in developing countries 242 Nicole Shilkofski Clinical skills assessment for paediatric postgraduate physicians 244 Joseph O. Lopreiato The challenge of doctors in difficulty: using simulated healthcare contexts to develop a national assessment programme 246 Kevin Stirling, Jean Ker and Fiona Anderson Simulation for remote and rural practice 250 Jerry Morse, Jean Ker and Sarah Race The use of incognito standardized patients in general practice 252 Jan-Joost Rethans Integration of simulation-based training for the trauma team in a university hospital 253 Anne-Mette Helsø and Doris Østergaard Conclusion 254 14 The future for simulation 258 Horizon scanning: the impact of technological change 259 Iliana Harrysson, Rajesh Aggarwal and Ara Darzi Guiding the role of simulation through paradigm shifts in medical education 267 Viren N. Naik and Stanley J. Hamstra The future of training in simulation 273 Ronnie Glavin Index 283

Read more less

Book SynopsisWritten by experts in the field, this book describes the Personal Network architecture and its various components This book focuses on networking and security aspects of Personal Networks (PNs). Given a single user, the authors propose an architecture for PNs in which devices are divided into one of two types of nodes: personal nodes and foreign nodes. Furthermore, the authors demonstrate the ways in which PNs can be formed in a self-organized and secure way, how they can be interconnected using infrastructure networks, how multiple PNs can be connected, and how their services and resources can be shared. In addition, the book shows how security and ease-of-use can be achieved through automatic configuration and how mobility can be supported through adaptability and self-organization. The motivations for the PN concept, the PN architecture, its functionalities and features, as well as future challenges are covered in depth. Finally, the authors consider the potential applicaTable of ContentsForeword. Preface. List of Abbreviations. 1 The Vision of Personal Networks. 1.1 Past, Present, and Future Telecommunication. 1.2 Personal Networks. 1.3 Some Typical PN Use-Case Scenarios. 1.4 Federations of Personal Networks. 1.5 Early Personal Network Implementations. 1.6 Expected Impact. 1.7 Summary. 2 Personal Networks User Requirements. 2.1 Ubiquitous Networking. 2.2 Heterogeneous Hardware Constraints. 2.3 Quality of Service and Reliability. 2.4 Name, Service, and Content Management. 2.5 Context Awareness. 2.6 Being Cognitive. 2.7 Security and Trust. 2.8 Privacy. 2.9 Usability. 2.10 Other Requirements. 2.11 Jane Revisited. 2.12 Summary. 3 Trends in Personal Networks. 3.1 Wireless Communications. 3.2 Ad Hoc Networking. 3.3 WWRF Book of Visions. 3.4 Ubiquitous and Pervasive Computing and Communication. 3.5 Ambient Networks. 3.6 IST PACWOMAN and SHAMAN. 3.7 Personal Distributed Environment. 3.8 MyNet. 3.9 P2P Universal Computing Consortium. 3.10 More Trends. 3.11 Personal Networks and Current Trends. 3.12 Summary. 4 The Personal Network Architecture. 4.1 Terminology. 4.2 Personal and Foreign Nodes. 4.3 The Three Level Architecture View. 4.4 Personalization of Nodes. 4.5 Cluster Organization. 4.6 Personal Network Organization. 4.7 Foreign Communication. 4.8 Higher Layer Support Systems. 4.9 Federations of Personal Networks. 4.10 Discussion. 4.11 Summary. 5 Cluster Formation and Routing. 5.1 What is a Cluster? 5.2 Mobile Ad Hoc Network Technologies. 5.3 Cluster Formation and Maintenance. 5.4 Intra-Cluster Routing. 5.5 Summary. 6 Inter-Cluster Tunneling and Routing. 6.1 Inter-Cluster Tunneling Requirements. 6.2 IP Mobility. 6.3 PN Addressing. 6.4 Infrastructure Support. 6.5 Inter-Cluster Tunneling. 6.6 Inter-Cluster Routing. 6.7 Summary. 7 Foreign Communication. 7.1 Requirements for Foreign Communication. 7.2 Setting up Communication with Foreign Nodes. 7.3 Bridging Inside and Outside Protocols. 7.4 Mobility and Gateway Node Handover. 7.5 Summary. 8 Personal Network Application Support Systems. 8.1 Required PN Application Support. 8.2 Design of a PN Application Support System. 8.3 Service Discovery and Management Implementation. 8.4 An Implementation of Context Management. 8.5 Summary. 9 Personal Network Security. 9.1 Device Personalization. 9.2 Establishment of Secure Communication. 9.3 Secure Foreign Communication. 9.4 Anonymity. 9.5 Summary. 10 Personal Network Federations. 10.1 Examples. 10.2 Types of Federations. 10.3 Requirements. 10.4 Architecture of a Federation. 10.5 Life Cycle of a Federation. 10.6 Federation Access Control. 10.7 Federation Implementation Approaches. 10.8 Security. 10.9 Summary. 11 Personal Network Prototypes. 11.1 The TU Delft Prototype. 11.2 The PNP2008 Prototypes. 11.3 The MAGNET Prototype. 11.4 Summary. 12 The Future of Personal Networks. 12.1 Are We There Yet? 12.2 Future Directions. Appendix A Terminology. A.1 Connectivity Abstraction Level. A.2 Network Abstraction Level. A.3 Application and Service Abstraction Level. A.4 Personal Network Federations. References. Related Websites. Index.

Read more less

Book SynopsisStatistical pattern recognition relates to the use of statistical techniques for analysing data measurements in order to extract information and make justified decisions. It is a very active area of study and research, which has seen many advances in recent years. Applications such as data mining, web searching, multimedia data retrieval, face recognition, and cursive handwriting recognition, all require robust and efficient pattern recognition techniques. This third edition provides an introduction to statistical pattern theory and techniques, with material drawn from a wide range of fields, including the areas of engineering, statistics, computer science and the social sciences. The book has been updated to cover new methods and applications, and includes a wide range of techniques such as Bayesian methods, neural networks, support vector machines, feature selection and feature reduction techniques.Technical descriptions and motivations are provided, and the techniques are illustTrade Review“In the end I must add that this book is so appealing that I often found myself lost in the reading, pausing the overview of the manuscript in order to look more into some presented subject, and not being able to continue until I had finished seeing all about it.” (Zentralblatt MATH, 1 December 2012)Table of ContentsPreface xix Notation xxiii 1 Introduction to Statistical Pattern Recognition 1 1.1 Statistical Pattern Recognition 1 1.1.1 Introduction 1 1.1.2 The Basic Model 2 1.2 Stages in a Pattern Recognition Problem 4 1.3 Issues 6 1.4 Approaches to Statistical Pattern Recognition 7 1.5 Elementary Decision Theory 8 1.5.1 Bayes’ Decision Rule for Minimum Error 8 1.5.2 Bayes’ Decision Rule for Minimum Error – Reject Option 12 1.5.3 Bayes’ Decision Rule for Minimum Risk 13 1.5.4 Bayes’ Decision Rule for Minimum Risk – Reject Option 15 1.5.5 Neyman–Pearson Decision Rule 15 1.5.6 Minimax Criterion 18 1.5.7 Discussion 19 1.6 Discriminant Functions 20 1.6.1 Introduction 20 1.6.2 Linear Discriminant Functions 21 1.6.3 Piecewise Linear Discriminant Functions 23 1.6.4 Generalised Linear Discriminant Function 24 1.6.5 Summary 26 1.7 Multiple Regression 27 1.8 Outline of Book 29 1.9 Notes and References 29 Exercises 31 2 Density Estimation – Parametric 33 2.1 Introduction 33 2.2 Estimating the Parameters of the Distributions 34 2.2.1 Estimative Approach 34 2.2.2 Predictive Approach 35 2.3 The Gaussian Classifier 35 2.3.1 Specification 35 2.3.2 Derivation of the Gaussian Classifier Plug-In Estimates 37 2.3.3 Example Application Study 39 2.4 Dealing with Singularities in the Gaussian Classifier 40 2.4.1 Introduction 40 2.4.2 Na¨ive Bayes 40 2.4.3 Projection onto a Subspace 41 2.4.4 Linear Discriminant Function 41 2.4.5 Regularised Discriminant Analysis 42 2.4.6 Example Application Study 44 2.4.7 Further Developments 45 2.4.8 Summary 46 2.5 Finite Mixture Models 46 2.5.1 Introduction 46 2.5.2 Mixture Models for Discrimination 48 2.5.3 Parameter Estimation for Normal Mixture Models 49 2.5.4 Normal Mixture Model Covariance Matrix Constraints 51 2.5.5 How Many Components? 52 2.5.6 Maximum Likelihood Estimation via EM 55 2.5.7 Example Application Study 60 2.5.8 Further Developments 62 2.5.9 Summary 63 2.6 Application Studies 63 2.7 Summary and Discussion 66 2.8 Recommendations 66 2.9 Notes and References 67 Exercises 67 3 Density Estimation – Bayesian 70 3.1 Introduction 70 3.1.1 Basics 72 3.1.2 Recursive Calculation 72 3.1.3 Proportionality 73 3.2 Analytic Solutions 73 3.2.1 Conjugate Priors 73 3.2.2 Estimating the Mean of a Normal Distribution with Known Variance 75 3.2.3 Estimating the Mean and the Covariance Matrix of a Multivariate Normal Distribution 79 3.2.4 Unknown Prior Class Probabilities 85 3.2.5 Summary 87 3.3 Bayesian Sampling Schemes 87 3.3.1 Introduction 87 3.3.2 Summarisation 87 3.3.3 Sampling Version of the Bayesian Classifier 89 3.3.4 Rejection Sampling 89 3.3.5 Ratio of Uniforms 90 3.3.6 Importance Sampling 92 3.4 Markov Chain Monte Carlo Methods 95 3.4.1 Introduction 95 3.4.2 The Gibbs Sampler 95 3.4.3 Metropolis–Hastings Algorithm 103 3.4.4 Data Augmentation 107 3.4.5 Reversible Jump Markov Chain Monte Carlo 108 3.4.6 Slice Sampling 109 3.4.7 MCMC Example – Estimation of Noisy Sinusoids 111 3.4.8 Summary 115 3.4.9 Notes and References 116 3.5 Bayesian Approaches to Discrimination 116 3.5.1 Labelled Training Data 116 3.5.2 Unlabelled Training Data 117 3.6 Sequential Monte Carlo Samplers 119 3.6.1 Introduction 119 3.6.2 Basic Methodology 121 3.6.3 Summary 125 3.7 Variational Bayes 126 3.7.1 Introduction 126 3.7.2 Description 126 3.7.3 Factorised Variational Approximation 129 3.7.4 Simple Example 131 3.7.5 Use of the Procedure for Model Selection 135 3.7.6 Further Developments and Applications 136 3.7.7 Summary 137 3.8 Approximate Bayesian Computation 137 3.8.1 Introduction 137 3.8.2 ABC Rejection Sampling 138 3.8.3 ABC MCMC Sampling 140 3.8.4 ABC Population Monte Carlo Sampling 141 3.8.5 Model Selection 142 3.8.6 Summary 143 3.9 Example Application Study 144 3.10 Application Studies 145 3.11 Summary and Discussion 146 3.12 Recommendations 147 3.13 Notes and References 147 Exercises 148 4 Density Estimation – Nonparametric 150 4.1 Introduction 150 4.1.1 Basic Properties of Density Estimators 150 4.2 k-Nearest-Neighbour Method 152 4.2.1 k-Nearest-Neighbour Classifier 152 4.2.2 Derivation 154 4.2.3 Choice of Distance Metric 157 4.2.4 Properties of the Nearest-Neighbour Rule 159 4.2.5 Linear Approximating and Eliminating Search Algorithm 159 4.2.6 Branch and Bound Search Algorithms: kd-Trees 163 4.2.7 Branch and Bound Search Algorithms: Ball-Trees 170 4.2.8 Editing Techniques 174 4.2.9 Example Application Study 177 4.2.10 Further Developments 178 4.2.11 Summary 179 4.3 Histogram Method 180 4.3.1 Data Adaptive Histograms 181 4.3.2 Independence Assumption (Naïve Bayes) 181 4.3.3 Lancaster Models 182 4.3.4 Maximum Weight Dependence Trees 183 4.3.5 Bayesian Networks 186 4.3.6 Example Application Study – Naïve Bayes Text Classification 190 4.3.7 Summary 193 4.4 Kernel Methods 194 4.4.1 Biasedness 197 4.4.2 Multivariate Extension 198 4.4.3 Choice of Smoothing Parameter 199 4.4.4 Choice of Kernel 201 4.4.5 Example Application Study 202 4.4.6 Further Developments 203 4.4.7 Summary 203 4.5 Expansion by Basis Functions 204 4.6 Copulas 207 4.6.1 Introduction 207 4.6.2 Mathematical Basis 207 4.6.3 Copula Functions 208 4.6.4 Estimating Copula Probability Density Functions 209 4.6.5 Simple Example 211 4.6.6 Summary 212 4.7 Application Studies 213 4.7.1 Comparative Studies 216 4.8 Summary and Discussion 216 4.9 Recommendations 217 4.10 Notes and References 217 Exercises 218 5 Linear Discriminant Analysis 221 5.1 Introduction 221 5.2 Two-Class Algorithms 222 5.2.1 General Ideas 222 5.2.2 Perceptron Criterion 223 5.2.3 Fisher’s Criterion 227 5.2.4 Least Mean-Squared-Error Procedures 228 5.2.5 Further Developments 235 5.2.6 Summary 235 5.3 Multiclass Algorithms 236 5.3.1 General Ideas 236 5.3.2 Error-Correction Procedure 237 5.3.3 Fisher’s Criterion – Linear Discriminant Analysis 238 5.3.4 Least Mean-Squared-Error Procedures 241 5.3.5 Regularisation 246 5.3.6 Example Application Study 246 5.3.7 Further Developments 247 5.3.8 Summary 248 5.4 Support Vector Machines 249 5.4.1 Introduction 249 5.4.2 Linearly Separable Two-Class Data 249 5.4.3 Linearly Nonseparable Two-Class Data 253 5.4.4 Multiclass SVMs 256 5.4.5 SVMs for Regression 257 5.4.6 Implementation 259 5.4.7 Example Application Study 262 5.4.8 Summary 263 5.5 Logistic Discrimination 263 5.5.1 Two-Class Case 263 5.5.2 Maximum Likelihood Estimation 264 5.5.3 Multiclass Logistic Discrimination 266 5.5.4 Example Application Study 267 5.5.5 Further Developments 267 5.5.6 Summary 268 5.6 Application Studies 268 5.7 Summary and Discussion 268 5.8 Recommendations 269 5.9 Notes and References 270 Exercises 270 6 Nonlinear Discriminant Analysis – Kernel and Projection Methods 274 6.1 Introduction 274 6.2 Radial Basis Functions 276 6.2.1 Introduction 276 6.2.2 Specifying the Model 278 6.2.3 Specifying the Functional Form 278 6.2.4 The Positions of the Centres 279 6.2.5 Smoothing Parameters 281 6.2.6 Calculation of the Weights 282 6.2.7 Model Order Selection 284 6.2.8 Simple RBF 285 6.2.9 Motivation 286 6.2.10 RBF Properties 288 6.2.11 Example Application Study 288 6.2.12 Further Developments 289 6.2.13 Summary 290 6.3 Nonlinear Support Vector Machines 291 6.3.1 Introduction 291 6.3.2 Binary Classification 291 6.3.3 Types of Kernel 292 6.3.4 Model Selection 293 6.3.5 Multiclass SVMs 294 6.3.6 Probability Estimates 294 6.3.7 Nonlinear Regression 296 6.3.8 Example Application Study 296 6.3.9 Further Developments 297 6.3.10 Summary 298 6.4 The Multilayer Perceptron 298 6.4.1 Introduction 298 6.4.2 Specifying the MLP Structure 299 6.4.3 Determining the MLP Weights 300 6.4.4 Modelling Capacity of the MLP 307 6.4.5 Logistic Classification 307 6.4.6 Example Application Study 310 6.4.7 Bayesian MLP Networks 311 6.4.8 Projection Pursuit 313 6.4.9 Summary 313 6.5 Application Studies 314 6.6 Summary and Discussion 316 6.7 Recommendations 317 6.8 Notes and References 318 Exercises 318 7 Rule and Decision Tree Induction 322 7.1 Introduction 322 7.2 Decision Trees 323 7.2.1 Introduction 323 7.2.2 Decision Tree Construction 326 7.2.3 Selection of the Splitting Rule 327 7.2.4 Terminating the Splitting Procedure 330 7.2.5 Assigning Class Labels to Terminal Nodes 332 7.2.6 Decision Tree Pruning – Worked Example 332 7.2.7 Decision Tree Construction Methods 337 7.2.8 Other Issues 339 7.2.9 Example Application Study 340 7.2.10 Further Developments 341 7.2.11 Summary 342 7.3 Rule Induction 342 7.3.1 Introduction 342 7.3.2 Generating Rules from a Decision Tree 345 7.3.3 Rule Induction Using a Sequential Covering Algorithm 345 7.3.4 Example Application Study 350 7.3.5 Further Developments 351 7.3.6 Summary 351 7.4 Multivariate Adaptive Regression Splines 351 7.4.1 Introduction 351 7.4.2 Recursive Partitioning Model 351 7.4.3 Example Application Study 355 7.4.4 Further Developments 355 7.4.5 Summary 356 7.5 Application Studies 356 7.6 Summary and Discussion 358 7.7 Recommendations 358 7.8 Notes and References 359 Exercises 359 8 Ensemble Methods 361 8.1 Introduction 361 8.2 Characterising a Classifier Combination Scheme 362 8.2.1 Feature Space 363 8.2.2 Level 366 8.2.3 Degree of Training 368 8.2.4 Form of Component Classifiers 368 8.2.5 Structure 369 8.2.6 Optimisation 369 8.3 Data Fusion 370 8.3.1 Architectures 370 8.3.2 Bayesian Approaches 371 8.3.3 Neyman–Pearson Formulation 373 8.3.4 Trainable Rules 374 8.3.5 Fixed Rules 375 8.4 Classifier Combination Methods 376 8.4.1 Product Rule 376 8.4.2 Sum Rule 377 8.4.3 Min, Max and Median Combiners 378 8.4.4 Majority Vote 379 8.4.5 Borda Count 379 8.4.6 Combiners Trained on Class Predictions 380 8.4.7 Stacked Generalisation 382 8.4.8 Mixture of Experts 382 8.4.9 Bagging 385 8.4.10 Boosting 387 8.4.11 Random Forests 389 8.4.12 Model Averaging 390 8.4.13 Summary of Methods 396 8.4.14 Example Application Study 398 8.4.15 Further Developments 399 8.5 Application Studies 399 8.6 Summary and Discussion 400 8.7 Recommendations 401 8.8 Notes and References 401 Exercises 402 9 Performance Assessment 404 9.1 Introduction 404 9.2 Performance Assessment 405 9.2.1 Performance Measures 405 9.2.2 Discriminability 406 9.2.3 Reliability 413 9.2.4 ROC Curves for Performance Assessment 415 9.2.5 Population and Sensor Drift 419 9.2.6 Example Application Study 421 9.2.7 Further Developments 422 9.2.8 Summary 423 9.3 Comparing Classifier Performance 424 9.3.1 Which Technique is Best? 424 9.3.2 Statistical Tests 425 9.3.3 Comparing Rules When Misclassification Costs are Uncertain 426 9.3.4 Example Application Study 428 9.3.5 Further Developments 429 9.3.6 Summary 429 9.4 Application Studies 429 9.5 Summary and Discussion 430 9.6 Recommendations 430 9.7 Notes and References 430 Exercises 431 10 Feature Selection and Extraction 433 10.1 Introduction 433 10.2 Feature Selection 435 10.2.1 Introduction 435 10.2.2 Characterisation of Feature Selection Approaches 439 10.2.3 Evaluation Measures 440 10.2.4 Search Algorithms for Feature Subset Selection 449 10.2.5 Complete Search – Branch and Bound 450 10.2.6 Sequential Search 454 10.2.7 Random Search 458 10.2.8 Markov Blanket 459 10.2.9 Stability of Feature Selection 460 10.2.10 Example Application Study 462 10.2.11 Further Developments 462 10.2.12 Summary 463 10.3 Linear Feature Extraction 463 10.3.1 Principal Components Analysis 464 10.3.2 Karhunen–Lo`eve Transformation 475 10.3.3 Example Application Study 481 10.3.4 Further Developments 482 10.3.5 Summary 483 10.4 Multidimensional Scaling 484 10.4.1 Classical Scaling 484 10.4.2 Metric MDS 486 10.4.3 Ordinal Scaling 487 10.4.4 Algorithms 490 10.4.5 MDS for Feature Extraction 491 10.4.6 Example Application Study 492 10.4.7 Further Developments 493 10.4.8 Summary 493 10.5 Application Studies 493 10.6 Summary and Discussion 495 10.7 Recommendations 495 10.8 Notes and References 496 Exercises 497 11 Clustering 501 11.1 Introduction 501 11.2 Hierarchical Methods 502 11.2.1 Single-Link Method 503 11.2.2 Complete-Link Method 506 11.2.3 Sum-of-Squares Method 507 11.2.4 General Agglomerative Algorithm 508 11.2.5 Properties of a Hierarchical Classification 508 11.2.6 Example Application Study 509 11.2.7 Summary 509 11.3 Quick Partitions 510 11.4 Mixture Models 511 11.4.1 Model Description 511 11.4.2 Example Application Study 512 11.5 Sum-of-Squares Methods 513 11.5.1 Clustering Criteria 514 11.5.2 Clustering Algorithms 515 11.5.3 Vector Quantisation 520 11.5.4 Example Application Study 530 11.5.5 Further Developments 530 11.5.6 Summary 531 11.6 Spectral Clustering 531 11.6.1 Elementary Graph Theory 531 11.6.2 Similarity Matrices 534 11.6.3 Application to Clustering 534 11.6.4 Spectral Clustering Algorithm 535 11.6.5 Forms of Graph Laplacian 535 11.6.6 Example Application Study 536 11.6.7 Further Developments 538 11.6.8 Summary 538 11.7 Cluster Validity 538 11.7.1 Introduction 538 11.7.2 Statistical Tests 539 11.7.3 Absence of Class Structure 540 11.7.4 Validity of Individual Clusters 541 11.7.5 Hierarchical Clustering 542 11.7.6 Validation of Individual Clusterings 542 11.7.7 Partitions 543 11.7.8 Relative Criteria 543 11.7.9 Choosing the Number of Clusters 545 11.8 Application Studies 546 11.9 Summary and Discussion 549 11.10 Recommendations 551 11.11 Notes and References 552 Exercises 553 12 Complex Networks 555 12.1 Introduction 555 12.1.1 Characteristics 557 12.1.2 Properties 557 12.1.3 Questions to Address 559 12.1.4 Descriptive Features 560 12.1.5 Outline 560 12.2 Mathematics of Networks 561 12.2.1 Graph Matrices 561 12.2.2 Connectivity 562 12.2.3 Distance Measures 562 12.2.4 Weighted Networks 563 12.2.5 Centrality Measures 563 12.2.6 Random Graphs 564 12.3 Community Detection 565 12.3.1 Clustering Methods 565 12.3.2 Girvan–Newman Algorithm 568 12.3.3 Modularity Approaches 570 12.3.4 Local Modularity 571 12.3.5 Clique Percolation 573 12.3.6 Example Application Study 574 12.3.7 Further Developments 575 12.3.8 Summary 575 12.4 Link Prediction 575 12.4.1 Approaches to Link Prediction 576 12.4.2 Example Application Study 578 12.4.3 Further Developments 578 12.5 Application Studies 579 12.6 Summary and Discussion 579 12.7 Recommendations 580 12.8 Notes and References 580 Exercises 580 13 Additional Topics 581 13.1 Model Selection 581 13.1.1 Separate Training and Test Sets 582 13.1.2 Cross-Validation 582 13.1.3 The Bayesian Viewpoint 583 13.1.4 Akaike’s Information Criterion 583 13.1.5 Minimum Description Length 584 13.2 Missing Data 585 13.3 Outlier Detection and Robust Procedures 586 13.4 Mixed Continuous and Discrete Variables 587 13.5 Structural Risk Minimisation and the Vapnik–Chervonenkis Dimension 588 13.5.1 Bounds on the Expected Risk 588 13.5.2 The VC Dimension 589 References 591 Index 637

Read more less

Book SynopsisStatistical pattern recognition relates to the use of statistical techniques for analysing data measurements in order to extract information and make justified decisions. It is a very active area of study and research, which has seen many advances in recent years.Trade Review"In the end I must add that this book is so appealing that I often found myself lost in the reading, pausing the overview of the manuscript in order to look more into some presented subject, and not being able to continue until I had finished seeing all about it.” (Zentralblatt MATH, 1 December 2012)Table of ContentsPreface xix Notation xxiii 1 Introduction to Statistical Pattern Recognition 1 1.1 Statistical Pattern Recognition 1 1.1.1 Introduction 1 1.1.2 The Basic Model 2 1.2 Stages in a Pattern Recognition Problem 4 1.3 Issues 6 1.4 Approaches to Statistical Pattern Recognition 7 1.5 Elementary Decision Theory 8 1.5.1 Bayes’ Decision Rule for Minimum Error 8 1.5.2 Bayes’ Decision Rule for Minimum Error – Reject Option 12 1.5.3 Bayes’ Decision Rule for Minimum Risk 13 1.5.4 Bayes’ Decision Rule for Minimum Risk – Reject Option 15 1.5.5 Neyman–Pearson Decision Rule 15 1.5.6 Minimax Criterion 18 1.5.7 Discussion 19 1.6 Discriminant Functions 20 1.6.1 Introduction 20 1.6.2 Linear Discriminant Functions 21 1.6.3 Piecewise Linear Discriminant Functions 23 1.6.4 Generalised Linear Discriminant Function 24 1.6.5 Summary 26 1.7 Multiple Regression 27 1.8 Outline of Book 29 1.9 Notes and References 29 Exercises 31 2 Density Estimation – Parametric 33 2.1 Introduction 33 2.2 Estimating the Parameters of the Distributions 34 2.2.1 Estimative Approach 34 2.2.2 Predictive Approach 35 2.3 The Gaussian Classifier 35 2.3.1 Specification 35 2.3.2 Derivation of the Gaussian Classifier Plug-In Estimates 37 2.3.3 Example Application Study 39 2.4 Dealing with Singularities in the Gaussian Classifier 40 2.4.1 Introduction 40 2.4.2 Na¨ive Bayes 40 2.4.3 Projection onto a Subspace 41 2.4.4 Linear Discriminant Function 41 2.4.5 Regularised Discriminant Analysis 42 2.4.6 Example Application Study 44 2.4.7 Further Developments 45 2.4.8 Summary 46 2.5 Finite Mixture Models 46 2.5.1 Introduction 46 2.5.2 Mixture Models for Discrimination 48 2.5.3 Parameter Estimation for Normal Mixture Models 49 2.5.4 Normal Mixture Model Covariance Matrix Constraints 51 2.5.5 How Many Components? 52 2.5.6 Maximum Likelihood Estimation via EM 55 2.5.7 Example Application Study 60 2.5.8 Further Developments 62 2.5.9 Summary 63 2.6 Application Studies 63 2.7 Summary and Discussion 66 2.8 Recommendations 66 2.9 Notes and References 67 Exercises 67 3 Density Estimation – Bayesian 70 3.1 Introduction 70 3.1.1 Basics 72 3.1.2 Recursive Calculation 72 3.1.3 Proportionality 73 3.2 Analytic Solutions 73 3.2.1 Conjugate Priors 73 3.2.2 Estimating the Mean of a Normal Distribution with Known Variance 75 3.2.3 Estimating the Mean and the Covariance Matrix of a Multivariate Normal Distribution 79 3.2.4 Unknown Prior Class Probabilities 85 3.2.5 Summary 87 3.3 Bayesian Sampling Schemes 87 3.3.1 Introduction 87 3.3.2 Summarisation 87 3.3.3 Sampling Version of the Bayesian Classifier 89 3.3.4 Rejection Sampling 89 3.3.5 Ratio of Uniforms 90 3.3.6 Importance Sampling 92 3.4 Markov Chain Monte Carlo Methods 95 3.4.1 Introduction 95 3.4.2 The Gibbs Sampler 95 3.4.3 Metropolis–Hastings Algorithm 103 3.4.4 Data Augmentation 107 3.4.5 Reversible Jump Markov Chain Monte Carlo 108 3.4.6 Slice Sampling 109 3.4.7 MCMC Example – Estimation of Noisy Sinusoids 111 3.4.8 Summary 115 3.4.9 Notes and References 116 3.5 Bayesian Approaches to Discrimination 116 3.5.1 Labelled Training Data 116 3.5.2 Unlabelled Training Data 117 3.6 Sequential Monte Carlo Samplers 119 3.6.1 Introduction 119 3.6.2 Basic Methodology 121 3.6.3 Summary 125 3.7 Variational Bayes 126 3.7.1 Introduction 126 3.7.2 Description 126 3.7.3 Factorised Variational Approximation 129 3.7.4 Simple Example 131 3.7.5 Use of the Procedure for Model Selection 135 3.7.6 Further Developments and Applications 136 3.7.7 Summary 137 3.8 Approximate Bayesian Computation 137 3.8.1 Introduction 137 3.8.2 ABC Rejection Sampling 138 3.8.3 ABC MCMC Sampling 140 3.8.4 ABC Population Monte Carlo Sampling 141 3.8.5 Model Selection 142 3.8.6 Summary 143 3.9 Example Application Study 144 3.10 Application Studies 145 3.11 Summary and Discussion 146 3.12 Recommendations 147 3.13 Notes and References 147 Exercises 148 4 Density Estimation – Nonparametric 150 4.1 Introduction 150 4.1.1 Basic Properties of Density Estimators 150 4.2 k-Nearest-Neighbour Method 152 4.2.1 k-Nearest-Neighbour Classifier 152 4.2.2 Derivation 154 4.2.3 Choice of Distance Metric 157 4.2.4 Properties of the Nearest-Neighbour Rule 159 4.2.5 Linear Approximating and Eliminating Search Algorithm 159 4.2.6 Branch and Bound Search Algorithms: kd-Trees 163 4.2.7 Branch and Bound Search Algorithms: Ball-Trees 170 4.2.8 Editing Techniques 174 4.2.9 Example Application Study 177 4.2.10 Further Developments 178 4.2.11 Summary 179 4.3 Histogram Method 180 4.3.1 Data Adaptive Histograms 181 4.3.2 Independence Assumption (Na¨ive Bayes) 181 4.3.3 Lancaster Models 182 4.3.4 Maximum Weight Dependence Trees 183 4.3.5 Bayesian Networks 186 4.3.6 Example Application Study – Na¨ive Bayes Text Classification 190 4.3.7 Summary 193 4.4 Kernel Methods 194 4.4.1 Biasedness 197 4.4.2 Multivariate Extension 198 4.4.3 Choice of Smoothing Parameter 199 4.4.4 Choice of Kernel 201 4.4.5 Example Application Study 202 4.4.6 Further Developments 203 4.4.7 Summary 203 4.5 Expansion by Basis Functions 204 4.6 Copulas 207 4.6.1 Introduction 207 4.6.2 Mathematical Basis 207 4.6.3 Copula Functions 208 4.6.4 Estimating Copula Probability Density Functions 209 4.6.5 Simple Example 211 4.6.6 Summary 212 4.7 Application Studies 213 4.7.1 Comparative Studies 216 4.8 Summary and Discussion 216 4.9 Recommendations 217 4.10 Notes and References 217 Exercises 218 5 Linear Discriminant Analysis 221 5.1 Introduction 221 5.2 Two-Class Algorithms 222 5.2.1 General Ideas 222 5.2.2 Perceptron Criterion 223 5.2.3 Fisher’s Criterion 227 5.2.4 Least Mean-Squared-Error Procedures 228 5.2.5 Further Developments 235 5.2.6 Summary 235 5.3 Multiclass Algorithms 236 5.3.1 General Ideas 236 5.3.2 Error-Correction Procedure 237 5.3.3 Fisher’s Criterion – Linear Discriminant Analysis 238 5.3.4 Least Mean-Squared-Error Procedures 241 5.3.5 Regularisation 246 5.3.6 Example Application Study 246 5.3.7 Further Developments 247 5.3.8 Summary 248 5.4 Support Vector Machines 249 5.4.1 Introduction 249 5.4.2 Linearly Separable Two-Class Data 249 5.4.3 Linearly Nonseparable Two-Class Data 253 5.4.4 Multiclass SVMs 256 5.4.5 SVMs for Regression 257 5.4.6 Implementation 259 5.4.7 Example Application Study 262 5.4.8 Summary 263 5.5 Logistic Discrimination 263 5.5.1 Two-Class Case 263 5.5.2 Maximum Likelihood Estimation 264 5.5.3 Multiclass Logistic Discrimination 266 5.5.4 Example Application Study 267 5.5.5 Further Developments 267 5.5.6 Summary 268 5.6 Application Studies 268 5.7 Summary and Discussion 268 5.8 Recommendations 269 5.9 Notes and References 270 Exercises 270 6 Nonlinear Discriminant Analysis – Kernel and Projection Methods 274 6.1 Introduction 274 6.2 Radial Basis Functions 276 6.2.1 Introduction 276 6.2.2 Specifying the Model 278 6.2.3 Specifying the Functional Form 278 6.2.4 The Positions of the Centres 279 6.2.5 Smoothing Parameters 281 6.2.6 Calculation of the Weights 282 6.2.7 Model Order Selection 284 6.2.8 Simple RBF 285 6.2.9 Motivation 286 6.2.10 RBF Properties 288 6.2.11 Example Application Study 288 6.2.12 Further Developments 289 6.2.13 Summary 290 6.3 Nonlinear Support Vector Machines 291 6.3.1 Introduction 291 6.3.2 Binary Classification 291 6.3.3 Types of Kernel 292 6.3.4 Model Selection 293 6.3.5 Multiclass SVMs 294 6.3.6 Probability Estimates 294 6.3.7 Nonlinear Regression 296 6.3.8 Example Application Study 296 6.3.9 Further Developments 297 6.3.10 Summary 298 6.4 The Multilayer Perceptron 298 6.4.1 Introduction 298 6.4.2 Specifying the MLP Structure 299 6.4.3 Determining the MLP Weights 300 6.4.4 Modelling Capacity of the MLP 307 6.4.5 Logistic Classification 307 6.4.6 Example Application Study 310 6.4.7 Bayesian MLP Networks 311 6.4.8 Projection Pursuit 313 6.4.9 Summary 313 6.5 Application Studies 314 6.6 Summary and Discussion 316 6.7 Recommendations 317 6.8 Notes and References 318 Exercises 318 7 Rule and Decision Tree Induction 322 7.1 Introduction 322 7.2 Decision Trees 323 7.2.1 Introduction 323 7.2.2 Decision Tree Construction 326 7.2.3 Selection of the Splitting Rule 327 7.2.4 Terminating the Splitting Procedure 330 7.2.5 Assigning Class Labels to Terminal Nodes 332 7.2.6 Decision Tree Pruning – Worked Example 332 7.2.7 Decision Tree Construction Methods 337 7.2.8 Other Issues 339 7.2.9 Example Application Study 340 7.2.10 Further Developments 341 7.2.11 Summary 342 7.3 Rule Induction 342 7.3.1 Introduction 342 7.3.2 Generating Rules from a Decision Tree 345 7.3.3 Rule Induction Using a Sequential Covering Algorithm 345 7.3.4 Example Application Study 350 7.3.5 Further Developments 351 7.3.6 Summary 351 7.4 Multivariate Adaptive Regression Splines 351 7.4.1 Introduction 351 7.4.2 Recursive Partitioning Model 351 7.4.3 Example Application Study 355 7.4.4 Further Developments 355 7.4.5 Summary 356 7.5 Application Studies 356 7.6 Summary and Discussion 358 7.7 Recommendations 358 7.8 Notes and References 359 Exercises 359 8 Ensemble Methods 361 8.1 Introduction 361 8.2 Characterising a Classifier Combination Scheme 362 8.2.1 Feature Space 363 8.2.2 Level 366 8.2.3 Degree of Training 368 8.2.4 Form of Component Classifiers 368 8.2.5 Structure 369 8.2.6 Optimisation 369 8.3 Data Fusion 370 8.3.1 Architectures 370 8.3.2 Bayesian Approaches 371 8.3.3 Neyman–Pearson Formulation 373 8.3.4 Trainable Rules 374 8.3.5 Fixed Rules 375 8.4 Classifier Combination Methods 376 8.4.1 Product Rule 376 8.4.2 Sum Rule 377 8.4.3 Min, Max and Median Combiners 378 8.4.4 Majority Vote 379 8.4.5 Borda Count 379 8.4.6 Combiners Trained on Class Predictions 380 8.4.7 Stacked Generalisation 382 8.4.8 Mixture of Experts 382 8.4.9 Bagging 385 8.4.10 Boosting 387 8.4.11 Random Forests 389 8.4.12 Model Averaging 390 8.4.13 Summary of Methods 396 8.4.14 Example Application Study 398 8.4.15 Further Developments 399 8.5 Application Studies 399 8.6 Summary and Discussion 400 8.7 Recommendations 401 8.8 Notes and References 401 Exercises 402 9 Performance Assessment 404 9.1 Introduction 404 9.2 Performance Assessment 405 9.2.1 Performance Measures 405 9.2.2 Discriminability 406 9.2.3 Reliability 413 9.2.4 ROC Curves for Performance Assessment 415 9.2.5 Population and Sensor Drift 419 9.2.6 Example Application Study 421 9.2.7 Further Developments 422 9.2.8 Summary 423 9.3 Comparing Classifier Performance 424 9.3.1 Which Technique is Best? 424 9.3.2 Statistical Tests 425 9.3.3 Comparing Rules When Misclassification Costs are Uncertain 426 9.3.4 Example Application Study 428 9.3.5 Further Developments 429 9.3.6 Summary 429 9.4 Application Studies 429 9.5 Summary and Discussion 430 9.6 Recommendations 430 9.7 Notes and References 430 Exercises 431 10 Feature Selection and Extraction 433 10.1 Introduction 433 10.2 Feature Selection 435 10.2.1 Introduction 435 10.2.2 Characterisation of Feature Selection Approaches 439 10.2.3 Evaluation Measures 440 10.2.4 Search Algorithms for Feature Subset Selection 449 10.2.5 Complete Search – Branch and Bound 450 10.2.6 Sequential Search 454 10.2.7 Random Search 458 10.2.8 Markov Blanket 459 10.2.9 Stability of Feature Selection 460 10.2.10 Example Application Study 462 10.2.11 Further Developments 462 10.2.12 Summary 463 10.3 Linear Feature Extraction 463 10.3.1 Principal Components Analysis 464 10.3.2 Karhunen–Lo`eve Transformation 475 10.3.3 Example Application Study 481 10.3.4 Further Developments 482 10.3.5 Summary 483 10.4 Multidimensional Scaling 484 10.4.1 Classical Scaling 484 10.4.2 Metric MDS 486 10.4.3 Ordinal Scaling 487 10.4.4 Algorithms 490 10.4.5 MDS for Feature Extraction 491 10.4.6 Example Application Study 492 10.4.7 Further Developments 493 10.4.8 Summary 493 10.5 Application Studies 493 10.6 Summary and Discussion 495 10.7 Recommendations 495 10.8 Notes and References 496 Exercises 497 11 Clustering 501 11.1 Introduction 501 11.2 Hierarchical Methods 502 11.2.1 Single-Link Method 503 11.2.2 Complete-Link Method 506 11.2.3 Sum-of-Squares Method 507 11.2.4 General Agglomerative Algorithm 508 11.2.5 Properties of a Hierarchical Classification 508 11.2.6 Example Application Study 509 11.2.7 Summary 509 11.3 Quick Partitions 510 11.4 Mixture Models 511 11.4.1 Model Description 511 11.4.2 Example Application Study 512 11.5 Sum-of-Squares Methods 513 11.5.1 Clustering Criteria 514 11.5.2 Clustering Algorithms 515 11.5.3 Vector Quantisation 520 11.5.4 Example Application Study 530 11.5.5 Further Developments 530 11.5.6 Summary 531 11.6 Spectral Clustering 531 11.6.1 Elementary Graph Theory 531 11.6.2 Similarity Matrices 534 11.6.3 Application to Clustering 534 11.6.4 Spectral Clustering Algorithm 535 11.6.5 Forms of Graph Laplacian 535 11.6.6 Example Application Study 536 11.6.7 Further Developments 538 11.6.8 Summary 538 11.7 Cluster Validity 538 11.7.1 Introduction 538 11.7.2 Statistical Tests 539 11.7.3 Absence of Class Structure 540 11.7.4 Validity of Individual Clusters 541 11.7.5 Hierarchical Clustering 542 11.7.6 Validation of Individual Clusterings 542 11.7.7 Partitions 543 11.7.8 Relative Criteria 543 11.7.9 Choosing the Number of Clusters 545 11.8 Application Studies 546 11.9 Summary and Discussion 549 11.10 Recommendations 551 11.11 Notes and References 552 Exercises 553 12 Complex Networks 555 12.1 Introduction 555 12.1.1 Characteristics 557 12.1.2 Properties 557 12.1.3 Questions to Address 559 12.1.4 Descriptive Features 560 12.1.5 Outline 560 12.2 Mathematics of Networks 561 12.2.1 Graph Matrices 561 12.2.2 Connectivity 562 12.2.3 Distance Measures 562 12.2.4 Weighted Networks 563 12.2.5 Centrality Measures 563 12.2.6 Random Graphs 564 12.3 Community Detection 565 12.3.1 Clustering Methods 565 12.3.2 Girvan–Newman Algorithm 568 12.3.3 Modularity Approaches 570 12.3.4 Local Modularity 571 12.3.5 Clique Percolation 573 12.3.6 Example Application Study 574 12.3.7 Further Developments 575 12.3.8 Summary 575 12.4 Link Prediction 575 12.4.1 Approaches to Link Prediction 576 12.4.2 Example Application Study 578 12.4.3 Further Developments 578 12.5 Application Studies 579 12.6 Summary and Discussion 579 12.7 Recommendations 580 12.8 Notes and References 580 Exercises 580 13 Additional Topics 581 13.1 Model Selection 581 13.1.1 Separate Training and Test Sets 582 13.1.2 Cross-Validation 582 13.1.3 The Bayesian Viewpoint 583 13.1.4 Akaike’s Information Criterion 583 13.1.5 Minimum Description Length 584 13.2 Missing Data 585 13.3 Outlier Detection and Robust Procedures 586 13.4 Mixed Continuous and Discrete Variables 587 13.5 Structural Risk Minimisation and the Vapnik–Chervonenkis Dimension 588 13.5.1 Bounds on the Expected Risk 588 13.5.2 The VC Dimension 589 References 591 Index 637

Read more less

Book SynopsisMore and more Agile projects are seeking architectural roots as they struggle with complexity and scale - and they're seeking lightweight ways to do it. This book helps you to find your own path.Trade Review'...a book of advice that is broad, enabling, and concrete. (Lean Magazine, January 2010).Table of ContentsAbout the Authors xii Preface xiii 1 Introduction 1 1.1 The Touchstones: Lean and Agile 1 1.2 Lean Architecture and Agile Feature Development 4 1.3 Agile Production 7 1.3.1 Agile Builds on Lean 7 1.3.2 The Scope of Agile Systems 8 1.3.3 Agile and DCI 9 1.4 The Book in a Very Small Nutshell 10 1.5 Lean and Agile: Contrasting and Complementary 11 1.5.1 The Lean Secret 14 1.6 Lost Practices 14 1.6.1 Architecture 15 1.6.2 Handling Dependencies between Requirements 15 1.6.3 Foundations for Usability 16 1.6.4 Documentation 16 Code Does Not Stand Alone 17 Capturing the ‘‘Why’’ 19 1.6.5 Common Sense, Thinking, and Caring 19 1.7 What this Book is Not About 21 1.8 Agile, Lean – Oh, Yeah, and Scrum and Methodologies and Such 22 1.9 History and Such 24 2 Agile Production in a Nutshell 27 2.1 Engage the Stakeholders 27 2.2 Define the Problem 29 2.3 Focusing on What the System Is: The Foundations of Form 30 2.4 Focusing on What the System Does: The System Lifeblood 32 2.5 Design and Code 33 2.6 Countdown: 3, 2, 1. . . 34 3 Stakeholder Engagement 35 3.1 The Value Stream 35 3.1.1 End Users and Other Stakeholders as Value Stream Anchors 36 3.1.2 Architecture in the Value Stream 37 3.1.3 The Lean Secret 38 3.2 The Key Stakeholders 41 3.2.1 End Users 43 Psyching Out the End Users 44 Don’t Forget Behavior 46 The End User Landscape 47 3.2.2 The Business 47 A Special Note for Managers 48 3.2.3 Customers 50 . . . As Contrasted with End Users 50 ‘‘Customers’’ in the Value Stream 52 3.2.4 Domain Experts 52 No Ivory Tower Architects 53 Experts in Both Problemand Solution Domains 54 3.2.5 Developers and Testers 55 3.3 Process Elements of Stakeholder Engagement 57 3.3.1 Getting Started 58 3.3.2 Customer Engagement 60 3.4 The Network of Stakeholders: Trimming Wasted Time 61 3.4.1 Stovepipe Versus Swarm 61 3.4.2 The First Thing You Build 64 3.4.3 Keep the Team Together 65 3.5 No Quick Fixes, but Some Hope 66 4 Problem Definition 67 4.1 What’s Agile about Problem Definitions? 68 4.2 What’s Lean about Problem Definitions? 68 4.3 Good and Bad Problem Definitions 70 4.4 Problems and Solutions 72 4.5 The Process Around Problem Definitions 73 4.5.1 Value the Hunt Over the Prize 73 4.5.2 Problem Ownership 74 4.5.3 Creeping Featurism 75 4.6 Problem Definitions, Goals, Charters, Visions, and Objectives 76 4.7 Documentation? 77 5 What the System Is, Part 1: Lean Architecture 79 5.1 Some Surprises about Architecture 80 5.1.1 What’s Lean about This? 82 Deliberation and ‘‘Pull’’ 83 Failure-Proof Constraints or Poka-Yoke 83 The Lean Mantras of Conservation, Consistency, and Focus 84 5.1.2 What’s Agile about Architecture? 84 It’s All About Individuals and Interactions 84 Past Excesses 85 Dispelling a Couple of Agile Myths 86 5.2 The First Design Step: Partitioning 88 5.2.1 The First Partition: Domain Form Versus Behavioral Form 89 5.2.2 The Second Partitioning: Conway’s Law 90 5.2.3 The Real Complexity of Partitioning 93 5.2.4 Dimensions of Complexity 94 5.2.5 Domains: A Particularly Interesting Partitioning 94 5.2.6 Back to Dimensions of Complexity 96 5.2.7 Architecture and Culture 100 5.2.8 Wrap-Up on Conway’s Law 100 5.3 The Second Design Step: Selecting a Design Style 100 5.3.1 Contrasting Structuring with Partitioning 102 5.3.2 The Fundamentals of Style: Commonality and Variation 104 5.3.3 Starting with Tacit Commonality and Variation 105 5.3.4 Commonality, Variation, and Scope 108 5.3.5 Making Commonalities and Variations Explicit 111 Commonality Categories 112 Next Steps 114 5.3.6 The Most Common Style: Object Orientation 114 Just What is Object Orientation? 115 5.3.7 Other Styles within the Von NeumannWorld 117 5.3.8 Domain-Specific Languages and Application Generators 120 The State of the Art in DSLs 121 DSLs’ Place in Architecture 121 5.3.9 Codified Forms: Pattern Languages 122 5.3.10 Third-Party Software and Other Paradigms 124 5.4 Documentation? 127 5.4.1 The Domain Dictionary 128 5.4.2 Architecture Carryover 128 5.5 History and Such 129 6 What the System Is, Part 2: Coding It Up 131 6.1 The Third Step: The Rough Framing of the Code 131 6.1.1 Abstract Base Classes 133 6.1.2 Pre-Conditions, Post-Conditions, and Assertions 137 Static Cling 142 6.1.3 Algorithmic Scaling: The Other Side of Static Assertions 144 6.1.4 Form Versus Accessible Services 146 6.1.5 Scaffolding 147 6.1.6 Testing the Architecture 149 Usability Testing 149 Architecture Testing 149 6.2 Relationships in Architecture 153 6.2.1 Kinds of Relationship 153 6.2.2 Testing the Relationships 155 6.3 Not Your Old Professor’s OO 155 6.4 How much Architecture? 159 6.4.1 Balancing BUFD and YAGNI 159 6.4.2 One Size Does Not Fit All 160 6.4.3 When Are You Done? 160 6.5 Documentation? 162 6.6 History and Such 163 7 What the System Does: System Functionality 165 7.1 What the System Does 166 7.1.1 User Stories: A Beginning 166 7.1.2 Enabling Specifications and Use Cases 167 7.1.3 Helping Developers, Too 169 7.1.4 Your Mileage may Vary 170 7.2 Who is Going to Use Our Software? 171 7.2.1 User Profiles 171 7.2.2 Personas 171 7.2.3 User Profiles or Personas? 172 7.2.4 User Roles and Terminology 173 7.3 What do the Users Want to Use Our Software for? 173 7.3.1 Feature Lists 173 7.3.2 Dataflow Diagrams 174 7.3.3 Personas and Scenarios 174 7.3.4 Narratives 174 7.3.5 Behavior-Driven Development 175 7.3.6 Now that We’re Warmed Up. . . 175 Prototypes 176 Towards Foundations for Decisions 176 Known and Unknown Unknowns 176 Use Cases as a Decision Framework 177 7.4 Why Does the User Want to Use Our Software? 177 7.5 Consolidation of What the System Does 178 7.5.1 The Helicopter View 181 Habits: The Developer View and the User View 182 Trimming the Scope 185 7.5.2 Setting the Stage 186 7.5.3 Play the Sunny Day Scenario 187 Business Rules 191 7.5.4 Add the Interesting Stuff 193 7.5.5 Use Cases to Roles 200 Roles from the Use Case 201 Bridging the Gap between the Business and the Programmer 202 7.6 Recap 203 7.6.1 Support the User’s Workflow 203 7.6.2 Support Testing Close to Development 203 7.6.3 Support Efficient Decision-Making about Functionality 204 7.6.4 Support Emerging Requirements 204 7.6.5 Support Release Planning 204 7.6.6 Support Sufficient Input to the Architecture 205 7.6.7 Support the Team’s Understanding of What to Develop 205 7.7 ‘‘It Depends’’: When Use Cases are a Bad Fit 206 7.7.1 Classic OO: Atomic Event Architectures 206 7.8 Usability Testing 208 7.9 Documentation? 209 7.10 History and Such 211 8 Coding It Up: Basic Assembly 213 8.1 The Big Picture: Model-View-Controller-User 214 8.1.1 What is a Program? 214 8.1.2 What is an Agile Program? 215 8.1.3 MVC in More Detail 217 8.1.4 MVC-U: Not the End of the Story 217 A Short History of Computer Science 218 Atomic Event Architectures 219 DCI Architectures 220 8.2 The Form and Architecture of Atomic Event Systems 220 8.2.1 Domain Objects 221 8.2.2 Object Roles, Interfaces, and the Model 221 Example 223 8.2.3 Reflection: Use Cases, Atomic Event Architectures, and Algorithms 224 8.2.4 A Special Case: One-to-Many Mapping of Object Roles to Objects 225 8.3 Updating the Domain Logic: Method Elaboration, Factoring, and Re-factoring 226 8.3.1 Creating New Classes and Filling in Existing Function Placeholders 227 Example 228 8.3.2 Back to the Future: This is Just Good Old-Fashioned OO 229 8.3.3 Analysis and Design Tools 229 8.3.4 Factoring 231 8.3.5 A Caution about Re-Factoring 231 8.4 Documentation? 231 8.5 Why All These Artifacts? 232 8.6 History and Such 233 9 Coding it Up: The DCI Architecture 235 9.1 Sometimes, Smart Objects Just Aren’t Enough 235 9.2 DCI in a Nutshell 236 9.3 Overview of DCI 238 9.3.1 Parts of the User Mental Model We’ve Forgotten 239 9.3.2 Enter Methodful Object Roles 240 9.3.3 Tricks with Traits 242 9.3.4 Context Classes: One Per Use Case 243 9.4 DCI by Example 246 9.4.1 The Inputs to the Design 246 9.4.2 Use Cases to Algorithms 247 9.4.3 Methodless Object Roles: The Framework for Identifiers 250 9.4.4 Partitioning the Algorithms Across Methodful Object Roles 253 Traits as a Building Block 253 In Smalltalk 253 In C++ 254 In Ruby 256 Coding it Up: C++ 257 Coding Up DCI in Ruby 259 9.4.5 The Context Framework 261 The Ruby Code 263 The C++ Code 265 Making ContextsWork 267 Habits: Nested Contexts in Methodful Object Roles 277 9.4.6 Variants and Tricks in DCI 283 Context Layering 283 Information Hiding 283 Selective Object Role Injection 284 9.5 Updating the Domain Logic 285 9.5.1 Contrasting DCI with the Atomic Event Style 286 9.5.2 Special Considerations for Domain Logic in DCI 287 9.6 Context Objects in the User Mental Model: Solution to an Age-Old Problem 290 9.7 Why All These Artifacts? 294 Why not Use Classes Instead of ‘‘Methodful Object Roles’’? 295 Why not Put the Entire Algorithm Inside of the Class with which it is Most Closely Coupled? 295 Then Why not Localize the Algorithm to a Class and Tie it to Domain Objects as Needed? 296 Why not Put the Algorithm into a Procedure, and Combine the Procedural Paradigm with the Object Paradigm in a Single Program? 296 If I Collect Together the Algorithm Code for a Use Case in One Class, Including the Code for All of its Deviations, Doesn’t the Context Become Very Large? 296 So, What do DCI and Lean Architecture Give Me? 297 And Remember. . . 297 9.8 Beyond C++: DCI in Other Languages 297 9.8.1 Scala 298 9.8.2 Python 299 9.8.3 C# 299 9.8.4 . . . and Even Java 299 9.8.5 The Account Example in Smalltalk 300 9.9 Documentation? 300 9.10 History and Such 301 9.10.1 DCI and Aspect-Oriented Programming 302 9.10.2 Other Approaches 302 10 Epilog 305 Appendix A Scala Implementation of the DCI Account Example 307 Appendix B Account Example in Python 311 Appendix C Account Example in C# 315 Appendix D Account Example in Ruby 321 Appendix E Qi4j 327 Appendix F Account Example in Squeak 331 F.1 Testing Perspective 333 F.2 Data Perspective 333 F.2.1 BB5Bank 333 F.2.2 BB5SavingsAccount 334 F.2.3 BB5CheckingAccount 334 F.3 Context Perspective 335 F.3.1 BB5MoneyTransferContext 335 F.4 Interaction (RoleTrait) Perspective 336 F.4.1 BB5MoneyTransferContextTransferMoneySource 336 F.4.2 BB5MoneyTransferContextMyContext 337 F.4.3 BB5MoneyTransferContextTransferMoneySink 337 F.5 Support Perspective (Infrastructure Classes) 337 F.5.1 BB1Context (common superclass for all contexts) 337 F.5.2 BB1RoleTrait (all RoleTraits are instances of this class) 339 Bibliography 341 Index 351

Read more less

Book Synopsis* Novel approach to the mathematics of problem solving, in particular how to do logical calculations. * Many of the problems are well-known from (mathematical) puzzle books. * The solution method in the book is new and more relevant to the true nature of problem solving in the modern IT-dominated world.Table of ContentsPreface xi PART I Algorithmic Problem Solving 1 CHAPTER 1 – Introduction 3 1.1 Algorithms 3 1.2 Algorithmic Problem Solving 4 1.3 Overview 5 1.4 Bibliographic Remarks 6 CHAPTER 2 – Invariants 7 2.1 Chocolate Bars 10 2.1.1 The Solution 10 2.1.2 The Mathematical Solution 11 2.2 Empty Boxes 16 2.2.1 Review 19 2.3 The Tumbler Problem 22 2.3.1 Non-deterministic Choice 23 2.4 Tetrominoes 24 2.5 Summary 30 2.6 Bibliographic Remarks 34 CHAPTER 3 – Crossing a River 35 3.1 Problems 36 3.2 Brute Force 37 3.2.1 Goat, Cabbage and Wolf 37 3.2.2 State-Space Explosion 39 3.2.3 Abstraction 41 3.3 Nervous Couples 42 3.3.1 What Is the Problem? 42 3.3.2 Problem Structure 43 3.3.3 Denoting States and Transitions 44 3.3.4 Problem Decomposition 45 3.3.5 A Review 48 3.4 Rule of Sequential Composition 50 3.5 The Bridge Problem 54 3.6 Conditional Statements 63 3.7 Summary 65 3.8 Bibliographic Remarks 65 CHAPTER 4 – Games 67 4.1 Matchstick Games 67 4.2 Winning Strategies 69 4.2.1 Assumptions 69 4.2.2 Labelling Positions 70 4.2.3 Formulating Requirements 72 4.3 Subtraction-Set Games 74 4.4 Sums of Games 78 4.4.1 A Simple Sum Game 79 4.4.2 Maintain Symmetry! 81 4.4.3 More Simple Sums 82 4.4.4 Evaluating Positions 83 4.4.5 Using the Mex Function 87 4.5 Summary 91 4.6 Bibliographic Remarks 92 CHAPTER 5 – Knights and Knaves 95 5.1 Logic Puzzles 95 5.2 Calculational Logic 96 5.2.1 Propositions 96 5.2.2 Knights and Knaves 97 5.2.3 Boolean Equality 98 5.2.4 Hidden Treasures 100 5.2.5 Equals for Equals 101 5.3 Equivalence and Continued Equalities 102 5.3.1 Examples of the Associativity of Equivalence 104 5.3.2 On Natural Language 105 5.4 Negation 106 5.4.1 Contraposition 109 5.4.2 Handshake Problems 112 5.4.3 Inequivalence 113 5.5 Summary 117 5.6 Bibliographic Remarks 117 CHAPTER 6 – Induction 119 6.1 Example Problems 120 6.2 Cutting the Plane 123 6.3 Triominoes 126 6.4 Looking for Patterns 128 6.5 The Need for Proof 129 6.6 From Verification to Construction 130 6.7 Summary 134 6.8 Bibliographic Remarks 134 CHAPTER 7 – Fake-Coin Detection 137 7.1 Problem Formulation 137 7.2 Problem Solution 139 7.2.1 The Basis 139 7.2.2 Induction Step 139 7.2.3 The Marked-Coin Problem 140 7.2.4 The Complete Solution 141 7.3 Summary 146 7.4 Bibliographic Remarks 146 CHAPTER 8 – The Tower of Hanoi 147 8.1 Specification and Solution 147 8.1.1 The End of the World! 147 8.1.2 Iterative Solution 148 8.1.3 Why? 149 8.2 Inductive Solution 149 8.3 The Iterative Solution 153 8.4 Summary 156 8.5 Bibliographic Remarks 156 CHAPTER 9 – Principles of Algorithm Design 157 9.1 Iteration, Invariants and Making Progress 158 9.2 A Simple Sorting Problem 160 9.3 Binary Search 163 9.4 Sam Loyd’s Chicken-Chasing Problem 166 9.4.1 Cornering the Prey 170 9.4.2 Catching the Prey 174 9.4.3 Optimality 176 9.5 Projects 177 9.6 Summary 178 9.7 Bibliographic Remarks 180 CHAPTER 10 – The Bridge Problem 183 10.1 Lower and Upper Bounds 183 10.2 Outline Strategy 185 10.3 Regular Sequences 187 10.4 Sequencing Forward Trips 189 10.5 Choosing Settlers and Nomads 193 10.6 The Algorithm 196 10.7 Summary 199 10.8 Bibliographic Remarks 200 CHAPTER 11 – Knight’s Circuit 201 11.1 Straight-Move Circuits 202 11.2 Supersquares 206 11.3 Partitioning the Board 209 11.4 Summary 216 11.5 Bibliographic Remarks 218 PART II Mathematical Techniques 219 CHAPTER 12 – The Language of Mathematics 221 12.1 Variables, Expressions and Laws 222 12.2 Sets 224 12.2.1 The Membership Relation 224 12.2.2 The Empty Set 224 12.2.3 Types/Universes 224 12.2.4 Union and Intersection 225 12.2.5 Set Comprehension 225 12.2.6 Bags 227 12.3 Functions 227 12.3.1 Function Application 228 12.3.2 Binary Operators 230 12.3.3 Operator Precedence 230 12.4 Types and Type Checking 232 12.4.1 Cartesian Product and Disjoint Sum 233 12.4.2 Function Types 235 12.5 Algebraic Properties 236 12.5.1 Symmetry 237 12.5.2 Zero and Unit 238 12.5.3 Idempotence 239 12.5.4 Associativity 240 12.5.5 Distributivity/Factorisation 241 12.5.6 Algebras 243 12.6 Boolean Operators 244 12.7 Binary Relations 246 12.7.1 Reflexivity 247 12.7.2 Symmetry 248 12.7.3 Converse 249 12.7.4 Transitivity 249 12.7.5 Anti-symmetry 251 12.7.6 Orderings 252 12.7.7 Equality 255 12.7.8 Equivalence Relations 256 12.8 Calculations 257 12.8.1 Steps in a Calculation 259 12.8.2 Relations between Steps 260 12.8.3 ‘‘If’’ and ‘‘Only If’’ 262 12.9 Exercises 264 CHAPTER 13 – Boolean Algebra 267 13.1 Boolean Equality 267 13.2 Negation 269 13.3 Disjunction 270 13.4 Conjunction 271 13.5 Implication 274 13.5.1 Definitions and Basic Properties 275 13.5.2 Replacement Rules 276 13.6 Set Calculus 279 13.7 Exercises 281 CHAPTER 14 – Quantifiers 285 14.1 DotDotDot and Sigmas 285 14.2 Introducing Quantifier Notation 286 14.2.1 Summation 287 14.2.2 Free and Bound Variables 289 14.2.3 Properties of Summation 291 14.2.4 Warning 297 14.3 Universal and Existential Quantification 297 14.3.1 Universal Quantification 298 14.3.2 Existential Quantification 300 14.4 Quantifier Rules 301 14.4.1 The Notation 302 14.4.2 Free and Bound Variables 303 14.4.3 Dummies 303 14.4.4 Range Part 303 14.4.5 Trading 304 14.4.6 Term Part 304 14.4.7 Distributivity Properties 304 14.5 Exercises 306 CHAPTER 15 – Elements of Number Theory 309 15.1 Inequalities 309 15.2 Minimum and Maximum 312 15.3 The Divides Relation 315 15.4 Modular Arithmetic 316 15.4.1 Integer Division 316 15.4.2 Remainders and Modulo Arithmetic 320 15.5 Exercises 322 CHAPTER 16 – Relations, Graphs and Path Algebras 325 16.1 Paths in a Directed Graph 325 16.2 Graphs and Relations 328 16.2.1 Relation Composition 330 16.2.2 Union of Relations 332 16.2.3 Transitive Closure 334 16.2.4 Reflexive Transitive Closure 338 16.3 Functional and Total Relations 339 16.4 Path-Finding Problems 341 16.4.1 Counting Paths 341 16.4.2 Frequencies 343 16.4.3 Shortest Distances 344 16.4.4 All Paths 345 16.4.5 Semirings and Operations on Graphs 347 16.5 Matrices 351 16.6 Closure Operators 353 16.7 Acyclic Graphs 354 16.7.1 Topological Ordering 355 16.8 Combinatorics 357 16.8.1 Basic Laws 358 16.8.2 Counting Choices 359 16.8.3 Counting Paths 361 16.9 Exercises 366 Solutions to Exercises 369 References 405 Index 407

Read more less

Book SynopsisMaking VHDL a simple and easy-to-use hardware description language Many engineers encountering VHDL (very high speed integrated circuits hardware description language) for the first time can feel overwhelmed by it. This book bridges the gap between the VHDL language and the hardware that results from logic synthesis with clear organisation, progressing from the basics of combinational logic, types, and operators; through special structures such as tristate buses, register banks and memories, to advanced themes such as developing your own packages, writing test benches and using the full range of synthesis types. This third edition has been substantially rewritten to include the new VHDL-2008 features that enable synthesis of fixed-point and floating-point hardware. Extensively updated throughout to reflect modern logic synthesis usage, it also contains a complete case study to demonstrate the updated features. Features to this edition include: a coTable of ContentsPreface xi List of Figures xv List of Tables xvii 1 Introduction 1 1.1 The VHDL Design Cycle 1 1.2 The Origins of VHDL 2 1.3 The Standardisation Process 3 1.4 Unification of VHDL Standards 4 1.5 Portability 4 2 Register-Transfer Level Design 7 2.1 The RTL Design Stages 8 2.2 Example Circuit 8 2.3 Identify the Data Operations 10 2.4 Determine the Data Precision 12 2.5 Choose Resources to Provide 12 2.6 Allocate Operations to Resources 13 2.7 Design the Controller 14 2.8 Design the Reset Mechanism 15 2.9 VHDL Description of the RTL Design 15 2.10 Synthesis Results 16 3 Combinational Logic 19 3.1 Design Units 19 3.2 Entities and Architectures 20 3.3 Simulation Model 22 3.4 Synthesis Templates 25 3.5 Signals and Ports 27 3.6 Initial Values 29 3.7 Simple Signal Assignments 30 3.8 Conditional Signal Assignments 31 3.9 Selected Signal Assignment 33 3.10 Worked Example 34 4 Basic Types 37 4.1 Synthesisable Types 37 4.2 Standard Types 37 4.3 Standard Operators 38 4.4 Type Bit 39 4.5 Type Boolean 39 4.6 Integer Types 41 4.7 Enumeration Types 46 4.8 Multi-Valued Logic Types 47 4.9 Records 48 4.10 Arrays 49 4.11 Aggregates, Strings and Bit-Strings 53 4.12 Attributes 56 4.13 More on Selected Signal Assignments 60 5 Operators 63 5.1 The Standard Operators 63 5.2 Operator Precedence 64 5.3 Boolean Operators 70 5.4 Comparison Operators 73 5.5 Shifting Operators 76 5.6 Arithmetic Operators 79 5.7 Concatenation Operator 84 6 Synthesis Types 85 6.1 Synthesis Type System 85 6.2 Making the Packages Visible 87 6.3 Logic Types – Std_Logic_1164 90 6.4 Numeric Types – Numeric_Std 95 6.5 Fixed-Point Types – Fixed_Pkg 105 6.6 Floating-Point Types – Float_Pkg 119 6.7 Type Conversions 134 6.8 Constant Values 144 6.9 Mixing Types in Expressions 146 6.10 Top-Level Interface 147 7 Std_Logic_Arith 151 7.1 The Std_Logic_Arith Package 151 7.2 Contents of Std_Logic_Arith 152 7.3 Type Conversions 161 7.4 Constant Values 162 7.5 Mixing Types in Expressions 164 8 Sequential VHDL 167 8.1 Processes 167 8.2 Signal Assignments 170 8.3 Variables 171 8.4 If Statements 172 8.5 Case Statements 177 8.6 Latch Inference 178 8.7 Loops 181 8.8 Worked Example 187 9 Registers 191 9.1 Basic D-Type Register 191 9.2 Simulation Model 192 9.3 Synthesis Model 193 9.4 Register Templates 195 9.5 Register Types 199 9.6 Clock Types 199 9.7 Clock Gating 200 9.8 Data Gating 201 9.9 Asynchronous Reset 203 9.10 Synchronous Reset 208 9.11 Registered Variables 210 9.12 Initial Values 211 10 Hierarchy 213 10.1 The Role of Components 213 10.2 Indirect Binding 214 10.3 Direct Binding 219 10.4 Component Packages 220 10.5 Parameterised Components 222 10.6 Generate Statements 225 10.7 Worked Examples 230 11 Subprograms 243 11.1 The Role of Subprograms 243 11.2 Functions 243 11.3 Operators 254 11.4 Type Conversions 258 11.5 Procedures 261 11.6 Declaring Subprograms 267 11.7 Worked Example 270 12 Special Structures 279 12.1 Tristates 279 12.2 Finite State Machines 284 12.3 RAMs and Register Banks 292 12.4 Decoders and ROMs 297 13 Test Benches 301 13.1 Test Benches 301 13.2 Combinational Test Bench 302 13.3 Verifying Responses 305 13.4 Clocks and Resets 307 13.5 Other Standard Types 310 13.6 Don’t Care Outputs 312 13.7 Printing Response Values 314 13.8 Using TextIO to Read Data Files 315 13.9 Reading Standard Types 318 13.10 TextIO Error Handling 319 13.11 TextIO for Synthesis Types 321 13.12 TextIO for User-Defined Types 322 13.13 Worked Example 325 14 Libraries 327 14.1 The Library 327 14.2 Library Names 328 14.3 Library Work 329 14.4 Standard Libraries 330 14.5 Organising Your Files 333 14.6 Incremental Compilation 335 15 Case Study 337 15.1 Specification 337 15.2 System-Level Design 338 15.3 RTL Design 340 15.4 Trial Synthesis 352 15.5 Testing the Design 353 15.6 Floating-Point Version 361 15.7 Final Synthesis 362 15.8 Generic Version 364 15.9 Conclusions 366 Appendix A Package Listings 369 A.1 Package Standard 369 A.2 Package Standard_Additions 373 A.3 Package Std_Logic_1164 380 A.4 Package Std_Logic_1164_Additions 383 A.5 Package Numeric_Std 389 A.6 Package Numeric_Std_Additions 393 A.7 Package Fixed_Float_Types 400 A.8 Package Fixed_Pkg 401 A.9 Package Float_Pkg 415 A.10 Package TextIO 429 A.11 Package Standard_Textio_Additions 431 A.12 Package Std_Logic_Arith 432 A.13 Package Math_Real 436 Appendix B Syntax Reference 439 B.1 Keywords 439 B.2 Design Units 440 B.3 Concurrent Statements 441 B.4 Sequential Statements 443 B.5 Expressions 444 B.6 Declarations 445 References 449 Index 451

Read more less

Book SynopsisSecurity of Energy supply is a major concern for all modern societies, intensified by skyrocketing demand in India and China and increasing international competition over fossil fuel deposits. Energy Security: An Interdisciplinary Approach gives A comparative analysis from both consumers'' and producers'' perspectives. It uniquely combines economics, geology, international relations, business, history, public management and political science, in one comprehensive volume, highlighting the vulnerabilities and need to move to more sustainable energy sources. The author provides a number of useful case studies to demonstrate the theory, including perspectives from consuming regions such as the United States, the European Union, and China, and from exporting regions; the Middle East, Africa, Russia and the Caspian Sea. Key features include: coverage on theoretical and empirical frameworks so readers are able to analyse concepts relevant to new laws and pTable of ContentsAbout the Author. Preface. Acknowledgements. List of Abbreviations. Glossary. 1 Introduction. 1.1 Energy Security. 1.2 Diversification of Energy Mix. 1.3 Conclusion. 2 United States. 2.1 Oil. 2.2 Natural Gas. 2.3 Coal. 2.4 Nuclear Power. 2.5 Ethanol. 2.6 The Quest for an Energy Strategy. 2.7 Conclusion: the Way Forward. 3 European Union. 3.1 The EU Energy Outlook. 3.2 Russia. 3.3 Central Asia/Caspian Sea Region. 3.4 Mediterranean Sea. 3.5 Gulf Cooperation Council. 3.6 Turkey. 3.7 Conclusion: the Way Ahead. 4 China. 4.1 Regulatory Authority. 4.2 Oil. 4.3 Coal. 4.4 Natural Gas. 4.5 Nuclear Power. 4.6 Renewable Energy. 4.7 Overseas Exploration and Production. 4.8 Conclusion. 5 Persian Gulf. 5.1 Socio-economic and Political Challenges. 5.2 Saudi Arabia. 5.3 Iran. 5.4 Iraq. 5.5 Conclusion: the Way Forward. 6 Africa. 6.1 Algeria. 6.2 Libya. 6.3 Egypt. 6.4 Sudan. 6.5 Angola. 6.6 Nigeria. 6.7 United States and Africa. 6.8 Europe and Africa. 6.9 Conclusion: the Way Ahead. 7 Caspian Sea. 7.1 Hydrocarbon Resources - An Assessment. 7.2 The Legal Status of the Caspian Sea. 7.3 Geopolitical Rivalry and Pipeline Diplomacy. 7.4 Conclusion: the Way Forward. 8 Russia. 8.1 Oil Sector. 8.2 Natural Gas. 8.3 The Energy Strategy - 2030. 8.4 The Arctic Hydrocarbons. 8.5 Russia-EU Energy Partnership. 8.6 Russia, the Middle East, and OPEC. 8.7 Energy Sector Organization. 8.8 Conclusion: the Way Forward. 9 OPEC and Gas-OPEC. 9.1 OPEC: History and Evolution. 9.2 OPEC: Objectives, Membership, and Organization. 9.3 OPEC Summits. 9.4 OPEC Long-Term Strategy. 9.5 Gas OPEC. 9.6 GECF and OPEC. 9.7 Oil vs. Gas. 9.8 Conclusion. 10 International Energy Agency. 10.1 The Founding of the IEA. 10.2 The International Energy Program. 10.3 Structure of the IEA. 10.4 Energy Security. 10.5 How Did the System Work?. 10.6 Conclusion. 11 Conclusion. 11.1 Energy Security. 11.2 The International Energy Forum (IEF). 11.3 Joint Oil Data Initiative. 11.4 Conclusion: the Way Forward. Index.

Read more less

Book SynopsisWritten by international experts in the field, this book covers the standards, architecture and deployment issues related to IP-based emergency services This book brings together contributions from experts on technical and operational aspects within the international standardisation and regulatory processes relating to routing and handling of IP-based emergency calls. Readers will learn how these standards work, how various standardization organizations contributed to them and about pilot projects, early deployment and current regulatory situation. Key Features: Provides an overview of how the standards related to IP-based emergency services work, and how various organizations contributed to them Focuses on SIP and IMS-based communication systems for the Internet Covers standards, architecture and deployment issues International focus, with coverage of the major national efforts in this area Written Trade Review“In addition, practitioners, product architects, and developers will find interesting and useful ideas. Many parts of the book can be recommended to experts working on standards and regulations.” (IEEE Communications Magazine, 1 February 2015) Table of ContentsList of Figures xiii List of Tables xvii List of Contributors xix Preface xxi Acknowledgments xxv Acronyms xxvii 1 Introduction 1 1.1 History 1 1.2 Overview 5 1.3 Building Blocks 8 1.3.1 Recognizing Emergency Calls 8 1.3.2 Obtaining and Conveying Location Information 9 1.3.3 Routing Emergency Calls 9 2 Location: Formats, Encoding and Protocols 11 2.1 Applying the PIDF-LO civicAddress Type to US Addresses 14 2.1.1 Introduction: The Context and Purpose of PIDF-LO and CLDXF 15 2.1.2 CLDXF Elements 17 2.1.3 Conclusion 30 2.2 DHCP as a Location Configuration Protocol (LCP) 31 2.2.1 What’s New in RFC 6225? 32 2.2.2 DHCPv4 and DHCPv6 Option Formats 32 2.2.3 Option Support 35 2.2.4 Latitude and Longitude Fields 36 2.2.5 Altitude 36 2.2.6 Datum 37 2.3 Geography Markup Language (GML) 37 2.3.1 Introduction 37 2.3.2 Overview of the OGC 38 2.3.3 The OGC Geography Markup Language (GML) 38 2.3.4 Conclusion 47 2.4 A Taxonomy of the IETF HELD Protocol 47 2.4.1 The LIS and HELD 48 2.4.2 LIS Discovery 48 2.4.3 Basic HELD 53 2.4.4 HELD Target Identities and Third-Party Requests 59 2.4.5 HELD Measurements 62 2.4.6 HELD as a Dereference Protocol 64 2.4.7 HELD Policy URIs 66 2.4.8 HELD Device Capabilities 69 2.5 OMA Enablers and Emergency Services 72 2.5.1 SUPL 73 2.5.2 MLS 84 2.5.3 MLP 85 2.5.4 LOCSIP 89 2.6 3GPP Location Protocols 92 2.6.1 Introduction 92 2.6.2 Location Technology in 3GPP Networks 93 2.6.3 Emergency Location Information in 3GPP CS Domain, Control Plane 100 2.6.4 Emergency Location Information in the IMS 100 3 Architectures 103 3.1 NENA i2 104 3.1.1 Background 104 3.1.2 The i2 Architecture 105 3.1.3 Regulatory Situation and Deployment Status 117 3.2 NENA i3 119 3.2.1 History 119 3.2.2 Emergency Services IP Networks 120 3.2.3 Signaling and Routing IP-Originated Calls 121 3.2.4 Legacy Wireline and Wireless Origination 122 3.2.5 Emergency Events 123 3.2.6 Routing Calls Within the ESInet 123 3.2.7 Provisioning the ECRF 124 3.2.8 PSAPs 125 3.2.9 Other i3 Features 126 3.3 IETF Emergency Services for Internet Multimedia 126 3.3.1 Introduction 126 3.3.2 Recognizing Emergency Calls 128 3.3.3 Obtaining and Conveying Location Information 128 3.3.4 Routing Emergency Calls 129 3.3.5 Obligations 130 3.3.6 LoST Mapping Architecture 132 3.3.7 Steps Toward an IETF Emergency Services Architecture 135 3.3.8 Summary 138 3.4 Emergency Services Support in WiFi Networks 139 3.4.1 Introduction 139 3.4.2 Location Configuration 140 3.4.3 Support for Emergency Services 141 3.4.4 Support for Emergency Alert Systems 142 3.5 WiMAX 142 3.5.1 The WiMAX Network Architecture 143 3.5.2 Network Architecture for Emergency Services Support 148 3.5.3 The Fundamental Building Blocks 150 3.5.4 Roaming Considerations and Network Entry 152 3.5.5 Limited Access 154 3.5.6 Location Support in WiMAX 157 3.5.7 Conclusion 163 3.6 3GPP 163 3.6.1 Introduction 163 3.6.2 Requirements 164 3.6.3 Emergency Calls in the CS Domain 169 3.6.4 Emergency Calls in PS Domain 176 3.6.5 Identified Overload Problems 189 4 Deployment Examples 193 4.1 Emergency Calling in Sweden 195 4.1.1 Introduction 195 4.1.2 Overview 196 4.1.3 Protocols for PSAP Interconnection 198 4.1.4 Protocol Standards 200 4.1.5 Media 201 4.1.6 Emergency Call Routing 201 4.1.7 Testing 201 4.1.8 Examples 201 4.2 UK Specification for Locating VoIP Callers 209 4.2.1 Introduction 209 4.2.2 The Regulatory Environment 209 4.2.3 Standards Development 210 4.2.4 The Current UK Emergency Services Structure 210 4.2.5 Principles Driving the Specification 211 4.2.6 Putting It All Together 213 4.2.7 Implications for Access Network Providers 215 4.3 Implementation of VoIP 9-1-1 Services in Canada 216 4.3.1 Regulatory Framework (About the CRTC) 217 4.3.2 Canada’s Telecom Profile 217 4.3.3 Interim Solution for Nomadic and Fixed/Non-Native VoIP 220 4.3.4 The (Defunct) Canadian i2 Proposal 222 4.3.5 VoIP Regulatory Processes, Decisions and Milestones 227 4.3.6 Lessons Learned 229 4.3.7 Conclusion 230 4.4 US/Indiana Wireless Direct Network Project 230 4.4.1 Background and History of the IWDN 231 4.4.2 The IWDN Crossroads Project 231 4.4.3 The IN911 IP Network 232 4.4.4 Conclusion 235 5 Security for IP-Based Emergency Services 237 5.1 Introduction 237 5.2 Communication Model 238 5.3 Adversary Models and Security Threats 240 5.4 Security Threats 241 5.4.1 Denial-of-Service Attacks 242 5.4.2 Attacks Involving the Emergency Identifier 242 5.4.3 Attacks Against the Mapping System 243 5.4.4 Attacks Against the Location Information Server 244 5.4.5 Swatting 245 5.4.6 Attacks to Prevent a Specific Individual From Receiving Aid 246 5.4.7 Attacks to Gain Information About an Emergency 246 5.4.8 Interfering With the LIS and LoST Server Discovery Procedure 246 5.4.9 Call Identity Spoofing 247 5.5 Countermeasures 248 5.5.1 Discovery 248 5.5.2 Secure Session Setup and Caller Identity 250 5.5.3 Media Exchange 251 5.5.4 Mapping Database Security 251 6 Emergency Services for Persons With Disabilities 253 6.1 What Is Specific with Communication for People with Disabilities? 253 6.1.1 Important Characteristics of Regular Voice Telephony 253 6.1.2 Important Characteristics of Accessible Conversational Services Suitable for People with Disabilities 254 6.2 Reality Today 255 6.3 Interpretation of the Term “Equivalent Service” 255 6.4 Sad History 256 6.5 Policy and Regulation Support 256 6.5.1 UN Convention on the Rights of Persons with Disabilities 256 6.5.2 The European Union Universal Service Directive 257 6.5.3 The Telecom Act and Public Procurement Act in the United States 257 6.5.4 Americans With Disability Act 257 6.5.5 Relay Service Regulation in the United States 258 6.6 Good Opportunities in IP-Based Services 258 6.7 Implementation Experience 260 7 Regulatory Situation 261 7.1 Regulatory Aspects of Emergency Services in the United States 262 7.1.1 Introduction 262 7.1.2 Background 262 7.1.3 E9-1-1 Requirements 263 7.2 Regulatory Aspects of Emergency Services in the European Union 266 7.2.1 Introduction 266 7.2.2 Regulatory Development of Emergency Services Under EU Law 267 7.2.3 Current Legal Framework 267 7.2.4 New Legal Framework 274 7.2.5 Emergency Regulation Outside of the EU Telecom Regulatory Framework 276 7.2.6 Conclusion 276 8 Research Projects and Pilots 279 8.1 REACH112: Responding to All Citizens Needing Help 280 8.1.1 Outline 280 8.1.2 Emergency Service Access 282 8.1.3 The Obstacles 284 8.1.4 Conclusion 288 8.2 PEACE: IP-Based Emergency Applications and Services for Next-Generation Networks 288 8.2.1 Introduction 288 8.2.2 Project Scope 289 8.2.3 Development Status 291 8.3 US Department of Transportation’s NG 9-1-1 Pilot Project 298 8.3.1 Overview 298 8.3.2 Proof-of-Concept Description 300 8.3.3 Testing 313 8.3.4 Conclusion 317 9 Organizations 321 9.1 ETSI EMTEL 322 9.1.1 Purpose of ETSI Special Committee EMTEL (Emergency Communications) 322 9.1.2 Main Features of EMTEL 322 9.1.3 Scope of ETSI SC EMTEL Work 323 9.1.4 Operation and Activities of SC EMTEL 324 9.1.5 EMTEL Evolution and Strategy 324 9.1.6 Vision for Future Emergency Services 325 9.2 NENA 326 9.3 EENA 327 9.3.1 What Is EENA? 327 9.3.2 What EENA Does? 327 9.3.3 What Are the EENA Memberships? 328 9.4 Ecma International 330 9.4.1 Ecma International 330 9.4.2 Ecma Technical Committee TC32 331 9.4.3 ECMA TR/101, Next Generation Corporate Networks (NGCN) – Emergency Calls 331 9.5 ATIS 332 9.5.1 Emergency Services Interconnection Forum (ESIF) 332 9.5.2 Next-Generation Emergency Services (NGES) Subcommittee 333 9.5.3 Example ESIF Issues 334 9.5.4 Summary 336 9.6 The NG9-1-1 Caucus and the NG9-1-1 Institute 336 9.7 COCOM EGEA 338 10 Conclusion and Outlook 341 10.1 Location 341 10.2 Architectures 342 10.3 Deployments 343 10.4 Security and Privacy 344 10.5 Emergency Services for Persons with Disabilities 344 10.6 Regulation 345 10.7 Research Projects and Pilots 345 10.8 Funding 346 References 349 Index 363

Read more less

Book SynopsisA comprehensive, encompassing and accessible text examining a wide range of key Wireless Networking and Localization technologies This book provides a unified treatment of issues related to all wireless access and wireless localization techniques. The book reflects principles of design and deployment of infrastructure for wireless access and localization for wide, local, and personal networking. Description of wireless access methods includes design and deployment of traditional TDMA and CDMA technologies and emerging Long Term Evolution (LTE) techniques for wide area cellular networks, the IEEE 802.11/WiFi wireless local area networks as well as IEEE 802.15 Bluetooth, ZigBee, Ultra Wideband (UWB), RF Microwave and body area networks used for sensor and ad hoc networks. The principles of wireless localization techniques using time-of-arrival and received-signal-strength of the wireless signal used in military and commercial applications in smart devices operating in urTable of ContentsPreface xv 1 Introduction 1 1.1 Introduction 1 1.2 Elements of Information Networks 3 1.3 Evolution of Wireless Access to the PSTN 17 1.4 Evolution of Wireless Access to the Internet 21 1.5 Evolution of Wireless Localization Technologies 27 1.6 Structure of this Book 29 Part I PRINCIPLES OF AIR–INTERFERENCE DESIGN 2 Characteristics of the Wireless Medium 39 2.1 Introduction 39 2.2 Modeling of Large-scale RSS, Path Loss, and Shadow Fading 45 2.3 Modeling of RSS Fluctuations and Doppler Spectrum 60 2.4 Wideband Modeling of Multipath Characteristics 72 2.5 Emerging Channel Models 79 Appendix A2: What Is the Decibel? 84 3 Physical Layer Alternatives forWireless Networks 99 3.1 Introduction 99 3.2 Physical Layer Basics: Data rate, Bandwidth, and Power 100 3.3 Performance in Multipath Wireless Channels 107 3.4 Wireless Transmission Techniques 112 3.5 Multipath Resistant Techniques 120 3.6 Coding Techniques for Wireless Communications 136 3.7 Cognitive Radio and Dynamic Spectrum Access 145 Appendix A3 145 4 Medium Access Methods 153 4.1 Introduction 153 4.2 Centralized Assigned-Access Schemes 155 4.3 Distributed Random Access for Data Oriented Networks 173 4.4 Integration of Voice and Data Traffic 195 Part II PRINCIPLES OF NETWORK INFRASTRUCTURE DESIGN 5 Deployment ofWireless Networks 217 5.1 Introduction 217 5.2 Wireless Network Architectures 218 5.3 Interference in Wireless Networks 224 5.4 Deployment of Wireless LANs 233 5.5 Cellular Topology, Cell Fundamentals, and Frequency Reuse 238 5.6 Capacity Expansion Techniques 248 5.7 Network Planning for CDMA Systems 268 5.8 Femtocells 270 6 Wireless Network Operations 275 6.1 Introduction 275 6.2 Cell Search and Registration 281 6.3 Mobility Management 283 6.4 Radio Resources and Power Management 301 7 Wireless Network Security 321 7.1 Introduction 321 7.2 Security in Wireless Local Networks 324 7.3 Security in Wireless Personal Networks 330 7.4 Security in Wide Area Wireless Networks 334 7.5 Miscellaneous Issues 340 Appendix A7: An Overview of Cryptography and Cryptographic Protocols 341 Part III WIRELESS LOCAL ACCESS 8 Wireless LANs 357 8.1 Introduction 357 8.2 Wireless Local Area Networks and Standards 363 8.3 IEEE 802.11 WLAN Operations 369 9 Low Power Sensor Networks 405 9.1 Introduction 405 9.2 Bluetooth 406 9.3 IEEE 802.15.4 and ZigBee 424 9.4 IEEE 802.15.6 Body Area Networks 434 10 GigabitWireless 447 10.1 Introduction 447 10.2 UWB Communications at 3.1–10.6 GHz 451 10.3 Gigabit Wireless at 60 GHz 467 Part IV WIDE AREA WIRELESS ACCESS 11 TDMA Cellular Systems 479 11.1 Introduction 479 11.2 What is TDMA Cellular? 480 11.3 Mechanisms to Support a Mobile Environment 486 11.4 Communication Protocols 491 11.5 Channel Models for Cellular Networks 501 11.6 Transmission Techniques in TDMA Cellular 508 11.7 Evolution of TDMA for Internet Access 512 12 CDMA Cellular Systems 519 12.1 Introduction 519 12.2 Why CDMA? 520 12.3 CDMA Based Cellular Systems 521 12.4 Direct Sequence Spread Spectrum 522 12.5 Communication Channels and Protocols in Example CDMA Systems 534 12.6 Cell Search, Mobility, and Radio Resource Management in CDMA 546 12.7 High Speed Packet Access 554 13 OFDM and MIMO Cellular Systems 561 13.1 Introduction 561 13.2 Why OFDM? 562 13.3 Multiple Input Multiple Output 572 13.4 WiMax 576 13.5 Long Term Evolution 582 13.6 LTE Advanced 591 Part V WIRELESS LOCALIZATION 14 Geolocation Systems 597 14.1 Introduction 597 14.2 What is Wireless Geolocation? 598 14.3 RF Location Sensing and Positioning Methodologies 602 14.4 Location Services Architecture for Cellular Systems 613 14.5 Positioning in Ad Hoc and Sensor Networks 620 15 Fundamentals of RF Localization 625 15.1 Introduction 625 15.2 Modeling of the Behavior of RF Sensors 626 15.3 Performance Bounds for Ranging 631 15.4 Wireless Positioning Algorithms 639 16 Wireless Localization in Practice 653 16.1 Introduction 653 16.2 Emergence of Wi-Fi Localization 653 16.3 Comparison of Wi-Fi Localization Systems 657 16.4 Practical TOA Measurement 665 16.5 Localization in the Absence of DP 669 16.6 Challenges in Localization inside the Human Body 675 References 687 Index 701

Read more less

Book SynopsisMost of the intriguing social phenomena of our time, such as international terrorism, social inequality, and urban ethnic segregation, are consequences of complex forms of agent interaction that are difficult to observe methodically and experimentally. This book looks at a new research stream that makes use of advanced computer simulation modelling techniques to spotlight agent interaction that allows us to explain the emergence of social patterns. It presents a method to pursue analytical sociology investigations that look at relevant social mechanisms in various empirical situations, such as markets, urban cities, and organisations. This book: Provides a comprehensive introduction to epistemological, theoretical and methodological features of agent-based modelling in sociology through various discussions and examples. Presents the pros and cons of using agent-based models in sociology. Explores agent-based models in combining quantitative and Trade Review“This book should be inserted into all sociological libraries as a vanguard for the rest of us - if it not torn to shreds by enraged sociologists it will very usefully inform them. Newcomers to ABM and even old hands, but especially those who have to survive within sociology, will find it a very valuable asset.” (Journal of Artificial Societies and Social Simulation, 1 January 2013) Table of ContentsPreface ix 1 What is agent-based computational sociology all about? 1 1.1 Predecessors and fathers 3 1.2 The main ideas of agent-based computational sociology 9 1.2.1 The primacy of models 9 1.2.2 The generative approach 11 1.2.3 The micro–macro link 13 1.2.4 Process and change 15 1.2.5 The unexcluded middle 16 1.2.6 Trans-disciplinarity 17 1.3 What are ABMs? 18 1.4 A classification of ABM use in social research 20 References 26 2 Cooperation, coordination and social norms 33 2.1 Direct reciprocity and the persistence of interaction 36 2.2 Strong reciprocity and social sanctions 42 2.3 Disproportionate prior exposure 49 2.4 Partner selection 54 2.5 Reputation 62 2.6 The emergence of conventions 69 References 78 3 Social influence 85 3.1 Segregation dynamics 88 3.2 Threshold behavior and opinions 97 3.3 Culture dynamics and diversity 103 3.4 Social reflexivity 109 References 122 4 The methodology 131 4.1 The method 134 4.2 Replication 140 4.2.1 The querelle about segregation 144 4.2.2 The querelle about trust and mobility 147 4.3 Multi-level empirical validation 151 References 159 5 Conclusions 165 References 172 Appendix A 175 A. 1 Research centers 175 A. 2 Scientific associations 177 A. 3 Journals 178 A. 4 Simulation tools 179 References 179 Appendix B 181 B. 1 Example I: Partner selection and dynamic networks (Boero, Bravo and Squazzoni 2010) 182 B. 2 Example II: Reputation (Boero et al. 2010) 211 References 234 Index 235

Read more less

Book SynopsisThis book is not only of practical value. It''s also a lot of fun to read. Michael Jackson, The Open University. Do you need to know how to create good requirements? Discovering Requirements offers a set of simple, robust, and effective cognitive tools for building requirements. Using worked examples throughout the text, it shows you how to develop an understanding of any problem, leading to questions such as: What are you trying to achieve? Who is involved, and how? What do those people want? Do they agree? How do you envisage this working? What could go wrong? Why are you making these decisions? What are you assuming? The established author team of Ian Alexander and Ljerka Beus-Dukic answer these and related questions, using a set of complementary techniques, including stakeholder analysis, goal modelling, context modelling, storytelling and scenario modelling, identifying rTable of ContentsAcknowledgements xv Foreword xvii Part I: Discovering Requirement Elements 1 1 Introduction 3 1.1 Summary 4 1.2 Why You Should Read This Book 4 1.3 Simple but Not Easy 6 1.4 Discovered, Not Found 7 1.4.1 Many Different Situations 9 1.5 A Softer Process, at First 12 1.6 More than a List of ‘The System Shalls’ 16 1.6.1 A Network of Requirement Elements 16 1.6.2 Discovery as Search 18 1.7 A Minimum of Process: The Discovery Cycle 18 1.8 The Structure of this Book 20 1.8.1 Part I: Discovering Requirement Elements 21 1.8.2 Part II: Contexts for Discovery 22 1.9 Further Reading 22 1.9.1 Books on ‘Softer’ Approaches 22 1.9.2 Books on the Philosophical Background 23 1.9.3 Books on ‘Harder’ Approaches 24 2 Stakeholders 27 2.1 Summary 28 2.2 Discovering Stakeholders 28 2.2.1 Operational Stakeholders within ‘The System’ 30 2.2.2 Stakeholders in the Containing System and Wider Environment 30 2.3 Identifying Stakeholders 37 2.3.1 From your Sponsor or Client 37 2.3.2 With a Template such as the Onion Model 37 2.3.3 By Comparison with Similar Projects 40 2.3.4 By Analysing Context 40 2.4 Managing Your Stakeholders 41 2.4.1 Engaging with Stakeholders 41 2.4.2 Keeping Track of Stakeholders 42 2.4.3 Analysing Influences 42 2.4.4 Prioritising Stakeholders 43 2.4.5 Involving Stakeholders 45 2.4.6 The Integrated Project Team 45 2.5 Validating Your List of Stakeholders 45 2.5.1 Things To Check the Stakeholder Analysis Against 46 2.6 The Bare Minimum of Stakeholder Analysis 46 2.7 Next Steps: Requirements from Stakeholders 46 2.8 Exercise 49 2.9 Further Reading 49 3 Goals 51 3.1 Summary 52 3.2 Discovering Goals 52 3.2.1 Worked Example: Goals for a Spacecraft 54 3.2.2 Worked Example: Goals for a Restaurant 57 3.2.3 Worked Example: Tram Goals and Trade-offs 59 3.2.4 Finding Solutions to Goal Conflicts 62 3.2.5 Contexts for Discovering Goals 63 3.2.6 The Negative Side 65 3.3 Documenting Goals 68 3.3.1 Drawing Goal Diagrams 69 3.3.2 Other Ways of Documenting Goals 69 3.4 Validating Goals 71 3.4.1 Things To Check Goals Against 73 3.5 The Bare Minimum of Goals 73 3.6 Next Steps 73 3.7 Exercises 73 3.8 Further Reading 74 3.8.1 Goals 74 3.8.2 The Negative Side 74 3.8.3 The i∗ Goal Modelling Notation 74 4 Context, Interfaces, Scope 75 4.1 Summary 76 4.2 Introduction 76 4.3 A ‘Soft Systems’ Approach for Ill-Defined Boundaries 77 4.3.1 You are Part of the Soft System you are Observing 78 4.3.2 From Stakeholders to Boundaries 79 4.3.3 Identifying Interfaces 83 4.3.4 Documenting Interfaces 84 4.3.5 Validating your Choice of Boundary 86 4.4 Switching to a ‘Hard Systems’ Approach for Known Events 87 4.4.1 The Traditional Context Diagram 87 4.4.2 Scope as a List of Events 87 4.4.3 Expressing Event-handling Functions 89 4.4.4 Strengths and Weaknesses of Context Diagrams 92 4.4.5 Validating Interfaces and Events 93 4.4.6 Things To Check Context and Interfaces Against 95 4.5 The Bare Minimum of Context 95 4.6 Next Steps 95 4.7 Exercise 95 4.8 Further Reading 96 4.8.1 Soft Approaches 96 4.8.2 Event-Driven Approaches 96 4.8.3 Writing Requirements 96 5 Scenarios 97 5.1 Summary 98 5.2 Discovering Scenarios 98 5.2.1 Interviews, storytelling 99 5.2.2 Scenario Workshops 101 5.2.3 Discovering Negative Scenarios 107 5.3 Documenting Scenarios 114 5.3.1 Index Cards, User Stories 115 5.3.2 Storyboards 116 5.3.3 Operational Scenarios 118 5.3.4 Use Cases 119 5.4 Summary 124 5.5 Validating Scenarios 124 5.5.1 Scenario Walkthroughs 124 5.5.2 Animation, Simulation, Prototyping 126 5.5.3 Things To Check Scenarios Against 127 5.6 The Bare Minimum of Scenarios 127 5.7 Next Steps 127 5.8 Exercises 128 5.9 Further Reading 128 5.9.1 Storytelling 128 5.9.2 Alternative Scenario Approaches 128 5.9.3 Running Scenario Workshops 129 5.9.4 The Principle of Commensurate Care 129 6 Qualities and Constraints 131 6.1 Summary 132 6.2 What are Qualities and Constraints? 132 6.2.1 A Rich Mixture 132 6.2.2 Qualities that Govern Choices 132 6.2.3 Constraints that Matter to People 133 6.3 Discovering Qualities and Constraints 133 6.3.1 Using Goals to Discover Qualities and Constraints 134 6.3.2 Stakeholder Analysis to Discover Qualities and Constraints 136 6.3.3 Using a Checklist to Discover Qualities and Constraints 136 6.4 Documenting Qualities and Constraints 141 6.4.1 Constraints 142 6.4.2 Development (Process) Qualities 146 6.4.3 Usage (Product) Qualities 147 6.5 Validating Qualities and Constraints 157 6.5.1 Things To Check Qualities and Constraints Against 158 6.6 The Bare Minimum of Qualities and Constraints 159 6.7 Next Steps 159 6.8 Exercises 159 6.9 Further Reading 160 7 Rationale and Assumptions 161 7.1 Summary 162 7.2 The Value of Rationale 162 7.3 Discovering Rationale and Assumptions 163 7.3.1 Asking Why 164 7.3.2 Looking for the word ‘will’ in vision statements, plans, etc 165 7.3.3 Rationalising a Set of Requirements 166 7.3.4 Inverting Risks 168 7.4 Documenting Rationale 169 7.4.1 Justification Text Field 171 7.4.2 Lists of Assumptions, Risks, Issues and Decisions 172 7.4.3 Traceability to Goals, Assumptions, etc 173 7.4.4 Rationale Models 178 7.4.5 The Goal Structuring Notation (GSN) 182 7.5 Validating Rationale and Assumptions 183 7.5.1 Rationale Walkthrough 184 7.5.2 Analysis of Traceability 184 7.5.3 Things To Check Rationale and Assumptions Against 186 7.6 The Bare Minimum of Rationale and Assumptions 187 7.7 Next Steps 187 7.8 Exercise 187 7.9 Further Reading 187 7.9.1 Discovering Assumptions 187 7.9.2 Reasoning 188 7.9.3 Modelling Rationale 188 7.9.4 Tracing to Goals 188 7.9.5 Goal Structuring Notation (GSN) 188 7.9.6 Satisfaction Arguments 188 8 Definitions 189 8.1 Summary 190 8.2 Discovering Definitions 190 8.2.1 Synonyms 191 8.2.2 Homonyms 193 8.3 Constructing the Project Dictionary 194 8.3.1 Acronyms 195 8.3.2 Definitions and Designations 195 8.3.3 Roles (Operational Stakeholders) 199 8.3.4 Data Definitions 201 8.3.5 Constraints as Data 202 8.4 Validating the Project Dictionary 204 8.4.1 Validating Data Models 205 8.4.2 Things To Check Definitions Against 206 8.5 The Bare Minimum of Definitions 206 8.6 Next Steps 206 8.7 Exercise 206 8.8 Further Reading 206 8.8.1 Definitions and Designations 206 8.8.2 Data Modelling 207 9 Measurements 209 9.1 Summary 210 9.2 Discovering and Documenting Acceptance Criteria 211 9.2.1 Acceptance Criteria for Behavioural Requirements 212 9.2.2 Acceptance Criteria for Qualities 216 9.2.3 Acceptance Criteria for Constraints 218 9.2.4 Verification Method 219 9.3 Validating Acceptance Criteria 222 9.3.1 Testing from Day One 222 9.4 Measuring Quality of Service (QoS) 223 9.4.1 Example Service: Office Carpeting 224 9.4.2 Two Opposite Approaches 225 9.4.3 A Spectrum of Service Approaches 226 9.4.4 Worked Example: QoS Measures for Food Preparation Services 228 9.5 Validating QoS Measures 230 9.5.1 Qualities of a Good QoS Measure 230 9.5.2 Will your QoS Measures Work? 231 9.5.3 Common QoS Measures 232 9.5.4 Validating QoS with Negative Scenarios 232 9.5.5 Things To Check Measurements Against 233 9.6 The Bare Minimum of Measurement 233 9.7 Next Steps 233 9.8 Exercise 233 9.9 Further Reading 233 10 Priorities 235 10.1 Summary 236 10.2 Two Kinds of Priority 236 10.3 Input Priority 237 10.3.1 Discovering Input Priority 237 10.3.2 Documenting Input Priority 241 10.3.3 Validating Input Priority 242 10.4 Output Priority 243 10.4.1 Discovering Output Priority 243 10.4.2 Documenting Output Priority 251 10.4.3 Validating Output Priority 253 10.5 Things To Check Priority Against 254 10.6 The Bare Minimum of Priorities 255 10.7 Next Steps 255 10.8 Exercise 255 10.9 Further Reading 255 10.9.1 Triage 255 10.9.2 Input Priority 256 10.9.3 Boston Matrix 256 10.9.4 Review Process 256 10.9.5 Life Cycles 256 Part II: Discovery Contexts 257 11 Requirements from Individuals 259 11.1 Summary 260 11.2 Introduction 260 11.3 Interviews 261 11.3.1 Planning an Interview Campaign 261 11.3.2 Planning Each Interview 267 11.3.3 Documenting Interviews 268 11.3.4 Validating Interview Findings 273 11.4 Observation and ‘Apprenticeship’ 274 11.4.1 Making Observations 274 11.4.2 Being ‘Talked Through’ Operations 276 11.4.3 Documenting Observations 277 11.4.4 Validating Observations 280 11.5 The Bare Minimum from Individuals 280 11.6 Exercises 280 11.7 Further Reading 281 11.7.1 Interviewing 281 11.7.2 Using Video 281 11.7.3 Observation 282 11.7.4 Tacit Knowledge 282 11.7.5 Standard Types of Systems Analysis 282 11.7.6 Informal Modelling Techniques 282 11.7.7 Philosophy 282 12 Requirements from Groups 283 12.1 Summary 284 12.2 The Goal of Group Work 284 12.2.1 Unique Capabilities 284 12.2.2 Obstacles 285 12.2.3 Mediating Group Work (on one site or many) 285 12.3 Workshops 286 12.3.1 Define Workshop Mission 286 12.3.2 Workshop Planning 287 12.3.3 Workshop Rehearsal 289 12.3.4 Workshop Setup 290 12.3.5 Workshop Recording 299 12.3.6 Validating Workshop Findings 302 12.4 Group Media 305 12.4.1 Project Wall 305 12.4.2 Project Website 306 12.4.3 Project Wiki 307 12.4.4 Modelling Tool 308 12.4.5 Requirements Management Tool 309 12.4.6 Groupware and Working at a Distance 310 12.4.7 The Role of Group Media 312 12.5 The Bare Minimum from Groups 314 12.6 Next Steps 314 12.7 Exercise 314 12.8 Further Reading 315 12.8.1 Workshops 315 12.8.2 Working in Groups 315 13 Requirements from Things 317 13.1 Summary 318 13.2 Requirements Prototyping 318 13.2.1 Purpose 319 13.2.2 Techniques 319 13.3 Reverse Engineering 330 13.3.1 From an Existing Product 330 13.4 Requirements Reuse 337 13.4.1 Type 1: Naïve Reuse 337 13.4.2 Type 2: Standardisation 338 13.4.3 Type 3: Product Lines 338 13.4.4 Tool Support for Reuse 338 13.5 Validating Requirements from Things 340 13.6 The Bare Minimum from Things 340 13.7 Exercises 340 13.8 Further Reading 340 13.8.1 Prototyping 340 14 Trade-offs 343 14.1 Summary 344 14.2 Optioneering: The Engineering of Trade-offs 344 14.2.1 The Requirements-First Life-Cycle Myth 344 14.2.2 An Optioneering Life Cycle 345 14.2.3 The Optioneering Process 350 14.2.4 Selecting the Winning Option 352 14.2.5 Optioneering with PCA: A Worked Example 360 14.3 Validating your Trade-offs 367 14.4 The Bare Minimum of Trade-offs 367 14.5 Next Steps 367 14.6 Exercises 368 14.7 Further Reading 369 14.7.1 Trade-offs 369 14.7.2 Statistics 370 14.7.3 PCA 370 14.7.4 Weighting Approaches 370 14.7.5 Analytic Hierarchy Process (AHP) 370 14.7.6 Quality Function Deployment (QFD) 370 14.7.7 Questions, Options, Criteria (QOC) 371 15 Putting it all Together 373 15.1 Summary 374 15.2 After Discovery 374 15.2.1 Everything Depends on the Requirements 374 15.2.2 Principles for the Requirements Chef 375 15.3 The Right Process for your Project 376 15.3.1 Case Study: A Retail IT Project 377 15.3.2 Case Study: Transport Planning 379 15.3.3 Requirements-Driven Project Management 381 15.4 Organising the Requirements Specification 385 15.4.1 Template 385 15.4.2 Levels 385 15.4.3 Can Use Cases Do Everything? 386 15.4.4 Organising Product Functions 386 15.4.5 Traditional ‘Shalls’ 387 15.4.6 Relating Requirements of Different Types 388 15.4.7 Conflicting Needs for Requirement Organisation 390 15.4.8 The Benefit of Requirements (Traceability) Tools 390 15.4.9 An Alternative View: Competing Approaches 391 15.5 The Bare Minimum of Putting it all Together 394 15.6 Further Reading 394 15.6.1 Choosing and Tailoring Development Life Cycles 394 15.6.2 Managing Projects From Requirements 395 15.6.3 Classics for Inspiration and Reflection 395 15.6.4 A Look Ahead 396 Appendix A: Exercise Answers and Hints 397 Appendix B: Getting the Level Right 405 Appendix C: Tools for Requirements Discovery 411 Appendix D: Template 423 Bibliography 429 Glossary 433 Index 445

Read more less

Book SynopsisThis book explores social mechanisms that drive network change and link them to computationally sound models of changing structure to detect patterns. This text identifies the social processes generating these networks and how networks have evolved. Reviews: this book is easy to read and entertaining, and much can be learned from it. Even if you know just abouteverything about large-scale and temporal networks, the book is a worthwhile read; you will learn a lot about SNA literature, patents, the US Supreme Court, and European soccer. (Social Networks) a clear and accessible textbook, balancing symbolic maths, code, and visual explanations. The authors' enthusiasm for the subject matter makes it enjoyable to read (JASSS)Table of ContentsPreface xiii 1 Temporal and Spatial Networks 1 1.1 Modern Social Network Analysis 1 1.2 Network Sizes 3 1.3 Substantive Concerns 3 1.3.1 Citation Networks 3 1.3.2 Other Types of Large Networks 7 1.4 Computational Methods 10 1.5 Data for Large Temporal Networks 12 1.5.1 The Main Datasets 12 1.5.2 Secondary Datasets 14 1.6 Induction and Deduction 16 2 Foundations of Methods for Large Networks 18 2.1 Networks 18 2.1.1 Descriptions of Networks 20 2.1.2 Degrees 21 2.1.3 Descriptions of Properties 21 2.1.4 Visualizations of Properties 22 2.2 Types of Networks 22 2.2.1 Temporal Networks 23 2.2.2 Multirelational Networks 25 2.2.3 Two-mode Networks 28 2.3 Large Networks 28 2.3.1 Small and Middle Sized Networks 29 2.3.2 Large Networks 30 2.3.3 Complexity of Algorithms 30 2.4 Strategies for Analyzing Large Networks 32 2.5 Statistical Network Measures 33 2.5.1 Using Pajek and R Together 35 2.5.2 Fitting Distributions 35 2.6 Subnetworks 37 2.6.1 Clusters, Clusterings, Partitions, Hierarchies 37 2.6.2 Contractions of Clusters 38 2.6.3 Subgraphs 40 2.6.4 Cuts 42 2.7 Connectivity Properties of Networks 46 2.7.1 Walks 46 2.7.2 Equivalence Relations and Partitions 47 2.7.3 Connectivity 48 2.7.4 Condensation 49 2.7.5 Bow-tie Structure of the Web Graph 50 2.7.6 The Internal Structure of Strong Components 51 2.7.7 Bi-connectivity and -connectivity 51 2.8 Triangular and Short Cycle Connectivities 53 2.9 Islands 54 2.9.1 Defining Islands 55 2.9.2 Some Properties of Islands 56 2.10 Cores and Generalized Cores 57 2.10.1 Cores 58 2.10.2 Generalized Cores 59 2.11 Important Vertices in Networks 61 2.11.1 Degrees, Closeness, Betweenness and Other Indices 63 2.11.2 Clustering 65 2.11.3 Computing Further Indices Through Functions 66 2.12 Transition to Methods for Large Networks 68 3 Methods for Large Networks 69 3.1 Acyclic Networks 71 3.1.1 Some Basic Properties of Acyclic Networks 71 3.1.2 Compatible Numberings: Depth and Topological Order 72 3.1.3 Topological Orderings and Functions on Acyclic Networks 74 3.2 SPC Weights in Acyclic Networks 75 3.2.1 Citation Networks 75 3.2.2 Analysis of Citation Networks 76 3.2.3 Search Path Count Method 77 3.2.4 Computing SPLC and SPNP Weights 77 3.2.5 Implementation Details 78 3.2.6 Vertex Weights 78 3.2.7 General Properties of Weights 79 3.2.8 SPC Weights 80 3.3 Probabilistic Flow in Acyclic Network 81 3.4 Nonacyclic Citation Networks 82 3.5 Two-mode Networks from Data Tables 84 3.5.1 Multiplication of Two-mode Networks 85 3.6 Bibliographic Networks 88 3.6.1 Co-authorship Networks 88 3.6.2 Collaboration Networks 89 3.6.3 Other Derived Networks 92 3.7 Weights 94 3.7.1 Normalizations of Weights 94 3.7.2 -Rings 94 3.7.3 4-Rings and Analysis of Two-mode Networks 95 3.7.4 Two-mode Cores 96 3.8 Pathfinder 96 3.8.1 Pathfinder Algorithms 100 3.8.2 Computing the Closure Over the Pathfinder Semiring 101 3.8.3 Spanish Algorithms 101 3.8.4 A Sparse Network Algorithm 102 3.9 Clustering, Blockmodeling, and Community Detection 102 3.9.1 The Louvain Method and VOS 102 3.10 Clustering Symbolic Data 103 3.10.1 Symbolic Objects Described with Distributions 103 3.10.2 The Leaders Method 105 3.10.3 An AgglomerativeMethod 107 3.11 Approaches to Temporal Networks 107 3.11.1 Journeys -- Walks in Temporal Networks 108 3.11.2 Measures 110 3.11.3 Problems and Algorithms 111 3.11.4 Evolution 114 3.12 Levels of Analysis 114 3.13 Transition to Substantive Topics 116 4 Scientific Citation and Other Bibliographic Networks 117 4.1 The Centrality Citation Network 117 4.2 Preliminary Data Analyses 118 4.2.1 Temporal Distribution of Publications 119 4.2.2 Degree Distributions of the Centrality Literature 121 4.2.3 Types of Works 124 4.2.4 The Boundary Problem 126 4.3 Transforming a Citation Network into an Acyclic Network 128 4.3.1 Checking for the Presence of Cycles 128 4.3.2 Dealing with Cycles in Citation Networks 133 4.4 The Most ImportantWorks 134 4.5 SPC Weights 134 4.5.1 Obtaining SPC Weights and Drawing Main Paths 135 4.5.2 The Main Path of the Centrality Citation Network 135 4.6 Line Cuts 139 4.7 Line Islands 141 4.7.1 The Main Island 143 4.7.2 A Geophysics and Meteorology Line Island 145 4.7.3 An Optical Network Line Island 150 4.7.4 A Partial Summary of Main Path and Line Island Results 154 4.8 Other Relevant Subnetworks for a Bounded Network 155 4.9 Collaboration Networks 157 4.9.1 Macros for Collaboration Networks 158 4.9.2 An Initial Attempt of Analyses of Collaboration Networks 159 4.10 A Brief Look at the SNA Literature SN5 Networks 160 4.11 On the Centrality and SNA Collaboration Networks 173 References 173 5 Citation Patterns in Temporal United States Patent Data 175 5.1 Patents 175 5.2 Supreme Court Decisions Regarding Patents 179 5.2.1 Co-cited Decisions 179 5.2.2 Citations Between Co-cited Decisions 182 5.3 The 1976--2006 Patent Data 183 5.4 Structural Variables Through Time 184 5.4.1 Temporally Specific Networks 184 5.4.2 Shrinking Specific Patent Citation Networks 186 5.4.3 Structural Properties 187 5.5 Some Patterns of Technological Development 188 5.5.1 Structural Properties of Temporally Specific Networks 190 5.6 Important Subnetworks 193 5.6.1 Line Islands 194 5.6.2 Line Islands with Patents Tagged by Keywords 196 5.6.3 Vertex Islands 201 5.7 Citation Patterns 202 5.7.1 Patents from 1976, Cited Through to 2006 204 5.7.2 Patents from 1987, Cited Through to 2006 209 5.8 Comparing Citation Patterns for Two Time Intervals 211 5.9 Summary and Conclusions 214 6 The US Supreme Court Citation Network 216 6.1 Introduction 217 6.2 Co-cited Islands of Supreme Court Decisions 219 6.3 A Native American Line Island 222 6.3.1 Forced Removal of Native American Populations 222 6.3.2 RegulatingWhites on Native American Lands 224 6.3.3 Curtailing the Authority of Native American Courts 224 6.3.4 Taxing Native Americans and Enforcing External Laws 225 6.3.5 The Presence of Non-Native Americans on Native American Lands 226 6.3.6 Some Later Developments 227 6.3.7 A Partial Summary 227 6.4 A ‘Perceived Threats to Social Order’ Line Island 228 6.4.1 Perceived Threats to Social Order 228 6.4.2 The Structures of the Threats to Social Order Line Island 230 6.4.3 Decisions Involving Communists and Socialists 230 6.4.4 Restrictions of Labor Groups Organizing 236 6.4.5 Restrictions of African Americans Organizing 237 6.4.6 Jehovah’sWitnesses as a Perceived Threat 239 6.4.7 Obscenity as a Threat to Social Order 243 6.5 Other Perceived Threats 246 6.6 The Dred Scott Decision 250 6.6.1 Citations from Dred Scott 251 6.6.2 Citations to Dred Scott 253 6.6.3 Methodological Implications of Dred Scott 260 6.7 Further Reflections on the Supreme Court Citation Network 261 7 Football as the World’s Game 263 7.1 A Brief Historical Overview 264 7.2 Football Clubs 264 7.3 Football Players 266 7.4 Football in England 267 7.5 Player Migrations 268 7.6 Institutional Arrangements and the Organization of Football 269 7.7 Court Rulings 271 7.8 Specific Factors Impacting Football Migration 272 7.9 Some Arguments and Propositions 272 7.10 Some Preliminary Results 278 7.10.1 The Non-English Presence in the EPL 279 7.10.2 Player Fitness 289 7.10.3 Starting Clubs for English Players 292 7.10.4 General Features of the Top Five European Leagues 295 7.10.5 Flows of Footballers into the Top European Leagues 301 7.11 Player Ages When Recruited to the EPL 303 7.12 A Partial Summary of Results 305 8 Networks of Player Movements to the EPL 308 8.1 Success in the EPL 308 8.2 The Overall Presence of Other Countries in the EPL 311 8.3 Network Flows of Footballers Between Clubs to Reach the EPL 312 8.3.1 Moving Directly into the EPL from Local and Non-local Clubs 313 8.3.2 Direct Moves of Players to the EPL from Non-EPL Clubs 315 8.4 Moves from EPL Clubs 318 8.4.1 The 1992--1996 Time Slice Flows with at Least Three Moves 318 8.4.2 The 1997--2001 Time Slice Flows with at Least Three Moves 322 8.4.3 The 2002--2006 Time Slice Flows with at Least Three Moves 323 8.5 Moves Solely Within the EPL 324 8.5.1 Loans 324 8.5.2 Transfers 326 8.6 All Trails of Footballers to the EPL 330 8.6.1 Counted Features of Trails to the EPL 331 8.6.2 Clustering Player Trails 335 8.6.3 Interpreting the Clusters of Player Careers 350 8.7 Summary and Conclusions 350 9 Mapping Spatial Diversity in the United States of America 353 9.1 Mapping Nations as Spatial Units of the United States 354 9.1.1 The Counties of the United States 357 9.2 Representing Networks in Space 359 9.3 Clustering with a Relational Constraint 360 9.3.1 Conditions for Hierarchical Clustering Methods 361 9.3.2 Clustering with a Relational Constraint 363 9.3.3 An AgglomerativeMethod for Relational Constraints 365 9.3.4 Hierarchies 367 9.3.5 Fast Agglomerative Clustering Algorithms 368 9.4 Data for Constrained Spatial Clustering 369 9.4.1 Discriminant Analysis for Garreau’s Nations 369 9.5 Clustering the US Counties with a Spatial Relational Constraint 374 9.5.1 The Eight Garreau Nations in the USA 375 9.5.2 The Ten Woodard Nations in the USA 379 9.6 Summary 381 10 On Studying Large Networks 382 10.1 Substance 382 10.2 Methods, Techniques, and Algorithms 384 10.3 Network Data 385 10.4 Surprises and Issues Triggered by Them 388 10.5 FutureWork 390 10.6 Two Final Comments 393 Appendix: Data Documentation 395 A.1 Bibliographic Networks 395 A.1.1 Centrality Literature Networks 397 A.1.2 SNA Literature 399 A.2 Patent Data 400 A.3 Supreme Court Data 401 A.4 Football Data 403 A.4.1 Core Data 403 A.4.2 Ancillary Data 413 A.5 The USA Spatial County Network 415 References 419 Person Index 428 Subject Index 432

Read more less

Book SynopsisBrimming with contributions from international specialists in industry and academia who have worked on the development of the JPEG2000 standard, The JPEG 2000 Suite provides a comprehensive reference on JPEG2000 and its extensions, examining current applications and future perspectives.Table of ContentsContributor Biographies. Foreword. Series Editor’s Preface. Preface. Acknowledgments. List of Acronyms. Part A. 1 JPEG 2000 Core Coding System (Part 1) (Majid Rabbani, Rajan L. Joshi, and Paul W. Jones). 1.1 Introduction. 1.2 JPEG 2000 Fundamental Building Blocks. 1.3 JPEG 2000 Bit-Stream Organization. 1.4 JPEG 2000 Rate Control. 1.5 Performance Comparison of the JPEG 2000 Encoder Options. 1.6 Additional Features of JPEG 2000 Part 1. Acknowledgments. References. 2 JPEG 2000 Extensions (Part 2) (Margaret Lepley, J. Scott Houchin, James Kasner, and Michael Marcellin). 2.1 Introduction. 2.2 Variable DC Offset. 2.3 Variable Scalar Quantization. 2.4 Trellis-Coded Quantization. 2.5 Precinct-Dependent Quantization. 2.6 Extended Visual Masking. 2.7 Arbitrary Decomposition. 2.8 Arbitrary Wavelet Transforms. 2.9 Multiple-Component Transform Extensions. 2.10 Nonlinear Point Transform. 2.11 Geometric Manipulation via a Code-Block Anchor Point (CBAP). 2.12 Single-Sample Overlap. 2.13 Region of Interest. 2.14 Extended File Format: JPX. 2.15 Extended Capabilities Signaling. Acknowledgments. References. 3 Motion JPEG 2000 and ISO Base Media File Format (Parts 3 and 12) (Joerg Mohr). 3.1 Introduction. 3.2 Motion JPEG 2000 and ISO Base Media File Format. 3.3 ISO Base Media File Format. 3.4 Motion JPEG 2000. References. 4 Compound Image File Format (Part 6) (Frederik Temmermans, Tim Bruylants, Simon McPartlin, and Louis Sharpe). 4.1 Introduction. 4.2 The JPM File Format. 4.3 Mixed Raster Content Model (MRC). 4.4 Streaming JPM Files. 4.5 Referencing JPM Files. 4.6 Metadata. 4.7 Boxes. 4.8 Profiles. 4.9 Conclusions. References. 5 JPSEC: Securing JPEG 2000 Files (Part 8) (Susie Wee and Zhishou Zhang). 5.1 Introduction. 5.2 JPSEC Security Services. 5.3 JPSEC Architecture. 5.4 JPSEC Framework. 5.5 What: JPSEC Security Services. 5.6 Where: Zone of Influence (ZOI). 5.7 How: Processing Domain and Granularity. 5.8 JPSEC Examples. 5.9 Summary. References. 6 JPIP – Interactivity Tools, APIs, and Protocols (Part 9) (Robert Prandolini). 6.1 Introduction. 6.2 Data-Bins. 6.3 JPIP Basics. 6.4 Client Request–Server Response. 6.5 Advanced Topics. 6.6 Conclusions. Acknowledgments. References. 7 JP3D – Extensions for Three-Dimensional Data (Part 10) (Tim Bruylants, Peter Schelkens, and Alexis Tzannes). 7.1 Introduction. 7.2 JP3D: Going Volumetric. 7.3 Bit-Stream Organization. 7.4 Additional Features of JP3D. 7.5 Compression performances: JPEG 2000 Part 1 versus JP3D. 7.6 Implications for Other Parts of JPEG 2000. Acknowledgments. References. 8 JPWL – JPEG 2000 Wireless (Part 11) (Frédéric Dufaux). 8.1 Introduction. 8.2 Background. 8.3 JPWL Overview. 8.4 Normative Parts. 8.5 Informative Parts. 8.6 Summary. Acknowledgments. References. Part B. 9 JPEG 2000 for Digital Cinema (Siegfried Fößel). 9.1 Introduction. 9.2 General Requirements for Digital Cinema. 9.3 Distribution of Digital Cinema Content. 9.4 Archiving of Digital Movies. 9.5 Future Use of JPEG 2000 within Digital Cinema. 9.6 Conclusions. Acknowledgments. References. 10 Security Applications for JPEG 2000 Imagery (John Apostolopoulos, Frédéric Dufaux, and Qibin Sun). 10.1 Introduction. 10.2 Secure Transcoding and Secure Streaming. 10.3 Multilevel Access Control. 10.4 Selective or Partial Encryption of Image Content. 10.5 Image Authentication. 10.6 Summary. Acknowledgments. References. 11 Video Surveillance and Defense Imaging (Touradj Ebrahimi and Frédéric Dufaux). 11.1 Introduction. 11.2 Scrambling. 11.3 Overview of a Typical Video Surveillance System. 11.4 Overview of a Video Surveillance System Based on JPEG 2000 and ROI Scrambling. 12 JPEG 2000 Application in GIS and Remote Sensing (Bernard Brower, Robert Fiete, and Roddy Shuler). 12.1 Introduction. 12.2 Geographic Information Systems. 12.3 Recommendations for JPEG 2000 Encoding. 12.4 Other JPEG 2000 Parts to Consider. References. 13 Medical Imaging (Alexis Tzannes and Ron Gut). 13.1 Introduction. 13.2 Background. 13.3 DICOM and JPEG 2000 Part 1. 13.4 DICOM and JPEG 2000 Part 2. 13.5 Example Results. 13.6 Image Streaming, DICOM, and JPIP. References. 14 Digital Culture Imaging (Greg Colyer, Robert Buckley, and Athanassios Skodras). 14.1 Introduction. 14.2 The Digital Culture Context. 14.3 Digital Culture and JPEG 2000. 14.4 Application – National Digital Newspaper Program. Acknowledgments. References. 15 Broadcast Applications (Hans Hoffman, Adi Kouadio, and Luk Overmeire). 15.1 Introduction – From Tape-Based to File-Based Production. 15.2 Broadcast Production Chain Reference Model. 15.3 Codec Requirements for Broadcasting Applications. 15.4 Overview of State-of-the-Art HD Compression Schemes. 15.5 JPEG 2000 Applications. 15.6 Multigeneration Production Processes. 15.7 JPEG 2000 Comparison with SVC. 15.8 Conclusion. References. 16 JPEG 2000 in 3-D Graphics Terrain Rendering (Gauthier Lafruit, Wolfgang Van Raemdonck, Klaas Tack, and Eric Delfosse). 16.1 Introduction. 16.2 Tiling: The Straightforward Solution to Texture Streaming. 16.3 View-Dependent JPEG 2000 Texture Streaming and Mipmapping. 16.4 JPEG 2000 Quality and Decoding Time Scalability for Optimal Quality–Workload Tradeoff. 16.5 Conclusion. References. 17 Conformance Testing, Reference Software, and Implementations (Peter Schelkens, Yiannis Andreopoulos, and Joeri Barbarien). 17.1 Introduction. 17.2 Part 4 – Conformance Testing. 17.3 Part 5 – Reference Software. 17.4 Implementation of the Discrete Wavelet Transform as Suggested by the JPEG 2000 Standard. 17.5 JPEG 2000 Hardware and Software Implementations. 17.6 Conclusions. Acknowledgments. References. 18 Ongoing Standardization Efforts (Touradj Ebrahimi, Athanassios Skodras, and Peter Schelkens). 18.1 Introduction. 18.2 JPSearch. 18.3 JPEG XR. 18.4 Advanced Image Coding and Evaluation Methodologies (AIC). References. Index.

Read more less

Book SynopsisWith the growth in social networking and the potential for larger and larger breaches of sensitive data,it is vital for all enterprises to ensure that computer users adhere to corporate policy and project staff design secure systems.Trade Review"...an engaging read." (Information Age, May 2009) "I found the book enjoyable and easy to read. It is very informative, and gives good references" (Infosecurity, June 2009) ‘For a big book-in size and in ambition- it's most readable.' (Professional Security, September 2010).Table of ContentsAcknowledgements xvii Foreword xix Introduction xxi 1 Power to the people 1 The power is out there . . . somewhere 1 An information-rich world 2 When in doubt, phone a friend 3 Engage with the public 4 The power of the blogosphere 4 The future of news 5 Leveraging new ideas 5 Changing the way we live 6 Transforming the political landscape 7 Network effects in business 8 Being there 9 Value in the digital age 9 Hidden value in networks 10 Network innovations create security challenges 12 You’ve been de-perimeterized! 14 The collapse of information management 15 The shifting focus of information security 15 The external perspective 17 A new world of openness 18 A new age of collaborative working 19 Collaboration-oriented architecture 20 Business in virtual worlds 21 Democracy . . . but not as we know it 22 Don’t lock down that network 23 The future of network security 24 Can we trust the data? 25 The art of disinformation 27 The future of knowledge 28 The next big security concern 30 Learning from networks 31 2 Everyone makes a difference 33 Where to focus your efforts 33 The view from the bridge 34 The role of the executive board 35 The new threat of data leakage 36 The perspective of business management 38 The role of the business manager 39 Engaging with business managers 40 The role of the IT function 41 Minding your partners 42 Computer users 43 Customers and citizens 44 Learning from stakeholders 44 3 There’s no such thing as an isolated incident 47 What lies beneath? 47 Accidents waiting to happen 48 No system is foolproof 49 Visibility is the key 49 A lesson from the safety field 50 Everyone makes mistakes 52 The science of error prevention 53 Swiss cheese and security 54 How significant was that event? 55 Events are for the record 56 When an event becomes an incident 57 The immediacy of emergencies 57 When disaster strikes 58 When events spiral out of control 58 How the response process changes 59 No two crises are the same 60 One size doesn’t fit all 61 The limits of planning 62 Some assets are irreplaceable 63 It’s the process, not the plan 63 Why crisis management is hard 64 Skills to manage a crisis 65 Dangerous detail 67 The missing piece of the jigsaw 67 Establish the real cause 68 Are you incubating a crisis? 69 When crisis management becomes the problem 70 Developing a crisis strategy 70 Turning threats into opportunities 71 Boosting market capitalization 72 Anticipating events 73 Anticipating opportunities 74 Designing crisis team structures 75 How many teams? 76 Who takes the lead? 77 Ideal team dynamics 77 Multi-agency teams 78 The perfect environment 79 The challenge of the virtual environment 80 Protocols for virtual team working 81 Exercising the crisis team 81 Learning from incidents 83 4 Zen and the art of risk management 85 East meetsWest 85 The nature of risks 86 Who invented risk management? 87 We could be so lucky 88 Components of risk 89 Gross or net risk? 90 Don’t lose sight of business 91 How big is your appetite? 92 It’s an emotional thing 93 In the eye of the beholder 94 What risk was that? 96 Living in the past 96 Who created that risk? 97 It’s not my problem 98 Size matters 99 Getting your sums right 99 Some facts are counterintuitive 101 The loaded dice 101 The answer is 42 103 It’s just an illusion 103 Context is king 104 Perception and reality 105 It’s a relative thing 107 Risk, what risk? 107 Something wicked this way comes 108 The black swan 109 Double jeopardy 110 What type of risk? 111 Lessons from the process industries 112 Lessons from cost engineering 113 Lessons from the financial sector 113 Lessons from the insurance field 115 The limits of percentage play 116 Operational risk 116 Joining up risk management 117 General or specific? 119 Identifying and ranking risks 120 Using checklists 122 Categories of risks 122 It’s a moving target 123 Comparing and ranking risks 124 Risk management strategies 125 Communicating risk appetite 126 Risk management maturity 127 There’s more to security than risk 128 It’s a decision support tool 129 The perils of risk assessment 130 Learning from risk management 131 5 Who can you trust? 133 An asset or a liability? 133 People are different 134 The rule of four 135 The need to conform 136 Understand your enemies 137 The face of the enemy 137 Run silent, run deep 138 Dreamers and charmers 139 The unfashionable hacker 140 The psychology of scams 142 Visitors are welcome 142 Where loyalties lie 144 Signs of disloyalty 144 The whistleblower 145 Stemming the leaks 146 Stamping out corruption 147 Know your staff 148 We know what you did 149 Reading between the lines 151 Liberty or death 153 Personality types 154 Personalities and crime 156 The dark triad 157 Cyberspace is less risky 157 Set a thief 159 It’s a glamour profession 160 There are easier ways 160 I just don’t believe it 161 Don’t lose that evidence 162 They had it coming 163 The science of investigation 164 The art of interrogation 165 Secure by design 167 Science and snake oil 167 The art of hypnosis 169 The power of suggestion 170 It’s just an illusion 171 It pays to cooperate 172 Artificial trust 173 Who are you? 173 How many identities? 175 Laws of identity 176 Learning from people 178 6 Managing organization culture and politics 181 When worlds collide 181 What is organization culture? 182 Organizations are different 184 Organizing for security 186 Tackling ‘localitis’ 186 Small is beautiful 187 In search of professionalism 188 Developing careers 190 Skills for information security 191 Information skills 192 Survival skills 194 Navigating the political minefield 195 Square pegs and round holes 196 What’s in a name? 197 Managing relationships 199 Exceeding expectations 200 Nasty or nice 201 In search of a healthy security culture 202 In search of a security mindset 204 Who influences decisions? 205 Dealing with diversity 206 Don’t take yes for an answer 207 Learning from organization culture and politics 208 7 Designing effective awareness programs 211 Requirements for change 211 Understanding the problem 212 Asking the right questions 213 The art of questionnaire design 214 Hitting the spot 215 Campaigns that work 216 Adapting to the audience 217 Memorable messages 218 Let’s play a game 220 The power of three 221 Creating an impact 222 What’s in a word? 224 Benefits not features 225 Using professional support 226 The art of technical writing 227 Marketing experts 228 Brand managers 229 Creative teams 230 The power of the external perspective 230 Managing the media 231 Behavioural psychologists 232 Blogging for security 233 Measuring your success 234 Learning to conduct campaigns 235 8 Transforming organization attitudes and behaviour 237 Changing mindsets 237 Reward beats punishment 238 Changing attitudes 240 Scenario planning 241 Successful uses of scenarios 242 Dangers of scenario planning 243 Images speak louder 244 A novel approach 245 The balance of consequences 245 The power of attribution 248 Environments shape behaviour 248 Enforcing the rules of the network 250 Encouraging business ethics 251 The art of on-line persuasion 251 Learning to change behaviour 252 9 Gaining executive board and business buy-in 255 Countering security fatigue 255 Money isn’t everything 256 What makes a good business case? 257 Aligning with investment appraisal criteria 257 Translating benefits into financial terms 258 Aligning with IT strategy 259 Achieving a decisive result 259 Key elements of a good business case 260 Assembling the business case 261 Identifying and assessing benefits 261 Something from nothing 263 Reducing project risks 263 Framing your recommendations 264 Mastering the pitch 264 Learning how to make the business case 266 10 Designing security systems that work 269 Why systems fail 269 Setting the vision 270 What makes a good vision? 270 Defining your mission 272 Building the strategy 274 Critical success factors for effective governance 275 The smart approach to governance 276 Don’t reinvent the wheel 276 Look for precedents from other fields 277 Take a top down approach 277 Start small, then extend 278 Take a strategic approach 278 Ask the bigger question 279 Identify and assess options 280 Risk assessment or prescriptive controls? 280 In a class of their own 282 Not all labels are the same 283 Guidance for technology and people 284 Designing long-lasting frameworks 285 Applying the fourth dimension 286 Do we have to do that? 287 Steal with caution 289 The golden triangle 290 Managing risks across outsourced supply chains 291 Models, frameworks and architectures 292 Why we need architecture 293 The folly of enterprise security architectures 294 Real-world security architecture 295 The 5Ws (and one H) 296 Occam’s Razor 297 Trust architectures 298 Secure by design 299 Jericho Forum principles 299 Collaboration-oriented architecture 300 Forwards not backwards 301 Capability maturity models 301 The power of metrics 302 Closing the loop 303 The importance of ergonomics 305 It’s more than ease of use 305 The failure of designs 306 Ergonomic methods 307 A nudge in the right direction 308 Learning to design systems that work 308 11 Harnessing the power of the organization 311 The power of networks 311 Surviving in a hostile world 311 Mobilizing the workforce 312 Work smarter, not harder 313 Finding a lever 313 The art of systems thinking 314 Creating virtuous circles 315 Triggering a tipping point 315 Identifying key influencers 316 In search of charisma 318 Understanding fashion 318 The power of context 319 The bigger me 320 The power of the herd 321 The wisdom of crowds 322 Unlimited resources – the power of open source 323 Unlimited purchasing power 324 Let the network to do the work 324 Why is everything getting more complex? 325 Getting to grips with complexity 327 Simple can’t control complex 327 Designing freedom 329 A process-free world 330 The power of expressive systems 331 Emergent behaviour 332 Why innovation is important 332 What is innovation? 333 What inspires people to create? 335 Just one idea is enough 335 The art of creative thinking 336 Yes, you can 336 Outside the box 337 Innovation environments 339 Turning ideas into action 339 Steps to innovation heaven 340 The road ahead 341 Mapping the future 342 Learning to harness the power of the organization 344 In conclusion 347 Bibliography 353 Index 357

Read more less

Book SynopsisGraphical models are of increasing importance in applied statistics, and in particular in data mining. Providing a self-contained introduction and overview to learning relational, probabilistic, and possibilistic networks from data, this second edition of Graphical Models is thoroughly updated to include the latest research in this burgeoning field, including a new chapter on visualization. The text provides graduate students, and researchers with all the necessary background material, including modelling under uncertainty, decomposition of distributions, graphical representation of distributions, and applications relating to graphical models and problems for further research.Trade Review“The text provides graduate students, and researchers with all the necessary background material, including modelling under uncertainty, decomposition of distributions, graphical representation of distributions, and applications relating to graphical models and problems for further research.” (Zentralblatt Math, 1 August 2013) "All of the necessary background is provided, with material on modeling under uncertainty and imprecision modeling, decomposition of distributions, graphical representation of distributions, applications relating to graphical models, and problems for further research." (Book News, December 2009)Table of ContentsPreface. 1 Introduction. 1.1 Data and Knowledge. 1.2 Knowledge Discovery and Data Mining. 1.3 Graphical Models. 1.4 Outline of this Book. 2 Imprecision and Uncertainty. 2.1 Modeling Inferences. 2.2 Imprecision and Relational Algebra. 2.3 Uncertainty and Probability Theory. 2.4 Possibility Theory and the Context Model. 3 Decomposition. 3.1 Decomposition and Reasoning. 3.2 Relational Decomposition. 3.3 Probabilistic Decomposition. 3.4 Possibilistic Decomposition. 3.5 Possibility versus Probability. 4 Graphical Representation. 4.1 Conditional Independence Graphs. 4.2 Evidence Propagation in Graphs. 5 Computing Projections. 5.1 Databases of Sample Cases. 5.2 Relational and Sum Projections. 5.3 Expectation Maximization. 5.4 Maximum Projections. 6 Naive Classifiers. 6.1 Naive Bayes Classifiers. 6.2 A Naive Possibilistic Classifier. 6.3 Classifier Simplification. 6.4 Experimental Evaluation. 7 Learning Global Structure. 7.1 Principles of Learning Global Structure. 7.2 Evaluation Measures. 7.3 Search Methods. 7.4 Experimental Evaluation. 8 Learning Local Structure. 8.1 Local Network Structure. 8.2 Learning Local Structure. 8.3 Experimental Evaluation. 9 Inductive Causation. 9.1 Correlation and Causation. 9.2 Causal and Probabilistic Structure. 9.3 Faithfulness and Latent Variables. 9.4 The Inductive Causation Algorithm. 9.5 Critique of the Underlying Assumptions. 9.6 Evaluation. 10 Visualization. 10.1 Potentials. 10.2 Association Rules. 11 Applications. 11.1 Diagnosis of Electrical Circuits. 11.2 Application in Telecommunications. 11.3 Application at Volkswagen. 11.4 Application at DaimlerChrysler. A Proofs of Theorems. A.1 Proof of Theorem 4.1.2. A.2 Proof of Theorem 4.1.18. A.3 Proof of Theorem 4.1.20. A.4 Proof of Theorem 4.1.26. A.5 Proof of Theorem 4.1.28. A.6 Proof of Theorem 4.1.30. A.7 Proof of Theorem 4.1.31. A.8 Proof of Theorem 5.4.8. A.9 Proof of Lemma .2.2. A.10 Proof of Lemma .2.4. A.11 Proof of Lemma .2.6. A.12 Proof of Theorem 7.3.1. A.13 Proof of Theorem 7.3.2. A.14 Proof of Theorem 7.3.3. A.15 Proof of Theorem 7.3.5. A.16 Proof of Theorem 7.3.7. B Software Tools. Bibliography. Index.

Read more less

Book SynopsisBetfair is the world's leading online betting exchange. Launched in 2000, its annual revenues reached 145m in 2006. In the last year, Betfair has more than doubled its number of registered users. Since the first edition of the book was published, the total number of Betfair websites has risen to 18, and an Australian exchange has launched. The services Betfair offer have also expanded, including a telephone betting operation and new games including poker, blackjack and baccarat. This is the definitive insider's guide to playing and winning on Betfair. Written by Betfair insiders it gives you the full picture of how Betfair works; it explains the terms and jargon, helps you get started on the site, introduces every type of play including poker and the Betfair Casino - and offers tips and insider know-how that both newcomers seasoned Betfair punters can use to maximise returns.Table of ContentsIntroduction 1 Part I: Starting Out 7 Chapter 1: Introducing Betfair 9 Chapter 2: Starting Out with the Essentials 17 Chapter 3: Managing Your Account 27 Chapter 4: Choosing Your Market 39 Part II: Let’s Get Betting 51 Chapter 5: Placing Your First Bet 53 Chapter 6: Doing the Maths 65 Chapter 7: Betting In-Play 73 Chapter 8: Using Betfair Starting Price 85 Chapter 9: Poker, Games, and Casino 91 Part III: Getting Serious 97 Chapter 10: Finding Information 99 Chapter 11: Low-Risk Betting: Trading 103 Chapter 12: Low-Risk Betting: Arbing 117 Chapter 13: Going Pro 123 Chapter 14: Controlling Your Gambling 139 Part IV: The Part of Tens 145 Chapter 15: Ten Common Mistakes to Avoid 147 Chapter 16: Ten Top Tips 153 Chapter 17: Ten Sporting Information Sources 159 Chapter 18: Ten (or So) Most Amazing Markets 165 Chapter 19: Ten Betfair Firsts 173 Appendix: Glossary 179 Index 183

Read more less

Book SynopsisA comprehensive research guide for both quantitative and qualitative research methods Written by a team of authorities in human-computer interaction (HCI) and usability, this pedagogical guide walks you through the methods used in HCI and examines what are considered to be appropriate research practices in the field.Table of ContentsAbout the Authors xvii Acknowledgments xviii Preface xix 1 Introduction 1 1.1 Changes in topics of HCI research over time 3 1.2 Shifts in measurement in HCI 5 1.3 Inherent conflicts in HCI 9 1.4 Interdisciplinary nature of HCI research 11 1.5 Communicating your ideas 13 1.6 Research and usability testing 14 Summary of Chapters 14 Discussion Questions 15 Research Design Exercise 16 References 16 2 Experimental Research 19 2.1 Types of behavioral research 20 2.2 Research hypotheses 22 2.2.1 Null hypothesis and alternative hypothesis 23 2.2.2 Dependent and independent variables 25 2.2.3 Typical independent variables in HCI research 25 2.2.4 Typical dependent variables in HCI research 26 2.3 Basics of experimental research 27 2.3.1 Components of an experiment 27 2.3.2 Randomization 28 2.4 Significance tests 30 2.4.1 Why do we need them? 30 2.4.2 Type I and Type II errors 32 2.4.3 Controlling the risks of Type I and Type II errors 34 2.5 Limitations of experimental research 34 Summary 36 Discussion Questions 37 Research Design Exercises 38 References 38 3 Experimental Design 41 3.1 What needs to be considered when designing experiments? 43 3.2 Determining the basic design structure 44 3.3 Investigating a single independent variable 45 3.3.1 Between-group design and within-group design 46 3.3.2 Choosing the appropriate design approach 49 3.4 Investigating more than one independent variable 53 3.4.1 Factorial design 53 3.4.2 Split-plot design 54 3.4.3 Interaction effects 55 3.5 Reliability of experimental results 57 3.5.1 Random errors 57 3.5.2 Systematic errors 57 3.6 Experimental procedures 63 Summary 64 Discussion Questions 65 Research Design Exercises 65 References 66 4 Statistical Analysis 69 4.1 Preparing data for statistical analysis 70 4.1.1 Cleaning up data 70 4.1.2 Coding data 71 4.1.3 Organizing data 73 4.2 Descriptive statistics 73 4.2.1 Measures of central tendency 73 4.2.2 Measures of spread 74 4.3 Comparing means 74 4.4 T tests 76 4.4.1 Independent-samples t test 76 4.4.2 Paired-samples t test 76 4.4.3 Interpretation of t test results 77 4.4.4 Two-tailed t tests and one-tailed t tests 78 4.5 Analysis of variance 78 4.5.1 One-way ANOVA 79 4.5.2 Factorial ANOVA 80 4.5.3 Repeated measures ANOVA 82 4.5.4 ANOVA for split-plot design 83 4.6 Assumptions of t tests and F tests 86 4.7 Identifying relationships 86 4.8 Regression 89 4.9 Nonparametric statistical tests 91 4.9.1 Chi-square test 92 4.9.2 Other non-parametric tests 94 Summary 94 Discussion Questions 95 Research Design Exercises 96 Team Exercises 96 References 96 5 Surveys 99 5.1 Introduction 100 5.2 Benefits and drawbacks of surveys 101 5.3 Goals and targeted users for survey research 102 5.4 Probabilistic sampling 103 5.4.1 Stratification 105 5.4.2 Response size 106 5.4.3 Errors 106 5.5 Non-probabilistic sampling 107 5.5.1 Demographic data 107 5.5.2 Oversampling 108 5.5.3 Random sampling of usage, not users 109 5.5.4 Self-selected surveys 109 5.5.5 Uninvestigated populations 109 5.6 Developing survey questions 111 5.6.1 Open-ended questions 111 5.6.2 Closed-ended questions 112 5.6.3 Common problems with survey questions 113 5.7 Overall survey structure 113 5.8 Existing surveys 115 5.9 Paper or online surveys? 116 5.10 Testing the survey tool 118 5.11 Response rate 119 5.12 Data analysis 120 Summary 121 Discussion Questions 121 Research Design Exercise 122 References 122 6 Diaries 125 6.1 Introduction 126 6.2 Why do we use diaries in HCI research? 127 6.3 Participants for a diary study 130 6.4 What type of diary? 132 6.4.1 Feedback diary 132 6.4.2 Elicitation diary 133 6.4.3 Hybrid feedback and elicitation diary 134 6.5 Data collection for the diary study 134 6.6 Letting participants know when to record a diary entry 136 6.7 Analysis of diaries 137 Summary 138 Discussion Questions 138 Research Design Exercise 138 References 139 Appendix A Frustration Experience Form (Time Diary) 140 Appendix B Excel Time Diary Form 141 7 Case Studies 143 7.1 Introduction 144 7.2 Observing Sara: a case study of a case study 145 7.3 What is a case study? 147 7.3.1 In-depth investigation of a small number of cases 147 7.3.2 Examination in context 147 7.3.3 Multiple data sources 148 7.3.4 Emphasis on qualitative data and analysis 149 7.4 Goals of HCI case studies 150 7.4.1 Exploration 150 7.4.2 Explanation 151 7.4.3 Description 152 7.4.4 Demonstration 154 7.5 Types of case study 156 7.5.1 Intrinsic or instrumental 156 7.5.2 Single case or multiple cases 156 7.5.3 Embedded or holistic 160 7.6 Research questions and hypotheses 161 7.7 Choosing cases 163 7.8 Data collection 164 7.8.1 Data sources and questions 164 7.8.2 Collecting data 165 7.9 Analysis and interpretation 167 7.10 Writing up the study 168 7.11 Informal case studies 170 Summary 172 Discussion Questions 174 Research Design Exercises 174 References 175 8 Interviews and Focus Groups 177 8.1 Pros and cons of interviews 178 8.2 Applications of interviews in HCI research 180 8.2.1 Initial exploration 180 8.2.2 Requirements gathering 184 8.2.3 Evaluation and subjective reactions 186 8.3 Who to interview 187 8.4 Interview strategies 189 8.4.1 How much structure? 189 8.4.2 Focused and contextual interviews 191 8.5 Interviews vs focus groups 192 8.6 Types of question 194 8.7 Conducting an interview 197 8.7.1 Preparation 197 8.7.2 Recording the responses 198 8.7.3 During the interview 199 8.8 Electronically mediated interviews and focus groups 203 8.8.1 Telephone 204 8.8.2 Online 204 8.9 Analyzing interview data 206 8.9.1 What to analyze 207 8.9.2 How to analyze 208 8.9.3 Validity 212 8.9.4 Reporting Results 212 Summary 213 Discussion Questions 214 Research Design Exercises 214 References 215 9 Ethnography 217 9.1 Introduction 218 9.2 What is ethnography? 219 9.3 Ethnography in HCI 221 9.4 Conducting ethnographic research 224 9.4.1 Selecting a site or group of interest 225 9.4.2 Participating: choosing a role 227 9.4.3 Building relationships 230 9.4.4 Making contact 231 9.4.5 Interviewing, observing, analyzing, repeating, and theorizing 232 9.4.6 Reporting results 236 9.5 Some examples 237 9.5.1 Home settings 237 9.5.2 Work settings 238 9.5.3 Educational settings 239 9.5.4 Ethnographies of mobile and ubiquitous systems 240 9.5.5 Virtual ethnography 241 Summary 246 Discussion Questions 246 Research Design Exercises 247 References 248 10 Usability Testing 251 10.1 What is usability testing? 252 10.2 How does usability testing relate to traditional research? 254 10.3 Types of usability testing or usability inspections 256 10.3.1 Expert-based testing 256 10.3.2 Automated usability testing 258 10.4 User-based testing 260 10.4.1 Types of usability testing 260 10.4.2 Stages of usability testing 262 10.4.3 How many users are sufficient? 263 10.4.4 Locations for usability testing 264 10.4.5 Task list 268 10.4.6 Measurement 270 10.4.7 The testing session 271 10.4.8 Making sense of the data 274 10.5 Other variations on usability testing 275 Summary 276 Discussion Questions 276 Research Design Exercise 277 References 277 11 Analyzing Qualitative Data 281 11.1 Introduction 282 11.2 Stages of qualitative analysis 282 11.3 Grounded theory 283 11.4 Content analysis 285 11.4.1 What is content? 286 11.4.2 Why do we need to collect text or multimedia information? 286 11.4.3 Questions to consider before content analysis 287 11.5 Analyzing text content 289 11.5.1 Procedure 289 11.5.2 Identifying coding categories 290 11.5.3 Coding the text 292 11.5.4 Ensuring high-quality analysis 294 11.6 Analyzing multimedia content 300 Summary 301 Discussion Questions 302 Research Design Exercise 303 References 303 12 Automated Data Collection Methods 307 12.1 Exploiting existing tools 308 12.1.1 Web logs 309 12.1.2 Stored application data 315 12.2 Using software to observe and record 317 12.2.1 Web proxies 317 12.2.2 Instrumented software 321 12.2.3 Custom-built software 324 12.2.4 Handling stored data 327 12.2.5 Keystroke and activity loggers 328 12.2.6 Analyzing log files 329 12.3 Hybrid data collection methods 330 12.4 Automated interface evaluation 333 12.5 Challenges of computerized data collection 333 Summary 336 Discussion Questions 337 Research Design Exercises 338 References 339 13 Measuring the Human 343 13.1 Eye tracking 344 13.2 Physiological tools 350 13.2.1 Physiological data 351 13.2.2 Challenges in data collection and interpretation 356 13.3 Examples of physiological research in HCI 359 Summary 361 Discussion Questions 362 Research Design Exercise 363 References 363 14 Working with Human Subjects 367 14.1 Identifying potential participants 368 14.1.1 Which subjects? 369 14.1.2 How many subjects? 371 14.1.3 Recruiting participants 373 14.2 Care and handling of research participants 376 14.2.1 Protecting participants 376 14.2.2 Informed consent 381 14.2.3 Institutional review boards 384 14.2.4 Potentially deceptive research? 387 14.2.5 General concerns 388 14.3 Online research 389 14.3.1 Appropriate topics for online research 389 14.3.2 Recruiting 389 14.3.3 Study design 391 14.3.4 Ethical concerns 391 14.3.5 Data collection 392 Summary 393 Discussion Questions 394 Research Design Exercises 395 References 396 15 Working with Research Participants with Impairments 399 15.1 Introduction 400 15.2 How many participants? 401 15.2.1 Small sample sizes 401 15.2.2 Distributed research 401 15.2.3 In-depth case studies 402 15.3 Proxy users 403 15.4 Multi-Population Studies 404 15.5 Recruiting users through community partners 405 15.6 Pilot studies 407 15.7 Scheduling users with impairments 408 15.8 Documentation for users with impairments 409 15.8.1 Human subjects forms 409 15.8.2 Research documentation 410 15.9 Differing levels of ability 412 15.10 Bringing extra computer parts 413 15.11 Payment 415 Summary 415 Discussion Questions 415 Research Design Exercise 416 References 416 Index 419

Read more less