Description
Book SynopsisShows managers how to inform themselves in order to make less risky, profitable business decisions. This book shows you how to measure those things in your own business, government agency or other organization that, until now, you may have considered "immeasurable," including customer satisfaction, organizational flexibility, and technology risk.
Table of ContentsPreface to the Third Edition xiii
Acknowledgments xix
About the Author xxi
Part I The Measurement Solution Exists 1
Chapter 1 The Challenge of Intangibles 3
The Alleged Intangibles 4
Yes, I Mean Anything 5
The Proposal: It’s about Decisions 7
A “Power Tools” Approach to Measurement 10
A Guide to the Rest of the Book 11
Chapter 2 An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily 15
How an Ancient Greek Measured the Size of Earth 16
Estimating: Be Like Fermi 17
Experiments: Not Just for Adults 20
Notes on What to Learn from Eratosthenes, Enrico, and Emily 25
Notes 27
Chapter 3 The Illusion of Intangibles: Why Immeasurables Aren’t 29
The Concept of Measurement 30
The Object of Measurement 37
The Methods of Measurement 40
Economic Objections to Measurement 48
The Broader Objection to the Usefulness of “Statistics” 52
Ethical Objections to Measurement 55
Reversing Old Assumptions 58
Notes 65
Part II Before You Measure 69
Chapter 4 Clarifying the Measurement Problem 71
Toward a Universal Approach to Measurement 73
The Unexpected Challenge of Defining a Decision 74
If You Understand It, You Can Model It 80
Getting the Language Right: What “Uncertainty” and “Risk” Really Mean 83
An Example of a Clarified Decision 84
Notes 90
Chapter 5 Calibrated Estimates: How Much Do You Know Now? 93
Calibration Exercise 95
Calibration Trick: Bet Money (or Even Just Pretend To) 101
Further Improvements on Calibration 104
Conceptual Obstacles to Calibration 106
The Effects of Calibration Training 111
Notes 118
Chapter 6 Quantifying Risk through Modeling 123
How Not to Quantify Risk 123
Real Risk Analysis: The Monte Carlo 125
An Example of the Monte Carlo Method and Risk 127
Tools and Other Resources for Monte Carlo Simulations 136
The Risk Paradox and the Need for Better Risk Analysis 140
Notes 143
Chapter 7 Quantifying the Value of Information 145
The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss 146
The Value of Information for Ranges 149
Beyond Yes/No: Decisions on a Continuum 156
The Imperfect World: The Value of Partial Uncertainty Reduction 159
The Epiphany Equation: How the Value of Information Changes Everything 166
Summarizing Uncertainty, Risk, and Information Value: The Pre-Measurements 171
Notes 172
Part III Measurement Methods 173
Chapter 8 The Transition: From What to Measure to How to Measure 175
Tools of Observation: Introduction to the Instrument of Measurement 177
Decomposition 180
Secondary Research: Assuming You Weren’t the First to Measure It 184
The Basic Methods of Observation: If One Doesn’t Work, Try the Next 186
Measure Just Enough 188
Consider the Error 189
Choose and Design the Instrument 194
Notes 196
Chapter 9 Sampling Reality: How Observing Some Things Tells Us about All Things 197
Building an Intuition for Random Sampling: The Jelly Bean Example 199
A Little about Little Samples: A Beer Brewer’s Approach 200
Are Small Samples Really “Statistically Significant”? 204
When Outliers Matter Most 208
The Easiest Sample Statistics Ever 210
A Biased Sample of Sampling Methods 214
Experiment 226
Seeing Relationships in the Data: An Introduction to Regression Modeling 235
Notes 243
Chapter 10 Bayes: Adding to What You Know Now 247
The Basics and Bayes 248
Using Your Natural Bayesian Instinct 257
Heterogeneous Benchmarking: A “Brand Damage” Application 263
Bayesian Inversion for Ranges: An Overview 267
The Lessons of Bayes 276
Notes 282
Part IV Beyond the Basics 285
Chapter 11 Preference and Attitudes: The Softer Side of Measurement 287
Observing Opinions, Values, and the Pursuit of Happiness 287
A Willingness to Pay: Measuring Value via Trade-Offs 292
Putting It All on the Line: Quantifying Risk Tolerance 296
Quantifying Subjective Trade-Offs: Dealing with Multiple Conflicting Preferences 299
Keeping the Big Picture in Mind: Profit Maximization versus Purely Subjective Trade-Offs 302
Notes 304
Chapter 12 The Ultimate Measurement Instrument: Human Judges 307
Homo Absurdus: The Weird Reasons behind Our Decisions 308
Getting Organized: A Performance Evaluation Example 313
Surprisingly Simple Linear Models 315
How to Standardize Any Evaluation: Rasch Models 316
Removing Human Inconsistency: The Lens Model 320
Panacea or Placebo?: Questionable Methods of Measurement 325
Comparing the Methods 333
Example: A Scientist Measures the Performance of a Decision Model 335
Notes 336
Chapter 13 New Measurement Instruments for Management 339
The Twenty-First-Century Tracker: Keeping Tabs with Technology 339
Measuring the World: The Internet as an Instrument 342
Prediction Markets: A Dynamic Aggregation of Opinions 346
Notes 353
Chapter 14 A Universal Measurement Method: Applied Information Economics 357
Bringing the Pieces Together 358
Case: The Value of the System That Monitors Your Drinking Water 362
Case: Forecasting Fuel for the Marine Corps 367
Case: Measuring the Value of ACORD Standards 373
Ideas for Getting Started: A Few Final Examples 378
Summarizing the Philosophy 384
Notes 385
Appendix Calibration Tests (and Their Answers) 387
Index 397
