Quick Summary
The book explores "noise"—unwanted variability in human judgment—as a pervasive and neglected source of error, distinct from bias. Using analogies and noise audits in various fields like justice, medicine, and business, it reveals that noise is often "scandalously high" and far more impactful than commonly perceived, accumulating rather than cancelling out. The text details how noise arises from psychological heuristics, individual cognitive styles, group dynamics, and the inherent limits of human matching operations. It advocates for "decision hygiene" strategies like structured assessments, independent judgments, and algorithmic tools to reduce noise, arguing that while zero noise may be impractical, recognizing and actively combating it is crucial for improving fairness, accuracy, and efficiency in professional decisions.
Key Ideas
Noise, or unwanted variability in judgment, is a major, often overlooked component of human error.
Noise audits reveal "scandalously high" levels of variability in professional judgments across many fields.
System noise comprises level noise (average severity), pattern noise (idiosyncratic reactions), and occasion noise (transient factors).
Algorithms and structured decision-making processes consistently outperform human intuition in reducing noise and improving accuracy.
"Decision hygiene" strategies, such as independent assessments and aggregation, are crucial for mitigating noise and improving fairness.
INTRODUCTION Two Kinds of Error
Human error comprises bias, a systematic deviation from a true target, and noise, which is random scatter. Using a shooting analogy, the text highlights that noise, unlike bias, can be measured even if the true answer is unknown. Despite being "scandalously high" in professional judgments across various fields, noise is often neglected in error discussions.
noise is "scandalously high" in many real-world judgments made by professionals.
PART I Finding Noise
This section establishes that noise is inherent in all judgments, often at surprisingly high levels. It introduces the crucial concept of a noise audit, a technique used to quantitatively measure unwanted disagreement among professionals in areas like criminal sentencing and insurance underwriting.
PART II Your Mind Is a Measuring Instrument
Judgment is fundamentally described as a form of measurement performed by the human mind. The ultimate goal is accuracy, with imperfections manifesting as both bias and noise. This inherent variability is universal in human performance, confirming that noise implies error in predictive judgments.
PART III NOISE IN PREDICTIVE JUDGMENTS
This part focuses on predictive judgments, which are unique because their accuracy can be objectively evaluated. Predictive accuracy is quantified using the Percent Concordant (PC) and the correlation coefficient (r), which measures the shared determinants between a prediction and its actual outcome.
PART IV HOW NOISE HAPPENS
This section delves into how mental mechanisms generate both noise and bias. It highlights that System 1 (fast, intuitive thinking) operations, especially heuristics and psychological biases, often lead to errors. These biases produce noise when judges apply them inconsistently or to varying degrees.
Psychological biases always lead to error.
PART V IMPROVING JUDGMENTS
This section outlines strategies to enhance the quality of professional judgments by reducing noise. A key approach is decision hygiene, a set of preventive techniques designed to reduce various unspecified errors. It also covers selecting better judges, using rules/algorithms, and structuring complex judgments.
The recommended approach to noise reduction is decision hygiene, a concept analogous to preventive measures like handwashing.
PART VI OPTIMAL NOISE
This part addresses the complexities of noise reduction, acknowledging objections such as cost, the risk of introducing new biases, and the perceived loss of dignity or individualized consideration. It explores the tension between strict rules and human discretion, noting that eliminating all noise might not always be optimal.
REVIEW AND CONCLUSION Judgments
Judgment is human measurement, prone to bias (systematic error) and noise (unwanted divergence). Noise, often greater than bias, degrades fairness and credibility. Key strategies for reduction include decision hygiene principles like prioritizing accuracy, taking the outside view, structuring tasks, and aggregating independent judgments. The optimal noise level might not be zero, but organizations must recognize its magnitude.
EPILOGUE A Less Noisy World
The epilogue envisions a world where noise is actively combated through practices like routine noise audits, algorithms, structured assessments, and aggregated judgments. Such efforts would lead to significant societal benefits, including enhanced fairness, improved public safety and health outcomes, and considerable financial savings by preventing avoidable errors caused by variability.
APPENDIX A How to Conduct a Noise Audit
A noise audit is a structured process to quantify judgment variability. It requires executive commitment, realistic case materials, anonymous professional evaluations, and statistical analysis. The audit identifies deficiencies and proposes specific decision hygiene and debiasing procedures to improve organizational performance.
APPENDIX B A Checklist for a Decision Observer
This appendix provides a checklist for a trained decision observer, whose role is to identify real-time biases during group decision-making. The checklist helps detect flaws in judgment approach, risks of premature closure, and issues in information processing, thereby mitigating the impact of consequential biases.
APPENDIX C Correcting Predictions
Correcting predictions involves adjusting intuitive "matching predictions" to account for regression to the mean. This four-step procedure, which involves an intuitive guess, the outside view (class mean), estimating correlation, and proportional adjustment, ensures greater accuracy by making predictions more conservative and less extreme, thereby minimizing overall error.
Frequently Asked Questions
What is the core difference between bias and noise in judgment?
Bias refers to a systematic error, where judgments consistently deviate in a specific direction from the true value. Noise, conversely, represents random scatter or unwanted variability among judgments, even when the average is accurate. Both contribute to overall error.
Why is noise often neglected compared to bias?
Noise is frequently ignored because the human mind naturally seeks causal explanations, making bias (a clear cause) more salient. Noise, being statistical and unpredictable, is harder to detect and is often suppressed by organizations or individual professionals who assume agreement.
What is a "noise audit" and why is it important?
A noise audit is a procedure where multiple professionals independently evaluate identical cases. It's crucial because it quantifies the extent of unwanted variability (noise) in judgments, revealing the magnitude of error and providing a baseline for implementing improvement strategies.
How do algorithms compare to human judgment in terms of noise?
Algorithms inherently eliminate noise because they apply rules consistently. While algorithms can carry algorithmic bias if trained on flawed data, well-designed algorithms often outperform human judges in both accuracy and non-discrimination by removing the pattern and occasion noise human judgment suffers from.
What is "decision hygiene" and how does it help reduce noise?
Decision hygiene refers to preventive measures that reduce a wide, unspecified range of errors in judgment. Analogous to handwashing, these techniques — like structuring judgments, taking an outside view, and aggregating independent assessments — aim to reduce noise without necessarily identifying specific biases.