A base rate is the prevalence of an item of interest in a population. In medicine, it would be the prevalence of a disease in a group of people. In information security, it might be the prevalence of sql injection flaws in web applications or the prevalence of malware in the population of downloaded *.exe files. Without an estimate of the base rate, it isn’t possible to talk meaningfully about detection rates (true positives) or false positives. Those who do so commit the “base rate fallacy. If the base rate is known, then a Fourfold table, also called a 2 x 2 table or matrix, is a mechanism that helps us understand the correct probabilities of True Positive, False Positive, True Negative, and False Negative events and avoid the base rate fallacy. Understanding these probabilities enables us to evaluate the claims of many types of security technologies, including the effectiveness of antivirus software, web application scanners, and IDS/IPS systems. • The base rate fallacy will be explained and demonstrated. • Gigerenzer’s Natural Frequencies Technique for Avoiding the Base Rate Fallacy • Examples of why base rates apply to information risk management:
* Common Vulnerability Scoring System (CVSS)
* The Distinction between Inherent Risk vs. Residual Risk
* Intrusion Detection Systems
* Vendor Management, Hosting Providers, and SOC 2 (formerly SAS70) Audit Reports
Patrick Florer has worked in information technology for 32 years. In addition, during 17 of those 32 years, he worked a parallel track in medical outcomes research, analysis, and the creation of evidence-based guidelines for medical treatment. His roles have included IT operations, programming, and systems analysis. From 1986 until now, he has worked as an independent consultant, helping customers with strategic development, analytics, risk analysis, and decision analysis. He is a cofounder of Risk Centric Security and currently serves as Chief Technology Officer.