Many industries, provide consumers with data about the quality, content, and cost of ownership of products, but the software industry leaves consumers with very little data to act upon. In fact when it comes to how secure or weak a product is from a security perspective, there is no meaningful consumer facing data. There has long been a call for the establishment of an independent organization to address this need.
Last year, Mudge (from DARPA, Google, and L0pht fame) announced that after receiving a phone call from the White House he was leaving his senior position inside Google to create a non-profit organization to address this issue. This effort, known as CITL, is akin to Consumer Reports in its methodologies. While the media has dubbed it a "CyberUL", there is no focus on certifications or seals of approval, and no opaque evaluation metrics. Rather, like Consumer Reports, the goal is to evaluate software according to metrics and measurements that allow quantitative comparison and evaluation by anyone from a layperson, CFO, to security expert.
How? A wide range of heuristics that attackers use to identify which targets are hard or soft against new exploitation has been codified, refined, and enhanced. Some of these techniques are quite straightforward and even broadly known, while others are esoteric tradecraft. To date, no one has applied all of these metrics uniformly across an entire software ecosystem before and shared the results. For the first time, a peak at the Cyber Independent Testing Lab's metrics, methodologies, and preliminary results from assessing the software quality and inherent vulnerability in over 100,000 binary applications on Windows, Linux, and OS X will be revealed. All accomplished with binaries only.
Sometimes the more secure product is actually the cheaper, and quite often the security product is the most vulnerable. There are plenty of surprises like these that are finally revealed through quantified measurements. With this information, organizations and consumers can finally make informed purchasing decisions when it comes the security of their products, and measurably realize more hardened environments. Insurance groups are already engaging CITL, as are organizations focused on consumer safety. Vendors will see how much better or worse their products are in comparison to their competitors. Even exploit developers have demonstrated that these results enable bug-bounty arbitrage.
That recommendation you made to your family members last holiday about which web browser they should use to stay safe (or that large purchase you made for your industrial control systems)? Well, you can finally see if you chose a hard or soft target… with the data to back it up.
Mudge is the Director of CITL. He has contributed significantly to disclosureand education on information and security vulnerabilities over the past 25years. In addition to pioneering buffer overflow work, the security work hehas released contained early examples of flaws in the following areas: codeinjection, race conditions, side-channel attacks, exploitation of embeddedsystems, and cryptanalysis of commercial systems. He was the original authorof the password cracking software L0phtCrack, Anti-Sniff, and L0phtWatch. In2010, Mudge accepted a position as a program manager at DARPA where he oversawcyber security R&D; and re-built the Agency's approach to cyber securityresearch. In 2013 Mudge went to work for Google where he was the DeputyDirector of their Advanced Technology & Projects division. Most recently,after conversations with the White House, Mudge stood up the non-profit CyberIndependent Testing Laboratory inspired by efforts such as Consumer's Union(Consumer Reports). He is the recipient of the Secretary of DefenseExceptional Civilian Service Award medal, an honorary Plank Owner of the USNavy Destroyer DDG-85, was inducted into the Order of Thor, the US Army'sAssociation of Cyber Military Professionals, recognized as a vital contributorto the creation of the US Cyber Corps (SfS PDD-63), and has received othercommendations from the CIA and from the Executive Office of the President ofthe United States.
Sarah Zatko is the Chief Scientist at CITL, a partner at L0pht Holdings, LLC,and a member of the US Army's Order of Thor. She has presented her research onthe integration of security into CS curriculum at Shmoocon and Hope. That workis also published in IEEE Security & Privacy. She holds a degree inmathematics from MIT and a Master's in computer science from BostonUniversity.