Many industries, provide consumers with data about the quality, content, and cost of ownership of products, but the software industry leaves consumers with very little data to act upon. In fact when it comes to how secure or weak a product is from a security perspective, there is no meaningful consumer facing data. There has long been a call for the establishment of an independent organization to address this need. Last year, Mudge (from DARPA, Google, and L0pht fame) announced that after receiving a phone call from the White House he was leaving his senior position inside Google to create a non-profit organization to address this issue. This effort, known as CITL, is akin to Consumer Reports in its methodologies. While the media has dubbed it a "CyberUL", there is no focus on certifications or seals of approval, and no opaque evaluation metrics. Rather, like Consumer Reports, the goal is to evaluate software according to metrics and measurements that allow quantitative comparison and evaluation by anyone from a layperson, CFO, to security expert.
How? A wide range of heuristics that attackers use to identify which targets are hard or soft against new exploitation has been codified, refined, and enhanced. Some of these techniques are quite straightforward and even broadly known, while others are esoteric tradecraft. To date, no one has applied all of these metrics uniformly across an entire software ecosystem before and shared the results.
For the first time, a peek at the Cyber Independent Testing Lab’s metrics, methodologies, and preliminary results from assessing the software quality and inherent vulnerability in over 100,000 binary applications on Windows, Linux, and OS X will be revealed. All accomplished with binaries only. Sometimes the more secure product is actually the cheaper, and quite often the security product is the most vulnerable.
There are plenty of surprises like these that are finally revealed through quantified measurements. With this information, organizations and consumers can finally make informed purchasing decisions when it comes the security of their products, and measurably realize more hardened environments. Insurance groups are already engaging CITL, as are organizations focused on consumer safety. Vendors will see how much better or worse their products are in comparison to their competitors. Even exploit developers have demonstrated that these results enable bug-bounty arbitrage. That recommendation you made to your family members last holiday about which web browser they should use to stay safe (or that large purchase you made for your industrial control systems)? Well, you can finally see if you chose a hard or soft target… with the data to back it up.
Mudge Zatko is the Director of CITL. He has contributed significantly to disclosure and education on information and security vulnerabilities. In addition to pioneering buffer overflow work, the security work he has released contained early examples of flaws in the following areas: code injection, race conditions, side-channel attacks, exploitation of embedded systems, and cryptanalysis of commercial systems. He was the original author of the password cracking software L0phtCrack, Anti-Sniff, and L0phtWatch. In 2010 Mudge accepted a position as a program manager at DARPA where he oversaw cyber security R&D, and re-built the Agency’s approach to cyber security research. In 2013 Mudge went to work for Google where he was the Deputy Director of their Advanced Technology & Projects division. Most recently, after conversations with the White House, Mudge stood up the non-profit Cyber Independent Testing Laboratory inspired by efforts such as Consumer’s Union. He is the recipient of the Secretary of Defense Exceptional Civilian Service Award medal, an honorary Plank Owner of the US Navy Destroyer DDG-85, was inducted into the Order of Thor, the US Army’s Association of Cyber Military Professionals, recognized as a vital contributor to the creation of the US Cyber Corps (SfS PDD-63), and has received other commendations from the CIA and from the Executive Office of the President of the United States
Sarah Zatko s the Chief Scientist at CITL, a partner at L0pht Holdings, LLC, and a member of the US Army’s Order of Thor. She has presented her research on the integration of security into CS curriculum at Shmoocon and Hope. That work is also published in IEEE Security & Privacy. She holds a degree in mathematics from MIT and a Master's in computer science from Boston University.