Many industries, provide consumers with data about the quality, content, and cost of ownership of products, but the software industry leaves consumers with very little data to act upon. In fact when it comes to how secure or weak a product is from a security perspective, there is no meaningful consumer facing data. There has long been a call for the establishment of an independent organization to address this need.
Last year, Mudge (from DARPA, Google, and L0pht fame) announced that after receiving a phone call from the White House he was leaving his senior position inside Google to create a non-profit organization to address this issue. This effort, known as CITL, is akin to Consumer Reports in its methodologies. While the media has dubbed it a "CyberUL", there is no focus on certifications or seals of approval, and no opaque evaluation metrics. Rather, like Consumer Reports, the goal is to evaluate software according to metrics and measurements that allow quantitative comparison and evaluation by anyone from a layperson, CFO, to security expert.
How? A wide range of heuristics that attackers use to identify which targets are hard or soft against new exploitation has been codified, refined, and enhanced. Some of these techniques are quite straightforward and even broadly known, while others are esoteric tradecraft. To date, no one has applied all of these metrics uniformly across an entire software ecosystem before and shared the results. For the first time, a peak at the Cyber Independent Testing Lab's metrics, methodologies, and preliminary results from assessing the software quality and inherent vulnerability in over 100,000 binary applications on Windows, Linux, and OS X will be revealed. All accomplished with binaries only.
Sometimes the more secure product is actually the cheaper, and quite often the security product is the most vulnerable. There are plenty of surprises like these that are finally revealed through quantified measurements. With this information, organizations and consumers can finally make informed purchasing decisions when it comes the security of their products, and measurably realize more hardened environments. Insurance groups are already engaging CITL, as are organizations focused on consumer safety. Vendors will see how much better or worse their products are in comparison to their competitors. Even exploit developers have demonstrated that these results enable bug-bounty arbitrage.
That recommendation you made to your family members last holiday about which web browser they should use to stay safe (or that large purchase you made for your industrial control systems)? Well, you can finally see if you chose a hard or soft target… with the data to back it up.