IPS devices are now an accepted, integral part of a defense-in-depth InfoSec strategy; by strategically positioning them on the network, attacks can be blocked before they ever reach their intended targets. But with the explosion of public exploits, polymorphic malware and an ever-increasing attack surface, how can IPS devices keep up? They all seem to have heuristic detection capabilities, which are supposed to protect you from unknown exploits, and frequent updates to protect against known vulnerabilities. But just how effective are those defenses? Sure, you can check out the Gartner magic quadrant or pay for the latest NSS Test report. Just because an IPS claims to protect you from a vulnerability doesn'tmean thats the case. In this talk, I'll talk about some of the strengths and weakness of IPS devices, as well entire classes of exploits that cause serious problems for IPS devices. While I happen to work for a company sells a very expensive device for testing IPS devices (which is where the data and my opinions come from), I plan to focus on how the same testing methodologies can be applied and the results can be duplicated using open-source tools.