Virtual Breakpoints for x86_64

Efficient, reliable trapping of execution in a program at the desired location is a linchpin technique for dynamic malware analysis. The progression of debuggers and malware is akin to a game of cat and mouse – each are constantly in a state of trying to thwart one another. At the core of most efficient debuggers today is a combination of virtual machines and traditional binary modification breakpoints (int3). In this paper, we present a design for Virtual Breakpoints — a modification to the x86 MMU which brings breakpoint management into hardware alongside page tables. In this paper we demonstrate the fundamental abstraction failures of current trapping methods, and rebuild the mechanism from the ground up. Our design incorporates the lessons learned from 40 years of virtualization and debugger design to deliver fast, reliable trapping without the pitfalls of traditional binary modification.

Presented by