x86 processors contain a surprising amount of built-in memory translation logic, which is driven by various data tables with intricate entry formats, and can produce various kinds of traps and other interesting computational effects. These features are mostly relics of earlier, more civilized times, when Jedi Knights tried to protect the Old Republic OS’s with segmentation, supervisor bits, and hardware task support, but were defeated by processor de-optimizations and performance concerns and left unused by both Windows and UNIX systems – and explored only by hackers. While the rest of the world programs only the x86 CPU with the provided instructions, clever neighbours like the PaX team instead program the MMU to enforce security policy.
We will show that the MMU is in fact a Turing-complete processor in its own right and demonstrate some tools that help to unleash its computational power. Furthermore, we will show some design suggestions (and possibly a FPGA prototype) to make the virtual memory system more suitable and easier to use as an enforcer of runtime policy.
Julian Bangert is a junior studying computer science at Dartmouth College. When he is not working on new defence mechanisms or dropping off waterfalls in his Kayak, he is a neighbourly cowboy in the ranges of Northern Appalachia, capturing specimens for his professors Sergey Bratus weird machine zoo.
Sergey Bratus is a Research Assistant Professor of Computer Science at Dartmouth College. He tries to help fellow academics to understand the value and relevance of hacker research. It is his ambition to collect and classify all kinds of weird machines; he is also a member of the http://langsec.org conspiracy to eliminate large classes of bugs.