x86 processors contain a surprising amount of built-in memory translation logic, which is driven by various data tables with intricate entry formats, and can produce various kinds of traps and other interesting computational effects. These features are mostly relics of earlier, more civilized times, when Jedi Knights tried to protect the Old Republic OS’s with segmentation, supervisor bits, and hardware task support, but were defeated by processor de-optimizations and performance concerns and left unused by both Windows and UNIX systems – and explored only by hackers. While the rest of the world programs only the x86 CPU with the provided instructions, clever neighbours like the PaX team instead program the MMU to enforce security policy.
We will show that the MMU is in fact a Turing-complete processor in its own right and demonstrate some tools that help to unleash its computational power. Furthermore, we will show some design suggestions (and possibly a FPGA prototype) to make the virtual memory system more suitable and easier to use as an enforcer of runtime policy.