The Life of Binaries

Creator: Xeno Kovah

License: Creative Commons: Attribution, Share-Alike (http://creativecommons.org/licenses/by-sa/3.0/)

Class Prerequisites: Recommended, but not required, to have taken Introduction to x86 and Intermediate x86

Lab Requirements: Requires a Windows system with Visual C++ Express Edition, Windows DDK or WDK kernel module compilation environment, and WinDbg. Requires a Windows guest OS running in VMWare Player or VMWare Server in order to do kernel debugging with WinDbg from the host OS.

Class Textbook: None

Recommended Class Duration: 3 days (class previously taught in 2 days which was too little)

Creator Available to Teach In-Person Classes: Yes

Author Comments:

Topics include but are not limited to: •Scanning and tokenizing source code. •Parsing a grammar. •Different targets for x86 assembly object files generation. (E.g. relocatable vs. position independent code). •Linking object files together to create a well-formed binary. •Detailed descriptions of the high level similarities and low level differences between the Windows PE and Linux ELF binary formats. (NOTE: we didn't get to this in the class where the video was recorded, but the materials are in the slides) •How an OS loads a binary into memory and links it on the fly before executing it.

Along the way we discuss the relevance of security at different stages of a binary’s life, from the tricks that can be played by a malicious compiler, to how viruses really work, to the way which malware “packers” duplicate OS process execution functionality, to the benefit of a security-enhanced OS loader which implements address space layout randomization (ASLR).

Lab work includes: •Manipulating compiler options to change the type of assembly which is output •Manipulating linker options to change the structure of binary formats •Reading and understanding PE files with PEView •Reading and understanding ELF files with Readelf (NOTE: we didn't get to this in the class where the video was recorded, but the materials are in the slides) •Using WinDbg and/or GDB to watch the loader dynamically link an executable •Using Thread Local Storage (TLS) to obfuscate control flow and serve as a basic anti-debug mechanism •Creating a simple example virus for PE •Analyze the changes made to the binary format when a file is packed with UPX •Using the rootkit technique of Import Address Table (IAT) hooking to subvert the integrity of a program’s calls to external libraries, allowing files to be hidden.

Knowledge of this material is recommended, but not required, for future classes such as Rootkits, but is required for reverse engineering.

A student Q&A forum has been set up at http://www.reddit.com/r/OST_LifeOfBinaries.

To submit any suggestions, corrections, or explanations of things I didn’t know the reasons for, please email me at the address included in the slides.

Presented by