Exploits Tutorial: The Memory Concepts

In order to compromise a computer system, you need to first find a vulnerability in it’s defenses and successfully exploit the vulnerability. (Un-)Fortunately modern computer systems employs multiple different defenses in order to stop you. In this series we learn of couple different types exploits and how we can exploit them.

Computer Science 101

When program starts, operating system loads program to the memory from some kind of storage. During the loading Operating System parses the executable file and places program code and static data on proper places on memory and initializes stack and registers. After program has been properly initialized the execution flow is then passed to the executable entry point.

Instructions and machine code

The Intel developed x86 (32 bit) (and the AMD developed x64-64) instruction sets are most common in modern desktop operating systems. Assembly language represents those instructions as pure as it can by not adding or redacting anything in process (C compiler usually optimizes programmers code).

Intel format of assembler language which NASM assembler uses has command <destination>, <source> syntax. There is a another variant called AT&T syntax, but were not using it in this tutorial. AT&T syntax is used by GNU Assembler (GAS). Let’s take a look on what assembly code looks like…

1
2
3
4
5
6
7
8
9
10

; Semicolon marks start of the comment in the assembly syntax
; Assembly code is usually heavily commented since it is much harder to interpret than
; Other higher level programming languages.
;
TestLabel: ; Labels acts as jump destinations for your code
mov eax, ebx ; Store contents of ebx register to eax register
nop ; No operation instruction, processor idles
cmp eax, ebx ; Compares values between eax and ebx registers
je TestLabel ; If last cmp or test instruction result was equal, jump to label "TestLabel"
jmp OtherLabel; Do unconditional jump to label "OtherLabel" and continue execution from there

Sample assembly coded program in Linux looks as following. Don’t mind if it’s hard follow, we dig in to the details later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

;
; This program runs in 32-bit protected mode.
; build: nasm -f elf -F stabs name.asm
; link: ld -o name name.o
;
; In 64-bit long mode you can use 64-bit registers (e.g. rax instead of eax, rbx instead of ebx, etc.)
; Also change "-f elf " for "-f elf64" in build command.
;
section .data ; section for initialized data
str: db 'Hello world!', 0Ah ; message string with new-line char at the end (10 decimal)
str_len: equ $ - str ; calcs length of string (bytes) by subtracting the str's start address
; from this address ($ symbol)
;
section .text ; this is the code section
global _start ; _start is the entry point and needs global scope to be 'seen' by the
; linker --equivalent to main() in C/C++
_start: ; definition of _start procedure begins here
mov eax, 4 ; specify the sys_write function code (from OS vector table)
mov ebx, 1 ; specify file descriptor stdout --in gnu/linux, everything's treated as a file,
; even hardware devices
mov ecx, str ; move start _address_ of string message to ecx register
mov edx, str_len ; move length of message (in bytes)
int 80h ; interrupt kernel to perform the system call we just set up -
; in gnu/linux services are requested through the kernel
mov eax, 1 ; specify sys_exit function code (from OS vector table)
mov ebx, 0 ; specify return code for OS (zero tells OS everything went fine)
int 80h ; interrupt kernel to perform system call (to exit)

Original source code from Wikipedia.

Registers

Intel CPU’s have multiple different registers which are used in different cases. Below we quickly recap what and where each of these general registers are used. Modern CPU’s may contain plenty of extra registers besides these 9.

Name	Description	32bit register	64bit register	16bit register	8 bit register
Accumalator Register	Although basic calculations can use any of the two registers, the EAX register is required to be used in specialized calculations (multiplication, division). Also many of the basic calculation instructions are optimized to use EAX register.	EAX	RAX	AX	AL
Base Register	The base register acts as general purpose pointer. The name base register derives from 16-bit world when it was the only register which could act as pointer. In 32-bit world, any of these registers can act as memory offset or pointer.	EBX	RBX	BX	BL
Counter Register	The ECX register is used in loops as index. Many looping and counting operands (LOOP, LOOPX, LOOPNZ, JCXZ) use ECX register as input. Only difference between higher level index variable and ECX is that usually in higher level languages loop value increases (index grows) but ECX does opposite and decreases in every loop cycle.	ECX	RCX	CX	CL
Data Register	The EDX register is used to read or write data to port and along with the EAX in the calculations.	EDX	RDX	DX	DL
Source Index Register	Used at reading of a data from the memory.	ESI	RSI	SI	SIL
Destination Index Register	Used at writing of a data to the memory.	EDI	RDI	DI	DIL
Base Pointer Register	The EBP stores address of a parameters of the function in stack.	EBP	RBP	BP	BPL
Stack Pointer Register	The ESP Contains location pointing to the top of the stack	ESP	RSP	SP	SPL
Instruction Pointer Register	The EIP register points always to the next instruction to be executed. By modifying the contents of EIP, we can modify the execution of the program.	EIP	RIP	IP	IPL

64-bit, 32-bit, 16-bit and 8-bit registers of same type are actually same register, bitness only affects which part of the 64-bit register we are refering.

Memory

The amount of the memory the application can use depends on the bitness of the CPU. 32-bit CPU can address up to 2³² bytes of RAM which equals as 4 GB [4 294 967 296 / (1024 x 1024 x 1024)].

The 64-bit CPU in other hand can handle 2⁶⁴ bytes of RAM which is 16 EB (18 446 744 073 709 551 616 bytes).

Virtual Memory

During the initialization of the application the operating system creates large continuous block of virtual memory to the application. Depending on the operating system, the virtual memory of the application is divided between Kernel space and User space. Kernel space is shared between all the applications (and the operating system) and applications can not access Kernel space memory directly. User space contains applications segments (data and code), heap and stack.

Kernel and User Space

In 32-bit CPU, split between Kernel and User space depends on operating system and operating systems configuration. Default Windows uses 2 GB to the Kernel Space and 2 GB to the User Space. Windows with “/3GB” switch and Linux by default settings use 3 GB of memory to User Space and 1 GB to Kernel Space.

Separation of Virtual and Physical Memory

Virtual Memory is not correlated to physical memory of computer and computer may have only a tiny fraction of virtual memory available in physical memory. Instead used virtual memory pages are mapped to physical memory by operating system memory manager. The virtual memory is backed by physical memory only after the application is using it.

Contents of the Virtual Memory

When operating system initializes application, it first creates applications own virtual memory area. Next it loads the program code (text segment of the application binary), initialized data (data segment), uninitialized data (BSS segment) and creates HEAP (dynamic data) and STACK (memory local to function or subroutine). Text, Data and BSS segments sizes are fixed and the sizes depends on the program. Heap and Stack in other hand can grow and shrink.

Stack

Stack is special FIFO data structure where program can store current subroutine (or functions) data which should be discarded after subroutine has run. Another main functions of the stack is keeping track of return address where execution flow should return after subroutine is done. Stack keeps growing when multiple subroutines or functions are nested. When recursive function returns, stack shrinks as subroutines complete and release data.

Example C/C++:

1
2
3
4
5
6

void function()
{
int x=64; //variable x with value 64 is pushed in to the stack
x = x+64; //variable x increased by 64
char *bytes = alloca(x); //Stack is allocated by x bytes (128 bytes)
} // Stack is free'd on exit

Example ASM:

1
2
3
4
5
6
7
8

mov eax, 64 ; Initialize eax with value 64
push eax; ; Push value 64 in to the stack
add eax, 64 ; Register eax is increased by 64
sub esp, 128 ; Grow stack by 128 bytes
;
; On exit we need to manually free the stack
add esp, 128 ; Stack is shrinked by 128 bytes
pop eax ; Get the last variable out and we're done

Heap

Heap is the memory area where data is allocated when program needs to store more persistent data which is used by multiple different subroutines or functions. Operating system manages the heap memory the program uses by allocating and freeing the memory as process requests. In Linux you can increase and decrease heap manually by using sbrk and brk system calls. Usually memory (heap) allocation is handled by higher level functions malloc and free which in turn uses brkand sbrk syscalls internally.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

#include <unistd.h>
#include <stdio.h>

int main(void)
{
printf("brk and sbrk method:\n");
void *heap = sbrk(0); // Get the heap address
printf("Heap Address: %p\n", heap); // And display it
brk(heap + 2*sizeof(int)); // Increase heap by size of two bytes
printf("Heap Address after memory allocation: %p\n", sbrk(0));
int *x = (int *)heap; //Get the address of the first int
*x = 1; //First int = 1
printf("Variable x address: %p, value: %d\n", x, *x);
int *y = (heap+sizeof(int)); // Get the address of the second int
*y = 2; // Second int = 2
printf("Variable y address: %p, value: %d\n", y, *y);
int z = *x + *y; //Some simple calculations
printf("Variable z (stack)address: %p, value: %d\n", &z, z);
brk(heap+1); //Deallocate memory and trash x & y
/* If trying to set heap to it's original value, Kernel will segfault
* the progrom. Hence, we need to set it at least one byte larger than
* the initial value.
* Tested on 4.13.0-kali1-amd64 kernel. */
printf("Heap Address after memory release: %p\n", sbrk(0));
/* Malloc and Free example */
printf("\nMalloc and Free method:\n");
int *a = (int*)malloc(sizeof(int));
int *b = (int*)malloc(sizeof(int));
int c;
*a = 1;
*b = 2;
c = *a + *b;
printf("Heap Address after memory allocation: %p\n", sbrk(0));
printf("Variable a address: %p, value: %d\n", a, *a);
printf("Variable b address: %p, value: %d\n", b, *b);
printf("Variable c address: %p, value: %d\n", &c, c);
free(a);
free(b);
printf("Heap Address after memory deallocation: %p\n", sbrk(0));
return 0; //Return 0 (Exit)
}

Program output is following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

brk and sbrk method:
Heap Address: 0x560049f9d000
Heap Address after memory allocation: 0x560049f9d008
Variable x address: 0x560049f9d000, value: 1
Variable y address: 0x560049f9d004, value: 2
Variable z (stack)address: 0x7ffdd9b08f94, value: 3
Heap Address after memory release: 0x560049f9d001

Malloc and Free mehtod:
Heap Address after memory allocation: 0x560049f9d001
Variable a address: 0x560049f7c420, value: 1
Variable b address: 0x560049f7c440, value: 2
Variable c address: 0x7ffdd9b08f90, value: 3
Heap Address after memory deallocation: 0x560049f9d001

Please not that memory addresses shown by the program varies by every run, just note the starting address, variable addresses (increasing memory address) and how the heap behaves. Also note that variable int z and int c is in the stack and so it is in the different memory segment (0x7f… vs 0x55…).

If you compare brk/sbrk and malloc/free methods you notice that malloc/free does not change heap size in this example. Instead it uses memory addresses already in heap which compiler probably has already allocated there.

Exploits Tutorial: The Memory Concepts