BlobRunner — Quickly Debug Shellcode Extracted During Malware Analysis

( Original text by LYDECKER BLACK )

BlobRunner is a simple tool to quickly debug shellcode extracted during malware analysis.
BlobRunner allocates memory for the target file and jumps to the base (or offset) of the allocated memory. This allows an analyst to quickly debug into extracted artifacts with minimal overhead and effort.

 

To use BlobRunner, you can download the compiled executable from the releases page or build your own using the steps below.
Building
Building the executable is straight forward and relatively painless.
Requirements

  • Download and install Microsoft Visual C++ Build Tools or Visual Studio

Build Steps

  • Open Visual Studio Command Prompt
  • Navigate to the directory where BlobRunner is checked out
  • Build the executable by running:
cl blobrunner.c

Building BlobRunner x64
Building the x64 version is virtually the same as above, but simply uses the x64 tooling.

  • Open x64 Visual Studio Command Prompt
  • Navigate to the directory where BlobRunner is checked out
  • Build the executable by running:
 cl /Feblobrunner64.exe /Foblobrunner64.out blobrunner.c

Usage
To debug:

  • Open BlobRunner in your favorite debugger.
  • Pass the shellcode file as the first parameter.
  • Add a breakpoint before the jump into the shellcode
  • Step into the shellcode
BlobRunner.exe shellcode.bin

Debug into file at a specific offset.

BlobRunner.exe shellcode.bin --offset 0x0100

Debug into file and don’t pause before the jump. Warning: Ensure you have a breakpoint set before the jump.

BlobRunner.exe shellcode.bin --nopause

Debugging x64 Shellcode
Inline assembly isn’t supported by the x64 compiler, so to support debugging into x64 shellcode the loader creates a suspended thread which allows you to place a breakpoint at the thread entry, before the thread is resumed.

Remote Debugging Shell Blobs (IDAPro)
The process is virtually identical to debugging shellcode locally — with the exception that the you need to copy the shellcode file to the remote system. If the file is copied to the same path you are running win32_remote.exe from, you just need to use the file name for the parameter. Otherwise, you will need to specify the path to the shellcode file on the remote system.

Shellcode Samples
You can quickly generate shellcode samples using the Metasploit tool msfvenom.
Generating a simple Windows exec payload.

msfvenom -a x86 --platform windows -p windows/exec cmd=calc.exe -o test2.bin

Feedback / Help

  • Any questions, comments or requests you can find us on twitter: @seanmw or @herrcore
  • Pull requests welcome!
Реклама

Linux Buffer Overflows x86 – Part 3 (Shellcoding)

( Original text by SubZero0x9 )

Hello guys, this is the part 3 of the Linux buffer Overflows x86. In this part we will learn what is shellcode, why do we use it and how do we use it. In the previous two blogs, we learnt about process memory, stack region, stack operations, stack registers, attempting buffer overflow, overwriting buffer, modifying return address, manipulating program flow and program execution.
If you have not read the first two blogs, here are the links Part1 and Part2.

The main purpose behind exploiting Buffer Overflow is to spawn a reverse shell or execute arbitrary commands at least. Even if we find a vulnerable binary with stack overflow and are able to overwrite the return address, we still have to tell the binary to execute commands on our behalf. But how do we do that? Suppose we want to spawn a shell of a vulnerable binary, but the code to spawn a shell is not present in the binary. So, even if we control the flow of the binary we really can’t modify the return address to a memory address which can execute the code which spawns a shell. We saw in the previous two blogs, we were able to overwrite the buffer with any arbitrary data. This means, when the program is loaded into memory all the operations are carried out on the stack. Any user input accepted will be stored on the stack and if the program accepts malicious input (remember insufficient bound checking) it also gets stored on the stack. So instead of placing random data, if we place the code which we want to execute on to the stack it will give us the results.
Simple Right? NO !!

We can’t simply go ahead and paste the code of spawning a shell onto the buffer, because when the program is running it is loaded into the system memory and it communicates with the processor for carrying out the operations through syscalls which is understood by kernel who actually communicates with the processor to get things done(well this is a simplified statement, the actual process is too complex). Now the processor doesn’t understand C,C++, JAVA, Python, etc. it only understands the machine code which it can directly execute. And We, human beings can’t understand machine code so we write programs in high level languages. Also, machine code is mostly regarded as the lowest level representation of compiled code. Another main reason we use machine code is because of the size. The space available in the stack for operations are very limited and machines codes are really compact in that sense.

SHELLCODE

So till now it is pretty clear that the code or payload used to exploit the buffer overflow vulnerability to execute arbitrary commands is called Shellcode. Its name is derived from the fact that it was initially used to spawn a root shell. Shellcode is basically a set of machine code or set of instructions injected into the buffer of the vulnerable program. The vulnerable program then executes the shellcode.

Shellcodes are machine codes. Basically, we write a set of instructions in a high level language or assembly language then convert into hexadecimal values (these values are just a bunch of opcodes and operands understood by the processor).

We cannot run shellcode directly, instead it needs a some sort of carrier (buffer, file, process, environment variables, etc) which will be read or executed by the vulnerable software. There are many shellcode execution strategies which are used depending on the exploitability and constraints of the vulnerable software.

Shellcodes are written mainly to force the program to do actions on the behalf of the attacker. The attacker can write a specific function which will do the intended action once loaded into the same address space of the vulnerable software. Every action (well almost) in an operating system is done by calling a syscall or system calls. The shellcode will also contain a syscall for a specific action which the attacker needs to perform. Syscalls are the most important thing in the shellcode as you can force the program to perform action just by triggering a syscall.

SYSCALLS

According to wikipedia, “In computing, a system call is the programmatic way in which a computer program requests a service from the kernel of the operating system it is executed on. ”

Linux’s security mechanism (the ring model) is divided into two spaces : User Space and Kernel space. Eevery process running in the userspace has its own virtual address space and cannot access any other processes address space until explicitly allowed. When the process in the userspace wants to access the kernel space to execute some operations, it cannot directly access it. No processes in the userspace can access the kernel space.

When an userspace process tries to access the kernel, an exception is raised and prevents the process from accessing the kernel space. The userspace processes uses the syscalls to communicate with the kernel processes to carry out the operations.

Syscalls are a bunch of functions which gives a userland process direct access to the kernel space to carryout low level functions. In simpler words, Syscalls are an interface between user mode and kernel mode.

In Linux, syscalls are generated using the software interrupt instruction, int 0x80. So, when a process in the user mode executes the int 0x80 instruction, the flow is transferred from the user mode to kernel mode by the CPU and the syscall is executed. Other than syscalls, there are standard library functions like the libc (which is extensively used in the linux OS). These standard library functions also call the syscall in some way or another (not everytime though). But we wont look into that as we are not going to use library functions in our shellcode. Also, using syscall is much nicer because there is no linking involved with any library as needed in other programming languages.

Note: The software interrupt int 0x80 was proving to be slower on the Pentium processors, so Linus introduced two new alternative namely SYSENTER/SYSEXIT instructions which were provided by all Pentium II+ processors. We will use the int 0x80 instruction in our shellcode however.

HOW TO USE SYSCALLS

When a process is in the userspace and the syscall instruction is executed, the interrupt handler i.e the software interrupt int 0x80 tells the CPU to change the execution from user mode to kernel mode. Now the system call handler is notified to call the routine which the userland process wanted. All the syscalls are stored in a syscall table, it is a big array which points to the memory address of the syscall routines. When the syscall is called, the syscall number is loaded into EAX register, this is how the system call handler comes to know which system call routine to execute. Keep in mind that the execution done here is carried out on the Kernel space stack and after the execution the control is given back to the userspace process and the remaining operation is carried out on userspace stack.

Here is the list of all the syscalls (x86):- http://syscalls.kernelgrok.com/

Each syscall is assigned a syscall number (an integer value) for basic naming convention. Every syscall has arguments which are necessary to execute the related operations. The syscall number is stored in the EAX register and the arguments are stored in the EBX , ECX , EDX , ESI and EDI respectively.

All the linux syscalls are well documented with instructions of how to implement them in your program. Usually we can use the syscall in normal C program or we can go ahead and do the same in an assembly language. Since we are writing our own shellcode, we have to dip our hands in the world of assembly language.

So what we will do is, write a C program with the syscall, disassemble the program, see the assembly instructions, write the assembly equivalent, make the necessary changes and extract the shellcode. Now we could have done this in assembly language directly but for getting the whole idea of how a program works from the high level to low level I am choosing this path.

Basic Assembly knowledge is needed for this, if you are new to Assembly then you can read the Assembly and Shellcoding blog series by my friend SLAER.

SPAWNING A SHELL

Lets write a shellcode for spawning a shell. We will use execve syscall for this operation.

First, lets take a look at the man page for execve..

The whole function looks like this:-

Here, filename is the command or program which we want to execute, in our case “/bin/sh”. Argv is an array of the argument string passed to the new program and it should contain the filename in its first index. Envp array can be NULL. Also the argvand envp array must end with a NULL.

We will initiate a char array spawnshell[2], with spawnshell[0]= “/bin/sh” and the next index being NULL i.e spawnshell[1]=NULL. We declared the last index as NULL because argv must end with a NULL byte.

So, execve will look like:

The C code for spawning a shell :

Compiling the program:

(We used static flag while compiling to prevent the dynamic linking as we want to disassemble the execve function.)

Running the executable we get a shell.

Lets disassemble the main function in our favorite GDB:

Here, the execve is called from the libc library, because we wrote a C program and used stdlib. Before calling the execve() function, the arguments required by it are stored on the stack. We will see the important instructions in detail:

sub $0x14, %esp -> Subtracting 14 from ESP, this is how the space is allocated on the stack.

1) movl $0x80bb868,-0x14(%ebp) -> Moving the memory address 0x80bb868 which contains “/bin/sh” on the stack. (You can see the value by using the command x/1s 0x80bb868 in GDB)

2) movl $0x0,-0x10(%ebp) -> Moving 0 on the stack at -0x10(%ebp), means the NULL byte after the “/bin/sh”.

3) mov -0x14(%ebp),%eax -> EAX now has the “/bin/sh”.

4) push $0x0 -> 0 is pushed onto the stack.

5) lea -0x14(%ebp),%edx -> The effective address of -0x14(%ebp) is loaded into EDX i.e EDX is pointing to the pointer of “/bin/sh”.

6) push %edx -> Pushing the value of EDX on the stack.

7) push %eax -> Pushing the value of EAX on the stack.

8)call 0x806c990 <execve> -> calling the function execve.

So what does this mean? Every numbered points are explained below in the same manner:-

1) filename = “/bin/sh”

2) filename = “/bin/sh” followed by NULL

3) argv[0]= “/bin/sh”

4) 0 is pushed for the envp

5) pointing to the pointer of “/bin/sh” i.e argv

6) EDX=argv

7) EAX= filename

8) execve function is called

Now the execve function according to the arguments pushed onto the stack will look like:-

execve(EAX, EDX,0)

So, now we know what to do with the assembly instructions. Lets write an assembly program for execve syscall.

Lets look at the execve syscall w.r.t assembly instructions:

Assembly program:-

Assemble this program:

Link this program:

Running the program we can see we get a shell.

Now, we want to extract the opcodes from the executable to form our shellcode. We will use objdump utility to display the object file.

Here, we can see in the right side, the assembly instructions that we wrote and in the left side opcodes for those instructions. Now we just have to copy these opcodes and execute it from C program which will act as a carrier for our shellcode.

Our Shellcode now is:

“\xc7\x05\xa8\x90\x04\x08\xa0\x90\x04\x08\xb8\x0b\x00\x00\x00\xbb\xa0\x90\x04\x08\xb9\xa8\x90\x04\x08\xba\xac\x90\x04\x08\xcd\x80\xb8\x01\x00\x00\x00\xbb\x0a\x00\x00\x00\xcd\x80”

As we already know we cannot execute shellcode directly, we will be needing a program to execute it for us. Shellcode is always loaded into the virtual address space of the target process which we want to exploit. Shellcode cant have its own process space as it cant be executed directly and it doesn’t makes sense to load the shellcode in any other address space other than the vulnerable software.

Just for demonstration we will load the shellcode in the address space of our own program and see whether it gets executed or not.

We will write a small C program which will execute our shellcode:

So, whats the program doing?

First, we are defining a char array shellcode with our actual shellcode in it. Then, in the main() function we are defining a variable ret which is an int pointer.

Then we are making the ret variable point to an address which is 8 bytes (2 int value)from its own address.

Remember this will happen on the stack, and the addresses on the stack moves from higher order memory to the lower order memory. So, by adding 8 bytes to the address of ret variable we are actually making it points towards the main()function.

Then we are assigning the address of our shellcode array to the address of the retvariable which is pointing to the address of main. So before the program ends it will execute our shellcode.

Lets compile the program:

Running the program we get Segmentation fault (core dumped)

So, we can see that our shellcode didn’t work. Lets take a look at what is wrong with our shellcode.

We can see there are a bunch of NULL bytes in the opcodes i.e ‘00’. Now one important thing, mostly the shellcode is loaded into the user input buffer and mostly it will be a char buffer or string buffer. If a NULL value occurs in the string, it terminates the string right there and doesn’t accept anything after the NULL termination. So our shellcode containing NULL values wont work. So we have to find a way to eliminate the NULL values.

Also, we are using hardcoded addresses in our shellcode. As we can see the mov instructions, all those addresses are machine specific hardcoded addresses. It is highly likely that it might not work in different versions of linux. So we have to deal with that also. One thing we can do is somehow load the starting address of our shellcode in the memory and using that reference we can make the necessary operations relative to this address. Now this is called as relative addressing. We will see this later in more detail.

Now the important part is to modify the assembly code with the two requirements mentioned above, getting rid of the NULL values and implementing relative addressing for avoiding hardcoded addresses .

REFINING THE SHELLCODE

So lets start…

From the above assembly program we know what instructions are needed to create our shellcode. In the above program we defined an ASCII string with “/bin/sh” and assigned the address of the string to another variable which will act as the pointer. But in the shellcode we will get the memory address of that variable and it will not work around different linux machines as the address may differ.

So in the manpages of the execve syscall it is clearly stated that, “The argv and envp arrays must each include a null pointer at the end of the array.” In our case envparray is already NULL, but the filename i.e the first index of argv pointer should also end with a NULL value. In the above assembly program we were defining separate variables for each argument of the syscall, but now we cant do that.

The execve syscall has three char pointer arguments. For the first argument we will have “/bin/shA”. We are appending A because we directly cant add a NULL value as it will hamper our shellcode, instead we will assign the index of A, a null value from the register. For the next char argument we will append four B’s as char is 4 bytes. Now the second argument is a pointer to the start of the address of the filename. Here we will assign the index of the starting B with the address of the starting memory address of filename. The third argument is the envp array which is a NULL pointer and we will append it with four C’s and then we will assign the index of the first C with a null value.

The representation of execve syscall arguments w.r.t registers will be like:

int execve(EBX, ECX, EDX); and the execve syscall number 11 will be loaded in the EAX register.

Below is the representation of the string which we will load along with the index in hex.

Lets take a look into the final assembly program for our shellcode.

Refined assembly program for Shellcode:

Now I will explain in detail what we have done.

-> We are starting our shellcode with a jmp instruction. jmp instruction will make an unconditional jump to the specified memory location. It will skip the actual shellcode and directly jump to the ShellcodeCall memory address.

-> In there the call instruction is executed which will directly point our Shellcode. Now what happens here is the most important thing. When the call instruction is executed the next instruction or memory address which contains the string “/bin/shABBBBCCCC” is pushed onto the top of the stack as the ret address. This is how we implement relative addressing.

->Now the pop instruction is executed. So the pop instruction will load the retaddress which has the string in the ESI register. The ESI register has the memory location which points to “/bin/shABBBBCCCC”. ESI register will be the reference point for all the operation which we will perform so the issue of hardcoded addresses will be resolved.

->Next we are XORing the EAX register to avoid the NULL bytes. We are basically resetting the EAX register without assigning a NULL value.

->Then we are loading the value 0 from the AL register (As we all know the ALregister is the lower part of the EAX address which is already 0 after the XORing) to the 7th index of the ESI register which will replace the A in our string with a null value.

-> Then we are loading the value of ESI i.e the start address of our string into the 8th index of ESI register replacing the B’s in our string.

->Then we are loading 0 from the AL register into the 12th index of our ESI register i.e replacing the C’s with 0’s.

-> Then we are loading the arguments of execve into the EBXECX and EDXregisters.

->Then we are loading the execve syscall number 11 into AL register again avoiding the NULL bytes because the higher part of the EAX register holds 0.

-> Lastly we make a software interrupt which will execute the syscall.

Now lets take a look at the opcodes after assembling and linking the assembly program.

As we can see there are no null bytes and the hardcoded addresses are also solved.

Now lets copy our shellcode into the C program which we wrote earlier to check whether our shellcode executes or not.

Lets run the compiled C program binary and see if it works.

VOILA !!!

We successfully spawned a shell with our own shellcode.

This was a very basic example of writing a working shellcode. Now there are many situations where you want take a reverse shell over the network and you have to write a shellcode for that. That would be a really challenging task to write a shellcode of that manner. This blog post only introduced what a shellcode looks like and how much pain you have to take to write a simple working shellcode.

In the next part, we will exploit a vulnerable binary and execute shellcode on that.

If you have any doubts or opinions, kindly express yourselves in the comment section. Thanks !!!

Linux Buffer Overflows x86 Part 2 ( Overwriting and manipulating the RETURN address)

( Original text by SubZero0x9 )

Hello Friends, this is the 2nd part of the Linux x86 Buffer Overflows. First of all I want to apologize for such a long delay after the First blog of this series, there were personal and professional issues going on in my life (Well who hasn’t got them? Meh?).

In the previous part we learned about the Process Memory, Stack Region, Stack Operations, Stack Registers, Attempting Buffer Overflow, Overwriting Buffer, ESP and EIP. In case you missed it, here is the link to the Part 1:

https://movaxbx.ru/2018/11/06/getting-started-with-linux-buffer-overflows-x86-part-1-introduction/

Quick Recap to the Part 1:

-> We successfully exploited the insufficient bound checking by giving input which was larger than the buffer size, which led to the overflow.

-> We successfully attempted to overwrite the buffer and the EIP (Instruction Pointer).

-> We used GDB debugger to examine the buffer for our input and where it was placed on the stack.

Till now, we already know how to attempt a buffer overflow on a vulnerable binary. We already have the control of EIP and ESP.

So, Whats Next ?

Till now we have only overwritten the buffer. We haven’t really exploit anything yet. You may have read articles or PoC where Buffer Overflow is used to escalate privileges or it is used to get a reverse shell over the network from a vulnerable HTTP Server. Buffer Overflow is mainly used to execute arbitrary commands on the vulnerable system.

So, we will try to execute arbitrary commands by using this exploit technique.

In this blog, we will mainly focus on how to overwrite the return address to manipulate the flow of control and program execution.

Lets see this in more detail….

We will use a simple program to manipulate its intended output by changing the RETURN address.

(The following example is same as the AlephOne’s famous article on Stack Smashing with some modifications for a better and easy understanding)

Program:

 

This is a very simple program where the variable random is assigned a value ‘0’, then a function named function is called, the flow is transferred to the function()and it returns to the main() after executing its instructions. Then the variable random is assigned a value ‘1’ and then the current value of random is printed on the screen.

Simple program, we already know its output i.e 1.

But, what if we want to skip the instruction random=1; ?

That means, when the function() call returns to the main(), it will not perform the assignment of value ‘1’ to random and directly print the value of random as ‘0’.

So how can we achieve this?

Lets fire up GDB and see how the stack looks like for this program.

(I won’t be explaining each and every instruction, I have already done this in the previous blog. I will only go through the instructions which are important to us. For better understanding of how the stack is setup, refer to the previous blog.)

In GDB, first we disassemble the main function and look where the flow of the program is returned to main() after the function() call is over and also the address where the assignment of value ‘1’ to random takes place.

-> At the address 0x08048430, we can see the value ‘0’ is assigned to random.

-> Then at the address 0x08048437 the function() is called and the flow is redirected to the actual place where the function() resides in memory at 0x804840b address.

-> After the execution of function() is complete, it returns the flow to the main()and the next instruction which gets executed is the assignment of value ‘1’ to random which is at address 0x0804843c.

->If we want to skip past the instruction random=1; then we have to redirect the flow of program from 0x08048437 to 0x08048443. i.e after the completion of function() the EIP should point to 0x08048443 instead of 0x0804843c.

To achieve this, we will make some slight modifications to the function(), so that the return address should point to the instruction which directly prints the randomvariable as ‘0’.

Lets take a look at the execution of the function() and see where the EIP points.

-> As of now there is only a char array of 5 bytes inside the function(). So, nothing important is going on in this function call.

-> We set a breakpoint at function() and step through each instruction to see what the EIP points after the function() is exectued completely.

-> Through the command info regsiters eip we can see the EIP is pointing to the address 0x0804843c which is nothing but the instruction random=1;

Lets make this happen!!!

From the previous blog, we came to know about the stack layout. We can see how the top of the stack is aligned, 8 bytes for the buffer, 4 bytes for the EBP, 4 bytes for the RET address and so on (If there is a value passed as parameter in a function then it gets pushed first onto the stack when the function is called).

-> When the function() is called, first the RET address gets pushed onto the stack so the program flow can be maintained after the execution of function().

->Then the EBP gets pushed onto the stack, so the EBP can act as the offset reference point for other instructions to access and calculate the memory addresses.

-> We have char array buffer of 5 bytes, but the memory is allocated in terms of WORD. Basically 1 word=4 bytes. Even if you declare a variable or array of 2 bytes it will be allocated as 4 bytes or 1 word. So, here 5 bytes=2 words i.e 8 bytes.

-> So, in our case when inside the function(), the stack will probably be setup as buffer + EBP + RET

-> Now we know that after 12 bytes, the flow of the program will return to main()and the instruction random = 1; will be executed. We want to skip past this instruction by manipulating the RET address to somehow point to the address after 0x0804843c.

->We will now try to manipulate the RET address by doing something like below :-

 

Here, we are introducing an int pointer ret, and we are assigning it the starting address of buffer + 12 .i.e (wait lets see this in a diagram)

-> So, the ret pointer now ends exactly at the start where the return value is actually stored. Now, we just want to add ret pointer with some value which will change the return value in such a manner that it will skip past the return = 1;instruction.

Lets just go back to the disassembled main function and calculate how much we have to add to the ret pointer to actually skip past the assignment instruction.

-> So, we already know we want to skip pass the instruction at address 0x0804843c to address 0x08048443. By subtracting these two address we get 7. So we will add 7 to the return address so that it will point to the instruction at address 0x08048443.

-> And now we will add 7 to the ret pointer.

 

So our final code should look like:

 

So, now we will compile our code.

 

Since we are adding code and re-compiling the program, the memory addresses will get changed.

Lets take a look at the new memory addresses.

-> Now the assignment instruction random = 1; is at address 0x08048452 and the next address which we want the return address to point is 0x08048459. The difference between these addresses is still 7. So we are good till now.

Hopefully by executing this we will get the output as “The value of random is: 0”

HOLY HELL!!!

Why did it gave the Segmentation fault (core dumped)? That means maybe the process is trying to access a memory that doesn’t exist or something.

Note:- Well this was really tricky for me atleast. Computers works in mysterious ways, it is not just mathematics, numbers and science ( Yeah I am being dramatic)

Lets fire up our binary in GDB and see whats wrong.

-> As you can see we set a breakpoint at line 4 and run the program. The breakpoint hits and the program halts. Then we examine the stack top and what we see, the distance between the start of the buffer and the start of return address is not 12, its 13. (It doesn’t even make sense mathematically). Three full block adds up for 12 bytes (4 bytes each) and 1 byte for that ‘A’ i.e 41.

-> Our buffer had value “ABCDE” in it. And A in ASCII hex is 41. From 41 to just right before of the return address 0x08048452, the count is 13 which traditionally is not correct. I still don’t know why the difference is 13 instead of 12, if anyone knows kindly tell me in the comments section.

-> So we just change our code from ret = buffer + 12; to ret = buffer + 13;

Now the final code will look like this:

 

Now, we re-compile it and run and hope its now going to give a desired output.

Before continuing lets see this again in GDB:

> So, we can see now after 7 is added to the ret pointer, the return address is now 0x08048459 which means we have successfully skipped past the assignment instruction and overwritten the return address to manipulate our way around the program flow to alter the program execution.

Till now we learnt a very important part of buffer overflow i.e how to manipulate and overwrite the return address to alter the program execution flow.

Now, lets another example where we can overwrite the return address in more similar ‘buffer overflow’ fashion.

We will use the program from protostar buffer overflow series. You can find more about it here: https://exploit-exercises.com/protostar/

 

Small overview of a program:-

->There is a function win() which has some message saying “code flow successfully changed”. So we assume this is the goal.
-> Then there is the main(), int pointer fp is declared and defined to ‘0’ along with char array buffer with size 64. The program asks for user input using the getsfunction and the input is stored in the array buffer.
-> Then there is an if statement where if fp is not equal to zero then it calls the function pointer jumping to some memory location.

Now we already know that gets function is vulnerable to buffer overflow (visit previous blog) and we have to execute the win() function. So by not wasting time lets find out the where the overflow happens.

First we will compile the code:

 

We know that the char array buffer is of 64 bytes. So lets feed input accordingly.

So, as we can see after the 64th byte the overflow occurs and the 65th ‘A’ i.e 41 in ASCII Hex is overwriting the EIP.

Lets confirm this in GDB:

-> As we see the buffer gets overflowed and the EIP is overwritten with ‘A’ i.e 41.

Now lets find the address of the win() function. We can find this with GDB or by objdump.

Objdump is a tool which is used to display information about object files. It comes with almost every Linux distribution.

The -d switch stands for disassemble. When we use this, every function used in a binary is listed, but we just wanted the address of win() function so we just used grep to find our required string in the output.

So it tells us that the win() function’s starting address is 0804846b. To achieve our goal we just have to pass this address after the 64th byte so that the EIP will get overwritten by this address and execute the win() function for us.

But since our machine is little endian (Find more about it here)we have to pass the address in a slightly different manner (basically in reverse). So the address 0804846b will become \x6b\x84\x04\x08 where the \x stands for HEX value.

Now lets just form our input:

As we can see the win() function is successfully executed. The EIP is also pointing the address of win().

Hence, we have successfully overwritten the return address and exploited the buffer overflow vulnerability.

Thats all for this blog. I am sorry but this blog also went quite long, but I will try to keep the next blog much shorter. And also, in the next blog we will try to play with shellcode.

 

 

Getting Started with Linux Buffer Overflows x86 – Part 1 (Introduction)

( Original text by SubZero0x9 )

Hello Friends, this series of blog posts will purely focus on Buffer Overflows. When I started my journey in Infosec, this one topic fascinated me as much as it frightened me. When I read some of the blogs related to Buffer Overflows, it really seemed as some High-level gibber jabber containing C code, Assembly and some black terminals (Yeah I am talking about GDB). Over the period of time and preparing for OSCP, I started to learn about Buffer Overflows in detail referring to the endless materials on Web scattered over different planets. So, I will try to explain Buffer Overflow in depth and detail so everyone reading this blog can understand what actually a Buffer Overflow is.

In this blog, we will understand the basic fundamentals behind the Buffer Overflow vulnerability. Buffer Overflow is a memory corruption attack which involves memory, stack, buffers to name a few. We will go through each of this and understand why really Buffer Overflows takes place in the first place. We will focus on 32-bit architecture.

A little heads up: This blog is going to be a lengthy one because to understand the concepts of Buffer Overflow, we have to understand the process memory, stack, stack operations, assembly language and how to use a debugger. I will try to explain as much as I can. So I strongly suggest to stick till the end and it will surely clear your concepts. Also, from the next blog I will try to keep it short :p :p

So lets get started….
Before getting to stack and buffers, it is really important to understand the Process Memory Organization. Process Memory is the main playground where it all happens. In theory, the Process memory where the program/process resides is quite a complex place. We will see the basic part which we need for Buffer Overflow. It consists of three main regions: Text, Data and Stack.

Text: The Text region contains the Program Code (basically instructions) of the executable or the process which will be executed. The Text area is marked as read-only and writing any data to this area will result into Segmentation Violation (Memory Protection Mechanism).

Data: The Data region consists of the variables which are declared inside the program. This area has both initialized (data defined while coding) and uninitialized (data declared while coding) data. Static variables are stored in this section.

Stack: While executing a program, there are many function and procedure calls along with many JUMP instructions which after the functions work is done has to return to its next intended place. To carry out this operation, to execute a program, the memory has an area called Stack. Whenever the CALL instruction is used to call a function, the stack is used. The Stack is basically a data structure which works on the LIFO (Last In First Out) principle. That means the last object entering the stack is the first object to get out. We will see how a stack works in detail below.

To understand more about the Process Memory read this fantastic article: https://www.bottomupcs.com/elements_of_a_process.xhtml

So Lets see an Assembly Code Skeleton to understand more about how the program is executed in Process Memory.


Here, as we can see the assembly program skeleton has three sections: .data, .bss and .txt

-> .txt section contains the assembly code instructions which resides in the Text section of the process memory.

-> .data section contains the initialized data or lets say defined variables or data types which resides in the Data section of the process memory.

-> .bss section contains the uninitialized data or lets say declared variables that will be used later in the program execution which also resides in the Data section of the process memory

However in a traditionally compiled program there may be many sections other than this.

Now the most important part….

HOLY STACK!!!

As we discussed earlier all the dirty work is done on the Stack. Stack is nothing but a region of memory where data is temporarily stored to carry out some operations. There are mainly two operations performed by stack: PUSH and POPPUSH is used to add the object on top of the stack, POP is used to remove the object from top of the stack.

But why stack is used in the first place?

Most of the programs have functions and procedures in them. So during the program execution flow, when the function is called, the flow of control is passed to the stack. That means when the function is called, all the operations which will take place inside the function will be carried out on the Stack. Now, when we talk about flow of control, after when the execution of the function is done, the stack has to return the flow of control to the next instruction after which the function was called in the program. This is a very important feature of the stack.

So, lets see how the Stack works. But before that, lets get familiar with some stack pointers and registers which actually carries out everything on the Stack.

ESP (Extended Stack Pointer): ESP is a stack register which points to the top of the stack.

EBP (Extended Base Pointer): EBP is a stack register which points to the top of the current frame when a function is called. EBP generally points to the return address. EBP is really essential in the stack operations because when the function is called, function arguments and local variables are stored onto the stack. As the stack grows the offset of both the function arguments and variables changes with respect to ESP. So ESP is changed many times and it is difficult for the compiler to keep track, hence EBP was introduced. When the function is called, the value of ESP is copied into EBP, thus making EBP the offset reference point for other instructions to access and calculate the memory addresses.

EIP (Extended Instruction Pointer): EIP is a stack register which tells the processor about the address of the next instruction to execute.

RET (return) address: Return address is basically the address to which the flow of control has to be passed after the stack operation is finished.

Stack Frame: A stack frame is a region on stack which contains all the function parameters, local variables, return address, value of instruction pointer at the time of a function call.

Okay, so now lets see the stack closely by executing a C program. For this blog post we will be using the program challenge ‘stack0’ from Protostar exploit series, which is a stack based buffer overflow challenge series.
You can find more about Protostar exploit series here -> https://exploit-exercises.com/protostar/

The program:

Whats the program about ?
An int variable modified is declared and a char array buffer of 64 bytes is declared. Then modified is set to 0 and user input is accepted in buffer using gets() function. The value of modified is checked, if its anything other than 0 then the message “you have changed the ‘modified’ variable” will be printed or else “Try again?” will be printed.

Lets execute the program once and see what happens.

So after executing the program, it asks for user input, after giving the string “IamGrooooot” it displays “Try again?”. By seeing the output it is clear that modified variable is still 0.

Now lets try to debug this executable in a debugger, we will be using GDB throughout this blogpost series (Why? Because its freaking awesome).

Just type gdb ./executable_name to execute the program in the debugger.

The first thing we do after firing up gdb is disassembling the main() function of our program.

We can see the main() function now, its in assembly language. So lets try to understand what actually this means.

From the address 0x080483f4 the main() function is getting started.
-> Since there is no arguments passed in the main function, directly the EBP is pushed on the stack.
As we know EBP is the base pointer on the stack, the Stack pushes some starting address onto the EBP and saves it for later purposes.

-> Next, the value of ESP is moved into EBP. The value of ESP is saved into EBP. This is done so that most of the operations carried out by the function arguments and the variable changes the ESP and it is difficult for the compiler to keep track of all the changes in ESP.

-> Many times the compiler add some instructions for better alignment of the stack. The instruction at the address 0x080483f7 is masking the ESP and adding some trailing zeros to the ESP. This instruction is not important to us.

-> The next instruction at address 0x080483fa is subtracting 0x60 hex value from the ESP. Now the ESP is pointing to a far lower address from the EBP(As we know the stack grows down the memory). This gap between the ESP and EBP is basically the Stack frame, where the operations needed to execute the program is done.

-> Now the instruction at address 0x080483fd is from where all our C code will make sense. Here we can see the instruction movl $0x0,0x5c(%esp) where the value 0 is moved into ESP + the offset 0x5c, that means 0 is moved to the address [ESP+0x5c]. This is same as in our program, modified=0.

-> From address 0x08048405 to 0x0804840c are the instructions with accepts the user input.
The instruction lea 0x1c(%esp),%eax is loading the effective address i.e [ESP+0x1c] into EAX register. This address is pointing to the char array buffer on the stack. The lea and mov instructions are almost same, the only difference is the leainstruction copies the address of the register and offset into the destination instead of the content(which the mov instruction does).

-> The instruction mov %eax,(%esp) is copying the address stored in EAX register into ESP. So the top of the stack is pointing to the address 0x1c(%esp). Another important thing is that the function parameters are stored in the ESP. Assuming that the next instruction is calling gets function, the gets function will write the data to a char array. In this case the ESP is pointing to the address of the char buffer and the gets function writes the data to ESP(i.e the char array at the address 0x1c(%esp) ).

-> From the address 0x08048411 to 0x0804842e, the if condition is carried out. The instruction mov 0x5c(%esp),%eax is copying the value of modified variable i.e 0 into EAX. Then the test instruction is checking whether the value of modifiedis changed or not. If the value is not changed i.e the je instruction’s output is equal then the flow of control will jump to 0x08048427 where the message “Try again?” will be printed. If the value is changed then flow of control will be normal and the next instruction will be executed, thus printing the message “you have changed the ‘modified’ variable”.

-> Now all the opertions are done. But the stack is as it is and for the program to be completed the flow control has to be passed to the RET address which is stored on the stack. But till now only variables and addresses has been pushed on the stack but nothing has been popped. At the address 0x08048433 the leave instruction is executed. The leave instruction is used to “free” the stack frame. If we see the disassemble main() the first two instructions push %ebp and push %esp,%ebp, these instructions basically sets up the stack frame. Now the leave instruction does exactly the opposite of what the first two instructions did, mov %ebp,%esp and pop %ebp. So these two instructions free ups stack frame and the EIP points to the RET address which will give the flow of control to the address which was next after the called function. In this case the RET value is not pointing to anything because our program ends here.

So till now we have seen what our C code in Assembly means, it is really important to understand these things because when we debug or lets say Reverse Engineer some binary and stuff, this understanding of how closely the memory and stack works really comes in handy.

Now we will see the stack operations in GDB. For those who will be doing debugging and reversing for the first time it may feel overwhelming seeing all these instructions(believe me, I used to go nuts sometimes), so for the moment we will focus only on the part which is required for this series to understand, like where is our input being written, how the memory addresses can be overwritten and all those stuffs….

Here comes the mighty GDB !!!!

In GDB we will list the program, so we can know where we want to set a breakpoint.

We will set the breakpoint for line number 7,11,13Lets run it in GDB..-> After we run it in GDB, we can see the breakpoints which we set is now hit, basically it is interrupting the program execution and halting the program flow at the given breakpoint.

-> We step to next instruction by typing s in the prompt, we can again see the next breakpoint gets hit.

-> At this point we check both our stack registers ESP and EIP. We look these two registers in two different ways. We check the ESP using the x switch which is for examine memory. We can see the ESP is pointing to the stack address 0xbffff0b0and the value it contains is 0x00000000 i.e 0. We can easily assume that, this 0 is the same 0 which gets assigned to the modified variable.

-> We check the EIP by the command info registers eip (you can see info about all the registers by simply typing info registers). We see the EIP is pointing to the memory address 0x8048405 which is nothing but the address of next instruction to be executed.

Now we step next and see what happens.-> When we step through the next instruction it asks us for the input. Now we give the input as random sets of ‘A’ and then we can see our third breakpoint which we set earlier is hit.

-> We again check the ESP and EIP. We clearly see the ESP is changed and pointing to different stack address. EIP now is pointing to the next instruction which is going to be executed.

Now lets see, the input which we gave where does it goes?-> By using the examine memory switch i.e x we see the contents of ESP. We are viewing the 24 words of content on the stack in Hex (thats why x/24xw $esp). We can see our input ‘A’ that is 41 in hex (according to the ASCII standards) on the stack(highlighted portion). So we can see that our input is being written on the stack.

Again lets step through the next instruction.-> In the previous step we saw that the instruction if(modified !=0) is going to be executed. Now lets go back to the section where we saw the assembly instructions equivalent of the C program in detail. We can see the instruction test %eax,%eaxwill be executed. So we already know it will compare if the modified value has changed or not.

How do we see that ?

-> We simply check the EAX register by typing info registers eax. We can see that the value of EAX is 0x0 i.e 0, that means the value of modified variable is still 0. Now we know that the message “Try again?” is going to be printed. The EIP also confirms this, the output of EIP points to 0x8048427 which if you look at the disassembled main function, you can see that it is calling the second puts function which has the message “Try again?”.

Lets step and move towards the exit of the program.-> When we step through the next instruction, it gives us the message “Try again?”. Then we check the EIP it points to 0x8048433, which is nothing but the leaveinstruction. Again we step through, we can the program exiting and terminated with the exit system call that is in the libc.s0.6 shared library files.

WOAH!!!!!

Till now, we saw the how the process memory works, how the programs gets loaded into the memory and how the stack operations are done when the program gets executed. Now this was more than enough to understand what Buffer Overflow really is.

BUFFER OVERFLOW

Finally we can get started with the topic. Lets ask ourselves two very simple questions:

What is a Buffer?
-> Buffer is a temporary storage place in memory to store data.

What is Buffer Overflow?
-> When a data written to a buffer that is larger than the actual buffer size and due to improper bounds checking it gets overflowed and overwrites the adjacent memory addresses/locations.

So, its time to get our hands dirty by smashing the stack. But before that, for this blogpost series we will only focus on the Stack based overflows. Also the examples which we are going to see may not be vulnerable to buffer overflow because the newer system kernels handle all these things in a very effifcient way. If you are using a newer system, for e.g Ubuntu 16.04 LTS you have to go and disable the ASLR bit to off as it is set to protect from the Memory Corruption attacks.
To disable it simply type : echo 0 > /proc/sys/kernel/randomize_va_space in your terminal.

Also you have to compile the program using gcc with stack smashing detect feature disabled. To do this compile your program using:

We will use the earlier program which we used for understanding the stack. This program is vulnerable to buffer overflow. As we can see the the program is using gets function to accept the user input. Now, this gets function has some serious security issues. Lets see the man page for gets.As, we can see the highlighted part it says “Never use gets(). Because it is impossible to tell without knowing the data in advance how many characters gets() will read, and because gets() will continue to store characters past the end of the buffer, it is extremely dangerous to use. It has been used to break computer security. Use fgets() instead”.

From here we can understand there is actually no bound checks happening when the user input is taken through the gets function (Extremely Dangerous Right?).

Now lets look what the program is and what the challenge of the program is all about.
-> We already discussed that if the modified variable is not 0, then the message “you have changed the ‘modified’ variable” will be printed. But if we look at the program, there is no way the modified variable’s value can be changed. So how it can be done?

-> The line where it takes user input and writes into char array buffer is actually our way to go and change the value of modified. The gets(buffer); is the vulnerable code.

In the code we can see the modified variable assignment and the input of char array buffer is next to each other,

This means when this two instructions will be executed by the processor the modified variable and buffer array will be adjacent to each other in the stack frame.

So, what does this means?
-> When we feed input to the buffer more than it is capable of, the extra input which we feed will get overwritten to the adjacent memory location, in this case the memory location pointing to the modified variable. Thus, the modified variable will be no longer be 0 and the success message will be printed.

Due to buffer overflow the above scenario was possible. Lets see in more detail.

First we will try to execute the program with some random input and see where the overflow happens.-> We already know the char array buffer is of 64 bytes. So we try to enter 60, 62, 64 times ‘A’ to our program. As, we can see the modified variable is not changed and the failure message is printed.

->But when we enter 65 A’s to our program, the value of modified variable mysteriously changes and the success message is printed.

->The buffer overflow has happened after the 64th byte of input and it overwrites the memory location after that i.e where the modified variable is stored.

Lets load our program in GDB and see how the modified variable’s memory location got overwritten.-> As we can see, the breakpoints 1 and 2 got hit, then we check the value of modified by typing x/xw $esp+0x5c (stack register + offset). If we see in the disassembled main function we can see the value 0 gets assigned to modifiedvariable through this instruction: movl $0x0,0x5c(%esp), that’s why we checked the value of modified variable by giving the offset along with the stack register ESP. The value of modified variable at stack location 0xbffff10c is 0x00000000.

->After stepping through, its asking us to enter the input. Now we know 65th byte is the point where the buffer gets overflowed. So we enter 65 A’s and then check the stack frame.

-> As we can see our input A i.e 41 is all over the stack. But we are only concerned with the adjacent memory location where the modified variable is there. By quickly checking the modified variable we can see the value of the stack address pointing to the modified variable 0xbffff10c is changed from 0x00000000 to 0x00000041.

-> This means when the buffer overflow took place it overwritten the adjacent memory location 0xbffff10c to 0x00000041.

-> As we step through the next instruction we can see the success message “you have changed the ‘modified’ variable” printed on the screen.

This was all possible because there was insufficient bounds checking when the user input was being written in the char array buffer. This led to overflow and the adjacent memory location (modified variable) got overwritten.

Voila !!!

We successfully learned the fundamentals of process memory, Stack operations and Buffer Overflow in detail. Now, this was only the concept of how buffer overflow takes place. We still haven’t exploited this vulnerability to actually exploit the system. In the next blog we will see how to execute arbitrary commands through Shellcode using this Buffer Overflow vulnerablity.

Till then, go and learn as much as possible about Assembly and GDB, because we are going to use this extensively in the future blogposts.

WoW64 internals …re-discovering Heaven’s Gate on ARM

( Original text by PetrBenes )

WoW64 — aka Windows (32-bit) on Windows (64-bit) — is a subsystem that enables 32-bit Windows applications to run on 64-bit Windows. Most people today are familiar with WoW64 on Windows x64, where they can run x86 applications. WoW64 has been with us since Windows XP, and x64 wasn’t the only architecture where WoW64 has been available — it was available on IA-64architecture as well, where WoW64 has been responsible for emulating x86. Newly, WoW64 is also available on ARM64, enabling emulation of both x86 and ARM32 appllications.

MSDN offers brief article on WoW64 implementation details. We can find that WoW64 consists of (ignoring IA-64):

  • Translation support DLLs:
    • wow64.dll: translation of Nt* system calls (ntoskrnl.exe / ntdll.dll)
    • wow64win.dll: translation of NtGdi*NtUser* and other GUI-related system calls (win32k.sys / win32u.dll)
  • Emulation support DLLs:
    • wow64cpu.dll: support for running x86 programs on x64
    • wowarmhw.dll: support for running ARM32 programs on ARM64
    • xtajit.dll: support for running x86 programs on ARM64

Besides Nt* system call translation, the wow64.dll provides the core emulation infrastructure.

If you have previous experience with reversing WoW64 on x64, you can notice that it shares plenty of common code with WoW64 subsystem on ARM64. Especially if you peeked into WoW64 of recent x64 Windows, you may have noticed that it actually contains strings such as SysArm32 and that some functions check against IMAGE_FILE_MACHINE_ARMNT (0x1C4) machine type:

Wow64SelectSystem32PathInternal
Wow64SelectSystem32PathInternal found in wow64.dll on Windows x64
Wow64ArchGetSP
Wow64ArchGetSP found in wow64.dll on Windows x64

WoW on x64 systems cannot emulate ARM32 though — it just apparently shares common code. But SysX8664 and SysArm64 sound particularly interesting!

Those similarities can help anyone who is fluent in x86/x64, but not that much in ARM. Also, HexRays decompiler produce much better output for x86/x64 than for ARM32/ARM64.

Initially, my purpose with this blogpost was to get you familiar with how WoW64 works for ARM32 programs on ARM64. But because WoW64 itself changed a lot with Windows 10, and because WoW64 shares some similarities between x64 and ARM64, I decided to briefly get you through how WoW64 works in general.

Everything presented in this article is based on Windows 10 — insider preview, build 18247.

Terms

Througout this article I’ll be using some terms I’d like to explain beforehand:

  • ntdll or ntdll.dll — these will be always refering to the native ntdll.dll (x64 on Windows x64, ARM64 on Windows ARM64, …), until said otherwise or until the context wouldn’t indicate otherwise.
  • ntdll32 or ntdll32.dll — to make an easy distinction between native and WoW64 ntdll.dllany WoW64 ntdll.dll will be refered with the *32 suffix.
  • emu or emu.dll — these will represent any of the emulation support DLLs (one of wow64cpu.dll,wowarmhw.dllxtajit.dll)
  • module!FunctionName — refers to a symbol FunctionName within the module. If you’re familiar with WinDbg, you’re already familiar with this notation.
  • CHPE — “compiled-hybrid-PE”, a new type of PE file, which looks as if it was x86 PE file, but has ARM64 code within them. CHPE will be tackled in more detail in the x86 on ARM64 section.
  • The terms emulation and binary-translation refer to the WoW64 workings and they may be used interchangeably.

Kernel

This section shows some points of interest in the ntoskrnl.exe regarding to the WoW64 initialization. If you’re interested only in the user-mode part of the WoW64, you can skip this part to the Initialization of the WoW64 process.

Kernel (initialization)

Initalization of WoW64 begins with the initialization of the kernel:

  • nt!KiSystemStartup
  • nt!KiInitializeKernel
  • nt!InitBootProcessor
  • nt!PspInitPhase0
  • nt!Phase1Initialization
    • nt!IoInitSystem
      • nt!IoInitSystemPreDrivers
      • nt!PsLocateSystemDlls

nt!PsLocateSystemDlls routine takes a pointer named nt!PspSystemDlls, and then calls nt!PspLocateSystemDll in a loop. Let’s figure out what’s going on here:

PspSystemDlls (x64)
PspSystemDlls (x64)
PspSystemDlls (ARM64)
PspSystemDlls (ARM64)

nt!PspSystemDlls appears to be array of pointers to some structure, which holds some NTDLL-related data. The order of these NTDLLs corresponds with this enum (included in the PDB):

typedef enum _SYSTEM_DLL_TYPE
{
PsNativeSystemDll = 0,
PsWowX86SystemDll = 1,
PsWowArm32SystemDll = 2,
PsWowAmd64SystemDll = 3,
PsWowChpeX86SystemDll = 4,
PsVsmEnclaveRuntimeDll = 5,
PsSystemDllTotalTypes = 6,
} SYSTEM_DLL_TYPE;
view rawSYSTEM_DLL_TYPE.h hosted with ❤ by GitHub

Remember this enum, we’ll be needing it again in a while.

Now, let’s look how such structure looks like:

SystemDllData (x64)
SystemDllData (x64)
SystemDllData (ARM64)
SystemDllData (ARM64)

The nt!PspLocateSystemDll function intializes fields of this structure. The layout of this structure isn’t unfortunatelly in the PDB, but you can find a reconstructed version in the appendix.

Now let’s get back to the nt!Phase1Initialization — there’s more:

  • ...
  • nt!Phase1Initialization
    • nt!Phase1InitializationIoReady
      • nt!PspInitPhase2
      • nt!PspInitializeSystemDlls

nt!PspInitializeSystemDlls routine takes a pointer named nt!NtdllExportInformation. Let’s look at it:

NtdllExportInformation (x64)
NtdllExportInformation (x64)
NtdllExportInformation (ARM64)
NtdllExportInformation (ARM64)

It looks like it’s some sort of array, again, ordered by the enum _SYSTEM_DLL_TYPE mentioned earlier. Let’s examine NtdllExports:

NtdllExportInformation (x64)
NtdllExportInformation (x64)

Nothing unexpected — just tuples of function name and function pointer. Did you notice the difference in the number after the NtdllExports field? On x64 there is 19 meanwhile on ARM64 there is 14. This number represents number of items in NtdllExports — and indeed, there is slightly different set of them:

x64 ARM64
(0) LdrInitializeThunk (0) LdrInitializeThunk
(1) RtlUserThreadStart (1) RtlUserThreadStart
(2) KiUserExceptionDispatcher (2) KiUserExceptionDispatcher
(3) KiUserApcDispatcher (3) KiUserApcDispatcher
(4) KiUserCallbackDispatcher (4) KiUserCallbackDispatcher
(5) KiUserCallbackDispatcherReturn
(5) KiRaiseUserExceptionDispatcher (6) KiRaiseUserExceptionDispatcher
(6) RtlpExecuteUmsThread
(7) RtlpUmsThreadYield
(8) RtlpUmsExecuteYieldThreadEnd
(9) ExpInterlockedPopEntrySListEnd (7) ExpInterlockedPopEntrySListEnd
(10) ExpInterlockedPopEntrySListFault (8) ExpInterlockedPopEntrySListFault
(11) ExpInterlockedPopEntrySListResume (9) ExpInterlockedPopEntrySListResume
(12) LdrSystemDllInitBlock (10) LdrSystemDllInitBlock
(13) RtlpFreezeTimeBias (11) RtlpFreezeTimeBias
(14) KiUserInvertedFunctionTable (12) KiUserInvertedFunctionTable
(15) WerReportExceptionWorker (13) WerReportExceptionWorker
(16) RtlCallEnclaveReturn
(17) RtlEnclaveCallDispatch
(18) RtlEnclaveCallDispatchReturn

We can see that ARM64 is missing Ums (User-Mode Scheduling) and Enclave functions. Also, we can see that ARM64 has one extra function: KiUserCallbackDispatcherReturn.

On the other hand, all NtdllWow*Exports contain the same set of function names:

NtdllWowExports (ARM64)
NtdllWowExports (ARM64)

Notice names of second fields of these “structures”: PsWowX86SharedInformation,PsWowChpeX86SharedInformation, … If we look at the address of those fields, we can see that they’re part of another array:

PsWowX86SharedInformation (ARM64)
PsWowX86SharedInformation (ARM64)

Those addresses are actually targets of the pointers in the NtdllWow*Exports structure. Also, those functions combined with PsWow*SharedInformation might give you hint that they’re related to this enum (included in the PDB):

typedef enum _WOW64_SHARED_INFORMATION
{
SharedNtdll32LdrInitializeThunk = 0,
SharedNtdll32KiUserExceptionDispatcher = 1,
SharedNtdll32KiUserApcDispatcher = 2,
SharedNtdll32KiUserCallbackDispatcher = 3,
SharedNtdll32RtlUserThreadStart = 4,
SharedNtdll32pQueryProcessDebugInformationRemote = 5,
SharedNtdll32BaseAddress = 6,
SharedNtdll32LdrSystemDllInitBlock = 7,
SharedNtdll32RtlpFreezeTimeBias = 8,
Wow64SharedPageEntriesCount = 9,
} WOW64_SHARED_INFORMATION;

Notice how the order of the SharedNtdll32BaseAddress corellates with the empty field in the previous screenshot (highlighted). The set of WoW64 NTDLL functions is same on both x64 and ARM64.

(The C representation of this data can be found in the appendix.)

Now we can tell what the nt!PspInitializeSystemDlls function does — it gets image base of each NTDLL (nt!PsQuerySystemDllInfo), resolves all Ntdll*Exports for them (nt!RtlFindExportedRoutineByName). Also, only for all WoW64 NTDLLs (if ((SYSTEM_DLL_TYPE)SystemDllType > PsNativeSystemDll)) it assigns the image base to the SharedNtdll32BaseAddress field of the PsWow*SharedInformation array (nt!PspWow64GetSharedInformation).

Kernel (create process)

Let’s talk briefly about process creation. As you probably already know, the native ntdll.dll is mapped as a first DLL into each created process. This applies for all architectures — x86x64 and also for ARM64. The WoW64 processes aren’t exception to this rule — the WoW64 processes share the same initialization code path as native processes.

  • nt!NtCreateUserProcess
  • nt!PspAllocateProcess
    • nt!PspSetupUserProcessAddressSpace
      • nt!PspPrepareSystemDllInitBlock
      • nt!PspWow64SetupUserProcessAddressSpace
  • nt!PspAllocateThread
    • nt!PspWow64InitThread
    • nt!KeInitThread // Entry-point: nt!PspUserThreadStartup
  • nt!PspUserThreadStartup
  • nt!PspInitializeThunkContext
    • nt!KiDispatchException

If you ever wondered how is the first user-mode instruction of the newly created process executed, now you know the answer — a “synthetic” user-mode exception is dispatched, with ExceptionRecord.ExceptionAddress = &PspLoaderInitRoutine, where PspLoaderInitRoutinepoints to the ntdll!LdrInitializeThunk. This is the first function that is executed in every process — including WoW64 processes.

Initialization of the WoW64 process

The fun part begins!

NOTE: Initialization of the wow64.dll is same on both x64 and ARM64. Eventual differences will be mentioned.

  • ntdll!LdrInitializeThunk
  • ntdll!LdrpInitialize
  • ntdll!_LdrpInitialize
  • ntdll!LdrpInitializeProcess
  • ntdll!LdrpLoadWow64

The ntdll!LdrpLoadWow64 function is called when the ntdll!UseWOW64 global variable is TRUE, which is set when NtCurrentTeb()->WowTebOffset != NULL.

It constructs the full path to the wow64.dll, loads it, and then resolves following functions:

  • Wow64LdrpInitialize
  • Wow64PrepareForException
  • Wow64ApcRoutine
  • Wow64PrepareForDebuggerAttach
  • Wow64SuspendLocalThread

NOTE: The resolution of these pointers is wrapped between pair of ntdll!LdrProtectMrdata calls, responsible for protecting (1) and unprotecting (0) the .mrdata section — in which these pointers reside. MRDATA (Mutable Read Only Data) are part of the CFG (Control-Flow Guard) functionality. You can look at Alex’s slides for more information.

When these functions are successfully located, the ntdll.dll finally transfers control to the wow64.dll by calling wow64!Wow64LdrpInitialize. Let’s go through the sequence of calls that eventually bring us to the entry-point of the “emulated” application.

  • wow64!Wow64LdrpInitialize
    • wow64!Wow64InfoPtr = (NtCurrentPeb32() + 1)
    • NtCurrentTeb()->TlsSlots[/* 10 */ WOW64_TLS_WOW64INFO] = wow64!Wow64InfoPtr
    • ntdll!RtlWow64GetCpuAreaInfo
    • wow64!ProcessInit
    • wow64!CpuNotifyMapViewOfSection // Process image
    • wow64!Wow64DetectMachineTypeInternal
    • wow64!Wow64SelectSystem32PathInternal
    • wow64!CpuNotifyMapViewOfSection // 32-bit NTDLL image
    • wow64!ThreadInit
    • wow64!ThunkStartupContext64TO32
    • wow64!Wow64SetupInitialCall
    • wow64!RunCpuSimulation
      • emu!BTCpuSimulate

Wow64InfoPtr is the first initialized variable in the wow64.dll. It contains data shared between 32-bit and 64-bit execution mode and its structure is not documented, although you can find this structure partialy restored in the appendix.

RtlWow64GetCpuAreaInfo is an internal ntdll.dll function which is called a lot during emulation. It is mainly used for fetching the machine type and architecture-specific CPU context (the CONTEXTstructure) of the emulated process. This information is fetched into an undocumented structure, which we’ll be calling WOW64_CPU_AREA_INFO. Pointer to this structure is then given to the ProcessInit function.

Wow64DetectMachineTypeInternal determines the machine type of the executed process and returns it. Wow64SelectSystem32PathInternal selects the “emulated” System32 directory based on that machine type, e.g. SysWOW64 for x86 processes or SysArm32 for ARM32 processes.

You can also notice calls to CpuNotifyMapViewOfSection function. As the name suggests, it is also called on each “emulated” call of NtMapViewOfSection. This function:

  • Checks if the mapped image is executable
  • Checks if following conditions are true:
    • NtHeaders->OptionalHeader.MajorSubsystemVersion == USER_SHARED_DATA.NtMajorVersion
    • NtHeaders->OptionalHeader.MinorSubsystemVersion == USER_SHARED_DATA.NtMinorVersion

If these checks pass, CpupResolveReverseImports function is called. This function checks if the mapped image exports the Wow64Transition symbol and if so, it assigns there a 32-bit pointer value returned by emu!BTCpuGetBopCode.

The Wow64Transition is mostly known to be exported by SysWOW64\ntdll.dll, but there are actually multiple of Windows’ WoW DLLs which exports this symbol (will be mentioned later). You might be already familiar with the term “Heaven’s Gate” — this is where the Wow64Transition will point to on Windows x64 — a simple far jump instruction which switches into long-mode (64-bit) enabled code segment. On ARM64, the Wow64Transition points to a “nop” function.

NOTE: Because there are no checks on the ImageName, the Wow64Transition symbol is resolved for all executable images that passes the checks mentioned earlier. If you’re wondering whether Wow64Transition would be resolved for your custom executable or DLL — it indeed would!

The initialization then continues with thread-specific initialization by calling ThreadInit. This is followed by pair of calls ThunkStartupContext64TO32(CpuArea.MachineType, CpuArea.Context, NativeContext) and Wow64SetupInitialCall(&CpuArea) — these functions perform the necessary setup of the architecture-specific WoW64 CONTEXT structure to prepare start of the execution in the emulated environment. This is done in the exact same way as if ntoskrnl.exe would actually executed the emulated application — i.e.:

  • setting the instruction pointer to the address of ntdll32!LdrInitializeThunk
  • setting the stack pointer below the WoW64 CONTEXT structure
  • setting the 1st parameter to point to that CONTEXT structure
  • setting the 2nd parameter to point to the base address of the ntdll32

Finally, the RunCpuSimulation function is called. This function just calls BTCpuSimulate from the binary-translator DLL, which contains the actual emulation loop that never returns.

wow64!ProcessInit

  • wow64!Wow64ProtectMrdata // 0
  • wow64!Wow64pLoadLogDll
    • ntdll!LdrLoadDll // "%SystemRoot%\system32\wow64log.dll"

wow64.dll has also it’s own .mrdata section and ProcessInit begins with unprotecting it. It then tries to load the wow64log.dll from the constructed system directory. Note that this DLL is never present in any released Windows installation (it’s probably used internally by Microsoft for debugging of the WoW64 subsystem). Therefore, load of this DLL will normally fail. This isn’t problem, though, because no critical functionality of the WoW64 subsystem depends on it. If the load would actually succeed, the wow64.dll would try to find following exported functions there:

  • Wow64LogInitialize
  • Wow64LogSystemService
  • Wow64LogMessageArgList
  • Wow64LogTerminate

If any of these functions wouldn’t be exported, the DLL would be immediately unloaded.

If we’d drop custom wow64log.dll (which would export functions mentioned above) into the %SystemRoot%\System32 directory, it would actually get loaded into every WoW64 process. This way we could drop a custom logging DLL, or even inject every WoW64 process with native DLL!

For more details, you can check my injdrv project which implements injection of native DLLs into WoW64 processes, or check this post by Walied Assar.

Then, certain important values are fetched from the LdrSystemDllInitBlock array. These contains base address of the ntdll32.dll, pointer to functions like ntdll32!KiUserExceptionDispatcherntdll32!KiUserApcDispatcher, …, control flow guard information and others.

Finally, the Wow64pInitializeFilePathRedirection is called, which — as the name suggests — initializes WoW64 path redirection. The path redirection is completely implemented in the wow64.dll and the mechanism is basically based on string replacement. The path redirection can be disabled and enabled by calling kernel32!Wow64DisableWow64FsRedirection & kernel32!Wow64RevertWow64FsRedirection function pairs. Both of these functions internally call ntdll32!RtlWow64EnableFsRedirectionEx, which directly operates on NtCurrentTeb()->TlsSlots[/* 8 */ WOW64_TLS_FILESYSREDIR] field.

wow64!ServiceTables

Next, a ServiceTables array is initialized. You might be already familiar with the KSERVICE_TABLE_DESCRIPTOR from the ntoskrnl.exe, which contains — among other things — a pointer to an array of system functions callable from the user-mode. ntoskrnl.exe contains 2 of these tables: one for ntoskrnl.exe itself and one for the win32k.sys, aka the Windows (GUI) subsystem. wow64.dll has 4 of them!

The WOW64_SERVICE_TABLE_DESCRIPTOR has the exact same structure as the KSERVICE_TABLE_DESCRIPTOR, except that it is extended:

typedef struct _WOW64_ERROR_CASE {
ULONG Case;
NTSTATUS TransformedStatus;
} WOW64_ERROR_CASE, *PWOW64_ERROR_CASE;
typedef struct _WOW64_SERVICE_TABLE_DESCRIPTOR {
KSERVICE_TABLE_DESCRIPTOR Descriptor;
WOW64_ERROR_CASE ErrorCaseDefault;
PWOW64_ERROR_CASE ErrorCase;
} WOW64_SERVICE_TABLE_DESCRIPTOR, *PWOW64_SERVICE_TABLE_DESCRIPTOR;

(More detailed definition of this structure is in the appendix.)

ServiceTables array is populated as follows:

  • ServiceTables[/* 0 */ WOW64_NTDLL_SERVICE_INDEX] = sdwhnt32
  • ServiceTables[/* 1 */ WOW64_WIN32U_SERVICE_INDEX] = wow64win!sdwhwin32
  • ServiceTables[/* 2 */ WOW64_KERNEL32_SERVICE_INDEX = wow64win!sdwhcon
  • ServiceTables[/* 3 */ WOW64_USER32_SERVICE_INDEX] = sdwhbase

NOTE: wow64.dll directly depends (by import table) on two DLLs: the native ntdll.dll and wow64win.dll. This means that wow64win.dll is loaded even into “non-Windows-subsystem” processes, that wouldn’t normally load user32.dll.

These two symbols mentioned above are the only symbols that wow64.dll requires wow64win.dll to export.

Let’s have a look at sdwhnt32 service table:

sdwhnt32 (x64)
sdwhnt32 (x64)
sdwhnt32JumpTable (x64)
sdwhnt32JumpTable (x64)
sdwhnt32Number (x64)
sdwhnt32Number (x64)

There is nothing surprising for those who already dealt with service tables in ntoskrnl.exe.sdwhnt32JumpTable contains array of the system call functions, which are traditionaly prefixed. WoW64 “system calls” are prefixed with wh*, which honestly I don’t have any idea what it stands for — although it might be the case as with Zw* prefix — it stands for nothing and is simply used as an unique distinguisher.

The job of these wh* functions is to correctly convert any arguments and return values from the 32-bit version to the native, 64-bit version. Keep in mind that that it not only includes conversion of integers and pointers, but also content of the structures. Interesting note might be that each of the wh* functions has only one argument, which is pointer to an array of 32-bit values. This array contains the parameters passed to the 32-bit system call.

As you could notice, in those 4 service tables there are “system calls” that are not present in the ntoskrnl.exe. Also, I mentioned earlier that the Wow64Transition is resolved in multiple DLLs. Currently, these DLLs export this symbol:

  • ntdll.dll
  • win32u.dll
  • kernel32.dll and kernelbase.dll
  • user32.dll

The ntdll.dll and win32u.dll are obvious and they represent the same thing as their native counterparts. The service tables used by kernel32.dll and user32.dll contain functions for transformation of particular csrss.exe calls into their 64-bit version.

It’s also worth noting that at the end of the ntdll.dll system table, there are several functions with NtWow64* calls, such as NtWow64ReadVirtualMemory64NtWow64WriteVirtualMemory64 and others. These are special functions which are provided only to WoW64 processes.

One of these special functions is also NtWow64CallFunction64. It has it’s own small dispatch table and callers can select which function should be called based on its index:

Wow64FunctionDispatch64 (x64)
Wow64FunctionDispatch64 (x64)

NOTE: I’ll be talking about one of these functions — namely Wow64CallFunctionTurboThunkControl — later in the Disabling Turbo thunks section.

wow64!Wow64SystemServiceEx

This function is similar to the kernel’s nt!KiSystemCall64 — it does the dispatching of the system call. This function is exported by the wow64.dll and imported by the emulation DLLs. Wow64SystemServiceEx accepts 2 arguments:

  • The system call number
  • Pointer to an array of 32-bit arguments passed to the system call (as mentioned earlier)

The system call number isn’t just an index, but also contains index of a system table which needs to be selected (this is also true for ntoskrnl.exe):

typedef struct _WOW64_SYSTEM_SERVICE
{
USHORT SystemCallNumber : 12;
USHORT ServiceTableIndex : 4;
} WOW64_SYSTEM_SERVICE, *PWOW64_SYSTEM_SERVICE;

This function then selects ServiceTables[ServiceTableIndex] and calls the appropriate wh*function based on the SystemCallNumber.

Wow64SystemServiceEx (x64)
Wow64SystemServiceEx (x64)

NOTE: In case the wow64log.dll has been successfully loaded, the Wow64SystemServiceEx function calls Wow64LogSystemServiceWrapper (wrapper around wow64log!Wow64LogSystemService function): once before the actual system call and one immediately after. This can be used for instrumentation of each WoW64 system call! The structure passed to Wow64LogSystemService contains every important information about the system call — it’s table index, system call number, the argument list and on the second call, even the resulting NTSTATUS! You can find layout of this structure in the appendix(WOW64_LOG_SERVICE).

Finally, as have been mentioned, the WOW64_SERVICE_TABLE_DESCRIPTOR structure differs from KSERVICE_TABLE_DESCRIPTOR in that it contains ErrorCase table. The code mentioned above is actually wrapped in a SEH __try/__except block. If whService raise an exception, the __exceptblock calls Wow64HandleSystemServiceError function. The function looks if the corresponding service table which raised the exception has non-NULL ErrorCase and if it does, it selects the appropriate WOW64_ERROR_CASE for the system call. If the ErrorCase is NULL, the values from ErrorCaseDefault are used. The NTSTATUS of the exception is then transformed according to an algorithm which can be found in the appendix.

wow64!ProcessInit (cont.)

  • ...
  • wow64!CpuLoadBinaryTranslator // MachineType
    • wow64!CpuGetBinaryTranslatorPath // MachineType
      • ntdll!NtOpenKey // "\Registry\Machine\Software\Microsoft\Wow64\"
      • ntdll!NtQueryValueKey // "arm" / "x86"
      • ntdll!RtlGetNtSystemRoot // "arm" / "x86"
      • ntdll!RtlUnicodeStringPrintf // "%ws\system32\%ws"

As you’ve probably guessed, this function constructs path to the binary-translator DLL, which is — on x64 — known as wow64cpu.dll. This DLL will be responsible for the actual low-level emulation.

\Registry\Machine\Software\Microsoft\Wow64\x86 (x64)
\Registry\Machine\Software\Microsoft\Wow64\x86 (x64)
\Registry\Machine\Software\Microsoft\Wow64\arm (ARM64)
\Registry\Machine\Software\Microsoft\Wow64\arm (ARM64)
\Registry\Machine\Software\Microsoft\Wow64\x86 (ARM64)
\Registry\Machine\Software\Microsoft\Wow64\x86 (ARM64)

We can see that there is no wow64cpu.dll on ARM64. Instead, there is xtajit.dll used for x86 emulation and wowarmhw.dll used for ARM32 emulation.

NOTE: The CpuGetBinaryTranslatorPath function is same on both x64 and ARM64 except for one peculiar difference: on Windows x64, if the \Registry\Machine\Software\Microsoft\Wow64\x86 key cannot be opened (is missing/was deleted), the function contains a fallback to load wow64cpu.dll. On Windows ARM64, though, it doesn’t have such fallback and if the registry key is missing, the function fails and the WoW64 process is terminated.

wow64.dll then loads one of the selected DLL and tries to find there following exported functions:

BTCpuProcessInit (!) BTCpuProcessTerm
BTCpuThreadInit BTCpuThreadTerm
BTCpuSimulate (!) BTCpuResetFloatingPoint
BTCpuResetToConsistentState BTCpuNotifyDllLoad
BTCpuNotifyDllUnload BTCpuPrepareForDebuggerAttach
BTCpuNotifyBeforeFork BTCpuNotifyAfterFork
BTCpuNotifyAffinityChange BTCpuSuspendLocalThread
BTCpuIsProcessorFeaturePresent BTCpuGetBopCode (!)
BTCpuGetContext BTCpuSetContext
BTCpuTurboThunkControl BTCpuNotifyMemoryAlloc
BTCpuNotifyMemoryFree BTCpuNotifyMemoryProtect
BTCpuFlushInstructionCache2 BTCpuNotifyMapViewOfSection
BTCpuNotifyUnmapViewOfSection BTCpuUpdateProcessorInformation
BTCpuNotifyReadFile BTCpuCfgDispatchControl
BTCpuUseChpeFile BTCpuOptimizeChpeImportThunks
BTCpuNotifyProcessExecuteFlagsChange BTCpuProcessDebugEvent
BTCpuFlushInstructionCacheHeavy

Interestingly, not all functions need to be found — only those marked with the “(!)”, the rest is optional. As a next step, the resolved BTCpuProcessInit function is called, which performs binary-translator-specific process initialization. We’ll cover that in later section.

At the end of the ProcessInit function, wow64!Wow64ProtectMrdata(1) is called, making .mrdatanon-writable again.

wow64!ThreadInit

  • wow64!ThreadInit
    • wow64!CpuThreadInit
      • NtCurrentTeb32()->WOW32Reserved = BTCpuGetBopCode()
      • emu!BTCpuThreadInit

ThreadInit does some little thread-specific initialization, such as:

  • Copying CurrentLocale and IdealProcessor values from 64-bit TEB into 32-bit TEB.
  • For non-WOW64_CPUFLAGS_SOFTWARE emulators, it calls CpuThreadInit, which:
    • Performs NtCurrentTeb32()->WOW32Reserved = BTCpuGetBopCode().
    • Calls emu!BTCpuThreadInit().
  • For WOW64_CPUFLAGS_SOFTWARE emulators, it creates an event, which added intoAlertByThreadIdEventHashTable and set to NtCurrentTeb()->TlsSlots[18]. This event is used for special emulation of NtAlertThreadByThreadId and NtWaitForAlertByThreadId.

NOTE: The WOW64_CPUFLAGS_MSFT64 (1) or the WOW64_CPUFLAGS_SOFTWARE (2) flag is stored in the NtCurrentTeb()->TlsSlots[/* 10 */ WOW64_TLS_WOW64INFO], in the WOW64INFO.CpuFlags field. One of these flags is always set in the emulator’s BTCpuProcessInit function (mentioned in the section above):

  • wow64cpu.dll sets WOW64_CPUFLAGS_MSFT64 (1)
  • wowarmhw.dll sets WOW64_CPUFLAGS_MSFT64 (1)
  • xtajit.dll sets WOW64_CPUFLAGS_SOFTWARE (2)

x86 on x64

Entering 32-bit mode

  • ...
  • wow64!RunCpuSimulation
    • wow64cpu!BTCpuSimulate
      • wow64cpu!RunSimulatedCode

RunSimulatedCode runs in a loop and performs transitions into 32-bit mode either via:

  • jmp fword ptr[reg] — a “far jump” that not only changes instruction pointer (RIP), but also the code segment register (CS). This segment usually being set to 0x23, while 64-bit code segment is 0x33
  • synthetic “machine frame” and iret — called on every “state reset”

NOTE: Explanation of segmentation and “why does it work just by changing a segment register” is beyond scope of this article. If you’d like to know more about “long mode” and segmentation, you can start here.

Far jump is used most of the time for the transition, mainly because it’s faster. iret on the other hand is more powerful, as it can change CSSSEFLAGSRSP and RIP all at once. The “state reset” occurs when WOW64_CPURESERVED.Flags has WOW64_CPURESERVED_FLAG_RESET_STATE (1) bit set. This happens during exception (see wow64!Wow64PrepareForException and wow64cpu!BTCpuResetToConsistentState). Also, this flag is cleared on every emulation loop (using btr — bit-test-and-reset).

Start of the RunSimulatedCode (x64)
Start of the RunSimulatedCode (x64)

You can see the simplest form of switching into the 32-bit mode. Also, at the beginning you can see that TurboThunkDispatch address is moved into the r15 register. This register stays untouched during the whole RunSimulatedCode function. Turbo thunks will be explained in more detail later.

Leaving 32-bit mode

The switch back to the 64-bit mode is very similar — it also uses far jumps. The usual situation when code wants to switch back to the 64-bit mode is upon system call:

NtMapViewOfSection (x64)
NtMapViewOfSection (x64)

The Wow64SystemServiceCall is just a simple jump to the Wow64Transition:

Wow64SystemServiceCall (x64)
Wow64SystemServiceCall (x64)

If you remember, the Wow64Transition value is resolved by the wow64cpu!BTCpuGetBopCodefunction:

BTCpuGetBopCode - wow64cpu.dll (x64)
BTCpuGetBopCode — wow64cpu.dll (x64)

It selects either KiFastSystemCall or KiFastSystemCall2 based on the CpupSystemCallFast value.

The KiFastSystemCall looks like this (used when CpupSystemCallFast != 0):

  • [x86] jmp 33h:$+9 (jumps to the instruction below)
  • [x64] jmp qword ptr [r15+offset] (which points to CpupReturnFromSimulatedCode)

The KiFastSystemCall2 looks like this (used when CpupSystemCallFast == 0):

  • [x86] push 0x33
  • [x86] push eax
  • [x86] call $+5
  • [x86] pop eax
  • [x86] add eax, 12
  • [x86] xchg eax, dword ptr [esp]
  • [x86] jmp fword ptr [esp] (jumps to the instruction below)
  • [x64] add rsp, 8
  • [x64] jmp wow64cpu!CpupReturnFromSimulatedCode

Clearly, the KiFastSystemCall is faster, so why it’s not used used every time?

It turns out, CpupSystemCallFast is set to 1 in the wow64cpu!BTCpuProcessInit function if the process is not executed with the ProhibitDynamicCode mitigation policy and if NtProtectVirtualMemory(&KiFastSystemCall, PAGE_READ_EXECUTE) succeeds.

This is because KiFastSystemCall is in a non-executable read-only section (W64SVC) whileKiFastSystemCall2 is in read-executable section (WOW64SVC).

But the actual reason why is KiFastSystemCall in non-executable section by default and needs to be set as executable manually is, honestly, unknown to me. My guess would be that it has something to do with relocations, because the address in the jmp 33h:$+9 instruction must be somehow resolved by the loader. But maybe I’m wrong. Let me know if you know the answer!

Turbo thunks

I hope you didn’t forget about the TurboThunkDispatch address hanging in the r15 register. This value is used as a jump-table:

TurboThunkDispatch (x64)
TurboThunkDispatch (x64)

There are 32 items in the jump-table.

TurboDispatchJumpAddressStart (x64)
TurboDispatchJumpAddressStart (x64)

CpupReturnFromSimulatedCode is the first code that is always executed in the 64-bit mode when 32-bit to 64-bit transition occurs. Let’s recapitulate the code:

  • Stack is swapped,
  • Non-volatile registers are saved
  • eax — which contains the encoded service table index and system call number — is moved into the ecx
  • it’s high-word is acquired via ecx >> 16.
  • the result is used as an index into the TurboThunkDispatch jump-table

You might be confused now, because few sections above we’ve defined the service number like this:

typedef struct _WOW64_SYSTEM_SERVICE
{
USHORT SystemCallNumber : 12;
USHORT ServiceTableIndex : 4;
} WOW64_SYSTEM_SERVICE, *PWOW64_SYSTEM_SERVICE;

…therefore, after right-shifting this value by 16 bits we should get always 0, right?

It turns out, on x64, the WOW64_SYSTEM_SERVICE might be defined like this:

typedef struct _WOW64_SYSTEM_SERVICE
{
ULONG SystemCallNumber : 12;
ULONG ServiceTableIndex : 4;
ULONG TurboThunkNumber : 5; // Can hold values 0 — 31
ULONG AlwaysZero : 11;
} WOW64_SYSTEM_SERVICE, *PWOW64_SYSTEM_SERVICE;

Let’s examine few WoW64 system calls:

NtMapViewOfSection (x64)
NtMapViewOfSection (x64)
NtWaitForSingleObject (x64)
NtWaitForSingleObject (x64)
NtDeviceIoControlFile (x64)
NtDeviceIoControlFile (x64)

Based on our new definition of WOW64_SYSTEM_SERVICE, we can conclude that:

  • NtMapViewOfSection uses turbo thunk with index 0 (TurboDispatchJumpAddressEnd)
  • NtWaitForSingleObject uses turbo thunk with index 13 (Thunk3ArgSpNSpNSpReloadState)
  • NtDeviceIoControlFile uses turbo thunk with index 27 (DeviceIoctlFile)

Let’s finally explain “turbo thunks” in proper way.

Turbo thunks are an optimalization of WoW64 subsystem — specifically on Windows x64 — that enables for particular system calls to never leave the wow64cpu.dll — the conversion of parameters and return value, and the syscall instruction itself is fully performed there. The set of functions that use these turbo thunks reveals, that they are usually very simple in terms of parameter conversion — they receive numerical values or handles.

The notation of Thunk* labels is as follows:

  • The number specifies how many arguments the function receives
  • Sp converts parameter with sign-extension
  • NSp converts parameter without sign-extension
  • ReloadState will return to the 32-bit mode using iret instead of far jump, if WOW64_CPURESERVED_FLAG_RESET_STATE is set
  • QuerySystemTimeReadWriteFileDeviceIoctlFile, … are special cases

Let’s take the NtWaitForSingleObject and its turbo thunk Thunk3ArgSpNSpNSpReloadState as an example:

  • it receives 3 parameters
  • 1st parameter is sign-extended
  • 2nd parameter isn’t sign-extended
  • 3rd parameter isn’t sign-extended
  • it can switch to 32-bit mode using iret if WOW64_CPURESERVED_FLAG_RESET_STATE is set

When we cross-check this information with its function prototype, it makes sense:

NTSTATUS
NTAPI
NtWaitForSingleObject(
_In_ HANDLE Handle,
_In_ BOOLEAN Alertable,
_In_ PLARGE_INTEGER Timeout
);

The sign-extension of HANDLE makes sense, because if we pass there an INVALID_HANDLE_VALUE, which happens to be 0xFFFFFFFF (-1) on 32-bits, we don’t want to convert this value to 0x00000000FFFFFFFF, but 0xFFFFFFFFFFFFFFFF.

On the other hand, if the TurboThunkNumber is 0, the call will end up in theTurboDispatchJumpAddressEnd which in turn calls wow64!Wow64SystemServiceEx. You can consider this case as the “slow path”.

Disabling Turbo thunks

On Windows x64, the Turbo thunk optimization can be actually disabled!

In one of the previous sections I’ve been talking about ntdll32!NtWow64CallFunction64 andwow64!Wow64CallFunctionTurboThunkControl functions. As with any other NtWow64* function, NtWow64CallFunction64 is only available in the WoW64 ntdll.dll. This function can be called with an index to WoW64 function in the Wow64FunctionDispatch64 table (you could see earlier).

The function prototype might look like this:

typedef enum _WOW64_FUNCTION {
Wow64Function64Nop,
Wow64FunctionQueryProcessDebugInfo,
Wow64FunctionTurboThunkControl,
Wow64FunctionCfgDispatchControl,
Wow64FunctionOptimizeChpeImportThunks,
} WOW64_FUNCTION;
NTSYSCALLAPI
NTSTATUS
NTAPI
NtWow64CallFunction64(
_In_ WOW64_FUNCTION Wow64Function,
_In_ ULONG Flags,
_In_ ULONG InputBufferLength,
_In_reads_bytes_opt_(InputBufferLength) PVOID InputBuffer,
_In_ ULONG OutputBufferLength,
_Out_writes_bytes_opt_(OutputBufferLength) PVOID OutputBuffer,
_Out_opt_ PULONG ReturnLength
);

NOTE: This function prototype has been reconstructed with the help of thewow64!Wow64CallFunction64Nop function code, which just logs the parameters.

We can see that wow64!Wow64CallFunctionTurboThunkControl can be called with an index of 2. This function performs some sanity checks and then passes callswow64cpu!BTCpuTurboThunkControl(*(ULONG*)InputBuffer).

wow64cpu!BTCpuTurboThunkControl then checks the input parameter.

  • If it’s 0, it patches every target of the jump table to point to TurboDispatchJumpAddressEnd(remember, this is the target that is called when WOW64_SYSTEM_SERVICE.TurboThunkNumber is 0).
  • If it’s non-0, it returns STATUS_NOT_SUPPORTED.

This means 2 things:

  • Calling wow64cpu!BTCpuTurboThunkControl(0) disables the Turbo thunks, and every system call ends up taking the “slow path”.
  • It is not possible to enable them back.

With all this in mind, we can achieve disabling Turbo thunks by this call:

#define WOW64_TURBO_THUNK_DISABLE 0
#define WOW64_TURBO_THUNK_ENABLE 1 // STATUS_NOT_SUPPORTED 🙁
ThunkInput = WOW64_TURBO_THUNK_DISABLE;
Status = NtWow64CallFunction64(Wow64FunctionTurboThunkControl,
0,
sizeof(ThunkInput),
&ThunkInput,
0,
NULL,
NULL);

What it might be good for? I can think of 3 possible use-cases:

  • If we deploy custom wow64log.dll (explained earlier), disabling Turbo thunks guarantees that we will see every WoW64 system call in our wow64log!Wow64LogSystemService callback. We wouldn’t see such calls if the Turbo thunks were enabled, because they would take the “fast path” inside of the wow64cpu.dll where the syscall would be executed.
  • If we decide to hook Nt* functions in the native ntdll.dll, disabling Turbo thunks guarantees that for each Nt* function called in the ntdll32.dll, the correspondint Nt* function will be called in the native ntdll.dll. (This is basically the same point as the previous one.)

    NOTE: Keep in mind that this only applies on system calls, i.e. on Nt* or Zw* functions. Other functions are not called from the 32-bit ntdll.dll to the 64-bit ntdll.dll. For example, if we hooked RtlDecompressBuffer in the native ntdll.dll of the WoW64 process, it wouldn’t be called on ntdll32!RtlDecompressBuffer call. This is because the full implementaion of the Rtl*functions is already in the ntdll32.dll.

  • We can “harmlessly” patch high-word moved to the eax in every WoW64 system call stub to 0. For example we could see in NtWaitForSingleObject there is mov eax, 0D0004h. If we patched appropriate 2 bytes in that instruction so that the instruction would become mov eax, 4h, the system call would still work.This approach can be used as an anti-hooking technique — if there’s a jump at the start of the function, the patch will break it. If there’s not a jump, we just disable the Turbo thunk for this function.

x86 on ARM64

Emulation of x86 applications on ARM64 is handled by an actual binary translation. Instead of wow64cpu.dll, the xtajit.dll (probably shortcut for “x86 to ARM64 JIT”) is used for its emulation. As with other emulation DLLs, this DLL is native (ARM64).

The x86 emulation on Windows ARM64 consists also of other “XTA” components:

  • xtac.exe — XTA Compiler
  • XtaCache.exe — XTA Cache Service

Execution of x86 programs on ARM64 appears to go way behind just emulation. It is also capable of caching already binary-translated code, so that next execution of the same application should be faster. This cache is located in the Windows\XtaCache directory which contains files in format FILENAME.EXT.HASH1.HASH2.mp.N.jc. These files are then mapped to the user-mode address space of the application. If you’re asking whether you can find an actual ARM64 code in these files — indeed, you can.

The whole “XTA” and its internals are not in the focus of this article, but they would definitely deserve a separate article.

Unfortunatelly, Microsoft doesn’t provide symbols to any of these xta* DLLs or executables. But if you’re feeling adventurous, you can find some interesting artifacts, like this array of structures inside of the xtajit.dll, which contains name of the function and its pointer. There are thousands of items in this array:

BT functions (before) (ARM64)
BT functions (before) (ARM64)

With a simple Python script, we can mass-rename all functions referenced in this array:

begin = 0x01800A8C20
end = 0x01800B7B4F
struct_size = 24
ea = begin
while ea < end:
ea += struct_size
name = idc.GetString(idc.Qword(ea))
idc.MakeName(idc.Qword(ea+8), name)
view raw2_IDA_BT_rename.py hosted with ❤ by GitHub

I’d like to thank Milan Boháček for providing me this script.

BT functions (after) (ARM64)
BT functions (after) (ARM64)
BT translated function list (ARM64)
BT translated function list (ARM64)

Windows\SyCHPE32 & Windows\SysWOW64

One thing you can observe on ARM64 is that it contains two folders used for x86 emulation. The difference between them is that SyCHPE32 contains small subset of DLLs that are frequently used by applications, while contents of the SysWOW64 folder is quite identical with the content of this folder on Windows x64.

The CHPE DLLs are not pure-x86 DLLs and not even pure-ARM64 DLLs. They are “compiled-hybrid-PE”s. What does it mean? Let’s see:

NtMapViewOfSection (CHPE) (ARM64)
NtMapViewOfSection (CHPE) (ARM64)

After opening SyCHPE32\ntdll.dll, IDA will first tell us — unsurprisingly — that it cannot download PDB for this DLL. After looking at randomly chosen Nt* function, we can see that it doesn’t differ from what we would see in the SysWOW64\ntdll.dll. Let’s look at some non-Nt* function:

RtlDecompressBuffer (CHPE) (ARM64)
RtlDecompressBuffer (CHPE) (ARM64)

We can see it contains regular x86 function prologue, immediately followed by x86 function epilogue and then jump somewhere, where it looks like that there’s just garbage.

My guess is that the reason for this prologue is probably compatibility with applications that check whether some particular functions are hooked or not — by checking if the first bytes of the function contain real prologue.

NOTE: Again, if you’re feeling adventurous, you can patch FileHeader.Machine field in the PE header to IMAGE_FILE_MACHINE_ARM64 (0xAA64) and open this file in IDA. You will see a whole lot of correctly resolved ARM64 functions. Again, I’d like to thank to Milan Boháček for this tip.

If your question is “how are these images generated?”, I would answer that I don’t know, but my bet would be on some internal version of Microsoft’s C++ compiler toolchain. This idea appears to be supported by various occurences of the CHPE keyword in the ChakraCore codebase.

ARM32 on ARM64

The loop inside of the wowarmhw!BTCpuSimulate is fairly simple compared to wow64cpu.dll loop:

DECLSPEC_NORETURN
VOID
BTCpuSimulate(
VOID
)
{
NTSTATUS Status;
PCONTEXT Context;
//
// Gets WoW64 CONTEXT structure (ARM32) using
// the RtlWow64GetCurrentCpuArea() function.
//
Status = CpupGetArmContext(&Context, NULL);
if (!NT_SUCCESS(Status))
{
RtlRaiseStatus(Status);
//
// UNREACHABLE
//
return;
}
for (;;)
{
//
// Switch to ARM32 mode and run the emulation.
//
NtCurrentTeb()->TlsSlots[/* 2 */ WOW64_TLS_INCPUSIMULATION] = TRUE;
CpupSwitchTo32Bit(Context);
NtCurrentTeb()->TlsSlots[/* 2 */ WOW64_TLS_INCPUSIMULATION] = FALSE;
//
// When we get here, it means ARM32 code performed a system call.
// Advance instruction pointer to skip the «UND 0F8h» instruction.
//
Context->Pc += 2;
//
// Set LSB (least significat bit) if ARM32 is executing in
// Thumb mode.
//
if (Context->Cpsr & 0x20) {
Context->Pc |= 1;
}
//
// Let wow64.dll emulate the system call. R12 has the system call
// number, Sp points to the stack which contains the system call
// arguments.
//
Context->R0 = Wow64SystemServiceEx(Context->R12, Context->Sp);
}
}

CpupSwitchTo32Bit does nothing else than saving the whole CONTEXT, performing SVC 0xFFFFinstruction and then restoring the CONTEXT.

nt!KiEnter32BitMode / SVC 0xFFFF

I won’t be explaining here how system call dispatching works in the ntoskrnl.exe — Bruce Dang already did an excellent job doing it. This section is a follow up on his article, though.

SVC instruction is sort-of equivalent of SYSCALL instruction on ARM64 — it basically enters the kernel mode. But there is a small difference between SYSCALL and SVC: while on Windows x64 the system call number is moved into the eax register, on ARM64 the system call number can be encoded directly into the SVC instruction.

SVC 0xFFFF (ARM64)
SVC 0xFFFF (ARM64)

Let’s peek for a moment into the kernel to see how is this SVC instruction handled:

  • nt!KiUserExceptionHandler
    • nt!KiEnter32BitMode
KiUserExceptionHandler (ARM64)
KiUserExceptionHandler (ARM64)
KiEnter32BitMode (ARM64)
KiEnter32BitMode (ARM64)

We can see that:

  • MRS X30, ELR_EL1 — current interrupt-return address (stored in ELR_EL1 system register) will be moved to the register X30 (link register — LR).
  • MSR ELR_EL1, X15 — the interrupt-return address will be replaced by value in the register X15(which is aliased to the instruction pointer register — PC — in the 32-bit mode).
  • ORR X16, X16, #0b10000 — bit [4] is being set in X16 which is later moved to the SPSR_EL1register. Setting this bit switches the execution mode to 32-bits.

Simply said, in the X15 register, there is an address that will be executed once we leave the kernel-mode and enter the user-mode — which happens with the ERET instruction at the end.

nt!KiExit32BitMode / UND #0xF8

Alright, we’re in the 32-bit ARM mode now, how exactly do we leave? Windows solves this transition via UND instruction — which is similar to the UD2 instruction on the Intel CPUs. If you’re not familiar with it, you just need to know that it is instruction that basically guarantees that it’ll throw “invalid instruction” exception which can OS kernel handle. It is defined-“undefined instruction”. Again there is the same difference between the UND and UD2 instruction in that the ARM can have any 1-byte immediate value encoded directly in the instruction.

Let’s look at the NtMapViewOfSection system call in the SysArm32\ntdll.dll:

NtMapViewOfSection (ARM64)
NtMapViewOfSection (ARM64)

Let’s peek into the kernel again:

  • nt!KiUser32ExceptionHandler
    • nt!KiFetchOpcodeAndEmulate
      • nt!KiExit32BitMode
KiEnter32BitMode (ARM64)
KiEnter32BitMode (ARM64)
KiEnter32BitMode (ARM64)
KiEnter32BitMode (ARM64)

Keep in mind that meanwhile the 32-bit code is running, it cannot modify the value of the previously stored X30 register — it is not visible in 32-bit mode. It stays there the whole time. Upon UND #0xF8 execution, following happens:

  • the KiFetchOpcodeAndEmulate function moves value of X30 into X24 register (not shown on the screenshot).
  • AND X19, X16, #0xFFFFFFFFFFFFFFC0 — bit [4] (among others) is being cleared in the X19register, which is later moved to the SPSR_EL1 register. Clearing this bit switches the execution mode back to 64-bits.
  • KiExit32BitMode then moves the value of X24 register into the ELR_EL1 register. That means when this function finishes its execution, the ERET brings us back to the 64bit code, right after the SVC 0xFFFF instruction.

NOTE: It can be noticed that Windows uses UND instruction for several purposes. Common example might also be UND #0xFE which is used as a breakpoint instruction (equivalent of __debugbreak() / int3)

As you could spot, 3 kernel transitions are required for emulation of the system call (SVC 0xFFFF, system call itself, UND 0xF8). This is because on ARM there doesn’t exist a way how to switch between 32-bit and 64-bit mode only in user-mode.

If you’re looking for “ARM Heaven’s Gate” — this is it. Put whatever function address you like into the X15 register and execute SVC 0xFFFF. Next instruction will be executed in the 32-bit ARM mode, starting with that address. When you feel you’d like to come back into 64-bit mode, simply execute UND #0xF8 and your execution will continue with the next instruction after the SVC 0xFFFF.

Appendix

////////////////////////////////////////////////////////////////////////////////
// General definitions.
////////////////////////////////////////////////////////////////////////////////
//
// Context flags.
// winnt.h (Windows SDK)
//
#define CONTEXT_i386 0x00010000L
#define CONTEXT_AMD64 0x00100000L
#define CONTEXT_ARM 0x00200000L
#define CONTEXT_ARM64 0x00400000L
//
// Machine type.
// winnt.h (Windows SDK)
//
#define IMAGE_FILE_MACHINE_TARGET_HOST 0x0001 // Useful for indicating we want to interact with the host and not a WoW guest.
#define IMAGE_FILE_MACHINE_I386 0x014c // Intel 386.
#define IMAGE_FILE_MACHINE_ARMNT 0x01c4 // ARM Thumb-2 Little-Endian
#define IMAGE_FILE_MACHINE_ARM64 0xAA64 // ARM64 Little-Endian
#define IMAGE_FILE_MACHINE_CHPE_X86 0x3A64 // Hybrid PE (defined in ntimage.h (WDK))
////////////////////////////////////////////////////////////////////////////////
// ntoskrnl.exe
////////////////////////////////////////////////////////////////////////////////
typedef struct _PS_NTDLL_EXPORT_ITEM {
PCSTR RoutineName;
PVOID RoutineAddress;
} PS_NTDLL_EXPORT_ITEM, *PPS_NTDLL_EXPORT_ITEM;
PS_NTDLL_EXPORT_ITEM NtdllExports[] = {
//
// 19 exports on x64
// 14 exports on ARM64
//
};
PVOID PsWowX86SharedInformation[Wow64SharedPageEntriesCount];
PS_NTDLL_EXPORT_ITEM NtdllWowX86Exports[] = {
{ «LdrInitializeThunk«,
&PsWowX86SharedInformation[SharedNtdll32LdrInitializeThunk] },
{ «KiUserExceptionDispatcher«,
&PsWowX86SharedInformation[SharedNtdll32KiUserExceptionDispatcher] },
{ «KiUserApcDispatcher«,
&PsWowX86SharedInformation[SharedNtdll32KiUserApcDispatcher] },
{ «KiUserCallbackDispatcher«,
&PsWowX86SharedInformation[SharedNtdll32KiUserCallbackDispatcher] },
{ «RtlUserThreadStart«,
&PsWowX86SharedInformation[SharedNtdll32RtlUserThreadStart] },
{ «RtlpQueryProcessDebugInformationRemote«,
&PsWowX86SharedInformation[SharedNtdll32pQueryProcessDebugInformationRemote] },
{ «LdrSystemDllInitBlock«,
&PsWowX86SharedInformation[SharedNtdll32LdrSystemDllInitBlock] },
{ «RtlpFreezeTimeBias«,
&PsWowX86SharedInformation[SharedNtdll32RtlpFreezeTimeBias] },
};
#ifdef _M_ARM64
PVOID PsWowArm32SharedInformation[Wow64SharedPageEntriesCount];
PS_NTDLL_EXPORT_ITEM NtdllWowArm32Exports[] = {
//
// …
//
};
PVOID PsWowAmd64SharedInformation[Wow64SharedPageEntriesCount];
PS_NTDLL_EXPORT_ITEM NtdllWowAmd64Exports[] = {
//
// …
//
};
PVOID PsWowChpeX86SharedInformation[Wow64SharedPageEntriesCount];
PS_NTDLL_EXPORT_ITEM NtdllWowChpeX86Exports[] = {
//
// …
//
};
#endif // _M_ARM64
//
// …
//
typedef struct _PS_NTDLL_EXPORT_INFORMATION {
PPS_NTDLL_EXPORT_ITEM NtdllExports;
SIZE_T Count;
} PS_NTDLL_EXPORT_INFORMATION, *PPS_NTDLL_EXPORT_INFORMATION;
//
// RTL_NUMBER_OF(NtdllExportInformation)
// == 6
// == (SYSTEM_DLL_TYPE)PsSystemDllTotalTypes
//
PS_NTDLL_EXPORT_INFORMATION NtdllExportInformation[PsSystemDllTotalTypes] = {
{ NtdllExports, RTL_NUMBER_OF(NtdllExports) },
{ NtdllWowX86Exports, RTL_NUMBER_OF(NtdllWowX86Exports) },
#ifdef _M_ARM64
{ NtdllWowArm32Exports, RTL_NUMBER_OF(NtdllWowArm32Exports) },
{ NtdllWowAmd64Exports, RTL_NUMBER_OF(NtdllWowAmd64Exports) },
{ NtdllWowChpeX86Exports, RTL_NUMBER_OF(NtdllWowChpeX86Exports) },
#endif // _M_ARM64
//
// { NULL, 0 } for the rest…
//
};
typedef struct _PS_SYSTEM_DLL_INFO {
//
// Flags.
// Initialized statically.
//
USHORT Flags;
//
// Machine type of this WoW64 NTDLL.
// Initialized statically.
// Examples:
// — IMAGE_FILE_MACHINE_I386
// — IMAGE_FILE_MACHINE_ARMNT
//
USHORT MachineType;
//
// Unused, always 0.
//
ULONG Reserved1;
//
// Path to the WoW64 NTDLL.
// Initialized statically.
// Examples:
// — «\\SystemRoot\\SysWOW64\\ntdll.dll»
// — «\\SystemRoot\\SysArm32\\ntdll.dll»
//
UNICODE_STRING Ntdll32Path;
//
// Image base of the DLL.
// Initialized at runtime by PspMapSystemDll.
// Equivalent of:
// RtlImageNtHeader(BaseAddress)->
// OptionalHeader.ImageBase;
//
PVOID ImageBase;
//
// Contains DLL name (such as «ntdll.dll» or
// «ntdll32.dll») before runtime initialization.
// Initialized at runtime by MmMapViewOfSectionEx,
// called from PspMapSystemDll.
//
union {
PVOID BaseAddress;
PWCHAR DllName;
};
//
// Unused, always 0.
//
PVOID Reserved2;
//
// Section relocation information.
//
PVOID SectionRelocationInformation;
//
// Unused, always 0.
//
PVOID Reserved3;
} PS_SYSTEM_DLL_INFO, *PPS_SYSTEM_DLL_INFO;
typedef struct _PS_SYSTEM_DLL {
//
// _SECTION* object of the DLL.
// Initialized at runtime by PspLocateSystemDll.
//
union {
EX_FAST_REF SectionObjectFastRef;
PVOID SectionObject;
};
//
// Push lock.
//
EX_PUSH_LOCK PushLock;
//
// System DLL information.
// This part is returned by PsQuerySystemDllInfo.
//
PS_SYSTEM_DLL_INFO SystemDllInfo;
} PS_SYSTEM_DLL, *PPS_SYSTEM_DLL;
////////////////////////////////////////////////////////////////////////////////
// ntdll.dll
////////////////////////////////////////////////////////////////////////////////
ULONG
RtlpArchContextFlagFromMachine(
_In_ USHORT MachineType
)
/*++
Routine description:
This routine translates architecture-specific CONTEXT
flag to the machine type.
Arguments:
MachineType — One of IMAGE_FILE_MACHINE_* values.
Return Value:
Context flag.
Note:
RtlpArchContextFlagFromMachine can be found only in
ntoskrnl.exe symbols, but from ntdll.dll disassembly
it is obvious that this function is present there
as well (probably __forceinline’d, or used as a macro).
—*/
{
switch (MachineType)
{
case IMAGE_FILE_MACHINE_I386:
return CONTEXT_i386;
case IMAGE_FILE_MACHINE_AMD64:
return CONTEXT_AMD64;
case IMAGE_FILE_MACHINE_ARMNT:
return CONTEXT_ARM;
case IMAGE_FILE_MACHINE_ARM64:
return CONTEXT_ARM64;
default:
return 0;
}
}
ULONG
RtlpGetLegacyContextLength(
_In_ ULONG ArchContextFlag,
_Out_opt_ PULONG SizeOfContext,
_Out_opt_ PULONG AlignOfContext
)
/*++
Routine description:
This routine determines size and alignment of the architecture-
-specific CONTEXT structure.
Arguments:
ArchContextFlag — Architecture-specific CONTEXT flag.
SizeOfContext — Receives sizeof(CONTEXT).
AlignOfContext — Receives __alignof(CONTEXT).
Return Value:
Alignment of the CONTEXT structure.
Note:
You can find corresponding DECLSPEC_ALIGN specifiers
for each CONTEXT structure in the winnt.h (Windows SDK).
By WOW64_CONTEXT_* here is meant an original CONTEXT
structure for the specific architecture (as CONTEXT
structures for other architectures are not available,
because it is selected during compile-time).
—*/
{
ULONG SizeOf = 0;
ULONG AlignOf = 0;
switch (ArchContextFlag)
{
case CONTEXT_i386:
SizeOf = sizeof(WOW64_CONTEXT_i386);
AlignOf = __alignof(WOW64_CONTEXT_i386); // 4
break;
case CONTEXT_AMD64:
SizeOf = sizeof(WOW64_CONTEXT_AMD64);
AlignOf = __alignof(WOW64_CONTEXT_AMD64); // 16
break;
case CONTEXT_ARM:
SizeOf = sizeof(WOW64_CONTEXT_ARM);
AlignOf = __alignof(WOW64_CONTEXT_ARM); // 8
break;
case CONTEXT_ARM64:
SizeOf = sizeof(WOW64_CONTEXT_ARM64);
AlignOf = __alignof(WOW64_CONTEXT_ARM64); // 16
break;
}
if (SizeOfContext) {
*SizeOfContext = SizeOf;
}
if (AlignOfContext) {
*AlignOfContext = AlignOf;
}
return AlignOf;
}
PULONG
RtlpGetContextFlagsLocation(
_In_ PCONTEXT_UNION Context,
_In_ ULONG ArchContextFlag
)
/*++
Routine description:
This routine returns pointer to the the «ContextFlags»
member of the CONTEXT structure.
Arguments:
Context — Architecture-specific CONTEXT structure.
ArchContextFlag — Architecture-specific CONTEXT flag.
Return Value:
Pointer to the the «ContextFlags» member.
—*/
{
//
// ContextFlags is always the first member of the
// CONTEXT struct — except for AMD64.
//
switch (ArchContextFlag)
{
case CONTEXT_i386:
return &Context->X86.ContextFlags; // Context + 0x00
case CONTEXT_AMD64:
return &Context->X64.ContextFlags; // Context + 0x30
case CONTEXT_ARM:
return &Context->ARM.ContextFlags; // Context + 0x00
case CONTEXT_ARM64:
return &Context->ARM64.ContextFlags; // Context + 0x00
default:
//
// Assume first member (Context + 0x00).
//
return (PULONG)Context;
}
}
//
// Architecture-specific WoW64 structure,
// holding the machine type and context
// structure.
//
#define WOW64_CPURESERVED_FLAG_RESET_STATE 1
typedef struct _WOW64_CPURESERVED {
USHORT Flags;
USHORT MachineType;
//
// CONTEXT has different alignment for
// each architecture and its location
// is determined at runtime (see
// RtlWow64GetCpuAreaInfo below).
//
// CONTEXT Context;
// CONTEXT_EX ContextEx;
//
} WOW64_CPURESERVED, *PWOW64_CPURESERVED;
typedef struct _WOW64_CPU_AREA_INFO {
PCONTEXT_UNION Context;
PCONTEXT_EX ContextEx;
PVOID ContextFlagsLocation;
PWOW64_CPURESERVED CpuReserved;
ULONG ContextFlag;
USHORT MachineType;
} WOW64_CPU_AREA_INFO, *PWOW64_CPU_AREA_INFO;
NTSTATUS
RtlWow64GetCpuAreaInfo(
_In_ PWOW64_CPURESERVED CpuReserved,
_In_ ULONG Reserved,
_Out_ PWOW64_CPU_AREA_INFO CpuAreaInfo
)
/*++
Routine description:
This routine returns architecture- and WoW64-specific
information based on the CPU-reserved region. It is
used mainly for fetching MachineType and the pointer
to the architecture-specific CONTEXT structure (which
is part of the WOW64_CPURESERVED structure). Because
the CONTEXT structure has different size and alignment
for each architecture, the pointer must be obtained
dynamically.
Arguments:
CpuReserved — WoW64 CPU-reserved region, usually located
at NtCurrentTeb()->TlsSlots[/* 1 */ WOW64_TLS_CPURESERVED]
Reserved — Unused. All callers set this argument to 0.
CpuAreaInfo — Receives the CPU-area information.
Return Value:
STATUS_SUCCESS — on success
STATUS_INVALID_PARAMETER — if CpuReserved contains invalid MachineType
*/
{
ULONG ContextFlag;
ULONG SizeOfContext;
ULONG AlignOfContext;
//
// In the ntdll.dll, this call is probably inlined, because
// RtlpArchContextFlagFromMachine symbol is not present there.
//
ContextFlag = RtlpArchContextFlagFromMachine(CpuReserved->MachineType);
if (!ContextFlag) {
return STATUS_INVALID_PARAMETER;
}
RtlpGetLegacyContextLength(ContextFlag, &SizeOfContext, &AlignOfContext);
//
// CpuAreaInfo->Context = &CpuReserved->Context;
// CpuAreaInfo->ContextEx = &CpuReserved->ContextEx;
//
CpuAreaInfo->Context = ALIGN_UP_POINTER_BY(
(PUCHAR)CpuArea + sizeof(WOW64_CPU_AREA),
AlignOfContext
);
CpuAreaInfo->ContextEx = ALIGN_UP_POINTER_BY(
(PUCHAR)Context + SizeOfContext + sizeof(CONTEXT_EX),
sizeof(PVOID)
);
CpuAreaInfo->ContextFlagsLocation = ContextFlagsLocation;
CpuAreaInfo->CpuArea = CpuArea;
CpuAreaInfo->ContextFlag = ContextFlag;
CpuAreaInfo->MachineType = CpuReserved->MachineType;
return STATUS_SUCCESS;
}
////////////////////////////////////////////////////////////////////////////////
// wow64.dll
////////////////////////////////////////////////////////////////////////////////
//
// WOW64INFO, based on:
// wow64t.h (WRK: https://github.com/mic101/windows/blob/master/WRK-v1.2/public/internal/base/inc/wow64t.h#L269)
//
#define WOW64_CPUFLAGS_MSFT64 0x00000001
#define WOW64_CPUFLAGS_SOFTWARE 0x00000002
typedef struct _WOW64INFO {
ULONG NativeSystemPageSize;
ULONG CpuFlags;
ULONG Wow64ExecuteFlags;
ULONG Unknown1;
USHORT NativeMachineType;
USHORT EmulatedMachineType;
} WOW64INFO, *PWOW64INFO;
//
// Thread Local Storage (TLS) support. TLS slots are statically allocated.
// wow64tls.h (WRK: https://github.com/mic101/windows/blob/master/WRK-v1.2/public/internal/base/inc/wow64tls.h#L23)
// Note: Not all fields probably matches their names on Windows 10.
//
#define WOW64_TLS_STACKPTR64 0 // contains 64-bit stack ptr when simulating 32-bit code
#define WOW64_TLS_CPURESERVED 1 // per-thread data for the CPU simulator
#define WOW64_TLS_INCPUSIMULATION 2 // Set when inside the CPU
#define WOW64_TLS_TEMPLIST 3 // List of memory allocated in thunk call.
#define WOW64_TLS_EXCEPTIONADDR 4 // 32-bit exception address (used during exception unwinds)
#define WOW64_TLS_USERCALLBACKDATA 5 // Used by win32k callbacks
#define WOW64_TLS_EXTENDED_FLOAT 6 // Used in ia64 to pass in floating point
#define WOW64_TLS_APCLIST 7 // List of outstanding usermode APCs
#define WOW64_TLS_FILESYSREDIR 8 // Used to enable/disable the filesystem redirector
#define WOW64_TLS_LASTWOWCALL 9 // Pointer to the last wow call struct (Used when wowhistory is enabled)
#define WOW64_TLS_WOW64INFO 10 // Wow64Info address (structure shared between 32-bit and 64-bit code inside Wow64).
#define WOW64_TLS_INITIAL_TEB32 11 // A pointer to the 32-bit initial TEB
#define WOW64_TLS_PERFDATA 12 // A pointer to temporary timestamps used in perf measurement
#define WOW64_TLS_DEBUGGER_COMM 13 // Communicate with 32bit debugger for event notification
#define WOW64_TLS_INVALID_STARTUP_CONTEXT 14 // Used by IA64 to indicate an invalid startup context. After startup, it stores a pointer to the context.
#define WOW64_TLS_SLIST_FAULT 15 // Used to retry RtlpInterlockedPopEntrySList faults
#define WOW64_TLS_UNWIND_NATIVE_STACK 16 // Forces an unwind of the native 64-bit stack after an APC
#define WOW64_TLS_APC_WRAPPER 17 // Holds the Wow64 APC jacket routine
#define WOW64_TLS_IN_SUSPEND_THREAD 18 // Indicates the current thread is in the middle of NtSuspendThread. Used by software CPUs.
#define WOW64_TLS_MAX_NUMBER 19 // Maximum number of TLS slot entries to allocate
typedef struct _WOW64_ERROR_CASE {
ULONG Case;
NTSTATUS TransformedStatus;
} WOW64_ERROR_CASE, *PWOW64_ERROR_CASE;
typedef struct _WOW64_SERVICE_TABLE_DESCRIPTOR {
//
// struct _KSERVICE_TABLE_DESCRIPTOR {
//
// //
// // Pointer to a system call table (array of function pointers).
// //
//
// PULONG_PTR Base;
//
// //
// // Pointer to a system call count table.
// // This field has been set only on checked (debug) builds,
// // where the Count (with the corresponding system call index)
// // has been incremented with each system call.
// // On non-checked builds it is set to NULL.
// //
//
// PULONG Count;
//
// //
// // Maximum number of items in the system call table.
// // In ntoskrnl.exe it corresponds with the actual number
// // of system calls. In wow64.dll it is set to 4096.
// //
//
// ULONG Limit;
//
// //
// // Pointer to a system call argument table.
// // The elements in this table actually contain how many
// // bytes on the stack are assigned to the function parameters
// // for a particular system call.
// // On 32-bit systems, if you divide this number by 4, you’ll
// // get the the number of arguments that the system call expects.
// //
//
// PUCHAR Number;
// };
//
KSERVICE_TABLE_DESCRIPTOR Descriptor;
//
// Extended fields of the WoW64 servie table:
// Wow64HandleSystemServiceError
//
WOW64_ERROR_CASE ErrorCaseDefault;
PWOW64_ERROR_CASE ErrorCase;
} WOW64_SERVICE_TABLE_DESCRIPTOR, *PWOW64_SERVICE_TABLE_DESCRIPTOR;
#define WOW64_NTDLL_SERVICE_INDEX 0
#define WOW64_WIN32U_SERVICE_INDEX 1
#define WOW64_KERNEL32_SERVICE_INDEX 2
#define WOW64_USER32_SERVICE_INDEX 3
#define WOW64_SERVICE_TABLE_MAX 4
WOW64_SERVICE_TABLE_DESCRIPTOR ServiceTables[WOW64_SERVICE_TABLE_MAX];
typedef struct _WOW64_LOG_SERVICE
{
PVOID Reserved;
PULONG Arguments;
ULONG ServiceTable;
ULONG ServiceNumber;
NTSTATUS Status;
BOOLEAN PostCall;
} WOW64_LOG_SERVICE, *PWOW64_LOG_SERVICE;
NTSTATUS
Wow64HandleSystemServiceError(
_In_ NTSTATUS ExceptionStatus,
_In_ PWOW64_LOG_SERVICE LogService
)
/*++
Routine description:
This routine transforms exception from native system
call to WoW64-compatible NTSTATUS.
Arguments:
ExceptionStatus — NTSTATUS raised from executing system call.
LogService — Information about the WoW64 system call.
Return Value:
Transformed NTSTATUS.
—*/
{
PWOW64_SERVICE_TABLE_DESCRIPTOR ServiceTable;
PWOW64_ERROR_CASE ErrorCaseTable;
ULONG ErrorCase;
NTSTATUS TransformedStatus;
ErrorCaseTable = ServiceTables[LogService->ServiceTable].ErrorCase;
if (!ErrorCaseTable)
{
ErrorCaseTable = &ServiceTables[LogService->ServiceTable].ErrorCaseDefault;
}
ErrorCase = ErrorCaseTable[LogService->ServiceNumber].ErrorCase;
TransformedStatus = ErrorCaseTable[LogService->ServiceNumber].TransformedStatus;
switch (ErrorCase)
{
case 0:
return ExceptionStatus;
case 1:
NtCurrentTeb()->LastErrorValue = RtlNtStatusToDosError(ExceptionStatus);
return ExceptionStatus;
case 2:
return TransformedStatus;
case 3:
NtCurrentTeb()->LastErrorValue = RtlNtStatusToDosError(ExceptionStatus);
return TransformedStatus;
default:
return STATUS_INVALID_PARAMETER;
}
}
view raw2_appendix.h hosted with ❤ by GitHub

References

How does one retrieve the 32-bit context of a Wow64 program from a 64-bit process on Windows Server 2003 x64?
http://www.nynaeve.net/?p=191

Mixing x86 with x64 code
http://blog.rewolf.pl/blog/?p=102

Windows 10 on ARM
https://channel9.msdn.com/Events/Build/2017/P4171

Knockin’ on Heaven’s Gate – Dynamic Processor Mode Switching
http://rce.co/knockin-on-heavens-gate-dynamic-processor-mode-switching/

Closing “Heaven’s Gate”
http://www.alex-ionescu.com/?p=300

PE-sieve is a light-weight tool that helps to detect malware running on the system

PE-sieve

PE-sieve is a light-weight tool that helps to detect malware running on the system, as well as to collect the potentially malicious material for further analysis. Recognizes and dumps variety of implants within the scanned process: replaced/injected PEs, shellcodes, hooks, and other in-memory patches.
Detects inline hooks, Process Hollowing, Process Doppelgänging, Reflective DLL Injection, etc.

uses library: https://github.com/hasherezade/libpeconv.git

Clone:

Use recursive clone to get the repo together with the submodule:

git clone --recursive https://github.com/hasherezade/pe-sieve.git

Latest builds*:

*those builds are available for testing and they may be ahead of the official release:

example: classic unmapping (2) vs remapping (3) — with remapping full virtual content of the section is preserved, so it helps i.e. if the full section was unpacked in memory, or if virtual caves were used


logo by Baran Pirinçal

Project: x86-devirt

Unpackme — x86 Virtualizer

Today, I am going to be going through how x86devirt works to disassemble and devirtualize the behaviour of code obfuscated using the x86virt virtual machine. I needed several tools to complete this task, the development of which will be covered in this article.

A code virtualizer protects code behaviour by retargeting some subroutines or sections of code from the x86/x64 platform (which is well understood and documented) into a (usually somewhat random) platform that we do not understand. Additionally, the tools we use to discover and analyze behaviour in executable code (such as radare2 or x64dbg) also do not understand it well. This makes identifying malicious behaviour, developing generic signatures or extracting other behavioral details from the code impossible without reversing the process in some way.

While it costs a large overhead in performance, obfuscation using code virtualization is very effective. Depending on the complexity of the protector/virtualizer, these packers can be very painful and tedious to work through.

Resources

You can see the final product (devirtualizer) of this article at the following GitHub URL: https://github.com/JeremyWildsmith/x86devirt

We are reverse engineering an unpackme by ReWolf at the following URL: https://tuts4you.com/download/1850/

This sample has been packed with an open-source application by ReWolf that is publicly posted on GitHub at the following URL: https://github.com/rwfpl/rewolf-x86-virtualizer

If you would like to look at a very simple example of a virtual machine, I have a project up on my GitHub that demonstrates the basic function of a VM Stub and you can see how exactly it works. The project is located at the following URL: https://github.com/JeremyWildsmith/StackLang

Tools / Knowledge

In this article I am going to use the following tools & knowledge:

  • The disassembler, written in the C++ programming language
  • x86 Intel Assembly
  • YARA Signatures
  • The udis86 library, used to disassemble x86 instructions
  • Python, used to automate x64dbg and the x64dbgpy plugin, as well as run Angr simulations to extract the jmp mappings
  • The x86virt unpackme sample by ReWolf on tuts4you.com (https://tuts4you.com/download/1850/)

How the x86virt VM Works

Before I dive into how x86devirt works, I am going to give a brief overview on my findings of how the x86virt VM works.

x86virt starts by taking the application to be protected and grabbing some subroutines that it has decided to protect. The x86 code for these subroutines is translated into an instruction set that uses the following format:

(Instruction Size)(Instruction Prefix)(Instruction Data)

  • Instruction Size — The size in bytes of the instruction
  • Instruction Prefix — This is 0xFFFF if the instruction is a VM instruction. Otherwise, the remaining bytes in the instruction data (and instruction prefix) should be interpreted as a x86 instruction
  • Instruction Data — If (Instruction Prefix) is 0xFFFF, this is the VM Opcode bytes followed by the operand bytes. Otherwise, these bytes are appended to the (Instruction Prefix) to form a valid x86 instruction.

For every VM Layer or virtualized target, x86virt randomizes the opcodes for that VM. So when we devirtualize, we need to map the opcodes to their respective VM instruction.

x86virt also takes instructions like the ones below:

mov eax, [esp + 0x8]

And translates them into something like this:

mov VMR, 0x8
add VMR, esp
mov eax, [VMR]

VMR is a register that only exists in the virtual machine, so during devirtualization, we need to interpret this and translate it back into its original form.

Finally, x86virt encrypts the entire instruction using an encryption algorithm that is, to some extent, randomized. Every time an instruction is executed, it is first decrypted and then interpreted. However, there is one consistency with instruction encryption between targets and VM layers: regardless of how the instruction was encrypted, in its encrypted form, the first byte XORed with the second byte will always give you the instruction length.

Another important note regarding instruction encryption is that the key to decrypt it is the address of that VM instruction relative to the start of the virtual function/code stub it belongs to. In other words, the key is the offset to that instruction in the function that has been virtualized. This means you cannot just blindly decrypt all the instructions in a function. You must do proper control flow analysis to determine where in memory the valid bytecode instructions are by identifying and following conditional or unconditional jumps.

So to disassemble an x86virt VM instruction, we must know:

  1. The size of the instruction
  2. The offset to that instruction
  3. The encryption algorithm (because it is somewhat random)

There is one last piece of the puzzle that we must consider when disassembling x86virt VM code. The conditional jumps for x86virt VM are encoded with a jump type operand. The part of the code that interprets the jump type operand is also somewhat random between VM layers and virtualized targets. We need to handle this case in our devirtualizer as well.

The x86devirt Disassembler

An important part to the x86-devirtualizer is the disassembler. The role of this module is to take a stub of virtualized, encrypted x86virt bytecode that has been extracted from the protected application and produce a NASM x86 Assembly translation of the encrypted bytecode. To do this, it needs a few pieces of information from the protected application:

  1. A dump of the decryption algorithm that the VM stub uses to decrypt VM instruction before interpreting them
  2. The mappings of opcodes to their respective behaviour, since the opcodes are randomized (i.e, 0x33 maps to add, 0x22 maps to mov)
  3. The mappings of jmp type operand to their respective jump behavior (i.e, jmp type 2 maps to je, 3 maps to jne, etc…)
  4. A dump of the function / stub of code to be devirtualized

We will see how this information is extracted by looking at the x86devirt.py x64dbg plugin later, but for now we will assume we have been provided this information.

The first step is to get the instruction length, decrypt the instruction and then identify whether we should interpret the instruction as an x86 instruction or a VM bytecode instruction. We see this being done in disassemblers’ decodeVmInstruction method:

unsigned int decodeVmInstruction(vector<DecodedVmInstruction>& decodedBuffer, uint32_t vmRelativeIp, VmInfo& vmInfo) {

    uint32_t instrLength = getInstructionLength(vmInfo.getBaseAddress() + vmRelativeIp, vmInfo);

    //Read with offset 1, to trim off instr length byte
    unique_ptr<uint8_t[]> instrBuffer = vmInfo.readMemory(vmInfo.getBaseAddress() + vmRelativeIp + 1, instrLength);
    vmInfo.decryptMemory(instrBuffer.get(), instrLength, vmRelativeIp);

    DecodedInstructionType_t instrType = DecodedInstructionType_t::INSTR_UNKNOWN;

    if(*reinterpret_cast<unsigned short*>(instrBuffer.get()) == 0xFFFF) {
        //Offset by 2 which removes the 0xFFFF part of the instruction.

        //Map instructions correctly
        instrBuffer[2] = vmInfo.getOpcodeMapping(instrBuffer[2]);
        decodedBuffer = disassembleVmInstruction(instrBuffer.get() + 2, instrLength - 2, vmRelativeIp, vmInfo);
    } else {
        decodedBuffer = disassemble86Instruction(instrBuffer.get(), instrLength, vmInfo.getBaseAddress() + vmRelativeIp);
    }

    return instrLength + 1;
}

We see that the size is extracted using the getInstructionLength method (this will simply XOR the first two bytes to get the length). After that, the instruction is decrypted by using the dumped decryption subroutine extracted from the protected application (a more proper approach would be to emulate the code rather than directly executing it). Finally, we examine the first word to identify how to decode the instruction (as an x86 instruction or as a VM instruction). If the instruction is a VM bytecode instruction, we need to look up the opcode in the opcode mapping to determine what behaviour it maps to.

The way we disassemble x86 instructions is by using the udis86 library, and also keeping some basic information about the disassembled instruction. You can see how that is done below:

vector<DecodedVmInstruction> disassemble86Instruction(const uint8_t* instrBuffer, uint32_t instrLength, const uint32_t instrAddress) {
    DecodedVmInstruction result;
    result.isDecoded = false;
    result.address = instrAddress;
    result.controlDestination = 0;
    result.size = instrLength;

    memcpy(result.bytes, instrBuffer, instrLength);

    ud_set_input_buffer(&ud_obj, instrBuffer, instrLength);
    ud_set_pc(&ud_obj, instrAddress);
    unsigned int ret = ud_disassemble(&ud_obj);
    strcpy(result.disassembled, ud_insn_asm(&ud_obj));

    if(ret == 0)
        result.type = DecodedInstructionType_t::INSTR_UNKNOWN;
    else
        result.type = (!strncmp(result.disassembled, "ret", 3) ? DecodedInstructionType_t::INSTR_RETN : DecodedInstructionType_t::INSTR_MISC);

    vector<DecodedVmInstruction> resultSet;
    resultSet.push_back(result);

    return resultSet;
}

When it comes to disassembling x86virt bytecode instructions, we do that with a different subroutine, disassembleVmInstruction. The purpose of this subroutine is fairly straightforward so I won’t bore you by reading the code line by line. However, some interesting cases are case 1 and case 2, which are essentially x86 instructions with VMR as the operand. It is also worth noting case 7 where the decoding of x86virt jump instructions are handled and case 16 which just signals for the VM to stop interpreting (and has no x86 equivalent)

Once we can disassemble the instructions, we need to properly identify where they are. As was previously mentioned, this requires some control flow analysis. During control flow analysis, the disassembler identifies the different blocks of code in a subroutine by using the getDisassembleRegions function. The getDisassembleRegions basically returns the regions of code in a method that can be reached using conditional or unconditional jumps inside of a virtualized function. We can see its behaviour below:

vector<DisassembledRegion> getDisassembleRegions(const uint32_t initialIp, VmInfo& vmInfo) {
    vector<DisassembledRegion> disassembledStubs;
    queue<uint32_t> stubsToDisassemble;
    stubsToDisassemble.push(initialIp);

    while(!stubsToDisassemble.empty()) {
        uint32_t vmRelativeIp = stubsToDisassemble.front() - vmInfo.getBaseAddress();
        stubsToDisassemble.pop();

        if(isInRegions(disassembledStubs, vmRelativeIp))
            continue;

        DisassembledRegion current;
        current.min = vmRelativeIp;

        bool continueDisassembling = true;
        while(vmRelativeIp <= vmInfo.getDumpSize() && continueDisassembling) {

            vector<DecodedVmInstruction> instrSet;

            vmRelativeIp += decodeVmInstruction(instrSet, vmRelativeIp, vmInfo);

            for(auto& instr : instrSet) {
                if(instr.type == DecodedInstructionType_t::INSTR_UNKNOWN) {
                    stringstream msg;
                    msg << "Unknown instruction encountered: 0x" << hex << ((unsigned long)instr.bytes[0]);
                    throw runtime_error(msg.str());
                }

                if(instr.type == DecodedInstructionType_t::INSTR_JUMP || instr.type == DecodedInstructionType_t::INSTR_CONDITIONAL_JUMP)
                    stubsToDisassemble.push(instr.controlDestination);

                if(instr.type == DecodedInstructionType_t::INSTR_STOP || instr.type == DecodedInstructionType_t::INSTR_RETN || instr.type == DecodedInstructionType_t::INSTR_JUMP)
                    continueDisassembling = false;
            }
        }

        current.max = vmRelativeIp;
        disassembledStubs.push_back(current);
    }

    //Now we must resolve all overlapping stubs
    for(auto it = disassembledStubs.begin(); it != disassembledStubs.end();) {
        if(isInRegions(disassembledStubs, it->min, it->max))
            disassembledStubs.erase(it++);
        else
            it++;
    }

    return disassembledStubs;
}

The getDisassembleRegions performs the following functionality:

  1. Disassemble the virtualized subroutine from its start address and continues to do so until it encounters a jump (conditional or unconditional) or a return.
  2. If a conditional jump is encountered, its destination address is queued up to be the next region to be disassembled and the disassembler continues executing.
  3. If an unconditional jump is encountered, the destination address is queued up and disassembling of the current block ends
  4. If a ret is encountered, disassembling of the current block ends.
  5. Loops until there are no more regions to be disassembled.

The problem with the above algorithm is that it will identify code such as what is seen below:

labelD:
...
...
labelA:
...
...
jmp labelB
...
...
labelB:
...
...
jz labelD
...

As having overlapping blocks of code, which will result in redundant blocks of code and thus redundant disassembled output. This was solved by testing for and removing smaller overlapping regions:

    //Now we must resolve all overlapping stubs
    for(auto it = disassembledStubs.begin(); it != disassembledStubs.end();) {
        if(isInRegions(disassembledStubs, it->min, it->max))
            disassembledStubs.erase(it++);
        else
            it++;
    }

    return disassembledStubs;

After we know the basic blocks of a subroutine that contain valid executable code, we can begin disassembling them. This is done in the disassembleStub routine:

bool disassembleStub(const uint32_t initialIp, VmInfo& vmInfo) {

    vector<DisassembledRegion> stubs = getDisassembleRegions(initialIp, vmInfo);

    //Needs to be sorted, otherwise (due to jump sizes) may not fit into original location
    //Sorting should match it with the way it was implemented.
    sort(stubs.begin(), stubs.end(), sortRegionsAscending);

    if(stubs.empty()) {
        printf(";No stubs detected to disassemble.. %d", stubs.size());
        return true;
    }

    vector<DecodedVmInstruction> instructions;
    for(auto& stub : stubs) {

        bool continueDisassembling = true;
        DecodedVmInstruction blockMarker;
        blockMarker.type = DecodedInstructionType_t::INSTR_COMMENT;
        strcpy(blockMarker.disassembled, "BLOCK");
        instructions.push_back(blockMarker);
        for(uint32_t vmRelativeIp = stub.min; continueDisassembling && vmRelativeIp < stub.max;) {

            vector<DecodedVmInstruction> instrSet;

            vmRelativeIp += decodeVmInstruction(instrSet, vmRelativeIp, vmInfo);

            for(auto& instr : instrSet) {
                if(instr.type == DecodedInstructionType_t::INSTR_UNKNOWN)
                    throw runtime_error("Unknown instruction encountered");

                if(instr.type == DecodedInstructionType_t::INSTR_STOP) {
                    continueDisassembling = false;
                    break;
                }

                instructions.push_back(instr);
            }

        }

        instructions.push_back(blockMarker);
    }

    for(auto& i : eliminateVmr(instructions)) {
        formatInstructionInfo(i);
    }

    return true;
}

An important note with this method is that it sorts the disassembled regions by their start address after doing the control flow analysis with getDisassembledRegions. This sorting must be done because the natural order was thrown out of whack by the queuing nature of the control flow analysis. Functionally, the order doesn’t really make a difference in a normal application because at the end of the day, the code is still going to execute the same way regardless of where the instructions are. However, the way in which the blocks are organized will change the size of the code once it is assembled in NASM due to the way jump instructions are encoded on the x86 platform. Essentially, the distance between the jump instructions and their destination addresses will change depending on the order of the code blocks in the function, and the distance will influence the size of the jump instruction. If the devirtualized code is not the size of its original form (i.e, before it is was passed into x86virt to be virtualized) or smaller, then it will not fit back into where it was ripped from. While functionally, it doesn’t matter that it isn’t in the «proper» location, it does matter later when we encounter multiple VM layers because our signatures will not match partial handlers etc.

Other than that, there isn’t anything too weird or noteworthy here until we encounter the call to eliminateVmr. Remember that I mentioned how x86virt creates a virtual register. We need to eliminate that because we cannot assemble that through NASM or produce valid x86 code while we have a virtual register. Below, we can see the behaviour of eliminateVmr:

vector<DecodedVmInstruction> eliminateVmr(vector<DecodedVmInstruction>& instructions) {
    auto itVmrStart = instructions.end();
    vector<DecodedVmInstruction> compactInstructionlist;

    for(auto it = instructions.begin(); it != instructions.end(); it++) {
        if(!strncmp("mov VMR,", it->disassembled, 8) && itVmrStart == instructions.end()) {
            itVmrStart = it;
        }else if(itVmrStart != instructions.end() && strstr(it->disassembled, "[VMR]") != 0)
        {
            for(auto listing = itVmrStart; listing != it+1; listing++) {
                DecodedVmInstruction comment = *listing;
                comment.type = INSTR_COMMENT;
                compactInstructionlist.push_back(comment);
            }
            compactInstructionlist.push_back(eliminateVmrFromSubset(itVmrStart, it + 1));
            itVmrStart = instructions.end();
        } else if (itVmrStart == instructions.end()) {
            compactInstructionlist.push_back(*it);
        }
    }

    return compactInstructionlist;
}

The way VMR is used in the virtualized code is fairly convenient. VMR is essentially used to calculate pointer addresses. For example, it only ever appears in a similar form to:

mov VMR, 0
add VMR, ecx
shl VMR, 2
add VMR, 15
mov eax, [VMR]

It always starts with operations on VMR and ends with VMR being dereferenced. This means that we can essentially replace all of those instructions with:

mov eax, [ecx * 2 + 15]

So eliminateVmr will look for a pattern where the destination operand is VMR and then some operations on VMR followed by a dereference on VMR. Everything between that pattern can always be simplified using the same algorithm. You can see the specifics of that algorithm in eliminateVmrFromSubset:

DecodedVmInstruction eliminateVmrFromSubset(vector<DecodedVmInstruction>::iterator start, vector<DecodedVmInstruction>::iterator end) {
    bool baseReg2Used = false;
    bool baseReg1Used = false;
    char baseReg1Buffer[10];
    char baseReg2Buffer[10];
    uint32_t multiplierReg1 = 1;
    uint32_t multiplierReg2 = 1;

    uint32_t offset = 0;

    for(auto it = start; it != end; it++) {
        char* dereferencePointer = 0;

        if(!strncmp(it->disassembled, "mov VMR, 0x", 11)) {
            offset = strtoul(&it->disassembled[11], NULL, 16);
            baseReg1Used = false;
            baseReg2Used = false;
            multiplierReg1 = multiplierReg2 = 1;
        } else if(!strncmp(it->disassembled, "mov VMR, ", 9)) {
            baseReg1Used = true;
            baseReg2Used = false;
            multiplierReg1 = multiplierReg2 = 1;
            offset = 0;
            strcpy(baseReg1Buffer, &it->disassembled[9]);
        } else if(!strncmp(it->disassembled, "add VMR, 0x", 11)) {
            offset += strtoul(&it->disassembled[11], NULL, 16);
        } else if(!strncmp(it->disassembled, "add VMR, ", 9)) {
            if(baseReg1Used) {
                baseReg2Used = true;
                strcpy(baseReg2Buffer, &it->disassembled[9]);
            } else {
                baseReg1Used = true;
                strcpy(baseReg1Buffer, &it->disassembled[9]);    
            }
        } else if(!strncmp(it->disassembled, "shl VMR, 0x", 11)) {
            uint32_t shift = strtoul(&it->disassembled[11], NULL, 16);
            offset = offset << shift;
            if(baseReg1Used) {
                multiplierReg1 = multiplierReg1 << shift;
            }
            if(baseReg2Used) {
                multiplierReg2 = multiplierReg2 << shift;
            }
        }
    }

    auto lastInstruction = end - 1;
    string reconstructInstr(lastInstruction->disassembled);
    stringstream reconstructed;

    reconstructed << "[";

    if(baseReg1Used) {
        if(multiplierReg1 != 1)
            reconstructed << "0x" << hex << multiplierReg1 << " * ";

        reconstructed << baseReg1Buffer;
    }

    if(baseReg2Used) {
        reconstructed << " + ";
        if(multiplierReg2 != 1)
            reconstructed << "0x" << hex << multiplierReg2 << " * ";

        reconstructed << baseReg2Buffer;
    }

    if(offset != 0 || !(baseReg1Used))
        reconstructed <<  " + 0x" << hex << offset;

    reconstructed << "]";

    reconstructInstr.replace(reconstructInstr.find("[VMR]"), 5, reconstructed.str());

    DecodedVmInstruction result;

    result.isDecoded = true;
    result.address = start->address;
    result.size = 0;
    result.type = lastInstruction->type;
    strcpy(result.disassembled, reconstructInstr.c_str());

    return result;
}

Once VMR is eliminated, all that is left to do is print out the disassembly, which we can see being done here in disassembleStub:

...
    for(auto& i : eliminateVmr(instructions)) {
        formatInstructionInfo(i);
    }
...

Generating a Jump Map with Angr

As I mentioned earlier, when it comes to the x86virt bytecode conditional / unconditional jump instruction handler, we need to extract the jump mappings (that is, which value in the jump type operand matches which type of jump). The way the x86virt jump handler works is that it takes the first operand (which is the jump type) and passes it into a somewhat randomly generated subroutine. This subroutine returns true if the EFLAGS are in a condition that permits jumping, or false otherwise.

Because this subroutine is not static and is a bit different for every VM Layer or virtualized target, we need some way of extracting these mappings out of that randomly generated subroutine. If you are not familiar with Angr or symbolic execution, I suggest you read a tiny bit on it before reading this section because the learning curve can be a bit steep.

The jump maps are extracted by running an Angr simulation on the jump decoder that was extracted from the protected application. The simulation is done in x86devirt_jmp.py and the dump of the decoder is provided by x86devirt.py (which we will get into later).

The way this was performed was first by creating a table of x86 jump types with EFLAG values that permit a jump to be taken for that jump type and EFLAG values that do not permit a jump to be taken. Additionally, all jump types in the table were prioritized.

Jump type priority worked by giving x86 jump types that test less flags a lower priority than jump types that test more flags. The reason jump types need to be prioritized is because there is overlap in the conditions that need to be checked for different jumps. An example of this is the JZ jump (ZF = 0) and the JA (ZF = 0 and CF = 0) jump. Essentially, if a set of x candidate x86 jump types can be mapped to a particular jump type y, then y should be mapped to the highest priority jump type in the set of x.

Below we see the list of possible jumps:

possibleJmps = [
    {
        "name": "jz",
        "must": [0x40],
        "not": [0x1, 0],
        "priority": 1
    },
    {
        "name": "jo",
        "must": [0x800],
        "not": [0],
        "priority": 1
    },
    ...
    ...

Below is the code responsible for mapping which emulated states permit jumping and which states do not, for all jump types (0-15):

def getJmpStatesMap(proj):
    statesMap = {}

    state = proj.factory.blank_state(addr=0x0)
    state.add_constraints(state.regs.edx >= 0)
    state.add_constraints(state.regs.edx <= 15)
    simgr = proj.factory.simulation_manager(state)
    r = simgr.explore(find=0xDA, avoid=0xDE, num_find=100)

    for state in r.found:
        val = state.solver.eval(state.regs.edx)
        val = val - 0xD
        val = val / 2

        if(not statesMap.has_key(val)):
            statesMap[val] = {"must": [], "not": []}

        statesMap[val]["must"].append(state)

    state = proj.factory.blank_state(addr=0x0)
    state.add_constraints(state.regs.edx >= 0)
    state.add_constraints(state.regs.edx <= 15)
    simgr = proj.factory.simulation_manager(state)
    r = simgr.explore(find=0xDE, avoid=0xDA, num_find=100)

    for state in r.found:
        val = state.solver.eval(state.regs.edx)
        val = val - 0xD
        val = val / 2

        statesMap[val]["not"].append(state)

    return statesMap

The method essentially performs the following:

  1. Iterate through all states that reach a positive/negative return (jump allowed or not allowed to be taken)
  2. Resolve the constraint on the jump type (Jump type is stored in EDX)
  3. Append that state to either the «must» set, which is states that reached a positive return permitting the jump to be taken (offset 0xDA in the dumped jump decoder code) or «not» (offset 0xDE in the dumped jump decoder code)

After we know which states permit jumping or restrict jumping for each jump type, we can begin testing the constraints on the EFLAGS register that allow for arriving at those states to determine which kind of x86 jump it maps to:

def decodeJumps(inputFile):
    proj = angr.Project(inputFile, main_opts={'backend': 'blob', 'custom_arch': 'i386'}, auto_load_libs=False)

    stateMap = getJmpStatesMap(proj)
    jumpMappings = {}
    for key, val in stateMap.iteritems():

        for jmp in possibleJmps:
            satisfiedMustsRemaining = len(jmp["must"])
            satisfiedNotsRemaining = len(jmp["not"])

            for state in val["must"]:
                for con in jmp["must"]:
                    if (state.solver.satisfiable(
                            extra_constraints=[state.regs.eax & controlFlowBits == con & controlFlowBits])):
                        satisfiedMustsRemaining -= 1;

            for state in val["not"]:
                for con in jmp["not"]:
                    if (state.solver.satisfiable(
                            extra_constraints=[state.regs.eax & controlFlowBits == con & controlFlowBits])):
                        satisfiedNotsRemaining -= 1;

            if(satisfiedMustsRemaining <= 0 and satisfiedNotsRemaining <= 0):
                if(not jumpMappings.has_key(key)):
                    jumpMappings[key] = []

                jumpMappings[key].append(jmp)

    finalMap = {}
    for key, val in jumpMappings.iteritems():
        maxPriority = 0;
        jmpName = "NOE FOUND"
        for j in val:
            if(j["priority"] > maxPriority):
                maxPriority = j["priority"]
                jmpName = j["name"]
        finalMap[jmpName] = key
        print("Mapped " + str(key) + " to " + jmpName)

    proj.terminate_execution()
    return finalMap

For each possible x86virt jump type, we test against each candidate x86 jump type to see if the x86 jump’s «not» and «must» sets can be satisfied accordingly by the restrictions on the EFLAGS registers in each state. If all «not»s and «must»s are satisfied, then the x86 jump is added as a possible candidate for that jump type.

Later, we iterate through all candidate x86 jumps for each jump type and choose the one with the highest priority to be mapped to it.

Finding the Signatures in the Protected Binary & Dumping Required Data

Finally, on to the last module needed to devirtualize code. x86devirt.py is the x64dbgpy Python plugin that instructs x64dbg on how to devirtualize the target.

x86devirt uses YARA rules to locate sections of code in the protected application, including the VM Stub, the instruction handlers, etc… You can see these YARA rules in VmStub.yara, VmRef.yara and instructions.yara:

  1. VmStub.yara is the YARA signature of the Virtual Machine interpreter.
  2. VmRef.yara is the YARA signature to detect where the application passes control off to the interpreter to begin interpreting a section of x86virt bytecode.
  3. instructions.yara is a set of YARA signatures for the different x86 instruction handlers.

An important note with these signatures is that they must match the original VM Stub and the devirtualized code generated by x86devirt. For example, consider a target that has been virtualized using two VM layers. After the first layer has been devirtualized, there will be a new VM stub in plain x86 form (that is, the second layer of virtualization). These signatures need to detect that second layer. However, the second layer was assembled using a different environment than the first layer and thus we need to take care for some special x86 instruction encodings in our signature (some x86 instructions have more than one way of being encoded). For example, consider:

add eax, ebx ; Encoded as 03C3
add eax, ebx ; Encoded as 01D8

So, when developing our signatures, we need to keep in mind that NASM could choose either encoding. With YARA, I just masked out instructions like these. If you are ever developing a signature with these constraints, please look into this more. There are plenty of resources on this topic: https://www.strchr.com/machine_code_redundancy

When it comes to devirtualizing x86virt, we must perform the following:

  1. Locate all VM Stubs that are present in plain x86 form using the vmStub.yara YARA signature
  2. Extract the decryption routine from the VM Stub
  3. Locate all references to that VM Stub (i.e, all areas where the VM is invoked to begin virtualizing code)
  4. Through each reference, extract the address where the virtualized bytecode is, and where the original code was ripped from
  5. Emulate part of the VM Stub to locate all instruction handlers and their opcodes
  6. Apply the YARA signatures to identify which handler (and subsequently, opcode) maps to which instruction behaviour and produce the instruction / opcode mappings
  7. Locate the JXX instruction handler and dump the part of the handler responsible for testing whether, given the jump type and state of the EFLAGS, the jump is taken. This is passed to x86devirt_jmp.py to extract the jump mappings
  8. For each reference, dump the virtualized code around that references and, with the jmp mappings, instruction mappings and the decryption routine, feed it to the x86virt-disassembler to be disassembled
  9. Finally, run NASM on the disassembler output to produce a x86 binary blob that can be written into where the virtualized code was ripped from, therefore restoring the virtualized function to its original form
  10. Loop back and search for any newly unveiled VM stubs

This process, once completed, will leave the x64dbg debugger in a state that allows the application to be cleanly dumped without any need for the VM stub.

(https://github.com/JeremyWildsmith)

CONTROL FLOW DEOBFUSCATION VIA ABSTRACT INTERPRETATION

I present some work that I did involving automatic deobfuscation of obfuscated control flow constructs with abstract interpretation.  Considering the image below, this project is responsible for taking graphs like the one on the left (where most of the «conditional» branches actually only go in one direction and are only present to thwart static analysis) and converting them into graphs like the one on the right.

Much work on deobfuscation relies on pattern-matching at least to some extent; I have coded such tools myself.  I have some distaste for such methods, since they stop working when the patterns change (they are «syntactic»).  I prefer to code my deobfuscation tools as generically («semantically») as possible, such that they capture innate properties of the obfuscation method in question, rather than hard-coding individual instances of the obfuscation.

The slides present a technique based on abstract interpretation, a form of static program analysis, for deobfuscating control flow transfers.  I translate the x86 code into a different («intermediate») language, and then perform an analysis based on three-valued logic over the translated code.  The end result is that certain classes of opaque predicates (conditional jumps that are either always taken or always not taken) are detected and resolved.  I have successfully used this technique to break several protections making use of similar obfuscation techniques.

Although I invented and implemented these techniques independently, given the wealth of work in program analysis, it wouldn’t surprise me to learn that the particular technique has been previously invented.  Proper references are appreciated.

Code is also included.  The source relies upon my Pandemic program analysis framework, which is not publicly available.  Hence, the code is for educational purposes only.  Nonetheless, I believe it is one of very few examples of publicly-available source code involving abstract interpretation on binaries.

PPTX presentationOCaml source code (for educational purposes only — does not include my framework.)

Reverse Engineering Advanced Programming Concepts

BOLO: Reverse Engineering — Part 2 (Advanced Programming Concepts)

Preface

Throughout this article we will be breaking down the following programming concepts and analyzing the decompiled assembly versions of each instruction:

  1. Arrays
  2. Pointers
  3. Dynamic Memory Allocation
  4. Socket Programming (Network Programming)
  5. Threading

For the Part 1 of the BOLO: Reverse Engineering series, please click here.

Please note: While this article uses IDA Pro to disassemble the compiled code, many of the features of IDA Pro (i.e. graphing, pseudocode translation, etc.) can be found in plugins and builds for other free disassemblers such as radare2. Furthermore, while preparing for this article I took the liberty of changing some variable names in the disassembled code from IDA presets like “v20” to what they correspond to in the C code. This was done to make each portion easier to understand. Finally, please note that this C code was compiled into a 64 bit executable and disassembled with IDA Pro’s 64 bit version. This can be especially seen when calculating array sizes, as the 32 bit registers (i.e. eax) are often doubled in size and transformed into 64 bit registers (i.e rax).

Ok, Let’s begin!

While Part 1 broke down and described basic programming concepts like loops and IF statements, this article is meant to explain more advanced topics that you would have to decipher when reverse engineering.

Arrays

Let’s begin with Arrays, First, let’s take a look at the code as a whole:

Basic Arrays — Code

Now, let’s take a look at the decompiled assembly as a whole:

Basic Arrays — Decompiled assembly overview

As you can see, the 12 lines of code turned into quite a large block of code. But don’t be intimidated! Remember, all we’re doing here is setting up arrays!

Let’s break it down bit by bit:

Declaring an array with a literal — disassembled

When initializing an array with an integer literal, the compiler simply initializes the length through a local variable.

EDIT: The above photo labeled “Declaring an array with a literal — disassembled” is actually labeled incorrectly. While yes, when initializing an array with an integer literal the compiler does first initialize the length through a local variable, the above screenshot is actually the initialization of a stack canary. Stack Canaries are used to detect overflow attacks that may, if unmitigated, lead to execution of malicious code. During compilation the compiler allocated enough space for the only litArray element that would be used, litArray[0] (see photo below labeled “local variables — Arrays” — as you can see, the only litArray element that was allocated for is litArray[0]). Compiler optimization can significantly enhance the speed of applications.
Sorry for the confusion!

local variables — Arrays
Declaring an array with a variable — code
Declaring an array with a variable — assembly
declaring an array with pre-defined objects — code
declaring an array with pre-defined objects — assembly

When declaring an array with pre-defined index definitions the compiler simply saves each pre-defined object into its own variable which represents the index within the array (i.e. objArray4 = objArray[4])

initializing an array index — code

 

initializing an array index — assembly

 

Much like declaring an array with pre-defined index definitions, when initializing (or setting) an index in an array, the compiler creates a new variable for said index.

retrieving an item from an array — code

 

retrieving an item from an array — assembly

 

When retrieving items from arrays, the item is taken from the index within the array and set to the desired variable.

creating a matrix with variables — code

Creating a matrix with variables — assembly

When creating a matrix, first the row and column sizes are set to their row and col variables. Next, the maximum and minimum indexes for the rows and columns are calculated and used to calculate the base location / overall size of the matrix in memory.

inputting to a matrix — code
inputting to a matrix — assembly

When inputting into a matrix, first the location of desired index is calculated using the matrix’s base location. Next, the contents of said index location is set to the desired input (i.e. 1337).

Retrieving from a matrix — code
Retrieving from a matrix — assembly

When retrieving from a matrix the same calculation as performed during the input sequence for the matrix index is performed again but instead of inputting something into the index, the index’s contents are retrieved and set to a desired variable (i.e. MatrixLeet).

Pointers

Now that we understand how arrays are used / look in assembly, let’s move on to pointers.

ointers — Code

Let’s break the assembly down now:

int num = 10 in assembly

First we set int num to 10

.

pointer = &num

Next we set the contents of the num variable (i.e. 10) to the contents of the pointer variable.

printf num — assembly

We print out the num variable.

printf *pointer — assembly

We print out the pointer variable.

printf address of num — assembly

We print out the address of the num variable by using the lea (load effective address) opcode instead of mov.

printf address of num using pointer variable — assembly

We print the address of the num variable through the pointer variable.

rintf address of pointer — assembly

we print the address of the pointer variable using the lea (load effective address) opcode instead of mov.

Dynamic Memory Allocation

The next item on our list is dynamic memory allocation. In this tutorial I will break down memory allocation using:

  1. malloc
  2. calloc
  3. realloc

malloc — dynamic memory allocation

First, let’s take a look at the code:

Dynamic memory allocation using malloc — code

 

In this function we allocate 11 characters using malloc and then copy “Hello World” into the allocated memory space.

Now, let’s take a look at the assembly:

Please note: Throughout the assembly you may see ‘nop’ instructions. these instructions were specifically placed by me during the preparation stage for this article so that I could easily navigate and comment throughout the assembly code.

dynamic memory allocation using malloc — assembly

When using malloc, first the size of the allocated memory (0x0B) is first moved into the edi register. Next, the _malloc system function is called to allocate memory. The allocated memory area is then stored in the ptr variable. Next, the “Hello World” string is broken down into “Hello Wo” and “rld” as it is copied into the allocated memory space. Finally, the newly copied “Hello World” string is printed out and the allocated memory is freed using the _free system function.

calloc — dynamic memory allocation

First, let’s take a look at the code:

dynamic memory allocation using calloc — code

Much like in the malloc technique, space for 11 characters is allocated and then the “Hello World” string is copied into said space. Then, the newly relocated “Hello World” is printed out and the allocated memory space is freed.

dynamic memory allocation using calloc — assembly

Dynamic memory allocation through calloc looks nearly identical to dynamic memory allocation through malloc when broken down into assembly.

First, space for 11 characters (0x0B) is allocated using the _calloc system function. Then, the “Hello World” string is broken down into “Hello Wo” and “rld” as it is copied into the newly allocated memory area. Next, the newly relocated “Hello World” string is printed out and the allocated memory area is freed using the _free system function.

realloc — dynamic memory allocation

First, let’s look at the code:

dynamic memory allocation using realloc — code

In this function, space for 11 characters is allocated using malloc. Then, “Hello World” is copied into the newly allocated memory space before said memory location is reallocated to fit 21 characters by using realloc. Finally, “1337 h4x0r @nonymoose” is copied into the newly reallocated space. Finally, after printing, the memory is freed.

Now, let’s take a look at the assembly:

Please note: Throughout the assembly you may see ‘nop’ instructions. these instructions were specifically placed by me during the preparation stage for this article so that I could easily navigate and comment throughout the assembly code.

dynamic memory allocation using realloc — assembly

First, memory is allocated using malloc precisely as it was in the above “malloc — dynamic memory allocation” section. Then, after printing out the newly relocated “Hello World” string, realloc (_realloc system call) is called on the ptr variable (that represents the mem_alloc variable in the code) and a size of 0x15 (21 in decimal) is passed in as well. Next, “1337 h4x0r @nonymoose” is broken down into “1337 h4x”, “0r @nony”, “moos”, and “e” as it is copied into the newly re-allocated memory space. Finally, the space is freed using the _free system call

Socket Programming

Next, we’ll cover socket programming by breaking down a very basic TCP client-server chat system.

Before we begin breaking down the server / client code, it is important to point out the following line of code at the top of the file:

define the Port number

This line defines the PORT variable as 1337. This variable will be used in both the client and the server as the network port used to create the connection.

Server

First, let’s look at the code:

Server — Code

First, the socket file descriptor ‘server’ is created with the AF_INET domain, the SOCK_STREAM type, and protocol code 0. Next, the socket options and the address is configured. Then, the socket is bound to the network address / port and the server begins to listen on said server with a maximum queue length of 3. Upon receiving a connection, the server accepts it into the sock variable and reads the transmitted value into the value variable. Finally, the server sends the serverhello string over the connection before the function returns.

Now, let’s break it down into assembly:

initiating the server variables

First, the server variables are created and initialized.

server = socket(…) — assembly

Next, the socket file descriptor ‘server’ is created by calling the _socket system function with the protocol, type, and domain settings passed through the edxesi, and edi registers respectively.

setockopt(…) — assembly

Then, setsockopt is called to set the socket options on the ‘server’ socket file descriptor.

address initialization — assembly

Next, the server’s address is initialized through adress.sin_familyaddress.sin_addr.s_addr, and address.sin_port.

bind(…) — assembly

Upon address and socket configuration, the server is bound to the network address using the _bind system call.

listen(…) — assembly

Once bound, the server listens on the socket by passing in the ‘server’ socket file descriptor and a max queue length of 3.

sock = accept(…) — assembly

Once a connection is made, the server accepts the socket connection into the sock variable.

value = read(…) — assembly

The server then reads the transmitted message into the value variable using the _read system call.

send(…) — assembly

Finally, the server sends the serverhello message through the variable (which represents serverhello in the code).

Client

First, let’s look at the code: 

Client — code

first, the socket file descriptor ‘sock’ is created with the AF_INET domain, SOCK_STREAM type, and protocol code 0. Next, memset is used to fill the memory area of server_addr with ‘0’s before address information is set using server_addr.sin_family and server_addr.sin_port. Next, the address information is converted from text to binary format using inet_pton before the client connects to the server. Upon connection, the client sends it’s helloclient string and then reads the server’s response into the value variable. Finally, the value variable is printed out and the function returns.

Now, let’s break down the assembly:

Client variable initialization — assembly

First, the client’s local variables are initialized.

sock = socket(…) — assembly

The ‘sock’ socket file descriptor is created by calling the _socket system function and passing in the protocol, type, and domain information through the edxesi, and edi registers respectively.

memset(…) — assembly

Next, the server_address variable (represented as ‘s’ in assembly) is filled with ‘0’s (0x30) using the _memset system call.

Client — address configuration — assembly

Then, the address information for the server is configured.

inet_pton(…) — assembly

Next, the address is translated from text to binary format using the _inet_pton system call. Please note that since no address was explicitly defined in the code, localhost (127.0.0.1) is assumed.

connect(…) — assembly

The client connects to the server using the _connect system call.

send(…) — assembly

Upon connecting, the client sends the helloClient string to the server.

value = read(…)

Finally, the client reads the server’s reply into the value variable using the _read system call.

Threading

Finally, we’ll cover the basics of threading in C.

First, let’s look at the code:

Threading — Code

As you can see, the program first prints “This is before the thread”, then creates a new thread that points to the *mythread function using the pthread_create function. Upon completion of the *mythread function (after sleeping for 1 second and printing “Hello from mythread”), the new thread is joined back the main thread using the pthread_join function and “This is after the thread” is printed.

Now, let’s break down the assembly:

printf “This is before the thread” — assembly

First, the program prints “This is before the thread”.

Creating a new thread — assembly

Next, a new thread is created with the _pthread_create system call. This thread points to mythread as it’s start routine.

The mythread function — assembly

As you can see, the mythread function simply sleeps for one second before printing “Hello from mythread”.

Please note: In the mythread function you will see two ‘nop’s. These were specifically placed for easier navigation during the preparation stage of this article.

joining the mythread function’s thread back to the main thread — assembly

Upon returning from the mythread function, the new thread is joined with the main thread using the _pthread_join function.

printf “This is after the thread” — assembly

Finally, “This is after the thread” is printed out and the function returns.

Closing Statements

I hope this article was able to shed some light on some more advanced programming concepts and their underlying assembly code. Now that we’ve covered all the major programming concepts, the next few articles in the BOLO: Reverse Engineering series will be dedicated to different types of attacks and vulnerable code so that you may be able to more quickly identify vulnerabilities and attacks within closed source programs through static analysis.