Original text by Gyorgy Miru

Samsung’s neural processing framework has received a lot of attention from the security community since its introduction. Hardware isolation vulnerabilities have been demonstrated, both on the NPU and DSP cores (1, 2), that could be used to compromise the kernel. The surrounding kernel code was also exploited by multiple researchers to gain local privilege escalation (1, 2). I, too, explored in a previous blog post how a kmalloc overflow within the Samsung NPU kernel driver can be exploited to gain arbitrary kernel read/write access. As a follow up work, I’ve decided to investigate Huawei’s implementation of their neural processing framework. Despite being the second largest vendor on the Android market, recently there have been lot fewer technical papers published about the security of their devices.

In this blog post I will briefly compare the architecture of the two frameworks from an application’s point of view. I will introduce the bugs I found within the Huawei NPU kernel driver, and the methods I used to exploit them, to gain arbitrary read-write kernel access. I will be focusing on the differences between the security posture of modern Huawei and Samsung kernels, with an emphasis on features that aim to prevent exploitation.

Neural Processing Subsystems

Huawei phones are shipped with a dedicated NPU (Neural-network Processing Unit) core since the Kirin 980 SoC. The NPU is part of the HiAI Mobile Computing Platform that allows developers to efficiently compute neural network operations based on offline models. The concepts are very similar to Samsung’s Neural SDK, however there are notable differences between the two subsystems’ architectures.

The Samsung NPU kernel driver was originally accessible to every application and their SDK directly used it to upload and operate on network models. Last year, in the November update, the access to the 

 NPU device was restricted to platform applications and the SDK is no longer available to third party developers. While the NPU has been restricted, the   device is still exposed to all untrusted applications and it was possible to upload arbitrary models and code until certain ioctls were removed in the February 2021 update. I am not aware of any publicly available libraries that utilize the DSP core and I don’t know how much of the original functionality is retained after the removal of these ioctl handlers.

On Huawei phones, the 

 and   devices are present to interact with the NPU kernel driver. The access to these devices is restricted by selinux, and the   system service is the only one that can operate on them. The   service exposes a series of binder APIs that allows network models to be uploaded and executed on the NPU core. These binder APIs are implemented by a client library that is available to application developers. Untrusted applications must rely on these binder APIs to access the features of the subsystem.

The research introduced in this article focuses on the NPU’s kernel driver implementation. The audit and the exploitation of the 

 is not discussed for now. I will assume that the   can be compromised through its binder APIs and the PoC can execute within its context. This is a strong assumption and a very significant difference compared to the Samsung research.

(While we are on the topic of introducing Huawei’s Da Vinci core — did you know that Da Vinci himself has famously suffered a nerve damage episode that has actually limited his abilities to paint, going as far as being a cause for leaving some of his most famous works like the Mona Lisa supposedly unfinished? I didn’t — but the BBC says so, so it must be true. Talk about hitting a nerve…)

Research Setup

Huawei only releases the source code of their kernels upon major version changes, and the source code might lag behind the up-to-date image version in terms of security updates. Nevertheless, the released sources can be modified and built. Once the bootloader is opened up to disable image verification, a custom compiled kernel can be flashed onto the device and booted.

In customizing my research kernel, I have installed the same kmalloc tracer that I used for the Samsung NPU research and instrumented the kernel to facilitate debugging and exploit development. The research was carried out on a P40 Pro (firmware version 10.1.0.176), however all devices with Kirin 980 and Kirin 990 chipsets should have been similarly affected. The final proof-of-concept exploit was verified on the original kernel image.

There is a “slight” issue here with unlocking bootloaders, however. Samsung is a convenient research platform as it supports bootloader unlocking and the flashing of modified kernels. Huawei, on the other hand, stopped providing bootloader unlock codes in 2017 and there is no official way to install altered kernel images on more recent devices. This meant that we needed to find a way to defeat Huawei’s secure boot. See the next section on a brief background on this.

Detour: Huawei Bootrom Exploitation

Over the course of the last year, together with my colleagues, Daniel Komaromy and Lorant Szabo, we have carried out extensive research into Huawei Kirin basebands. Much like with the kernel, our baseband vulnerability research has also eventually ran into the challenge of Huawei’s secure boot limitations.

Luckily, Lorant and Daniel have identified several bootloader vulnerabilities, some in the second stage bootloader and some in the bootrom code itself. In May 2020, we have sent Huawei several reports describing in total six different exploit chains, all of which gained full code execution at the EL3 level, providing full secure boot bypasses and therefore complete control of the SoC. All told, we have written exploits for 970, 980, 990 Kirin chipsets.

After a year, we have submitted our research white paper to Black Hat. Following the acceptance and the announcement of the talk, we did some more research and identified an additional exploitable vulnerability in bootrom code that is separate (found in a different layer of the bootloader stack) from those in the BH submission.

By this time, Huawei has published fixes for our non-bootrom bootloader vulnerabilities months ago. On the other hand, bootrom vulnerabilities are typically assumed “unfixable” — as seen in some infamous examples such as the “checkm8” iPhone bootrom vulnerability. Quite interestingly, following long discussions, Huawei actually identified a way to mitigate the bootrom vulnerabilities as well and published OTA updates for Kirin 990 devices at the end of last month! We want to give a shout-out to Huawei here for a bit of engineering ingenuity in creating a mitigation for bootrom issues as well as for the courage to go for a relatively high risk patch. Their commitment to finding a technical (as opposed to a legal) solution was refreshing to see.

After another month of delay, we are now disclosing the additional bootrom vulnerability that we have identified and reported to Huawei this May.

The advisory contains a brief summary of the vulnerability in the Huawei bootrom’s USB stack. For all additional context regarding our secure boot research (the reverse engineered details of the Huawei secure boot chain, all of our other bootloader vulnerabilities, explanations on how to exploit them for code execution, completing exploitation to gain full control of the device e.g. loading patched kernel images, etc) together with our research into remote baseband exploitation and sandbox escapes from the baseband to the Linux kernel and TrustZone — make sure to check out our presentation (and accompanying white paper) which will be released next week at Black Hat!

Final note: due to Covid reasons, the Black Hat presentation had to be recorded and finalized before we had the opportunity to analyze Huawei’s bootrom fix. Nonetheless, in the Q&A session of the talk, we’ll be able to get into details about Huawei’s final fixes as well — so make sure to join if you would have any questions about that 🙂

Everywhere I Go

During my initial code review I have discovered several bugs within the NPU driver code, including kernel structures disclosed in shared memory (CVE-2021-22389), an any call primitive (CVE-2021-22388), an oob write vmap write (CVE-2021-22412), a kmalloc double free (CVE-2021-22390), and a mmap overlap crash (CVE-2021-22415). (All of these vulnerabilities have been included in Huawei’s June 2021 security update.) The any call primitive is severely restricted by the Clang forward-edge CFI which is used by Huawei kernels. It should also be mentioned, that Huawei uses a fairly strong 

 implementation in their kernel. It verifies the provided source and destination sizes even when they are dynamically calculated. The verification check rejects unreasonably large sizes which negates the impact of many integer arithmetics-based under or overflows.

As a comparison the Galaxy S20 still uses Samsung’s homebrew CFI solution, where each function is preceded by a constant 4 byte tag and function pointer call sites check for these. This ensures that functions are called at their beginning and hijacked function pointers cannot call into the middle of a function. Clang’s CFI provides much stronger guarantees, it classifies the functions based on types and ensures that from each call site only functions with matching type can be called. For a complete description check out the relevant llvm documentation.

Out of these bugs the shared kernel structures and the double free seemed to be the most promising for exploitation purposes. Let’s take a closer look at these.

The Huawei NPU driver implements a custom mmap interface in 

. Based on the provided offset, three different regions can be mapped by the function.

• MAP_L2_BUFF
  : Maps the same “pad” physical page to user space as many times as requested
• MAP_CONTIGUOUS_MEM
  : Physically contiguous memory, cannot be mapped directly from user space
• MAP_INFO_SQ_CQ_MEM
  : Maps four different, preallocated memory regions to user space in 
  devdrv_info_sq_cq_mmap

static int devdrv_npu_map(struct file *filep, struct vm_area_struct *vma)

{

    struct npu_vma_mmapping *npu_vma_map = NULL;



  [...]

    npu_vma_map = (struct npu_vma_mmapping *)kzalloc (sizeof(struct npu_vma_mmapping), GFP_KERNEL); // [1]

    COND_RETURN_ERROR(npu_vma_map == NULL, -EINVAL, "alloc npu_vma_map fail\n");



    dev_id = proc_ctx->devid;

    vm_pgoff = vma->vm_pgoff;

    map_type = MAP_GET_TYPE(vm_pgoff);



    mutex_lock(&proc_ctx->map_mutex);

    NPU_DRV_WARN("map_type = %d memory mmap start. vm_pgoff=0x%lx, vm_start=0x%lx, vm_end=0x%lx\n",

        map_type, vm_pgoff, vma->vm_start, vma->vm_end);

    switch (map_type) {

    case MAP_RESERVED:

        ret = -EINVAL;

        break;

    case MAP_L2_BUFF:

        ret = devdrv_map_l2_buff(filep, vma, dev_id);

        list_add(&npu_vma_map->list, &proc_ctx->l2_vma_list);

        break;

    case MAP_CONTIGUOUS_MEM:

        share_num = MAP_GET_SHARE_NUM(vm_pgoff);

        ret = devdrv_map_cm(proc_ctx, vma, share_num, dev_id);

        list_add(&npu_vma_map->list, &proc_ctx->cma_vma_list);

        break;

    case MAP_INFO_SQ_CQ_MEM:

        ret = devdrv_info_sq_cq_mmap(dev_id, filep, vma); // [2]

        list_add(&npu_vma_map->list, &proc_ctx->sqcq_vma_list);

        break;

    default:

        NPU_DRV_ERR("map_type is error\n");

        ret = -EINVAL;

        break;

    }

    npu_vma_map->map_type = map_type;

    npu_vma_map->proc_ctx = proc_ctx;

    npu_vma_map->vma = vma;



    vma->vm_flags |= VM_DONTCOPY;

    vma->vm_ops = &npu_vm_ops;

    vma->vm_private_data = (void *)npu_vma_map; // [3]

    vma->vm_ops->open(vma);



    mutex_unlock(&proc_ctx->map_mutex);

    if (ret != 0)

        NPU_DRV_ERR("map_type = %d memory mmap failed\n", map_type);



    return ret;

}

Out of these, the third option is the most interesting, the layout of the mapped shared memory is the following:

|___SQ(32MB)___|____INFO(32MB)_____|__DOORBELL(32MB)___|___CQ(32MB)___|

The SQ (Submission Queue), CQ (Completion Queue) and doorbell regions are used to implement ring-buffers to send and receive data to and from the device. The info region however, contains structures that are used by the kernel to manage the device. These are inherently trusted by the kernel and some of them contain raw kernel pointers. The layout of the info memory is as follows:

| 64 * struct devdrv_ts_sq_info | 1 * struct devdrv_ts_cq_info | 2 * 64 * struct devdrv_stream_info |

This kernel memory can be mapped with read-write permissions from user space. Most of the device ioctls interact with these structures providing various kernel primitives. Exploring all of these would not be possible within the constraint of this article, I will only detail how I exploited one of the many resulting primitives.

The other equally strong vulnerability is the kmalloc double free within the same mmap interface. Once the 

 function creates the requested mapping it sets up custom operations for the new virtual memory area (VMA). The VMAs ( ) are structures within the kernel to represent various memory mappings belonging to a process. The   field can be set to a   structure, so a device, that implements the mmap interface, can define custom actions for certain memory events. This can be used to fault in pages, if specific parts of the mapping is accessed, or ensure synchronization with a backing device.

The NPU driver implements two of the custom handlers, the 

 simply logs when the VMA is created, and   is responsible for freeing up the allocated resources. The   function kmalloc allocates a 48 byte   structure at [1]. This is later saved in the   field of the freshly initialized VMA at [2]. The interesting part happens in the close function:

void npu_vm_close(struct vm_area_struct *vma)

{

    struct npu_vma_mmapping *npu_vma_map = NULL;

    struct devdrv_proc_ctx *proc_ctx = NULL;

    COND_RETURN_VOID(vma == NULL, "davinci munmap vma is null\n");



    NPU_DRV_DEBUG("davinci munmap: vma=0x%lx, vm_start=0x%lx, vm_end=0x%lx\n",

        vma, vma->vm_start, vma->vm_end);

    npu_vma_map = (struct npu_vma_mmapping *)vma->vm_private_data;

    COND_RETURN_VOID(npu_vma_map == NULL, "davinci mmap npu_vma_map is null\n");



    proc_ctx = (struct devdrv_proc_ctx *)npu_vma_map->proc_ctx;

    COND_RETURN_VOID(proc_ctx == NULL, "proc_ctx is null\n");



    if(npu_vma_map->map_type > MAP_RESERVED && npu_vma_map->map_type < MAP_MAX) {

        mutex_lock(&proc_ctx->map_mutex);

        list_del (&npu_vma_map->list);

        mutex_unlock(&proc_ctx->map_mutex);

    }

  // [4]

    kfree(npu_vma_map);

    vma->vm_private_data = NULL;

}

Whenever 

 is called the content of the   field is kfreed and the field is cleared ([4]). This behaviour is based on the assumption that virtual memory areas are immutable and the close operation can only be triggered once. In practice this is not the case, when virtual memory mapping is split a new VMA is created and the fields of the original are copied to it. The start and end addresses and size of both VMAs are adjusted to reflect the state after the split. This happens, for example, when part of a memory mapping is unmapped. The kernel first copies and splits the original VMA and calls the close callback only on the part that is being unmapped.

As a result it is possible to force a kfree on the same kernel virtual address arbitrary times by unmapping the mmapped device memory page by page. This can be turned into a use-after-free against any chosen victim that is allocated from the general 

 cache.

List of Hardenings, Hardening of Lists

Throughout the last couple of years there has been an abundance of kernel LPE publications that feature kmalloc double-frees and use-after-frees (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, …). There are well documented techniques, victim structures, spray and overwrite primitives that can be used to build such exploits. Initially, I started working on the shared structure vulnerability as I found it the more interesting bug. However, Huawei rated the double-free vulnerability the least impactful out of the reported bugs, which led me to believe there might be some exciting mitigations that prevent the exploitation of this vulnerability class on Huawei devices. As it turns out there is no such thing, regardless it was a good opportunity to compare the heap hardening features of the two vendor kernels.

As discussed in my previous post, the Galaxy S20 kernel was built without 

 enabled. This hardening feature protects the freelist pointers, that are stored within the freed heap objects, by XORing the list pointer with a cache specific, random value and the address of the free object. On Huawei this kernel configuration is turned on, preventing kmalloc use-after-frees and overflows to hijack the freelist pointer and force allocations from controlled locations.

Unlike Samsung, Huawei also enables the 

 hardening feature. As the name suggests, this randomizes the order in which objects are inserted into the freelist within a slab. As a result subsequent allocations are not necessarily consecutive in memory, instead they are random within the slab pages. This can provide a probabilistic mitigation against kmalloc overflows, where the effectiveness depends on the cache size. However, it does not provide much protection against our double free primitive as both the victim and the overflow object can be sprayed until the freed object is reclaimed.

The figure above shows the traces of consecutive allocations without freelist randomized, while the second figure shows them when the mitigation is enabled

The only Huawei specific heap mitigations I have encountered, are the 

 and   features. These provide a double free detection by tagging freed objects with a random canary. This can detect if a freed object is released again, however if the object is reclaimed by an allocation and freed again through a dangling pointer this protection is ineffective. I consider this more of a debugging than a mitigation option.

I used the multiple-free primitive to overwrite a chosen object within the 

 cache. The steps to achieve this are as straight forward as it gets.

1. Open the 
   /dev/davinci0
    device and mmap at least three pages of it
2. munmap the first pages causing the original 
   struct npu_vma_mmapping
    object to be freed, while retaining the dangling pointer
3. Spray the selected victim object which should reclaim the freed slot within the slab
4. munmap the second mapped page, causing the victim object to be freed through the dangling pointer
5. Spray controlled data with the usual 
   send[m]msg
    technique to overwrite the victim object

I implemented this and found that I was able to overwrite the victim reliably. The only additional trick required was to pin the main thread and the spraying threads on the same CPU core by the 

 syscall. Otherwise the per-CPU freelists could prevent the released object to be reclaimed by the sprayed objects. This was not an issue as the   selinux context has   permission and it is able to set its affinity mask. To complete the exploit using this bug is left as an exercise for the reader, along with the hiaiserver exploit of course 🙂

Exploiting NPU Shared Memory

Going back to shared kernel memory, let’s take a closer look at the structures that are mmapped to user space and how they are used by the kernel. The NPU kernel device relies on so called streams to manage input and output data flows. These streams can be allocated and released through the 

 and   ioctls. At the same time, there can only be 64 sink and 64 non-sink streams allocated at most. For each potential stream a   is reserved in the shared “info” memory region.

| 64 * struct devdrv_ts_sq_info | 1 * struct devdrv_ts_cq_info | 2 * 64 * struct devdrv_stream_info |

These structures are globally assigned to the various clients, who use the device, and they are pre-initialized upon device probe by the 

 function. Whenever a new stream is allocated by the alloc stream ioctl, the next free stream info is reserved for the new stream, and the associated stream id is returned.

struct devdrv_stream_info {

    int id;

    u32 devid;

    u32 cq_index;

    u32 sq_index;

    void *stream_sub; // points to a struct devdrv_stream_sub_info

    int pid;

    u32 strategy;

    u16 smmu_substream_id;

    u8 priority;

    u8 resv[5];

};

For each of these stream info structures there is an associated 

 structure pointed by the   field. These sub info structures are also initialised upon device probe and they reside within a large vmalloc buffer. Before moving on to the actual exploitation it is crucial to understand how these structures are used and managed by the kernel driver.

struct devdrv_stream_sub_info {

    u32 id;

    struct list_head list;

    void *proc_ctx;

};

The sub info is basically a linked list node and it is used to account for the free and allocated streams. There is a global list of free streams and a per-process list of allocated streams. The per-process list is used to clean up after a process, that failed to release all of its allocated resources.

When a client process calls the alloc stream ioctl the following chain of calls is executed. First 

 receives the parameters from user space, then   calls   ([5]) to retrieve a pointer to the next free stream info structure. Inside the   calls   which walks the global free sub info list and removes the first node. Note that the   does not have a back-pointer to its stream info, instead it has an id field. Once the id is allocated   is used to find the address of the stream info for the given id, within the “info” memory.

int devdrv_proc_alloc_stream(struct devdrv_proc_ctx *proc_ctx, u32 *stream_id, u32 strategy)

{

    struct devdrv_stream_info* stream_info = NULL;

    struct devdrv_stream_sub_info *stream_sub_info = NULL;



  [...]

  // [5]

    stream_info = devdrv_alloc_stream(stream_cq_id, strategy);



    stream_info->pid = proc_ctx->pid;

  // [6]

    stream_sub_info = (struct devdrv_stream_sub_info *)stream_info->stream_sub;

    

  if (strategy == STREAM_STRATEGY_SINK) {

  // [7]

        list_add(&stream_sub_info->list, &proc_ctx->sink_stream_list);

        proc_ctx->sink_stream_num++;

    } else {

  // [7]

        list_add(&stream_sub_info->list, &proc_ctx->stream_list);

        proc_ctx->stream_num++;

    }



    NPU_DRV_DEBUG("npu process_id = %d thread_id %d own sink stream num = %d, non sink stream num = %d now \n",

        proc_ctx->pid, current->pid, proc_ctx->sink_stream_num, proc_ctx->stream_num);



    *stream_id = (u32)stream_info->id;



    return 0;

}

In 

 once the stream info is retrieved its   field is accessed ([6]) to obtain a pointer to the associated sub info structure. This is supposed to point to the same sub info that was just unlinked from the global free list, however, as the stream info resides in the shared memory, it can be corrupted by the user space application. The newly retrieved sub info is added back to the process specific allocated stream list at [7].

At this point it is possible to control the 

 pointer of a selected stream info structure, through the user space “info” mapping. If alloc stream is called on the corrupted stream the hijacked sub info is linked into the allocated stream list.

The NPU driver also implements the 

 ioctl, which can be used to mmap a dma capable shared memory buffer into the user space process’s address space. The mapped memory is allocated with   that returns physically contiguous memory from the dma region on arm64. The beauty of this function is that it also maps the memory into the kernel virtual address space, however the returned vmalloc region address is not affected by KASLR, it only depends on the vmalloc allocation order. As the contiguous dma memory is allocated in   on device probe during kernel boot, the order of vmalloc allocations this early is deterministic, thus the kernel virtual address is stable and known. The result is an extremely strong primitive, a memory buffer at a know kernel virtual address, that can be mapped into user space.

The contiguous shared memory can be used to create a fake sub info node while redirecting the 

 pointer of a selected stream info at it. Once this stream is allocated, the fake sub info is linked into the per-process allocated stream list.

The figure denotes the state of the stream info structures after the alloc stream ioctl completes with the corrupted sub info

The per-process list head is contained within the 

 structure which is kmallocated from the kernel heap. After the linking in of the fake sub info the address of this structure is disclosed into the shared memory. The leaked address is a kernel virtual address from the linear mapping region. Before going any further with the exploitation let’s take a look at the different vendor KASLR implementations.

Kernel Memory 101

In order to have a meaningful discussion on the KASLR implementation we must look at the different regions within the virtual memory and how they are used. This will be a very superficial introduction to the ARM64 memory management, for accurate details please consult the kernel sources and documentation. A handy tool to observe the page table block entries and the kernel virtual address space on a running kernel is the 

 file. It can be enabled with the   and   kernel options. An extract from the Huawei kernel is presented below.

---[ Modules start ]---

---[ Modules end ]---

---[ vmalloc() Area ]---

0xffffff8008000000-0xffffff8008010000      64K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff8008011000-0xffffff8008012000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff8008013000-0xffffff8008014000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff8008015000-0xffffff8008016000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff8008017000-0xffffff8008018000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff8008019000-0xffffff800801a000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff800801b000-0xffffff800801c000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff800801d000-0xffffff800801e000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

0xffffff800801f000-0xffffff8008020000       4K PTE     RW NX SHD AF      UXN DEVICE/nGnRE

...

0xffffff8049000000-0xffffff8049001000       4K PTE     RW NX SHD AF      UXN MEM/NORMAL

// Kernel Image

0xffffff8428c80000-0xffffff8428e00000    1536K PTE     ro x  SHD AF  CON   UXN MEM/NORMAL

0xffffff8428e00000-0xffffff842a200000      20M PMD     ro x  SHD AF    BLK UXN MEM/NORMAL

0xffffff842a200000-0xffffff842a260000     384K PTE     ro x  SHD AF  CON   UXN MEM/NORMAL

0xffffff842a260000-0xffffff842a400000    1664K PTE     ro NX SHD AF      UXN MEM/NORMAL

0xffffff842a400000-0xffffff842aa00000       6M PMD     ro NX SHD AF    BLK UXN MEM/NORMAL

0xffffff842aa00000-0xffffff842aba0000    1664K PTE     ro NX SHD AF      UXN MEM/NORMAL

0xffffff842afa0000-0xffffff842b000000     384K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffff842b000000-0xffffff842c400000      20M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffff842c400000-0xffffff842c420000     128K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffff842c420000-0xffffff842c42c000      48K PTE     RW NX SHD AF      UXN MEM/NORMAL



0xffffffbebfdb0000-0xffffffbebfdba000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfdc8000-0xffffffbebfdd2000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfde0000-0xffffffbebfdea000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfdf8000-0xffffffbebfe02000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfe10000-0xffffffbebfe1a000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfe28000-0xffffffbebfe32000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfe40000-0xffffffbebfe4a000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfe58000-0xffffffbebfe62000      40K PTE     RW NX SHD AF      UXN MEM/NORMAL

0xffffffbebfe70000-0xffffffbebfff0000    1536K PTE     RW NX SHD AF      UXN MEM/NORMAL

---[ vmalloc() End ]---

---[ Fixmap start ]---

0xffffffbefe800000-0xffffffbefea00000       2M PMD     ro NX SHD AF    BLK UXN MEM/NORMAL

---[ Fixmap end ]---

---[ PCI I/O start ]---

---[ PCI I/O end ]---

---[ vmemmap start ]---

0xffffffbf2d000000-0xffffffbf35000000     128M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffbf4d000000-0xffffffbf4e000000      16M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

---[ vmemmap end ]---

---[ Linear Mapping ]---

0xffffffcb40000000-0xffffffcb40080000     512K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

// Kernel Image in linear memory

0xffffffcb40080000-0xffffffcb40200000    1536K PTE     ro NX SHD AF      UXN MEM/NORMAL

0xffffffcb40200000-0xffffffcb41e00000      28M PMD     ro NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcb41e00000-0xffffffcb41fa0000    1664K PTE     ro NX SHD AF      UXN MEM/NORMAL

0xffffffcb41fa0000-0xffffffcb42000000     384K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcb42000000-0xffffffcb50000000     224M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL

0xffffffcb50000000-0xffffffcb50c00000      12M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcb50c00000-0xffffffcb50d00000       1M PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcb59300000-0xffffffcb59400000       1M PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcb59400000-0xffffffcb5a000000      12M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcb5a000000-0xffffffcb5e000000      64M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL

0xffffffcb5f100000-0xffffffcb5f200000       1M PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcb5f200000-0xffffffcb60000000      14M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcb60000000-0xffffffcb6c000000     192M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL

0xffffffcb6c000000-0xffffffcb6c200000       2M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL



0xffffffcb6f000000-0xffffffcb6f100000       1M PTE     RW NX SHD AF  CON   UXN MEM/NORMAL



0xffffffcb71060000-0xffffffcb71200000    1664K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcb71200000-0xffffffcb72000000      14M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcb72000000-0xffffffcb80000000     224M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL

0xffffffcb80000000-0xffffffcbc0000000       1G PGD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcbc0000000-0xffffffcbe0000000     512M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL



0xffffffcbf1280000-0xffffffcbf1400000    1536K PTE     RW NX SHD AF  CON   UXN MEM/NORMAL

0xffffffcbf1400000-0xffffffcbf2000000      12M PMD     RW NX SHD AF    BLK UXN MEM/NORMAL

0xffffffcbf2000000-0xffffffcc20000000     736M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL



0xffffffcc40000000-0xffffffcd40000000       4G PGD     RW NX SHD AF    BLK UXN MEM/NORMAL



0xffffffd340000000-0xffffffd360000000     512M PMD     RW NX SHD AF  CON BLK UXN MEM/NORMAL

The virtual address examples are specific to ARM64 with 39 virtual address bits and 4 kb page tables. The first region (0xffffff8000000000 — 0xffffff8008000000) is reserved for kernel modules. It is a 128 MB kernel virtual memory area used to decouple modules from the kernel image thus prevent them from leaking the kernel base address. The second is the vmalloc region (0xffffff8008000000 — 0xffffffbebfff0000), this 256 GB address space is used to provide virtual address for various kernel resources. Pointers acquired with 

,  ,  ,  ,  , etc. are assigned a virtual address from this region. Consecutive virtual addresses don’t necessarily relate to contiguous physical pages, they are not even necessarily backed by RAM. Addresses are not randomized, they are reserved by the   call, that maintains an RB tree of unused areas within this region and uses a first fit algorithm to satisfy requests.

Still within the vmalloc region resides the virtual address of the kernel base image. Here, there is a significant difference between the Galaxy S20 and the P40 Pro. On Samsung, the kernel base image is loaded at a randomized address in the physical memory, with ~12 bits of entropy, however the virtual address is at a constant offset from the physical base. If either the physical or the virtual base is leaked the other can be calculated. On Huawei, the kernel is always loaded at a fixed physical address (0x80000), but the virtual base address is independently randomized with ~20 bits of entropy.

The 2 MB fixmap region (0xffffffbefe800000-0xffffffbefea00000) is used when a virtual address needs to be know at compile time, but the physical address can be changed later. As an example it is used by early ioremaps, when the kernel is already using the MMU, but the entire memory subsystem is not yet initialized. By definition the addresses in this region need to be known and cannot be randomized.

The vmemmap region (0xffffffbf00000000 — 0xffffffc000000000) is used to access the 

 array, where each physical memory page has an associated   entry. The size of this region is 4GB, however the actual size of the page array is significantly smaller (~128 MB). On the Galaxy S20 both the vmemmap and the linear mapping begins at a fixed address and they are not randomized. On the Huawei phones they are both randomized with the same seed, the addresses containing around 8 bits of entropy, consequently the base address of the vmemmap can be calculated from the linear map base and vice versa.

The linear map is used to address the entire physical memory, pages consecutive in the physical memory are also consecutive in this virtual memory range. It is sometimes also referred to as direct mapping or in security publications as physmap. The base address of this region must be gigabyte aligned which makes the address randomization rather coarse. There are also “holes” in this address space as the physical RAM’s bus addresses are not necessarily consecutive and there might be reserved RAM areas, outside of the scope of the kernel. Since the physical address of the kernel is not randomized on Huawei it can be located within the linear memory if the base address of the linear mapping is known.

The kernel divides the physical memory into zones, that can be used to satisfy different types of memory request. The largest zone is used to provide dynamic memory for the kernel whenever it needs to allocate memory for any reason. When the kernel requests consecutive pages with 

 or   their linear address is returned. Besides the kernel’s dynamic data the user space processes’ memory pages can also be accessed through the linear map, when they are swapped in.

Breaking KASLR

The 

 structure is allocated with kmalloc which uses   when creating new slabs thus the leaked address is a linear address. Unless we knew somehow the associated physical address it is not possible to directly calculate the linear map base from this leak. Kernel heap allocations are not deterministic enough to reasonably guess the physical location, in theory the leaked address could be anywhere within the region. The saving grace is the coarse 1 GB alignment of the linear mapping base address. Because of that, we don’t need to know the exact location of the object in the physical memory, instead it is enough to know at which GB offset it resides. Then the base address could be calculated by applying a mask and subtracting the GB offset from the leaked address.

Since the device only has 8 GB of physical memory even the most naive approach would yield a 12.5% success chance. Fortunately, this can be significantly improved by profiling heap allocations and massaging the zone allocator. The kernel uses the buddy allocator to reserve memory from a zone when 

 is called. The buddy allocator can only satisfy requests for power of two number of pages. It does so by maintaining a list of free blocks, searching it for the first large enough block and then splitting the block into halves until it matches the requested size.

The 

 structure is allocated when the   character device is opened. Before doing that, the PoC sprays page size kernel allocations to exhaust scattered free blocks in the zone allocator’s list. Then it sprays the target slab to force the   allocation into a freshly allocated slab. This way it is possible to have the object at a fairly deterministic physical location, where the GB offset can be guessed. The PoC’s success chance of the linear base leak is around 70% and the physical memory profile is model specific.

Of course, the double free vulnerability could be used to leak the actual kernel base virtual address with a very high probability. As explained in Longterm Security’s Exploiting a Single Instruction Race Condition in Binder blog post’s KASLR Leak section, 

 and   objects could be leveraged to do that.

Unsafe Unlink

At this point we have an allocated stream info where the 

 pointer points to a fake sub info structure in the shared memory, and it is linked into the per-process stream list. The address of the   structure is leaked, from which the linear mapping base can be calculated thus the linear address of the kernel image is known.

It is time to look at the stream release implementation. When the stream free ioctl is called, eventually the 

 function is executed. It takes the supplied stream id and finds the associated stream info structure in the “info” shared memory ([8]). Once it gets a hold of the stream it dereferences the   pointer ([9]), then it calls   on it ([10]). The   helper function does a simple double linked list unlink on the sub info node. Finally,   adds back the node into the global free list.

int devdrv_proc_free_stream(struct devdrv_proc_ctx* proc_ctx, u32 stream_id)

{

    struct devdrv_stream_info* stream_info = NULL;

    struct devdrv_stream_sub_info *stream_sub_info = NULL;

  [...]



    dev_id = proc_ctx->devid;

  // [8]

    stream_info = devdrv_calc_stream_info(dev_id, stream_id);

    if (stream_info == NULL) {

        NPU_DRV_ERR("stream_info is NULL. stream_id=%d\n", stream_id);

        return -1;

    }



  // [9]

    stream_sub_info = (struct devdrv_stream_sub_info*)stream_info->stream_sub;



    if (test_bit(stream_id, proc_ctx->stream_bitmap) == 0) {

        NPU_DRV_ERR(" has already been freed! stream_id=%d \n", stream_id);

        return -1;

    }



  // [10]

    list_del(&stream_sub_info->list);



    ret = devdrv_free_stream(proc_ctx->devid, stream_id, &sq_send_count);

    if (ret != 0) {

        NPU_DRV_ERR("npu process %d free stream_id %d failed \n", current->pid, stream_id);

        return -1;

    }

    proc_ctx->send_count += sq_send_count;



    bitmap_clear(proc_ctx->stream_bitmap, stream_id, 1);

    if (stream_id < DEVDRV_MAX_NON_SINK_STREAM_ID) {

        proc_ctx->stream_num--;

    } else {

        proc_ctx->sink_stream_num--;

    }



    NPU_DRV_DEBUG("npu process %d left stream num = %d sq_send_count = %d "

        "(if stream'sq has been released) now\n",

        current->pid, proc_ctx->stream_num, sq_send_count);

    return 0;

}

Samsung kernels are compiled with 

 kernel option set. This causes the   function to be executed when   is called on a   node. The check verifies that   and   pointers both point at the node that is to be deleted. On Huawei there is no such protection, the vanilla   function is called, which is susceptible to the unsafe unlink attack if the unlinked node is controlled.

static inline void __list_del(struct list_head * prev, struct list_head * next)

{

    next->prev = prev;

    WRITE_ONCE(prev->next, next);

}



static inline void list_del(struct list_head *entry)

{

    __list_del(entry->prev, entry->next);

    entry->next = LIST_POISON1;

    entry->prev = LIST_POISON2;

}

Remember that the hijacked sub info node is located in the mmapped memory that the user space can write. With both the prev and next pointer controlled it is possible to write a controlled value at an arbitrary kernel virtual address. The only caveat is that the value also needs to be a valid kernel pointer that can be written. This restricted value write can easily be turned into a truly arbitrary value write the following way. The prev pointer is set to point at the destination that needs to be written. The next pointer (the value to be written) can point into the contiguous shared memory at the known kernel virtual address, after where the sub info node is. This shared memory is more than a page in size, and contains no meaningful data, thus the least significant byte of the value pointer can be set to any selected value. The pointer would still remain within the bounds of the shared memory. The result is a completely controlled byte written at the target location and the next seven bytes trashed by the rest of the pointer value. Fortunately, arm64 supports unaligned access (conditions apply, certain memory types such as device do not support it), therefore the target address can be incremented and the whole process can be repeated.

The final primitive can write arbitrary number of consecutive bytes with a chosen value at a chosen kernel virtual address. The only drawback is that the next seven bytes after the final byte written will be overwritten by uncontrolled values.

Sharing is Caring

With an arbitrary write and known kernel data address there are many ways to finish the exploit. My goal was to reach a convenient, truly arbitrary read-write primitive and explore Huawei’s anti-root solutions later. As we have seen already the NPU kernel driver allocates most of the memory, that can be mapped to user space, upon device probe. As it turns out, the addresses of these different memory regions are stored in a global array.

struct devdrv_mem_info {

    phys_addr_t phy_addr;

    vir_addr_t virt_addr;

    size_t size;

};



struct devdrv_mem_info g_shm_desc[NPU_DEV_NUM][DEVDRV_MAX_MEM];

static struct devdrv_continuous_mem g_continuous_mem[NPU_DEV_NUM];

When the 

 mmap handler is called, it retrieves the physical address and the size of the different regions (sq, info, doorbell) from this array. It calls   to map the selected physical memory into the process’ VMA.

int devdrv_info_sq_cq_mmap(u8 dev_id, const struct file *filep, struct vm_area_struct *vma) {

  [...]

    phy_addr = g_shm_desc[dev_id][DEVDRV_DOORBELL_MEM].phy_addr;

    size = g_shm_desc[dev_id][DEVDRV_DOORBELL_MEM].size;

    COND_RETURN_ERROR(size <= 0, -ENOMEM, "npu dev %d illegal doorbell cfg size = %lu\n", dev_id, size);

    NPU_DRV_DEBUG("npu dev %d doorbell mem:user_virt_addr = 0x%lx, "

        "phys_addr = 0x%llx, size = %lu\n", dev_id, start_addr, phy_addr, size);



    err = remap_pfn_range(vma, start_addr, phy_addr >> PAGE_SHIFT, size, vma->vm_page_prot);

    COND_RETURN_ERROR(err != 0, -EINVAL, "npu dev_id %d doobell register mmap failed\n", dev_id);

  [...]

}

The layout of the 

 is quite lucky for the list unlink primitive as the   field that succeeds the   field is never used for most region types. This way the   can be rewritten and it doesn’t matter that the primitive trashes the next field. Even the size field can be overwritten for the doorbell entry as the next entry in the array is never used. By overwriting the doorbell regions entry in the   array it is possible to mmap any physical memory into the user space for reading and writing. While there is no explicit size check, a consecutive mapping is created 32 MB after the doorbell region, so at most 32 MB can be mapped.

Initially, I attempted to map the entire kernel text and overwrite it through the mapping. The mmap was successful and it was possible to read the kernel text, but write attempts caused unknown bus faults and consequently kernel panics. I strongly suspect that the kernel text is protected by the hypervisor, similarly to Galaxy kernels. I tried the same with the kernel page tables with similar results. Unboxing and comparing the hypervisors is a topic for a whole other blog post.

The PoC mmaps the entire kernel data section. This way, besides having read-write access to the entire kernel data section, it is really easy to reuse this mmap primitive as the 

 is within this map itself. Thus any 32 MB physical memory can be mapped to user space with relative ease.

Can We Get More Arbitrary?

Technically, at this point we already have an arbitrary kernel-read write primitive, however there are ways to improve it. The kernel data/bss section isn’t considered shared memory by any means. When the kernel data is written through the user space mapping the cached kernel addresses are not flushed, thus the kernel does not see the changes immediately. Any time the PoC wants to map a different 32 MB chunk and overwrites the doorbell descriptor it needs to wait until the previous value is evicted from the cache.

The final stage of the exploit walks the 

 structures until it finds its own  . It reads the   field that points to its   descriptor, that contains the address of its top level page table. As it turns out process page tables are not protected against any kind of modification. The PoC modifies its own page table this way to map the entire physical memory into its address space. The result is a truly complete and convenient arbitrary read-write primitive.

Demo

Here is a high definition video capture of the PoC doing its thing on a P40 Pro.

Summary

The emergence of hardware assisted neural computing led to the addition of new subsystems on both Huawei and Samsung devices. As is the case with many new and complex software components both implementation contained exploitable security vulnerabilities. While the identified vulnerabilities were more impactful in the Huawei driver, which led to an arguably less challenging exploit, the access control defaults were stricter on Huawei originally. I was also pleasantly surprised by the kernel mitigation features of the platform. The P40 kernel comes with freelist hardening and randomization enabled, the linear memory is randomized and the secure memcpy implementation is effective at preventing exploitable integer overflows. On the other hand Huawei lacks a linked list hardening feature, like the Galaxy S20’s debug list, that would prevent unlink attacks from a list node corruption.