Key Takeaways
- Persistent VPN authentication tokens are equally as vulnerable to session hijacking as browser session cookies and other device-resident credential material.
- Device requirements for VPN access may be defeated by reconstructing profiles from a working device and replaying them with a third-party VPN client.
- Network access, like any other initial access method, is not catastrophic on it’s own. Even without full visibility into the originating endpoint, network-based discovery and lateral movement techniques still offer valuable opportunities for detection and prevention.
Background
With the adoption of modern VPN or SASE solutions, always-on or long-lived VPN sessions is an increasingly common configuration. Less authentication means less user friction, and an always-on solution ensures consistent visibility into network traffic from endpoints. It’s a win win when done correctly.
An always-on or long-lived VPN configuration implies that the device must store some authentication material, i.e., a cookie, that is used by the VPN client to restore the connection without user interaction. Like any other cookie or credential material, this presents an opportunity for an adversary to steal and replay to gain access. In this post, we’ll investigate how one such product, Palo Alto’s GlobalProtect client, makes reasonable but ultimately defeatable efforts to secure such credential material.
Disclaimer
Before we continue:
- Adversaries using VPN for remote access isn’t new. The concept of stealing always-on VPN cookies probably isn’t new either, but I didn’t find much information published on the subject specifically.
- Nothing we’re exploring here is a “vulnerability”, and nothing about always-on VPN solutions like GlobalProtect is inherently broken. In fact, the added layer of protection was likely added in response to CVE-2019–1573. Customers have viable mitigation options to address token theft after the fact.
- This is my first real attempt at reverse engineering something that I considered somewhat complex. I am NOT a trained nor seasoned reverse engineer by any means, so anything misstated, misunderstood, or done inefficiently here is entirely attributable to me being a noob :) (constructive feedback and tips welcome).
With that out of the way, let’s dig in.
Finding the Cookies
Like all good research endeavors, we start with Google and RTFM. From official Palo Alto documentation, we learn that GlobalProtect client stores it’s cookies (called “Portal User Auth Cookies”) in a file in %LOCALAPPDATA%. Specifically: %LOCALAPPDATA%\Palo Alto Networks\GlobalProtect\PanPUAC_<hash>.dat
A quick look at this file in a hex editor reveals that the file is DPAPI protected, as denoted by it’s unique header, 01 00 00 00 D0 8C 9D DF 01 15 D1 11 8C 7A 00 C0 ...
Calling CryptUnprotectData on this file yields another unusable blob of high entropy (likely encrypted) data.
From this, we can make two assumptions:
- These .dat files are probably encrypted and/or serialized in some way before storage.
- The VPN client is using some hardcoded or derived key material to decrypt it.
And into the deep we go…
Reversing Strategy
As a fairly inexperienced reverse engineer, the thought of digging into cryptographic implementation used in closed-source software felt a little daunting. I probably did some things the hard way, but here’s what ultimately helped:
- IDA and Ghidra obviously are not going to decompile 1:1 with readable source, so I used dynamic analysis to help get a better understanding of certain functions. When supplementing static analysis with dynamic analysis, disable ASLR. After attaching to the process in x64dbg, locate the image’s base address and rebase the program accordingly in IDA. This makes it easier to set breakpoints on the functions you want to trace in the debugger.
- Verbose logging FTW! Thankfully, GlobalProtect has very verbose logging that we could use to find and trace relevant functions more easily. Search for debug strings in IDA, find the xrefs, and work backwards from there. This is exactly what I did below.
As we are investigating how the client manages stored cookies, I began by searching in IDA for any strings containing “cookie” or “decrypt.” This produced several results, with two standing out: one pointing to a function called UnserializePortalAuthCookie and another related to the decryption of a file.
Further down in the same function, we see logging that occurs after checking the result of another function call, suggesting that function is probably responsible for decrypting the cookie file, which I renamed accordingly in IDA.
Identifying AES Cipher and Recovering Key
Within the decryption function, we see a string/debug log reference to EVP_EncryptInit_ex and subsequent EVP_DecryptUpdate and EVP_DecryptFinal_Ex calls. Google searching these strings points us to the OpenSSL library, which gives us a hint that OpenSSL is probably statically linked to the service binary. We can confirm this by digging into some of the functions, and comparing them to open source. For example, the EVP_EncryptInit_ex function decompiles almost identically to the source function, confirming our suspicion:
Looking at the overall flow of the EVP function calls (compared to public examples) suggests an AES encryption cipher may be in use. Notably, EVP_EncryptInit_ex and EVP_DecryptInit_ex take the key and IV for initialization, so I set breakpoints on these function addresses to dump these values.
Through this, we recovered a probable key, C4 10 06 BC DB EF 66 83 B2 E7 38 7E A9 48 7A 77 C4 10 06 BC DB EF 66 83 B2 E7 38 7E A9 48 7A 77 and what appears to be a null IV. Notably, the key is a repeating 16 byte pattern.
At this point I just took a guess that it was AES-256-CBC (based on key length) and that turned out to be right. Using Cyberchef to test, we confirmed successful decryption of the portal configuration file.
To recap what we know and need to investigate further:
- The OpenSSL library is used for encryption and decryption.
- Stored .dat files are DPAPI protected and AES-256-CBC encrypted using a consistent key and a null IV.
- The AES-256 key is a repeating 16-byte pattern that does not appear in memory after the encrypt/decrypt functions complete, and is not present in the binary (i.e., this is not a hardcoded key.)
In this case, we should assume some key-derivation-function exists that we need to now find and reproduce.
Investigating Key Derivation
Upon revisiting the decryption function, we see another function is called before decryption which performs some complex byte operations through a series of nested functions. These functions were obtuse and unfriendly to read but luckily, the decompiled code for one of the functions provided a pretty strong hint at what could be happening here:
void __fastcall z_unknown_1(_DWORD *a1)
{
a1[1] = 0;
*a1 = 0;
a1[2] = 1732584193;
a1[3] = -271733879;
a1[4] = -1732584194;
a1[5] = 271733878;
}A quick Google search confirms that these values correspond to MD5 state variables. Recalling the AES key we previously observed, MD5 certainly makes sense as it will produce a 16-byte value. Again, reviewing public code samples for MD5 hashing in C/C++, we identify these functions as MD5_Init, MD5_Update and MD5_Final. With this information, the two key functions can be renamed as follows:
_BYTE *__fastcall z_get_pan_md5(_BYTE *out_buf)
{
int md5_state[24]; // [rsp+20h] [rbp-29h] BYREF
int pann_str[4]; // [rsp+80h] [rbp+37h] BYREF
qmemcpy(pann_str, "pannetwork", 10);
z_md5_init(md5_state);
z_md5_update((char *)md5_state, (char *)pann_str, 10);
z_md5_final(md5_state, out_buf);
return out_buf;
}
_BYTE *__fastcall z_get_md5_key(char *data_ptr, int data_len, _BYTE *out_buf)
{
char *pan_md5; // rax
int md5_state[24]; // [rsp+20h] [rbp-88h] BYREF
z_md5_init(md5_state);
z_md5_update(md5_state, data_ptr, data_len);
pan_md5 = z_get_pan_md5(out_buf);
z_md5_update(md5_state, pan_md5, 16);
z_md5_final(md5_state, out_buf);
return out_buf;
}So, what’s happening here? The code nests MD5 hashing, where the data passed to the z_get_md5_key function is concatenated with the MD5 hash of the hardcoded string pannetwork, then hashed again using MD5. Essentially, it’s performing MD5(input + MD5(“pannetwork”)). To determine what data is being passed into this hashing function, I set a breakpoint on z_get_md5_key and inspected the RCX and RDX registers, which hold the data pointer and data length, respectively.
In the RDX register we see the data length is 0x18 (24 bytes), and 24 byte string passed to the function is 01 04 00 00 00 00 00 05 15 00 00 00 EF C8 89 7F 22 AF 1E 09 04 2D C8 51…
Walking the calls back again in IDA, we find that the pointer passed to z_get_md5_key ultimately comes from a function which retrieves the computer SID using Win32 API calls (GetComputerNameExW -> LookupAccountNameW -> LookupAccountNameW -> ConvertSidToStringSidA), returning the computer SID in it’s hex representation. If any of these calls fail, the function resorts to using a hardcoded string global135protect.
Using python we can confirm that this is in fact the key derivation function:
import hashlib
sidbytes = bytearray([0x01, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x05,
0x15, 0x00, 0x00, 0x00, 0xEF, 0xC8, 0x89, 0x7F,
0x22, 0xAF, 0x1E, 0x09, 0x04, 0x2D, 0xC8, 0x51])
pannetwork_str = "pannetwork"
md5_pannetwork = hashlib.md5(pannetwork_str.encode()).digest()
finalbytes = sidbytes + md5_pannetwork
finalmd5 = hashlib.md5(finalbytes).hexdigest()
print(finalmd5)
c41006bcdbef6683b2e7387ea9487a77
This indeed produces the 16 byte value we see twice repeated in the AES key! So, our final KDF is:
MD5(ComputerSID + MD5("pannetwork")) + MD5(ComputerSID + MD5("pannetwork"))Now is probably a good time to share that I am not the first one to find this. While working on this project a colleague shared an article from Crowdstrike’s research team documenting this _exact_ key derivation function, but used in the context of a EoP exploit in the client’s update process.
https://www.crowdstrike.com/blog/exploiting-escalation-of-privileges-via-globalprotect-part-1/
If anything, we’ve at least confirmed that nothing has changed and showed our work.
Passing the Cookie
With a decrypted session cookie in hand, we can use the `openconnect` client which fully implements GlobalProtect’s protocol, including authentication with a portal auth cookie. After providing the recovered cookie at the prompt, this will authenticate us to the portal and prompt us to select a gateway to connect to (depending on the configuration, as some deployments use the portal as the gateway).
$ sudo openconnect --protocol=gp --user="example\\username" --usergroup=portal:portal-userauthcookie --os=win https://vpn.example.comAnd just like that, we’re on the wire! Except….
Host Information Profile Checks
Authenticating to the VPN is step one, but mature deployments should be expected to check the host against a set of compliance policies using GlobalProtect’s Host Information Profile (HIP) checks. Existing research into circumventing these checks is well established, but with the ability to decrypt data files we can take it a step further. GlobalProtect stores information on Host Information Profile reports in %PROGRAMFILES%\Palo Alto Networks\GlobalProtect :
| File | Purpose |
| --------------------- | ---------------------------------------------------------------------------- |
| HipPolicy.dat | All configured HIP checks performed by the agent (does not contain results). |
| HIP_AM_Report_V4.dat | Anti-malware policy check results. |
| HIP_BC_Report_V4.dat | Backup compliance check results. |
| HIP_DE_Report_V4.dat | Disk encryption check results. |
| HIP_DLP_Report_V4.dat | DLP check results. |
| HIP_FW_Report_V4.dat | Host based firewall check results. |
| HIP_PM_Report_V4 | Patch management check results. |
| PanGPS.log | PanGPS service log (may contain HIP related data). |
| PanGPHip.log | HIP specific log, usually contains full HIP XML profile. |All .dat files are AES encrypted using the same key, so we can just as easily decrypt these files on a compliant host and use them to reconstruct a working profile. We can also use the log file contents to piece together a working profile as well, as in some cases the entire HIP profile XML content is dumped in the PanGPHip.log file.
Luckily for us, openconnect supports passing HIP report data from a shell-script via the --csd-wrapper parameter. Fairly simply, we can take the XML recovered from the log/constructed from the .dat files above, and replace the relevant portions of the hipreport.sh script in the openconnect project. For example, if a compliance policy requires real time windows defender scanning and a recent scan, we might include the following:
<entry name="anti-malware">
<list>
<entry>
<ProductInfo>
<Prod vendor="Microsoft Corporation" name="Windows Defender" version="4.18.2304.8" defver="1.389.187.0" engver="1.1.20300.3" datemon="5" dateday="4" dateyear="2023" prodType="3" osType="1"/>
<real-time-protection>yes</real-time-protection>
<last-full-scan-time>$NOW</last-full-scan-time>
</ProductInfo>
</entry>
</list>
</entry>We can pass this during the connection setup as follows:
$ sudo openconnect --protocol=gp --user="example\\username" --usergroup=portal:portal-userauthcookie --os=win https://vpn.example.com --csd-wrapper ~/tools/custom-hips-profile.shOperationalizing
As EDR capabilities continue to improve, red teams are consistently challenged with identifying new ways to remain evasive on the endpoint. This has led to more widespread adoption of “stay off the land” or “live off the foreign land” tradecraft, reducing our footprint on the endpoint by tunneling our tools through an implant. There may be scenarios where this is impractical, and even tunneling standard offensive tooling like impacket over our C2 ‘s native SOCKS channel could risk detection by some leading EDR products. Novel Active Directory tradecraft such as the myriad of NTLM relay and object-takeover primitives introduced by SpecterOps can also be quite tedious to execute through an implant as it requires chaining multiple port forwards together, and In some cases require bind access to port 445 which is inaccessible in a non-privileged context.
Connecting our red team controlled devices to the VPN solves many of these challenges, and using an established VPN session cookie allows us to do so without requiring user credentials or MFA.
A PoC tool for decrypting and collecting all relevant files from a Windows GlobalProtect installation will be made available some time after this blog post, with countermeasures in the form of Sigma and Yara rules available immediately in the same repo.
https://github.com/rotarydrone/GlobalUnProtect
Defense Guidance
In addition to the countermeasures in the repository which signature the GlobalUnProtect and OpenConnect tools specifically, defenders may consider the following:
- Alert on users with more than a single active VPN connection from multiple locations: assuming users only have a single device, this should be trivial to accomplish using the device serial and IP information as signals.
- Use HIP profiles to prevent connections from unmanaged devices: while we demonstrated that it is possible to defeat HIP policies by relaying or reconstructing a profile from a working and compliant device, this attack scenario can only occur if an endpoint connected to the VPN is compromised. Ensuring that devices connecting to the VPN meet minimum hardening and AV/EDR requirements is still critical in reducing the likelihood of this risk being realized. Combining BYOD with corporate VPN access is a recipe for pain.
- Alert on HIP failures and devices out of compliance: despite demonstrating ways to defeat these policies, it is entirely possible that an attacker will not construct a working profile on the first attempt. Further, an adversary with credential-only access (e.g., through password spraying) will likely be unable to predict the HIP policy requirements.
- Defend and detect in depth: an adversary gaining network access, whether through VPN or other means, shouldn’t be seen as a catastrophic event. They still need to progress towards achieving their objectives. Tactics like network discovery and lateral movement offer numerous opportunities for detection and prevention, even if visibility into the originating endpoint is limited. Enforcing strict network segmentation, maintaining strong credential hygiene, and applying the principle of least privilege can significantly increase friction for attackers within your network.
References
- https://www.riskinsight-wavestone.com/en/2023/01/bypassing-host-security-checks-on-a-modern-vpn-solution/
- https://www.crowdstrike.com/blog/exploiting-escalation-of-privileges-via-globalprotect-part-1/
- https://www.infradead.org/openconnect/globalprotect.html
- https://medium.com/cyesec/no-portals-needed-79995d8f7e62