Logging https requests and responses of Apple system processes

May 31 2020 by Jeff Johnson

This is a follow-up to Catalina is checking notarization of unsigned executables. In that blog post, I explained how I was trying to compare packet traces of app notarization checks with packet traces of (apparent) shell script notarization checks. The traces looked the same. The request packets even had the exact same number of bytes. The only problem with the comparison was that the requests and responses use https and are thus encrypted. Is there a way of decrypting them?

I talked to someone who attempted MITM on the connections, but that failed because the system process (syspolicyd in this case) uses certificate pinning. So I started to wonder if macOS had some kind of logging facility that could be enabled for this. If you want to log https connections in your own app, you would set the environment variable CFNETWORK_DIAGNOSTICS when debugging. But how would you do that for an Apple system process such as syspolicyd, which runs as root? Fortunately, it turns out to be very easy! In Terminal, enter the following command:

sudo launchctl setenv CFNETWORK_DIAGNOSTICS 3

Then in Activity Monitor, quit the relevant system process. Even though syspolicyd runs as root, it politely quits when you ask. Quitting is necessary so that the process picks up the new environment variable next time it launches. After that, you're good to go! Just open Console apps to see the log messages.

Here are the notarization request headers when I launched a downloaded app for the first time on Catalina:

POST https://api.apple-cloudkit.com/database/1/com.apple.gk.ticket-delivery/production/public/records/lookup HTTP/1.1
Accept: */*
Content-Type: application/json
X-CloudKit-ContainerId: com.apple.gk.ticket-delivery
Accept-Encoding: gzip, deflate, br
Accept-Language: en-us
Content-Length: 77
X-Apple-Cache-Key: PFT-com.apple.gk.ticket-delivery-production-2/2/8d817db79d5c07d0deb7daf4908405f6a37c34b4
Cache-Control: max-age=300

The only real mystery here is the string 8d817db79d5c07d0deb7daf4908405f6a37c34b4. What does it mean? My educated guess was that it's the hash of the app. And my guess was right! In Terminal, the command codesign -d -vvv on the app path (yes, you need at least 3 verbose levels here) showed CDHash=8d817db79d5c07d0deb7daf4908405f6a37c34b4, among other info.

Here are the notarization response headers:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
X-Apple-Cache: false
X-Apple-Request-UUID: 9a6c7444-c6f3-4670-a5ce-ebc3d244fdef
Content-Encoding: gzip
Server: AppleHttpServer/ac9d40131a4d
X-Apple-CloudKit-Version: 1.0
Via: xrail:st13p00ic-zteu25203401.me.com:8301:19C449:grp60,631194250daa17e24277dea86cf30319:bea26b4d7667c62d282dd124414bdb26:Chicago
access-control-expose-headers: X-Apple-Request-UUID,X-Responding-Instance,Via
X-Responding-Instance: ckdatabasews:16308501:st42p63ic-ztfb18191701:8807:2009B656:7e090e9bd90bf91dce9664d7008dcf25f8c77165
Date: Sat, 30 May 2020 22:52:54 GMT
Content-Length: 2040
Apple-Originating-System: UnknownOriginatingSystem
Connection: keep-alive
Strict-Transport-Security: max-age=31536000; includeSubDomains;
apple-tk: false
apple-seq: 0

Unfortunately, the request and response bodies are not logged, just the headers. That's the major shortcoming of CFNETWORK_DIAGNOSTICS. However, the headers still tell us a lot.

What about the shell script notarization check? Decrypted, it looks the same as the app notarization check. The hash value is different, but the request content length is exactly the same. The response content length, on the on the other hand, is different, indeed significantly smaller. I don't see anything in the response headers that indicate the app is notarized and the script isn't, so I'm assuming that the actual verdict on whether they're notarized is contained in the response body JSON, which we don't have decrypted, sadly. Nonetheless, when the headers are decrypted, it's even clearer than it was in the previous blog post that the shell script is undergoing a notification check too.

By the way, using sudo launchctl setenv did not require disabling System Integrity Protection (SIP). I discovered this in further testing, because I started by disabling SIP, just in case. Interestingly, I also discovered that disabling SIP also completely disables the shell script notarization checks! For several minutes during testing I was baffled, because I couldn't even get syspolicyd to relaunch after quitting, and my shell script time tests were showing no delay at all. It wasn't until I tried to launch a regular app again, and syspolicyd also launched again, that I realized the shell script notarization checks were disabled along with SIP. (Security researchers take note: syspolicyd can be quit apparently without elevated privileges, and notarization checks don't occur without syspolicyd. Area for investigation?)

This technique of setting CFNETWORK_DIAGNOSTICS via launchctl should work for any system process, so I encourage you to go spelunking in the system https connections and see how much good stuff (or bad stuff) you can find!

Jeff Johnson (My apps, PayPal.Me)