Overview

Previously, I wrote an article introducing how HTTPS handshake works: HTTPS Handshake Process. However, that article was purely theoretical, describing how the HTTPS handshake process works in theory. Although it was quite helpful to me, when I actually analyzed HTTPS packets, I found that there was still a slight gap between theory and reality. So today, I will introduce it from a practical perspective, starting with real packets to see the entire process of HTTPS handshake.

Retracing the HTTPS Handshake Process

Before we dive into the practical part, let’s review the theoretical handshake process:

  1. Client generates a random number 1, sends the protocol version, supported cipher suites, and the random number to the Server by sending a hello packet.
  2. Server generates a random number 2, responds to the Client with the preferred cipher suite, server certificate, and random number 2 (in plaintext).
  3. Client generates a random number 3, the so-called “pre-master secret,” and sends it to the server encrypted with the public key in the server certificate.
  4. Server decrypts and obtains the “pre-master secret” using the private key (the only time the private key is used).
  5. Both Client and Server have client random number, server random number, and pre-master secret. Combining these three inputs produces the “session master secret,” which is used to encrypt all subsequent communication during the session.

Now let’s start with an overview of packet capture. I feel that this packet was captured very well because it matches the theoretical process one by one (😂):

Figure 1: HTTPS Handshake Process
Figure 1

Practical Handshake Process

Client Hello

Figure 2: Client Hello
Figure 2

So, the Client Hello is also a normal TCP packet, and the Server will respond with an ack response (without data).

Server Hello

Figure 3: Server Hello
Figure 3

Similarly, the Server Hello is also a normal TCP packet, and the Client will respond with an ack response.

Master Secret

Figure 4: Master Secret
Figure 4

Data Transmission

Figure 5: Data Transmitted over HTTPS
Figure 5

Advanced Features

Decrypting Packets

From the previous packets, we can see that packets 24 and 26 are labeled as Application Data. However, because they are encrypted, we cannot know their contents. This is important for debugging, as we are concerned with things like whether the HTTP header values are correct, what the values are, and how long the body is.

However, I did not find a convenient way to decrypt them. One verified method is to set an environment variable in the browser and obtain an sslkey log file for decryption:

  1. Close the browser you want to use for packet capture, such as Chrome or Firefox.
  2. Open a terminal and set the environment variable: export SSLKEYLOGFILE="/home/liuliqiang/ssl/sslkey.log"
  3. Open the browser in the terminal (must be in the same terminal as the environment variable for it to take effect):
    • For Mac: open /Applications/Firefox.app &
  4. Run tcpdump, for example: sudo tcpdump -s0 -nn -w decrypt-ssl.pcap dst host 192.168.1.3 or src host 192.168.1.3
    • Replace 192.168.1.3 with your server’s address
  5. Visit the address you want to capture in the browser, e.g., “http://3.localdomain.com“
  6. Stop tcpdump. You will find two files in your current directory:
    • sslkey.log: This is the sslkey log file generated by the browser for wireshark decryption.
    • decrypt-ssl.pcap: This is the packet data captured by tcpdump.

Next, open Wireshark:

  1. Open “Preferences…” -> “Protocols” -> “TLS” -> “(Pre)-Master-Secret log filename,” then add our sslkey.log file;

    Figure 6

  2. Open the decrypt-ssl.pcap file, and you will see that the data previously shown as “Application Data” is now in plain text:

    Figure 7

From the details below, we can see that Hypertext Transfer Protocol is under Transport Layer Security, indicating that this is encrypted data, which is now decrypted and viewable by us.

Other Methods

In addition to generating the sslkey log file, the curl command can also generate this file. However, sometimes we may not be able to obtain this file. In such cases, we can only hope that the encryption algorithm used here is relatively weak (compared to RSA):

  1. Open “Preferences…” -> “Protocols” -> “TLS” -> “RSA keys list,” then add our server certificate private key.

    Figure 8

  2. Then you can decrypt it using the private key.

However, I have not verified this method (😂) because I am using Let’s Encrypt certificates, which use the ECDHE_RSA encryption. The advantage of this encryption method is that even if your certificate is leaked, hackers cannot decrypt the HTTPS data they captured before the certificate was leaked, which is a problem with RSA encryption algorithm.

Ref