CHAPTER 04 35 MIN READ INTERMEDIATE

TLS in Practice: Handshakes, Certificates & Interception

If you have worked through the Fundamentals module, you already know how symmetric and asymmetric encryption work, what a hash function does, and roughly how a TLS handshake establishes a shared key. That is the math. This chapter is not about the math. It is about what TLS actually looks like when you open a packet capture: which fields are visible, which are opaque, and what an analyst can pull out of a TLS session without ever touching a private key. That distinction, reading TLS from the wire rather than deriving it from first principles, is a different skill and one you will use constantly in triage, threat hunting, and network monitoring.

TLS handshake SNI interception

The TLS 1.3 Handshake as Captured Packets

Open a packet capture of an HTTPS connection and watch the handshake unfold message by message. Each step below is what you will actually see land on the wire, in order, before the connection settles into ordinary encrypted traffic.

The Handshake, Message by Message

  1. ClientHello, client to server. Sent in plaintext, and unusually information-rich for something exchanged before any key agreement exists. It carries the TLS versions the client supports, the cipher suites it is willing to negotiate, a client-generated random value, its supported elliptic curve groups, and a key_share extension containing the client's ephemeral public key. It also carries the Server Name Indication field, naming the exact hostname the client wants to reach. All of this is readable in Wireshark with zero decryption.
  2. ServerHello, server to client. Also sent in plaintext in TLS 1.3, and smaller than the ClientHello. It confirms the negotiated TLS version, selects one cipher suite from the client's offered list, sends its own random value, and returns its own key_share so both sides can derive the same handshake secret independently.
  3. Encrypted server flight, server to client. The moment ServerHello lands, both endpoints can compute the handshake traffic keys. Everything the server sends next, EncryptedExtensions, its Certificate, CertificateVerify, and its own Finished message, is encrypted under that secret.
  4. Client Finished, client to server. Once both sides have sent their Finished messages, the handshake is complete.
  5. Application data phase. The connection switches to a separate set of application traffic keys derived from the handshake. From here, every record on the wire is opaque ciphertext unless you hold the keys, whether through a server-side key log file, a debugging build of the browser, or a TLS-terminating proxy.

Why TLS 1.3 Hides the Certificate

This is the detail that trips up analysts moving from TLS 1.2 capture habits to TLS 1.3. In TLS 1.2, the server's certificate travels in a clearly labeled, plaintext Certificate message you can click open in Wireshark. In TLS 1.3, the Certificate, CertificateVerify, and server Finished messages are encrypted, and for privacy reasons their outer record header is deliberately mislabeled as type 23, the same type used for application data.

Without session keys loaded into Wireshark, every one of these records just shows up as "Application Data." You cannot tell a handshake-finish message from actual payload by looking at the record type alone.

Faster By Design

A full 1-RTT TLS 1.3 handshake, ClientHello, ServerHello plus the encrypted server flight, and both Finished messages, typically completes in a single round trip. That is why TLS 1.3 connections establish visibly faster in a capture than TLS 1.2 ones did.

MessageSenderVisible in a Capture Without Keys
ClientHelloClientYes, fully plaintext including SNI and cipher list
ServerHelloServerYes, fully plaintext including selected cipher
EncryptedExtensions, Certificate, CertificateVerifyServerNo, shown only as "Application Data"
Finished (server and client)BothNo, shown only as "Application Data"
Application data recordsBothNo, opaque ciphertext

SNI and Certificate Validation

Quick glossary, key terms in this section:
  • SNI (Server Name Indication): the hostname field the client sends in the clear so the server knows which certificate to present.
  • ECH (Encrypted Client Hello): a newer extension that hides SNI behind encryption, not yet the default on most of the web.
  • SAN (Subject Alternative Name): the certificate field a hostname must match; the older CN field is deprecated and ignored by modern browsers.
  • CRL / OCSP: the two mechanisms for checking whether a certificate has been revoked, the second in real time or via a stapled response.

Why SNI Is Plaintext

SNI exists because a single IP address routinely hosts many different HTTPS sites behind a load balancer or CDN. Before the server can pick which certificate to present, it needs to know the target hostname, and it needs that information before a TLS session key exists to encrypt anything.

The result is a genuinely awkward property of an otherwise encrypted protocol: the exact domain name a user is connecting to sits in the clear in the very first packet of the handshake. ECH hides SNI behind an additional layer of encryption, but it is not yet the default on most of the web, so for the majority of traffic today, SNI remains one of the most reliable plaintext fields a network defender has.

SNI's Dual-Use Nature

That plaintext visibility cuts both ways operationally. A proxy, firewall, or DNS-based filtering product can enforce policy on SNI alone, blocking a category of sites without ever decrypting the session. A SOC analyst reviewing full packet capture can pivot on SNI to see every TLS connection made to a suspicious domain, correlate it against DNS logs, and build a timeline, all without needing decryption keys.

The tradeoff is that anyone else on the network path, an ISP, a coffee shop hotspot operator, a nation-state intercept point, gets the same visibility into which sites a user visits, even though the content of those visits stays encrypted.

What Certificate Chain Validation Actually Checks

Certificate validation is the client-side check that determines whether to trust the identity the server just presented. It covers four distinct things, and failing any one of them fails the whole check.

  • Signature chain to a trusted root. Validation walks from the server's leaf certificate up through any intermediate certificates to a root certificate the client already trusts, verifying each signature along the way. If that chain does not terminate at a certificate already sitting in the client's trust store, validation fails outright, exactly what happens with a self-signed certificate nobody has explicitly trusted.
  • Validity window. The certificate's notBefore and notAfter dates must cover the current time.
  • Hostname match. The hostname the client requested, the same one sent in SNI, must appear in the certificate's Subject Alternative Name extension. The older practice of matching against the Common Name field is deprecated; modern browsers ignore CN entirely.
  • Revocation status. The client checks a Certificate Revocation List, queries an OCSP responder in real time, or reads an OCSP staple the server attached during the handshake. A certificate can pass every other check and still be rejected if the issuing CA has revoked it.
Note: Because SNI is plaintext by default, treat it as one of the highest-value fields in any TLS-heavy packet capture. It gives you domain-level visibility into encrypted traffic for free, and it is the field most SNI-based web filters and many detection rules key off directly.

Cipher Suite Negotiation

How Negotiation Happens

Cipher suite negotiation happens entirely inside the two plaintext messages already covered in the handshake walkthrough.

  1. ClientHello lists every cipher suite the client is willing to use, ordered by the client's own preference.
  2. ServerHello picks exactly one from that list. Neither side proposes anything the other did not already advertise support for, so the negotiated suite is always the intersection of what both endpoints support, filtered through whichever side's preference policy wins.

TLS 1.3 Cipher Suites Are Simpler

TLS 1.3 narrowed the field considerably compared to TLS 1.2. Instead of the sprawling combinations of key exchange, authentication, bulk cipher, and MAC algorithm that TLS 1.2 cipher suite names encoded, TLS 1.3 cipher suites only describe the bulk encryption and hash algorithm, things like TLS_AES_128_GCM_SHA256 or TLS_CHACHA20_POLY1305_SHA256.

That is because every TLS 1.3 suite is required to use an AEAD cipher, and every handshake is required to provide forward secrecy through ephemeral key exchange. Key exchange group and signature algorithm are negotiated separately through their own extensions.

What a "Weak Cipher" Finding Actually Means

When a vulnerability scanner flags "weak cipher suite supported" or "deprecated TLS version enabled," it is reporting that the server's ServerHello logic is still willing to select something from an unsafe portion of its supported list if a client asks for it, not that every connection to that server is necessarily insecure.

  • TLS 1.0 or TLS 1.1 still accepted. Exposes the server to protocol-level weaknesses those versions carry, including susceptibility to attacks like POODLE and BEAST that exploited specific block-cipher-mode weaknesses no longer present in TLS 1.2 and 1.3.
  • RC4 or 3DES still offered. Both are ciphers with known cryptographic weaknesses.
  • Static RSA key exchange still offered. Common in older TLS 1.2 configurations, and it means that connection has no forward secrecy: if the server's private key is ever compromised, every past session captured and stored can retroactively be decrypted.

Operationally, these findings usually trace back to compatibility decisions rather than negligence. An organization keeps TLS 1.0 enabled because a legacy payment terminal or an old API client still requires it, and disabling it breaks that integration.

The fix is rarely "just disable it" without checking who still depends on it, but the finding is real. Every legacy cipher or protocol version left enabled is an option an attacker performing a downgrade attempt, or a passive collector hoping to decrypt traffic later, can try to steer a connection toward.

Mutual TLS

One-Way TLS vs Mutual TLS

Standard TLS is one-way by default. The server proves its identity to the client through its certificate chain, but the client proves nothing about its own identity at the TLS layer. Authentication of the human or application on the client side, if it happens at all, is handled afterward at the application layer, usually with a password or a token.

Mutual TLS (mTLS) adds a second identity check in the other direction: the server also demands and verifies a certificate from the client before the session is considered established.

The Extra Handshake Messages

Mechanically, this shows up in the handshake as an extra pair of messages. After the server sends its own certificate, it sends a CertificateRequest message telling the client it needs to authenticate too.

The client responds with its own Certificate message and a CertificateVerify message, a signature proving it holds the private key matching that certificate. The server validates the client's chain exactly the way the client validated the server's: signature chain to a trusted root, valid dates, and whatever identity match policy the server enforces. Only after both directions check out does the connection proceed.

Where mTLS Shows Up

mTLS shows up wherever both sides of a connection need cryptographic proof of who they are talking to, not just encryption. Service mesh architectures in microservices deployments, tools like Istio and Linkerd, issue every workload a short-lived certificate and enforce mTLS on every service-to-service call inside the cluster, so a compromised pod cannot simply call an internal API by reaching the right IP address.

Zero Trust network architectures lean on the same mechanism at a larger scale. Instead of trusting a request because it originated inside a corporate network perimeter, the architecture demands a valid client certificate as one of several factors before granting access, regardless of network location.

The Operational Cost

The operational cost of mTLS is certificate lifecycle management on both sides instead of one. Every client that needs to connect requires its own certificate, which means issuance, rotation, and revocation infrastructure has to scale to every service or device involved, not just to the handful of public-facing servers a standard TLS deployment needs to manage.

That overhead is exactly why mTLS tends to appear inside trust boundaries an organization fully controls, rather than on public-facing consumer traffic.

TLS Interception

A TLS interception proxy, sometimes called a TLS inspection or SSL inspection appliance, sits in the network path and does something that sounds contradictory: it decrypts and inspects encrypted traffic without breaking the connection from the user's point of view. It manages this by splitting one TLS session into two.

How the Split Session Works

  1. The client's TLS connection terminates at the proxy, not at the real destination.
  2. The proxy presents its own certificate for that domain, signed by an internal CA the organization has pushed into every managed endpoint's trust store ahead of time.
  3. The proxy opens a second, separate TLS connection to the actual destination server.
  4. The proxy decrypts the client's traffic and inspects or logs it in plaintext.
  5. The proxy re-encrypts the traffic going out over that second connection to the real destination.

What This Buys the Organization

From the endpoint's perspective, the browser padlock still shows a valid certificate, because the browser trusts the organization's internal CA the same way it trusts a public one. From the proxy's perspective, it now has full visibility into content that would otherwise be opaque: it can run malware scanning against downloaded files, apply data loss prevention rules against outbound uploads, and enforce content filtering based on the actual page content rather than just the SNI hostname.

Organizations deploy this specifically because SNI-only visibility, the kind covered earlier in this chapter, is not enough to catch malware delivered over HTTPS or sensitive data leaving inside an encrypted upload.

The Privacy and Trust Cost

Every connection passing through the proxy has its content exposed to whatever logging, retention, or human review policy the organization has in place, including traffic to banking sites, healthcare portals, or personal accounts an employee happens to access from a managed device. The interception CA's private key becomes an extremely high-value target: anyone who compromises it can mint trusted certificates for any domain as far as that organization's endpoints are concerned.

Certificate pinning, where an application hardcodes or tightly restricts which certificate it will accept for a given service, breaks under interception by design, since the pinned app rejects the proxy's substituted certificate. This is a common cause of specific apps mysteriously failing only on the corporate network.

Warning: TLS interception is a real man-in-the-middle from a purely technical standpoint, deployed deliberately and with the endpoint's trust store cooperating. Its legitimacy rests entirely on the organization owning the endpoint, disclosing the practice, and scoping what gets inspected and retained. The same mechanism used maliciously, without a legitimately trusted CA on the victim's device, is exactly how adversary-in-the-middle attacks against TLS work.

Spotting TLS Anomalies in Traffic

Quick glossary, key terms in this section:
  • Downgrade attack: forcing a connection into an older protocol version or weaker cipher suite so a known weakness can be exploited.
  • On-path attacker: an attacker positioned to interfere with the handshake in transit, without necessarily decrypting it.
  • JA3: a fingerprint of the client's ClientHello, identifying the TLS library or software that generated it.
  • JA3S: the same fingerprinting approach applied to the server's ServerHello.

The Downgrade Attack Pattern

A TLS downgrade attack tries to force a connection into using an older protocol version or a weaker cipher suite than both endpoints would actually prefer, specifically so the attacker can exploit a known weakness that only exists in the downgraded configuration. The classic pattern involves an on-path attacker interfering with the handshake, stripping out the extensions or version indicators a modern client sends, so the server falls back to a legacy negotiation path it still supports for compatibility.

This is precisely why the cipher suite and protocol version findings covered earlier matter operationally: every legacy option a server still accepts is a downgrade target, not just a theoretical weakness.

Spotting It From a Capture

Because a downgrade attack manipulates the plaintext portion of the handshake, ClientHello and ServerHello, before any encryption applies, it is detectable directly from a packet capture without decrypting anything. An analyst watching for downgrade activity looks for:

  • A negotiated TLS version lower than what the client's ClientHello indicated it supports.
  • Repeated handshake failures followed by a retry at a lower version.
  • A cipher suite selection that falls well outside what a legitimate, up-to-date server for that domain would normally offer.

JA3: Fingerprinting the Client

JA3 fingerprinting takes a different but related approach: instead of looking for an attack in progress, it identifies what software actually generated a given ClientHello. JA3 builds a fingerprint from the ordered list of values in the ClientHello, TLS version, cipher suites offered, extensions present, elliptic curves supported, and elliptic curve point formats, then MD5-hashes that concatenated string into a single fixed-length value.

Because different TLS libraries construct their ClientHello messages with subtly different field orderings and option sets, a JA3 hash reliably identifies the client software generating the connection: a specific browser version, a specific Python TLS library, or a specific malware family's custom TLS stack, independent of what domain it is connecting to or what content it sends.

JA3S and Correlating Signals

JA3S does the same thing from the server side, fingerprinting the ServerHello to identify server-side TLS stack behavior. The pairing becomes genuinely useful for detection when the two signals disagree with the story the traffic is trying to tell: a connection whose SNI claims to be a mainstream browser reaching a common cloud service, but whose JA3 hash matches a known malware C2 framework's TLS library rather than any real browser, is a strong behavioral tell.

Malware authors can and do randomize or spoof individual handshake fields to evade a specific JA3 hash, which is why JA3 works best as one signal correlated with others, SNI reputation, connection timing, destination ASN, rather than a standalone verdict.

Key Takeaways

  • ClientHello and ServerHello travel in plaintext even in TLS 1.3, exposing SNI, offered cipher suites, and the negotiated cipher suite to anyone watching the wire, no decryption required.
  • In TLS 1.3, the certificate and both Finished messages are encrypted and disguised as generic "Application Data" records, unlike TLS 1.2 where the certificate was a clearly labeled plaintext message.
  • Certificate validation checks four things: a trusted signature chain to a root CA, current validity dates, a hostname match against the Subject Alternative Name field, and revocation status via CRL or OCSP.
  • A weak-cipher or deprecated-TLS-version scanner finding means the server's ServerHello logic will still select an unsafe option if a client requests it, not that every connection to it is currently compromised.
  • mTLS adds a CertificateRequest and client Certificate/CertificateVerify exchange so both sides authenticate, common in service mesh and Zero Trust architectures where network location alone is not trusted.
  • JA3/JA3S fingerprint the TLS stack generating a handshake from field ordering and options; a mismatch between a connection's claimed identity and its JA3 hash is a strong signal worth correlating with other data.

Knowledge Check

Click an answer to reveal the explanation.

An analyst reviewing a TLS 1.3 packet capture without decryption keys wants to identify the certificate the server presented. What will they find?

TLS 1.3 encrypts the Certificate, CertificateVerify, and Finished messages under the handshake traffic secret immediately after ServerHello, and their record header is deliberately labeled as application data for privacy. Without session keys loaded into the analysis tool, these records are opaque, unlike TLS 1.2 where the certificate traveled in a clearly labeled plaintext message.

A network security team wants to block access to a category of websites without deploying a full TLS interception proxy. Which field lets them do this while the traffic stays encrypted?

SNI travels in plaintext in the ClientHello by default, before any encryption keys exist, because the server needs it to select the correct certificate. That plaintext hostname is exactly what SNI-based filtering products key off, letting them enforce domain-level policy without ever decrypting the session's content.

A connection's SNI claims to be reaching a well-known cloud storage provider, but its JA3 hash does not match any known browser or standard library and instead matches a hash previously associated with a malware family. What does this most strongly suggest?

A mismatch between a claimed destination and the JA3 fingerprint of the client generating the handshake is a classic indicator of malware attempting to blend into legitimate-looking traffic, since the underlying TLS library still leaves a distinguishable fingerprint in how it builds its ClientHello. It is a strong signal, not a standalone verdict, and should be correlated with other data before drawing a final conclusion.