Deep Dive into TLS Handshakes and Certificate Inspection

Transport Layer Security (TLS) is the backbone of encrypted communication on the web—powering HTTPS, secure emails, and VPNs. But what exactly happens during a TLS handshake, and how can you analyze it using Wireshark?

In this guide, we’ll break down each phase of the TLS handshake, show how to inspect certificates in Wireshark, and highlight what to look for when diagnosing issues or verifying security posture.

What is the TLS Handshake?

The TLS handshake is a negotiation process between a client (like a browser) and a server to establish a secure, encrypted connection.

Typical TLS Handshake Flow (TLS 1.2)

  • ClientHello: Client sends supported TLS versions, cipher suites, random value, and SNI (Server Name Indication).
  • ServerHello: Server selects TLS version, cipher suite, sends random value, and optionally its certificate.
  • Certificate: Server presents its certificate chain for authentication.
  • ServerHelloDone: Indicates server is done with initial negotiation.
  • ClientKeyExchange: Client sends key material for session encryption.
  • ChangeCipherSpec: Client and server switch to encrypted communication.
  • Finished: Both parties exchange a hash of the handshake to confirm success.

TLS 1.3 simplifies this process significantly by reducing round trips and encrypting more handshake data.

Capturing the TLS Handshake in Wireshark

Capture only TLS traffic to reduce noise:

tcp port 443

Open a browser and visit a site like https://example.com. Watch the packets appear.

Step-by-Step Analysis in Wireshark
  • Locate the ClientHello

    • Apply the display filter:

    tls.handshake.type == 1

    This shows only ClientHello packets.

    Look for:

    • TLS Version (e.g., TLS 1.2, TLS 1.3)
    • Supported Cipher Suites
    • Extensions like:
      • server_name (SNI)
      • supported_groups (Elliptic curve preferences)
      • signature_algorithms

    SNI is useful for identifying which domain the client is contacting in multi-host environments.

  • Examine the ServerHello

    • Filter:

    tls.handshake.type == 2

    This shows ServerHello packets.

    Check:

    • Selected cipher suite
    • Selected TLS version
    • Random value (used in key generation)
    • Any extensions returned by the server
  • Inspect the Server Certificate

    • Filter:

    tls.handshake.type == 11

    Expand the "Handshake Protocol: Certificate" section to view:

    • Subject CN (common name of the server)
    • Issuer CN (certificate authority)
    • Validity period
    • Public key algorithm
    • Certificate chain

    Verify:

    • The certificate is not expired
    • It matches the requested domain
    • It's signed by a trusted CA

    Tip: Wireshark decodes X.509 certs nicely—no need to export unless you want to analyze them further (e.g., with openssl).

  • Look for Session Resumption

    • Filter:

    tls.session_id_length > 0

    If the client and server reuse session parameters (session IDs or tickets), this speeds up future connections.

  • TLS 1.3 Differences

    • TLS 1.3 has only one round trip:

    • ClientHello includes almost everything (cipher suites, extensions, and key shares)
    • ServerHello is immediately followed by encrypted data

    To check if TLS 1.3 is used:

    tls.record.version == 0x0304

    and

    tls.handshake.type == 1 && tls.handshake.version == 0x0304
Detecting Problems or Weaknesses
  • Detecting Problems or Weaknesses

    • If the server supports or selects outdated ciphers (e.g., RC4, 3DES), that’s a red flag.

    Look under:

    ClientHello → Cipher Suites

    Or search with:

    tls.handshake.ciphersuite
  • Self-signed or Invalid Certificates

    • If the Issuer CN matches the Subject CN, the cert is self-signed.

    Look for alerts:

    tls.record.content_type == 21

    This filters TLS Alert messages, which may indicate issues like:

    • Unknown certificate authority
    • Expired certificates
    • Unsupported protocol versions
Exporting the Certificate for Manual Inspection

Right-click on the Certificate frame.

Choose Export Packet Bytes.

Save as .der format.

Inspect with:

openssl x509 -in cert.der -inform DER -text -noout
Best Practices for TLS Security

Enforce TLS 1.2 or higher

Use strong cipher suites like AES-GCM and ECDHE

Avoid deprecated features like SSLv3, RSA key exchange, and SHA-1

Ensure valid, trusted certificates with correct SANs (Subject Alternative Names)

Periodically scan public-facing endpoints with tools like SSL Labs