Introduction

After DNS resolves a hostname and routing delivers packets toward the selected destination, an application can use Transport Layer Security, commonly abbreviated as TLS, to establish a protected communication channel.

TLS is designed to protect client-server communication against:

• Passive observation of protected application data
• Unauthorized modification of protected messages
• Message forgery
• Impersonation when peer authentication is correctly configured and validated

TLS is used by HTTPS and can also protect email, database, messaging and application-specific protocols.

In the System Design curriculum, TLS is Topic 3.4 under Networking and Web Request Lifecycle. It follows routing and precedes HTTP/1.1, HTTP/2 and HTTP/3. The module's practical exercise traces DNS, TLS and HTTP activity using tools such as dig, curl and Wireshark.

Prerequisites

What Is TLS?

Transport Layer Security is a protocol for creating a secure channel between communicating peers.

TLS provides three primary security properties:

TLS vs HTTPS

HTTP request and response
          |
          v
TLS-protected records
          |
          v
Underlying transport
          |
          v
IP packets and network frames

TLS does not define HTTP resources, methods, status codes or cookies. HTTP does not independently provide TLS encryption and certificate validation.

TLS in the Request Lifecycle

User enters an HTTPS URL
          |
          v
DNS resolves the hostname
          |
          v
Routing selects a network path
          |
          v
Transport communication begins
          |
          v
TLS handshake
          |
          v
Server certificate validation
          |
          v
Protected HTTP request
          |
          v
Protected HTTP response

A failure at any earlier layer can prevent the TLS handshake from starting. A successful TLS handshake does not prove that the HTTP application will return a successful response.

Cryptographic Building Blocks

TLS combines several cryptographic mechanisms:

Symmetric and Asymmetric Cryptography

TLS does not generally encrypt all application data directly with the server's certificate public key. The handshake establishes shared traffic secrets, after which symmetric authenticated encryption protects the connection's records.

Digital Certificates

A digital certificate associates a public key with identity information and is signed by an issuing authority.

A TLS server certificate can contain:

• Subject identity information
• Subject Alternative Name entries
• Public key information
• Issuer information
• Validity period
• Serial number
• Key-usage restrictions
• Extended key-usage restrictions
• Signature algorithm
• Issuer's digital signature

Simplified Certificate Model

Certificate
+------------------------------------------------+
| Subject and names                              |
| Public key                                     |
| Issuer                                         |
| Validity period                                |
| Usage restrictions                             |
| Serial number                                  |
| Issuer signature                               |
+------------------------------------------------+

Certificate Chain

A server commonly sends its end-entity certificate and one or more intermediate certificates. The client attempts to build a valid chain to a trusted root.

Trusted Root Certificate
           |
           | Signs or authorizes
           v
Intermediate CA Certificate
           |
           | Signs
           v
Server Certificate
           |
           | Contains server public key
           v
TLS Endpoint

The trusted root is commonly obtained from the client's configured trust store rather than being trusted merely because the server transmits it.

Certificate Validation

A TLS client performs several checks before accepting a server certificate.

Validation can include:

• Building a chain to an accepted trust anchor
• Verifying certificate signatures
• Checking the certificate's validity period
• Matching the requested hostname against permitted certificate names
• Checking key-usage and extended-key-usage restrictions
• Applying certificate-policy requirements
• Evaluating revocation information according to client policy
• Rejecting unsupported or invalid cryptographic parameters

Hostname Validation

Requested hostname:

api.example.com


Certificate names:

api.example.com
www.example.com


Result:

api.example.com is represented by
an applicable certificate name.

The certificate is signed by a trusted authority,
so accept it for every hostname.

Verify both:

1. The certificate chains to an accepted trust anchor.
2. The requested hostname matches an applicable certificate name.

Certificate Validity Period

Certificates have a defined validity period. The client evaluates the certificate against an applicable current time.

Not valid before:
Certificate cannot yet be accepted for its intended use.

Valid period:
Certificate can be evaluated with the remaining checks.

Not valid after:
Certificate has expired.

Incorrect client or server clocks can contribute to certificate-validity errors. Time synchronization is therefore an operational dependency.

Certificate Revocation

A certificate can need to be rejected before its normal expiration because of key compromise, incorrect issuance or another policy reason.

Revocation-related mechanisms can include:

• Certificate Revocation Lists
• Online Certificate Status Protocol responses
• Stapled status information
• Client- or platform-specific revocation policies

Actual revocation behaviour depends on the client, operating system, application and configured policy.

TLS 1.3 Handshake

The TLS 1.3 handshake negotiates protocol parameters, establishes shared key material, authenticates the server in the normal server-authenticated case and confirms that both sides derived the expected handshake state.

Client                                      Server
  |                                            |
  | ClientHello                                |
  | Supported versions                         |
  | Supported algorithms                       |
  | Key share                                  |
  | Server name and application protocols      |
  |------------------------------------------->|
  |                                            |
  | ServerHello                                |
  | Selected version and parameters            |
  | Selected key share                         |
  |<-------------------------------------------|
  |                                            |
  | EncryptedExtensions                        |
  | Certificate                                |
  | CertificateVerify                          |
  | Finished                                   |
  |<-------------------------------------------|
  |                                            |
  | Validate certificate and handshake         |
  |                                            |
  | Finished                                   |
  |------------------------------------------->|
  |                                            |
  | Protected application data                 |
  |<==========================================>|

The exact messages depend on the authentication method, resumption, optional client authentication and negotiated extensions.

ClientHello

The ClientHello begins TLS negotiation and communicates capabilities and parameters supported by the client.

It can include:

• Supported TLS versions
• Supported cipher suites
• Supported cryptographic groups
• One or more key shares
• Signature algorithms
• Server Name Indication
• Application-Layer Protocol Negotiation values
• Session-resumption information
• Other TLS extensions

ServerHello

The ServerHello communicates the parameters selected by the server, including the negotiated TLS version, cipher suite and applicable key exchange information.

Client offers:

TLS versions:
- TLS 1.3
- TLS 1.2

Supported cipher suites:
- Suite A
- Suite B

Supported key groups:
- Group X
- Group Y


Server selects:

TLS 1.3
Suite B
Group X

Negotiation succeeds only when client and server have compatible supported parameters.

Key Exchange

TLS 1.3 commonly uses ephemeral Diffie-Hellman key exchange methods. The peers exchange public key shares and independently calculate shared secret material.

Client ephemeral private value
        +
Server public key share
        |
        v
Shared secret material


Server ephemeral private value
        +
Client public key share
        |
        v
Same shared secret material

The private ephemeral values are not transmitted. Derived secrets are processed through the TLS key schedule to produce handshake and application traffic keys.

Forward Secrecy

Ephemeral key exchange supports forward-secrecy properties. Compromise of a server's long-term certificate private key should not by itself reveal previously established session traffic keys created through secure ephemeral key exchange.

Forward secrecy does not protect a connection when:

• An endpoint was already compromised during the session
• Plaintext application data was logged or stored elsewhere
• Traffic keys were captured from endpoint memory
• The negotiated implementation or cryptographic mechanisms were defective

CertificateVerify

The CertificateVerify message contains a digital signature over applicable handshake transcript information.

In a typical server-authenticated handshake, this proves that the server controls the private key corresponding to its certificate's public key and binds the certificate identity to the current handshake.

Finished Message

The Finished message authenticates the handshake transcript using a key derived during the handshake.

Handshake messages
        |
        v
Transcript hash
        |
        v
Finished verification data
        |
        v
Peer verifies consistent handshake state

A successful Finished verification confirms that the peer derived the expected handshake secrets and observed the required transcript.

TLS Record Protection

After traffic secrets are available, TLS protects data through its record protocol.

Application plaintext
        |
        v
TLS record construction
        |
        v
Authenticated encryption
        |
        v
Protected record
        |
        v
Underlying transport

The receiving peer verifies and decrypts the record before delivering plaintext to the application.

Authenticated Encryption

TLS 1.3 uses authenticated-encryption mechanisms to provide confidentiality and integrity for protected records.

Inputs:

- Plaintext
- Traffic key
- Nonce
- Associated protocol data

Outputs:

- Ciphertext
- Authentication information

A receiver rejects a protected record when authentication verification fails.

Server Name Indication

Server Name Indication, commonly abbreviated as SNI, allows a client to indicate the intended server name during TLS negotiation.

One IP address:

192.0.2.50


Hosted names:

api.example.com
shop.example.com
portal.example.net


ClientHello SNI:

shop.example.com


Server:

Selects configuration and certificate
for shop.example.com

SNI supports hosting several secure names on one network address. The server must still return a certificate valid for the requested hostname.

Application-Layer Protocol Negotiation

Application-Layer Protocol Negotiation, abbreviated as ALPN, allows the client and server to select the application protocol that will use the protected connection.

Client offers:

- h2
- http/1.1


Server selects:

- h2


Result:

HTTP/2 communication uses the TLS connection.

ALPN is important when several application protocols can use the same address and port.

TLS 1.2 vs TLS 1.3

TLS Session Resumption

Session resumption allows peers with suitable previous session information to establish a new protected connection using a pre-shared-key-based mechanism.

Initial connection:

Full handshake
      |
      v
Server provides resumption information
      |
      v
Client stores applicable resumption state


Later connection:

Client presents resumption identity
      |
      v
Server accepts resumption
      |
      v
Shorter resumed handshake

Resumption can reduce:

• Handshake computation
• Certificate processing on a resumed connection
• Connection-establishment latency

Resumption state requires expiration, rotation, privacy and compromise controls.

0-RTT Early Data

TLS 1.3 can permit a resuming client to send early application data before the new handshake is fully complete.

Previous resumable session exists
        |
        v
Client sends ClientHello and early data
        |
        v
Server evaluates resumption and early-data policy
        |
        v
Handshake completes

Early data should not be used carelessly for operations such as:

• Creating an order
• Transferring funds
• Changing a password
• Submitting a non-idempotent workflow
• Performing another operation with an irreversible effect

Mutual TLS

In the common HTTPS model, the server presents a certificate and the client validates the server. Mutual TLS, commonly called mTLS, additionally authenticates the client using a client certificate.

Standard server authentication:

Client validates server certificate.


Mutual TLS:

Client validates server certificate.
Server validates client certificate.

mTLS can be used for:

• Service-to-service authentication
• Administrative interfaces
• Managed devices
• Partner integrations
• Applications requiring certificate-based client identity

Certificate authentication does not independently define business authorization. The application must still decide what the authenticated identity is allowed to do.

Authentication vs Authorization

TLS can contribute authentication evidence. Application authorization remains a separate control.

TLS Termination

TLS termination occurs at the component that decrypts the incoming TLS connection and processes or forwards the resulting application data.

Client
  |
  | TLS connection
  v
Load Balancer or Reverse Proxy
  |
  | Decrypted application traffic
  v
Backend Service

The connection from the terminating component to the backend can use:

• Plaintext within an explicitly accepted trust boundary
• A separate TLS connection
• Mutual TLS
• Another authenticated and protected transport

TLS Passthrough

With TLS passthrough, an intermediate network component forwards encrypted transport traffic without terminating the TLS session.

Client
  |
  | End-to-end TLS session
  v
Transport Forwarder
  |
  | Encrypted bytes remain protected
  v
Backend TLS Endpoint

Passthrough preserves TLS termination at the backend but limits the intermediate component's ability to inspect application-layer content.

TLS and Latency

TLS contributes handshake work before protected application data can be exchanged.

A simplified HTTPS latency model is:

\[ T_{HTTPS} = T_{DNS} + T_{transport} + T_{TLS} + T_{HTTP} + T_{application} \]

TLS latency can be affected by:

• Network round-trip time
• Protocol version
• Session resumption
• Certificate-chain size
• Certificate validation
• Cryptographic computation
• Packet loss and retransmission
• Server load
• Remote revocation-information access according to client policy

Persistent connections and resumption can avoid or reduce repeated full-handshake cost.

Certificate Renewal

Certificate renewal should be completed and validated before the active certificate expires.

A certificate-management process should cover:

• Inventory of certificates and owners
• Renewal automation
• Private-key protection
• Deployment validation
• Certificate-chain validation
• Rollback
• Expiry monitoring
• Emergency revocation and replacement

Private-key Protection

The private key is a critical authentication secret. Unauthorized control of the key can enable impersonation where the associated certificate is accepted.

Private-key controls can include:

• Restricting filesystem and administrative access
• Using an approved secrets-management system
• Using hardware-backed key protection where required
• Avoiding plaintext key copies in source control
• Rotating keys according to policy
• Auditing key access
• Maintaining an incident-response process

Application repository
|
+-- source code
+-- server certificate
+-- production-private-key.pem

Application repository:
- Contains no production private key

Approved secret or key system:
- Restricts access
- Audits access
- Supports rotation
- Provides key material only to authorized runtime components

Common TLS Failures

Inspect a TLS Endpoint with OpenSSL

Connect to an HTTPS Endpoint

openssl s_client \
  -connect example.com:443 \
  -servername example.com

The -servername option sends the intended server name for SNI.

Display the Certificate Chain

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -showcerts

Request TLS 1.3

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -tls1_3

Test ALPN Negotiation

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -alpn h2,http/1.1

Display Certificate Dates

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  < /dev/null 2> /dev/null |
openssl x509 -noout -dates

Display Certificate Details

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  < /dev/null 2> /dev/null |
openssl x509 -noout -subject -issuer -serial -fingerprint -sha256

Display Certificate Names

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  < /dev/null 2> /dev/null |
openssl x509 -noout -ext subjectAltName

Replace example names and ports with approved test targets. Output and available options depend on the installed OpenSSL version.

Inspect TLS with curl

Verbose HTTPS Request

curl -v https://example.com/

Measure Request Stages

curl -o /dev/null -sS \
  -w 'dns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\nfirst_byte=%{time_starttransfer}\ntotal=%{time_total}\n' \
  https://example.com/

Test a Specific Address while Preserving the Hostname

curl --resolve example.com:443:192.0.2.50 \
  https://example.com/

This can test a selected endpoint while retaining the URL hostname for SNI, certificate validation and the HTTP host context.

curl -k https://example.com/

Investigate the actual validation error:

- Requested hostname
- Certificate names
- Validity period
- Issuer and chain
- Client trust store
- Server-provided intermediates
- Client and server clocks

Disabling certificate validation can hide the exact trust or identity defect being investigated and should not become an application fix.

Packet Capture and Wireshark

Capture HTTPS Traffic

sudo tcpdump -i any -nn port 443

Wireshark Display Filters

tls

tls.handshake

tls.handshake.type == 1

tls.handshake.type == 2

tcp.port == 443

TLS 1.3 encrypts substantial handshake content after ServerHello. Packet capture without approved session-key material does not reveal protected application plaintext.

TLS Troubleshooting Workflow

1. Confirm the exact URL, hostname and port.
2. Resolve the hostname and record the selected address.
3. Confirm route and transport connectivity.
4. Connect with the correct SNI name.
5. Record the negotiated TLS version and cipher suite.
6. Inspect the certificate names and validity period.
7. Inspect the issuer and certificate chain.
8. Verify the client's trust-store configuration.
9. Check ALPN negotiation.
10. Measure handshake time separately from application time.
11. Compare the failing client with a working client.
12. Inspect load balancer, proxy and backend TLS boundaries.

Scenario: Hostname Mismatch

openssl s_client \
  -connect 192.0.2.50:443 \
  -servername api.example.com \
  -showcerts

Investigate:

• The hostname used by the client
• The SNI value sent during the handshake
• The certificate's Subject Alternative Name entries
• The certificate selected by the proxy or server
• Whether several virtual hosts share the same IP address
• Whether the client connected by IP address instead of hostname

Scenario: Incomplete Certificate Chain

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -showcerts

Investigate:

• The end-entity certificate
• The intermediate certificates sent by the server
• The trust anchor available to the client
• The order and relationship of certificates
• Differences between client trust stores

Scenario: TLS Handshake Is Slow

curl -o /dev/null -sS \
  -w 'dns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\ntotal=%{time_total}\n' \
  https://example.com/

Investigate:

• Network round-trip time
• Transport retransmissions
• Full handshake versus session resumption
• Certificate-chain size
• Server CPU and load
• Cryptographic parameter compatibility
• Revocation-processing behaviour
• Proxy and TLS-termination layers

Important TLS Metrics

Common TLS Mistakes

Recommended Test Cases

TLS Best Practices

Practice Exercise

Trace and evaluate the TLS handshake for an approved HTTPS endpoint.

Tasks

1. Resolve the endpoint's hostname.
2. Record the selected destination address and route.
3. Confirm that the target TCP port is reachable.
4. Connect with openssl s_client using the correct SNI name.
5. Record the negotiated TLS version.
6. Record the negotiated cipher suite.
7. Inspect the server certificate's names.
8. Inspect the issuer and certificate chain.
9. Record the certificate validity period.
10. Record the selected ALPN protocol.
11. Measure TLS connection time using curl.
12. Capture the handshake in an approved test environment.
13. Identify the TLS-termination component.
14. Document how traffic is protected beyond that component.

Result Template

Requested hostname:
<Fully qualified domain name>

Resolved destination:
<IPv4 or IPv6 address>

Port:
<TLS service port>

TLS endpoint:
<Server, proxy, gateway or load balancer>

Negotiated TLS version:
<Observed version>

Cipher suite:
<Observed suite>

Server certificate names:
<Subject Alternative Name entries>

Issuer:
<Certificate issuer>

Validity period:
<Not before and not after values>

Certificate chain:
<End-entity and intermediate certificates>

Trust result:
<Accepted or rejected with reason>

ALPN result:
<Selected application protocol>

TLS connection time:
<Measured time>

Termination boundary:
<Where TLS is decrypted>

Backend protection:
<TLS, mTLS, plaintext in approved boundary or other>

Observed failure:
<Optional certificate, protocol or connectivity failure>

Conclusion:
<Evidence-based explanation>

Frequently Asked Questions