Introduction

Hypertext Transfer Protocol, commonly abbreviated as HTTP, is an application-level protocol used for communication between clients, servers, proxies, gateways and other web intermediaries.

HTTP defines common semantics for:

• Identifying resources through URIs
• Sending requests
• Returning responses
• Applying request methods
• Communicating status codes
• Transferring representations
• Using request and response fields
• Caching responses
• Authentication and authorization challenges
• Content negotiation
• Conditional requests

HTTP/1.1, HTTP/2 and HTTP/3 preserve the main HTTP semantics but use different wire formats and transport mappings.

In your System Design curriculum, HTTP/1.1, HTTP/2 and HTTP/3 is Topic 3.5 under Networking and Web Request Lifecycle. It follows TLS and precedes cookies, sessions and real-time communication mechanisms. The module includes tracing DNS, TLS and HTTP activity with dig, curl and Wireshark.

Prerequisites

HTTP Semantics

The semantics of an HTTP request are primarily expressed through:

• The request method
• The target resource
• Request fields
• An optional request body

The semantics of an HTTP response are primarily expressed through:

• The status code
• Response fields
• An optional response body

HTTP Request

Conceptual Request

GET /courses/system-design HTTP/1.1
Host: www.example.com
Accept: text/html
User-Agent: ExampleClient/1.0

This request contains:

• The GET request method
• The /courses/system-design request target
• The HTTP version used for the textual representation
• Request fields describing the request context

HTTP Response

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1280
Cache-Control: max-age=300

<!doctype html>
<html>
...
</html>

This response contains:

• The 200 status code
• Response fields
• A blank line separating the field section from the content
• The representation body

HTTP/2 and HTTP/3 carry equivalent semantic components through binary framing rather than transmitting this exact HTTP/1.1 text format.

Common HTTP Methods

Status-code Classes

HTTP Version Overview

HTTP/1.1

HTTP/1.1 represents requests and responses using whitespace-delimited textual start lines and fields. Its message syntax, parsing and connection management are specified separately from the shared HTTP semantics.

HTTP/1.1 Message Structure

Request or status line
        |
        v
Header fields
        |
        v
Empty line
        |
        v
Optional message content

Request-line Structure

METHOD SP REQUEST-TARGET SP HTTP-VERSION

Status-line Structure

HTTP-VERSION SP STATUS-CODE SP REASON-PHRASE

Persistent Connections

HTTP/1.1 supports persistent connections, allowing several request-response exchanges to reuse one transport connection.

Without connection reuse:

Connect
Request 1
Response 1
Close

Connect
Request 2
Response 2
Close


With connection reuse:

Connect
Request 1
Response 1
Request 2
Response 2
Close later

Connection reuse can reduce:

• Repeated TCP connection establishment
• Repeated TLS handshakes
• Repeated congestion-control warm-up
• Connection-management overhead

HTTP/1.1 Message Length

An HTTP/1.1 recipient must determine where one message ends before it can correctly process later messages on a persistent connection.

Message framing can depend on:

• Whether the method or status permits content
• The Content-Length field
• Transfer coding
• Connection closure in applicable cases

Content-Length

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 27

{"status":"success"}

The field communicates the expected decimal number of content octets according to the message semantics.

Chunked Transfer Coding

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked

5
Hello
6
 World
0

Each chunk begins with its size in hexadecimal notation. A zero-size chunk ends the chunked content, followed by applicable trailer processing.

HTTP/1.1 Request Concurrency

HTTP/1.1 does not provide the multiplexing layer available in HTTP/2. Clients commonly use several connections to perform requests concurrently.

Connection 1:
Request A -> Response A

Connection 2:
Request B -> Response B

Connection 3:
Request C -> Response C

Several connections can improve concurrency but create additional transport, TLS, memory and congestion-control overhead.

HTTP/1.1 Pipelining

HTTP/1.1 pipelining allows a client to send several requests without waiting for each preceding response.

Client sends:

Request A
Request B
Request C


Server returns in request order:

Response A
Response B
Response C

A slow first response delays later responses on that same connection. This is application-layer head-of-line blocking.

HTTP/2

HTTP/2 provides an optimized binary expression of HTTP semantics. It adds a framing layer, field compression and multiple concurrent streams on one connection.

HTTP/2 introduces:

• Binary framing
• Multiplexed streams
• Concurrent request-response exchanges
• HPACK field compression
• Stream-level flow control
• Connection-level flow control
• Stream-specific and connection-level error handling

HTTP/2 Binary Framing

HTTP/2 communication is divided into frames. Each frame belongs to a connection and, where applicable, a stream.

HTTP request or response
        |
        v
One or more HTTP/2 frames
        |
        +--> HEADERS frame
        +--> DATA frame
        +--> Other control frames

Common HTTP/2 Frame Types

HTTP/2 Streams

A stream is an independent, bidirectional sequence of HTTP/2 frames within one connection. Each request-response exchange is associated with a stream.

One HTTP/2 connection
|
+-- Stream 1: Request A and Response A
|
+-- Stream 3: Request B and Response B
|
+-- Stream 5: Request C and Response C

Frames from different streams can be interleaved on the connection and reassembled using their stream identifiers.

Multiplexed Frame Sequence

HEADERS  Stream 1
HEADERS  Stream 3
DATA     Stream 1
HEADERS  Stream 5
DATA     Stream 3
DATA     Stream 5
DATA     Stream 1

A slow application response on one stream does not require the server to delay sending available frames for every other stream.

HPACK Field Compression

HTTP fields can be repetitive. HTTP/2 uses HPACK to compress field sections and reduce repeated field data.

Repeated requests:

:method: GET
:scheme: https
:authority: example.com
user-agent: ExampleBrowser
accept: text/html


HPACK:

- Uses indexed representations
- Uses static and dynamic tables
- Encodes repeated field names and values efficiently

HPACK maintains compression state associated with the HTTP/2 connection. Encoders and decoders must keep the required state synchronized.

HTTP/2 Flow Control

HTTP/2 provides flow control for DATA frames at both stream and connection levels.

Stream window:
Limits DATA sent on one stream.

Connection window:
Limits total DATA sent across the connection.

Flow control prevents one endpoint from sending DATA beyond the capacity advertised by the receiver.

HTTP/2 flow control does not replace TCP congestion control. They operate at different layers.

HTTP/2 and TCP Head-of-line Blocking

HTTP/2 removes HTTP/1.1's ordered-response limitation by multiplexing streams. However, all streams normally share one TCP byte stream.

HTTP/2 frames on one TCP connection:

Stream 1 frame
Stream 3 frame
Stream 5 frame
        |
        v
TCP byte stream
        |
        v
One TCP segment is lost
        |
        v
Later TCP bytes wait for recovery

TCP must restore missing byte-stream data before later bytes are delivered in order. Therefore, packet loss can temporarily delay multiple HTTP/2 streams sharing the connection.

HTTP/3

HTTP/3 maps HTTP semantics over QUIC. QUIC provides stream multiplexing, per-stream flow control, connection security and connection-establishment mechanisms over UDP.

HTTP/3 includes:

• HTTP semantics carried over QUIC
• Independent QUIC streams
• Binary HTTP/3 frames
• QPACK field compression
• QUIC connection and stream flow control
• TLS-based security integrated into QUIC
• Connection identifiers
• Support for connection migration under applicable conditions

What Is QUIC?

QUIC is a transport protocol built over UDP. It provides reliable streams, cryptographic protection, flow control, congestion control and transport connection management.

HTTP/3
  |
  v
QUIC streams and HTTP/3 frames
  |
  v
QUIC transport and TLS-based security
  |
  v
UDP datagrams
  |
  v
IP packets

Using UDP does not mean HTTP/3 behaves like a basic unreliable UDP application. QUIC implements transport-level reliability, congestion control and stream management above UDP.

HTTP/3 Streams

HTTP/3 request-response exchanges use QUIC streams. QUIC tracks reliability independently for stream data.

One QUIC connection
|
+-- Request Stream A
|
+-- Request Stream B
|
+-- Request Stream C
|
+-- Control Stream
|
+-- QPACK Encoder Stream
|
+-- QPACK Decoder Stream

Loss affecting data for one stream does not require unrelated stream data to wait for recovery in the same manner as several HTTP/2 streams sharing one TCP byte stream.

Reduced Cross-stream Head-of-line Blocking

Stream A packet is lost.

Stream A:
Waits for missing stream data.

Stream B:
Can continue when its data is available.

Stream C:
Can continue when its data is available.

This improves stream independence during packet loss. Application dependencies can still create waiting, and ordered data within the affected stream still requires the missing data to be recovered.

QPACK Field Compression

HTTP/3 uses QPACK to compress HTTP field sections. QPACK is designed for QUIC's stream model.

QPACK uses:

• A static table
• A dynamic table
• Indexed field representations
• Dedicated encoder and decoder instruction streams
• Mechanisms to manage compression dependencies

QPACK must balance compression efficiency against the risk of a request stream waiting for required dynamic-table state.

HTTP/3 Connection Establishment

QUIC integrates transport and TLS negotiation, allowing transport and security establishment to be coordinated.

Client                                      Server
  |                                            |
  | QUIC initial data and TLS ClientHello      |
  |------------------------------------------->|
  |                                            |
  | QUIC and TLS handshake response            |
  |<-------------------------------------------|
  |                                            |
  | Handshake completion                       |
  |                                            |
  | Protected HTTP/3 streams                   |
  |<==========================================>|

The exact exchange depends on address validation, previous connection state, negotiated parameters and packet loss.

Resumption and Early Data

QUIC can use TLS resumption information and can support early data when the connection and application policies permit it.

QUIC Connection Identifiers

QUIC uses connection identifiers so a connection is not identified only by one fixed combination of source and destination addresses and ports.

Initial path:

Client network A -> Server


Network changes:

Client network B -> Server


Connection identifier:

Allows the server to associate packets
with the existing QUIC connection,
subject to migration validation and policy.

This can support connection migration when the client's network path changes, provided the QUIC connection and endpoint policies permit it.

HTTP/3 Security

QUIC integrates TLS-based cryptographic protection. HTTP/3 is therefore used through the secure QUIC transport rather than an unencrypted HTTP/3 mode.

HTTP/3 still requires:

• Correct certificate validation
• Hostname validation
• Secure protocol configuration
• Application authorization
• Input validation
• Safe retry and replay handling

Detailed Version Comparison

Protocol Negotiation

HTTPS clients and servers can use TLS Application-Layer Protocol Negotiation, or ALPN, to select an application protocol.

Client offers:

h2
http/1.1


Server selects:

h2

HTTP/3 discovery and selection also depend on client support, server configuration and applicable protocol advertisement or connection mechanisms.

Proxies and Version Translation

An HTTP request can pass through several intermediaries using different HTTP versions on different connection segments.

Client
  |
  | HTTP/3
  v
Edge Proxy
  |
  | HTTP/2
  v
Application Gateway
  |
  | HTTP/1.1
  v
Backend Service

The end-user protocol does not prove which HTTP version is used between proxies and backend services.

Intermediaries must correctly translate:

• Request methods and targets
• Status codes
• HTTP fields
• Content framing
• Connection-specific information
• Streaming behaviour
• Errors and cancellation

HTTP Caching across Versions

HTTP caching semantics are shared across HTTP versions. A cache evaluates response fields, request directives, validators and freshness rules rather than selecting a policy solely from the protocol version.

HTTP/1.1 200 OK
Cache-Control: public, max-age=300
ETag: "course-v12"
Content-Type: text/html

The semantically equivalent fields can be carried in HTTP/2 or HTTP/3 field sections.

Retry and Idempotency

None of the HTTP versions guarantees exactly-once application execution.

Client sends POST request.

Server completes the operation.

Connection fails before the client
receives the response.

Client cannot determine from transport
failure alone whether the operation completed.

Applications should use idempotency keys, request identifiers, transactional state and safe retry policies where operations can be repeated after an uncertain outcome.

Request and Response Streaming

HTTP can transfer content incrementally rather than buffering the entire message before sending it.

Producer
   |
   | Generates content portions
   v
HTTP response stream
   |
   | Delivers available data
   v
Client consumer

Streaming design requires:

• Backpressure
• Bounded buffers
• Cancellation
• Timeouts
• Incomplete-message handling
• Content validation
• Clear application framing where required

Flow Control and Backpressure

Protocol flow control does not automatically protect an application from constructing an unbounded queue before data reaches the transport layer.

Performance Model

A simplified web request time can be represented as:

\[ T_{request} = T_{DNS} + T_{connection} + T_{TLS} + T_{request\ transfer} + T_{server} + T_{response\ transfer} \]

The contribution of each component depends on connection reuse, protocol version, packet loss, data size, server location, caching and workload.

When HTTP/2 Can Help

HTTP/2 can be useful when:

• Several requests target the same origin
• Fields are repetitive
• Connection reuse is effective
• Parallel exchanges are required
• Reducing the number of TCP connections is beneficial

HTTP/2 does not automatically improve every workload. Performance can still be limited by application processing, database access, large payloads, packet loss, poor prioritization or an overloaded origin.

When HTTP/3 Can Help

HTTP/3 can be useful when:

• Network packet loss affects multiplexed TCP performance
• Clients frequently change network paths
• Low-latency connection establishment is valuable
• Multiplexed independent streams are required
• Client, server and network environments support QUIC reliably

HTTP/3 can be limited by unsupported clients, blocked UDP traffic, middlebox behaviour, server configuration, CPU cost or fallback behaviour.

Security Considerations

HTTP-version differences do not remove the need for secure application design.

Security controls should include:

• TLS configuration and certificate validation
• Strict request framing
• Field and body-size limits
• Request timeout and rate-limit policies
• Authentication and authorization
• Input validation
• Safe proxy translation
• Protection against decompression resource exhaustion
• Bounded concurrent streams
• Logging without exposing secrets

Request Smuggling

Request smuggling can occur when intermediaries disagree about where one request ends and another begins.

Frontend interpretation:

Request A ends at position X.
Request B begins after position X.


Backend interpretation:

Request A ends at position Y.
Unexpected bytes become another request.

Defences include strict parsing, rejecting ambiguous message framing, normalizing protocol translations and maintaining consistent frontend and backend behaviour.

Compression-related Risks

HTTP/2 and HTTP/3 field compression maintains state and consumes memory and CPU.

Implementations should apply:

• Field-section size limits
• Dynamic-table limits
• Request-count and stream limits
• Processing deadlines
• Protocol-error handling
• Resource monitoring

Test HTTP Versions with curl

Request HTTP/1.1

curl --http1.1 -v https://example.com/

Request HTTP/2

curl --http2 -v https://example.com/

Request HTTP/3

curl --http3 -v https://example.com/

HTTP/2 and HTTP/3 support depends on the installed curl build and its linked protocol libraries. Review curl --version to see supported features.

Display curl Capabilities

curl --version

Show Response Headers

curl -i https://example.com/

Download without Printing the Body

curl -o /dev/null -sS https://example.com/

Measure Request Stages

curl -o /dev/null -sS \
  -w 'version=%{http_version}\ndns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\nfirst_byte=%{time_starttransfer}\ntotal=%{time_total}\n' \
  https://example.com/

Test a Specific Endpoint

curl --resolve example.com:443:192.0.2.50 \
  -v https://example.com/

The hostname remains available for TLS SNI, certificate validation and HTTP authority handling while the selected test address is used.

Inspect ALPN with OpenSSL

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -alpn h2,http/1.1

The output can show which application protocol was selected during TLS negotiation. OpenSSL's normal TLS-over-TCP client is not an HTTP/3 client.

Inspect Connections with ss

ss -tnp

ss -unp

TCP socket information can support HTTP/1.1 and HTTP/2 investigation. UDP socket information can support QUIC and HTTP/3 investigation, although identifying the application protocol can require additional evidence.

Capture HTTP-related Traffic

Capture TCP-based HTTPS

sudo tcpdump -i any -nn tcp port 443

Capture QUIC-related UDP Traffic

sudo tcpdump -i any -nn udp port 443

Wireshark Display Filters

http

http2

http3

quic

tls

tcp.port == 443

udp.port == 443

Display-filter support depends on the installed Wireshark version and captured protocol information.

HTTP Troubleshooting Workflow

1. Confirm the complete URL, including scheme, hostname, port and path.
2. Resolve the hostname and record the selected address.
3. Confirm the selected route.
4. Test transport connectivity.
5. Inspect the TLS handshake and certificate.
6. Record the negotiated application protocol.
7. Send the request with verbose HTTP output.
8. Record status code, fields and redirect behaviour.
9. Measure DNS, connect, TLS, first-byte and total times separately.
10. Compare HTTP/1.1, HTTP/2 and HTTP/3 where supported.
11. Inspect proxy and backend protocol versions separately.
12. Capture packets only when lighter diagnostic evidence is insufficient.

Scenario: HTTP/2 Is Not Negotiated

curl --http2 -v https://example.com/

openssl s_client \
  -connect example.com:443 \
  -servername example.com \
  -alpn h2,http/1.1

Investigate:

• Whether the curl build supports HTTP/2
• Whether the client offers h2 through ALPN
• Whether the server selects h2
• Whether a proxy terminates TLS before the origin
• Whether the endpoint is configured for HTTP/2
• Whether the test is using the expected hostname and endpoint

Scenario: HTTP/3 Falls Back

curl --version
curl --http3 -v https://example.com/
ss -unp

Investigate:

• Whether the client supports HTTP/3
• Whether the endpoint supports HTTP/3
• Whether UDP traffic is permitted
• Whether the client discovered the HTTP/3 endpoint
• Whether a proxy or firewall blocks QUIC
• Whether fallback to HTTP/2 or HTTP/1.1 succeeds

Scenario: Slow Web Page

curl -o /dev/null -sS \
  -w 'version=%{http_version}\ndns=%{time_namelookup}\nconnect=%{time_connect}\ntls=%{time_appconnect}\nfirst_byte=%{time_starttransfer}\ntotal=%{time_total}\n' \
  https://example.com/

Investigate:

• DNS latency
• Network connection latency
• TLS handshake latency
• Server time to first byte
• Response transfer time
• Number of resources requested
• Connection reuse
• Multiplexing behaviour
• Packet loss and retransmissions
• Cache effectiveness

Important HTTP Metrics

Common HTTP-version Mistakes

Recommended Test Cases

HTTP Best Practices

Practice Exercise

Compare HTTP/1.1, HTTP/2 and HTTP/3 against an approved test endpoint.

Tasks

1. Confirm which HTTP versions the installed curl build supports.
2. Resolve the endpoint hostname.
3. Record the selected address and route.
4. Inspect TLS and ALPN negotiation.
5. Request the same resource with HTTP/1.1.
6. Request the same resource with HTTP/2.
7. Request the same resource with HTTP/3 where supported.
8. Record the negotiated version for every request.
9. Measure DNS, connection, TLS, first-byte and total time.
10. Capture TCP and UDP traffic in an approved environment.
11. Introduce controlled latency and packet loss in a test environment.
12. Compare concurrent resource-transfer behaviour.
13. Test HTTP/3 fallback when UDP is unavailable.
14. Document the client-facing and backend protocol versions separately.

Comparison Template

Frequently Asked Questions