Introduction

Distributed applications frequently need one service to request work from another service. REST models these interactions primarily around resources and HTTP semantics. Remote Procedure Call, commonly abbreviated as RPC, models communication around procedures or methods that can be invoked remotely.

An RPC client calls a method using parameters. The RPC framework serializes the request, sends it across a network, invokes the corresponding server implementation and returns a result or error.

RPC systems commonly provide:

• Service and method definitions
• Typed request and response messages
• Serialization and deserialization
• Generated client and server code
• Network communication
• Deadlines and cancellation
• Structured status handling
• Authentication metadata
• Unary and streaming operations

gRPC is an RPC framework in which a service contract defines remotely callable methods, parameters and return types. gRPC commonly uses Protocol Buffers as both its Interface Definition Language and its binary message format.

In your System Design curriculum, RPC and gRPC follows REST in the API Design & Service Contracts module. The module also covers GraphQL, resource modelling, pagination, versioning, idempotency, authentication versus authorization and rate-limit semantics.

Prerequisites

What Is RPC?

Remote Procedure Call is a communication model in which a client invokes a procedure implemented by another process or service.

Simplified Architecture

Client Application
        |
        | Calls client stub
        v
Generated Client Stub
        |
        | Serializes request
        v
RPC Transport
        |
        | Network communication
        v
RPC Server
        |
        | Deserializes request
        v
Service Implementation
        |
        | Produces response
        v
Generated Server Code
        |
        | Serializes response
        v
Client receives result

Local Call vs Remote Call

REST vs RPC

What Is gRPC?

gRPC is a framework for defining services and invoking their methods across process and network boundaries.

A gRPC design commonly includes:

• A .proto contract file
• Service definitions
• RPC method definitions
• Protocol Buffer message schemas
• Generated client stubs
• Generated server interfaces
• HTTP/2-based transport
• gRPC metadata and status codes

.proto contract
      |
      v
Protocol Buffer compiler
      |
      +--> Generated client code
      |
      +--> Generated server code
      |
      +--> Generated message classes

Protocol Buffers

Protocol Buffers provide a language-neutral mechanism for defining and serializing structured data.

A Protocol Buffer contract defines:

• Messages
• Fields
• Field types
• Stable numeric field identifiers
• Enumerations
• Services
• RPC methods

Basic Message Definition

syntax = "proto3";

package customer.v1;

message Customer {
  int64 id = 1;
  string name = 2;
  string email = 3;
  CustomerStatus status = 4;
}

enum CustomerStatus {
  CUSTOMER_STATUS_UNSPECIFIED = 0;
  CUSTOMER_STATUS_ACTIVE = 1;
  CUSTOMER_STATUS_INACTIVE = 2;
}

The field numbers are part of the binary contract. They identify fields in serialized data and must be managed carefully during schema evolution.

Defining a gRPC Service

syntax = "proto3";

package customer.v1;

service CustomerService {
  rpc GetCustomer(
    GetCustomerRequest
  ) returns (
    GetCustomerResponse
  );

  rpc CreateCustomer(
    CreateCustomerRequest
  ) returns (
    CreateCustomerResponse
  );

  rpc UpdateCustomer(
    UpdateCustomerRequest
  ) returns (
    UpdateCustomerResponse
  );

  rpc DeleteCustomer(
    DeleteCustomerRequest
  ) returns (
    DeleteCustomerResponse
  );
}

message GetCustomerRequest {
  int64 customer_id = 1;
}

message GetCustomerResponse {
  Customer customer = 1;
}

message CreateCustomerRequest {
  string name = 1;
  string email = 2;
  string idempotency_key = 3;
}

message CreateCustomerResponse {
  Customer customer = 1;
}

message UpdateCustomerRequest {
  int64 customer_id = 1;
  string name = 2;
  string email = 3;
  int64 expected_version = 4;
}

message UpdateCustomerResponse {
  Customer customer = 1;
}

message DeleteCustomerRequest {
  int64 customer_id = 1;
}

message DeleteCustomerResponse {
}

The contract identifies remotely callable methods and the request and response types associated with every call.

Code Generation

The Protocol Buffer compiler and gRPC plugins generate language-specific code from the service contract.

protoc \
  --proto_path=./proto \
  --php_out=./generated \
  --grpc_out=./generated \
  --plugin=protoc-gen-grpc=/path/to/grpc_php_plugin \
  ./proto/customer/v1/customer_service.proto

Exact compiler options and plugins depend on the selected language, framework version and build environment.

Generated code can include:

• Request and response message classes
• Serialization and parsing logic
• Client stub methods
• Server-side service interfaces
• Type-safe accessors

gRPC over HTTP/2

Standard gRPC calls are carried through HTTP/2 framing. A gRPC request uses HTTP/2 headers and one or more length-prefixed messages.

gRPC Request

HTTP/2 request headers
        |
        v
Length-prefixed request message
        |
        v
End of request stream


gRPC Response

HTTP/2 response headers
        |
        v
Length-prefixed response message
        |
        v
gRPC trailers with final status

gRPC uses HTTP/2 capabilities such as multiplexed streams and long-lived connections while defining its own method, message and status conventions.

Length-prefixed Messages

+--------------------+----------------------+------------------+
| Compression Flag   | Message Length       | Serialized Data  |
| 1 byte             | 4 bytes              | N bytes          |
+--------------------+----------------------+------------------+

Each message identifies whether message compression is applied and carries the size of the serialized message body.

gRPC Call Types

Unary RPC

rpc GetCustomer(
  GetCustomerRequest
) returns (
  GetCustomerResponse
);

Client sends one request
        |
        v
Server performs operation
        |
        v
Server returns one response

Unary RPC resembles a traditional request-response method call.

Server-streaming RPC

rpc WatchOrder(
  WatchOrderRequest
) returns (
  stream OrderUpdate
);

Client sends one request
        |
        v
Server sends update 1
        |
        v
Server sends update 2
        |
        v
Server sends update 3
        |
        v
Server completes response stream

This is useful for:

• Progress updates
• Monitoring events
• Change notifications
• Large result streams

Client-streaming RPC

rpc UploadMeasurements(
  stream Measurement
) returns (
  UploadSummary
);

Client sends message 1
Client sends message 2
Client sends message 3
Client completes request stream
        |
        v
Server returns one summary response

This is useful for:

• Batch uploads
• Telemetry submission
• Incremental file processing
• Aggregating client events

Bidirectional Streaming

rpc Collaborate(
  stream CollaborationCommand
) returns (
  stream CollaborationEvent
);

Client                                    Server
  |                                          |
  | Request message 1                        |
  |----------------------------------------->|
  |                                          |
  | Response message 1                       |
  |<-----------------------------------------|
  |                                          |
  | Response message 2                       |
  |<-----------------------------------------|
  |                                          |
  | Request message 2                        |
  |----------------------------------------->|
  |                                          |
  | Independent message flow                 |
  |<========================================>|

Request and response streams operate independently. The application protocol must define message correlation, ordering, completion, acknowledgements and error behaviour.

Deadlines

Every remote call should use a finite deadline. A deadline defines how long the client is willing to wait for the complete operation.

Client starts RPC
      |
      v
Deadline begins
      |
      +--> Response arrives before deadline:
      |       return result
      |
      +--> Deadline expires:
              cancel local wait
              propagate cancellation where supported
              return deadline error

A deadline should account for:

• Connection establishment
• TLS negotiation
• Queueing
• Server processing
• Downstream calls
• Response transfer

Cancellation

Cancellation communicates that the caller no longer needs the operation. Server implementations should observe the cancellation context and stop unnecessary work when safe.

Client cancels call
        |
        v
gRPC runtime communicates cancellation
        |
        v
Server observes cancellation
        |
        +--> Work can stop safely:
        |       release resources
        |
        +--> Operation already committed:
                preserve committed outcome
                avoid unsafe rollback assumptions

Cancellation is cooperative. It does not guarantee that the remote operation was never started or completed.

Retries

A failed RPC can be retried only when the method contract and failure state make repetition safe.

RPC fails or times out
        |
        v
Can the operation be safely repeated?
        |
        +--> No:
        |       return uncertain outcome
        |
        +--> Yes:
                apply bounded retry
                with backoff and jitter

Retry design should define:

• Retryable status codes
• Maximum attempts
• Backoff policy
• Randomized jitter
• Complete operation deadline
• Idempotency requirements
• Retry ownership

Idempotency

Read operations such as GetCustomer can naturally support safe repetition. Creation, payment and workflow commands require explicit idempotency design.

message CreateOrderRequest {
  string idempotency_key = 1;
  int64 customer_id = 2;
  repeated OrderLine items = 3;
}

First call with key K:

Validate key
Create order
Store result for key K
Return order


Retry with key K and identical request:

Find stored result
Do not create another order
Return original result


Key K with different request:

Reject as a conflict

gRPC Status Codes

gRPC returns a structured status containing a status code and optional error information.

Error Contract

A service should return stable machine-readable error information without exposing internal stack traces, SQL statements or infrastructure details.

{
    "code": "INVALID_ARGUMENT",
    "message": "Request validation failed.",
    "details": [
        {
            "field": "email",
            "reason": "invalid_format"
        }
    ],
    "traceId": "trace-8f21"
}

An error contract should distinguish:

• Client validation errors
• Authentication failures
• Authorization failures
• Missing entities
• State conflicts
• Deadline expiration
• Temporary service unavailability
• Unexpected internal failures

Metadata

gRPC metadata carries additional call information using HTTP/2 fields.

Metadata can carry:

• Authentication credentials
• Correlation identifiers
• Tracing context
• Locale preferences
• Client-version information
• Application-specific request context

authorization: Bearer ACCESS_TOKEN
x-correlation-id: request-1042
traceparent: trace-context-value
accept-language: en-US

Authentication

Authentication establishes the identity of the calling service, user or workload.

Common approaches include:

• TLS server authentication
• Mutual TLS
• Bearer access tokens
• Signed workload credentials
• Service-mesh identity

Client establishes protected connection
        |
        v
Client presents credential
        |
        v
Server validates credential
        |
        v
Caller identity is established
        |
        v
Method authorization is evaluated

Authorization

Every protected method requires an authorization decision. Authorization should consider both the method and the target entity.

Authenticated caller
        |
        v
Calls GetCustomer(customer_id = 42)
        |
        v
Load customer 42
        |
        v
Evaluate permission for customer 42
        |
        +--> Allowed:
        |       return customer
        |
        +--> Denied:
                return permission error

Do not authorize access only because the request contains a syntactically valid entity identifier.

TLS and Mutual TLS

gRPC connections should use TLS when communication requires protected transport. Mutual TLS additionally validates client certificates.

Server-authenticated TLS:

Client validates server certificate.


Mutual TLS:

Client validates server certificate.
Server validates client certificate.

Certificate identity establishes an authenticated context. Business authorization remains a separate service decision.

Flow Control and Backpressure

Streaming methods need backpressure so a fast producer cannot create unlimited buffered data for a slow consumer.

Fast producer
      |
      v
Bounded application queue
      |
      v
gRPC and HTTP/2 flow control
      |
      v
Slow consumer

A streaming contract should define:

• Maximum message size
• Maximum queued messages
• Maximum queued bytes
• Consumer cancellation behaviour
• Producer waiting behaviour
• Whether data can be coalesced or discarded

Message-size Limits

Every incoming and outgoing message should have explicit size limits.

Unbounded messages can cause:

• Memory exhaustion
• Long serialization pauses
• High network latency
• Slow garbage collection
• Resource-exhaustion attacks

Large datasets should normally be paginated, streamed or transferred through a purpose-built storage and download mechanism.

Protocol Buffer Compatibility

Protocol Buffer contracts should evolve without changing the meaning of existing field numbers.

Compatible Additive Change

message Customer {
  int64 id = 1;
  string name = 2;
  string email = 3;

  string preferred_language = 4;
}

Do Not Reuse Removed Field Numbers

message Customer {
  int64 id = 1;
  string name = 2;

  reserved 3;
  reserved "email";
}

Reserving removed numbers and names prevents accidental reuse that could cause old and new clients to interpret data differently.

Field-number Rules

Field Presence

APIs must distinguish when necessary between:

• A field that was not sent
• A field sent with its default value
• A field explicitly cleared
• A field inherited from existing state

message UpdateCustomerRequest {
  int64 customer_id = 1;
  optional string name = 2;
  optional string email = 3;
}

Presence-aware fields help prevent partial-update APIs from treating an omitted value as an instruction to clear existing data.

Field Masks

Field masks can identify fields that a caller intends to update.

import "google/protobuf/field_mask.proto";

message UpdateCustomerRequest {
  Customer customer = 1;
  google.protobuf.FieldMask update_mask = 2;
  int64 expected_version = 3;
}

Customer values:

name = "Updated Name"
email = "new@example.com"


Update mask:

paths:
- name


Result:

Only name is changed.
Email retains its existing value.

Optimistic Concurrency

message UpdateCustomerRequest {
  Customer customer = 1;
  int64 expected_version = 2;
}

Client reads:

customer version = 7


Client submits update:

expected_version = 7


Server verifies:

Current version is 7:
Apply update and create version 8.

Current version is not 7:
Reject the stale update.

Version checking prevents one caller from unknowingly overwriting changes made by another caller.

Health Checking

A production gRPC service should expose health information through an appropriate health-checking contract.

Health checks can distinguish:

• Process liveness
• Service readiness
• Serving status
• Dependency degradation

A healthy process does not prove that every method and dependency is ready for production traffic.

Server Reflection

Server reflection can allow compatible tools to discover available gRPC services and descriptors at runtime.

Reflection is useful for authorized development and diagnostic tooling, but production exposure should follow the service's security and information- disclosure policy.

Load Balancing

gRPC commonly reuses long-lived HTTP/2 connections. Connection reuse affects how traffic is distributed across backend instances.

Client
  |
  | Long-lived HTTP/2 connection
  v
Load Balancer
  |
  v
Backend A


Without suitable balancing design:

Many RPC calls on that connection
can continue reaching Backend A.

A balancing design can consider:

• Client-side load balancing
• Proxy-based load balancing
• Name resolution
• Health checking
• Connection lifetime
• Backend draining
• Retry and failover policy

Graceful Shutdown

Deployment begins
      |
      v
Stop accepting new calls
      |
      v
Mark instance not ready
      |
      v
Allow in-flight calls to finish
      |
      v
Cancel calls after shutdown deadline
      |
      v
Close connections and stop process

Streaming calls require a defined drain, cancellation and reconnection policy.

Browser Compatibility

Standard browser APIs do not generally expose the complete gRPC HTTP/2 protocol directly. Browser applications commonly use gRPC-Web or a compatible gateway.

Browser
   |
   | gRPC-Web request
   v
gRPC-Web Proxy or Gateway
   |
   | Standard gRPC
   v
gRPC Backend Service

The gateway becomes part of the contract and must handle authentication, CORS, translation, limits, errors and observability correctly.

REST Gateway

Some architectures expose REST or JSON externally while using gRPC between backend services.

External Client
      |
      | REST and JSON
      v
API Gateway
      |
      | Translates request
      v
Internal gRPC Service

The translation contract should define:

• HTTP method and URI mapping
• JSON-to-Protobuf conversion
• Status-code mapping
• Error-detail mapping
• Authentication propagation
• Deadline propagation
• Streaming limitations

Interceptors

Interceptors can apply cross-cutting logic around client or server calls.

Common interceptor responsibilities include:

• Authentication
• Authorization context
• Correlation identifiers
• Tracing
• Metrics
• Structured logging
• Retry policy
• Validation

Observability

Useful gRPC telemetry includes:

• Calls per method
• Latency percentiles per method
• Status-code distribution
• Deadline-exceeded count
• Cancellation count
• Retry count
• Request and response message sizes
• Active streaming calls
• Messages per stream
• Authentication and authorization failures
• Backend connection state
• Queue depth and backpressure

Logs should identify the method, status, duration and trace context without recording credentials or sensitive message contents.

Test with grpcurl

List Available Services

grpcurl \
  api.example.com:443 \
  list

Describe a Service

grpcurl \
  api.example.com:443 \
  describe customer.v1.CustomerService

Invoke a Unary Method

grpcurl \
  -H 'authorization: Bearer ACCESS_TOKEN' \
  -d '{"customerId":"42"}' \
  api.example.com:443 \
  customer.v1.CustomerService/GetCustomer

Listing and describing services require server reflection or locally supplied descriptors. Do not place production credentials directly in shell history.

Inspect HTTP/2 Connections

ss -tnp

openssl s_client \
  -connect api.example.com:443 \
  -servername api.example.com \
  -alpn h2

These commands can confirm transport connections and ALPN negotiation, but a gRPC-aware client is required to validate method-level behaviour.

Troubleshooting Workflow

1. Confirm the package, service and method names.
2. Confirm the deployed contract version.
3. Resolve the server name and verify routing.
4. Confirm TCP and TLS connectivity.
5. Confirm HTTP/2 negotiation.
6. Inspect authentication metadata.
7. Record the gRPC status code and safe error details.
8. Check the call deadline and cancellation state.
9. Inspect message-size and flow-control limits.
10. Check retries and idempotency handling.
11. Use the trace identifier to inspect downstream calls.
12. Compare runtime behaviour with the deployed .proto contract.

Common RPC and gRPC Mistakes

Recommended Test Cases

RPC and gRPC Best Practices

Practice Exercise

Design a gRPC service for customer and order operations and compare it with the earlier REST API.

Requirements

1. Define versioned package names.
2. Define customer and order messages.
3. Implement a unary customer lookup.
4. Implement idempotent order creation.
5. Implement a server stream for order-status updates.
6. Implement a client stream for telemetry upload.
7. Apply finite deadlines.
8. Propagate cancellation to database and downstream operations.
9. Authenticate callers through protected metadata or mTLS.
10. Authorize every customer and order operation.
11. Apply request and response message-size limits.
12. Use expected versions for updates.
13. Return structured gRPC statuses.
14. Expose an authorized health-check service.
15. Add compatibility tests for the .proto contract.

Suggested Service Contract

syntax = "proto3";

package order.v1;

service OrderService {
  rpc GetOrder(
    GetOrderRequest
  ) returns (
    GetOrderResponse
  );

  rpc CreateOrder(
    CreateOrderRequest
  ) returns (
    CreateOrderResponse
  );

  rpc WatchOrder(
    WatchOrderRequest
  ) returns (
    stream OrderUpdate
  );
}

message GetOrderRequest {
  int64 order_id = 1;
}

message GetOrderResponse {
  Order order = 1;
}

message CreateOrderRequest {
  string idempotency_key = 1;
  int64 customer_id = 2;
  repeated OrderLine lines = 3;
}

message CreateOrderResponse {
  Order order = 1;
}

message WatchOrderRequest {
  int64 order_id = 1;
  int64 after_version = 2;
}

message OrderUpdate {
  int64 order_id = 1;
  int64 version = 2;
  OrderStatus status = 3;
  string occurred_at = 4;
}

message OrderLine {
  string product_id = 1;
  int32 quantity = 2;
}

message Order {
  int64 id = 1;
  int64 customer_id = 2;
  repeated OrderLine lines = 3;
  OrderStatus status = 4;
  int64 version = 5;
}

enum OrderStatus {
  ORDER_STATUS_UNSPECIFIED = 0;
  ORDER_STATUS_PENDING = 1;
  ORDER_STATUS_CONFIRMED = 2;
  ORDER_STATUS_PROCESSING = 3;
  ORDER_STATUS_COMPLETED = 4;
  ORDER_STATUS_CANCELLED = 5;
}

Frequently Asked Questions