Nginx proxy_cache for API Responses: Keys, Staleness, Invalidation

How nginx proxy_cache works for API responses

Nginx proxy caching stores upstream responses on local disk and serves later requests for the same resource without contacting the backend. For read-heavy API endpoints such as catalogs, configuration and public reference data, this cuts backend load and removes upstream latency from the hot path. It suits teams that already run nginx as a reverse proxy and want shared caching in front of services they control.

The request path is short. Nginx computes a cache key for the incoming request and looks it up in shared memory. On a HIT, it reads the stored response from disk and returns it. On a MISS, it forwards the request upstream, returns the response, and writes it to the cache if the response is cacheable.

Storage is declared once with proxy_cache_path:

• keys_zone names a shared memory zone and sets its size. The zone holds keys and metadata, not response bodies.
• levels sets the subdirectory hierarchy so that no single directory holds too many files.
• max_size caps disk usage. When the cache exceeds it, the cache manager removes the least recently used entries.
• inactive removes entries that have not been accessed within the period, even if they are still fresh.
• use_temp_path=off writes files directly into the cache directory, which avoids copying them from a separate temp path.

You then enable caching in a location with proxy_cache, which references the zone by name.

Each nginx instance keeps its own cache. Behind a load balancer, hit rates are per node, and a purge on one node does not affect the others. An undersized keys_zone or max_size causes early eviction, and the resulting low hit rate is easy to mistake for a poor key design. Disk-backed caching also adds I/O, so plan disk capacity and throughput as you would for any other storage workload.

Copy# http context: declare cache storage once
# levels=1:2      -> two-level subdirectories to keep directories small
# keys_zone       -> shared memory for keys and metadata (1 MB holds roughly 8,000 keys)
# max_size        -> disk cap; cache manager evicts least recently used entries beyond it
# inactive        -> evict entries not accessed for 60m, even if still fresh
# use_temp_path   -> write directly into the cache dir instead of copying from a temp path
proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:50m
                 max_size=5g inactive=60m use_temp_path=off;

upstream api_backend {
    server 10.0.0.10:8080;
    server 10.0.0.11:8080;
}

server {
    listen 80;
    server_name api.example.com;

    # Read-heavy endpoint: enable caching by referencing the zone name
    location /v1/catalog/ {
        proxy_cache api_cache;
        proxy_cache_key $scheme$proxy_host$request_uri;
        proxy_cache_valid 200 5m;    # cache successful responses for 5 minutes
        proxy_cache_methods GET HEAD; # only cache safe reads

        # Expose HIT / MISS / EXPIRED etc. to clients for debugging
        add_header X-Cache-Status $upstream_cache_status always;

        proxy_pass
    }

    # Writes and user-specific endpoints bypass the cache entirely
    location /v1/ {
        proxy_pass
    }
}

A minimal setup that declares a disk-backed cache with proxy_cache_path, enables it for a read-heavy API location with proxy_cache, and exposes HIT/MISS status through a response header.

Designing the cache key: what goes in and what gets normalized

The default key, $scheme$proxy_host$request_uri, assumes the URL alone determines the response. For most APIs that is false. A tenant ID resolved from a header or token, an API-Version header, or Accept choosing between JSON and CSV all change the body without changing the URI. Any input that changes the response but is missing from the key is a data leak: the first tenant's response is served to the next one.

The opposite mistake is quieter. Every variable you add multiplies the number of distinct entries. Put a raw Authorization header or session cookie in the key, and each user gets a private cache with a hit ratio near zero.

Normalize before keying. Nginx does not reorder query parameters, so ?a=1&b=2 and ?b=2&a=1 are separate entries. Tracking parameters such as utm_* fragment the cache the same way. Build the key from the specific $arg_* values that matter, and use map to collapse headers like Accept into a small set of canonical values.

Upstream Vary adds a second, hidden key. Nginx stores one variant per key. A request whose varied headers differ is a miss, and the new response replaces the cached one, so variants keep evicting each other. Vary: Authorization or Vary: User-Agent therefore destroys the hit ratio without showing up in your key definition. Vary: * disables caching entirely. Put the dimensions you need in the key explicitly. If the upstream emits Vary you have already accounted for, use proxy_ignore_headers Vary.

Copy# http context: collapse request inputs into a few canonical values
map $http_accept $api_format {
    default        json;   # absent, */*, or application/json
    ~*text/csv     csv;
}

# Tenant ID is set by a trusted auth layer, never taken from the client.
# If it is missing, skip the cache instead of sharing one entry.
map $http_x_tenant_id $no_tenant {
    ""       1;
    default  0;
}

proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:50m
                 max_size=2g inactive=10m use_temp_path=off;

server {
    listen 80;

    location /api/products {
        proxy_cache api_cache;

        # Build the key from each input that changes the response.
        # Name only the query args that matter, in a fixed order, so
        # ?page=2&limit=50 and ?limit=50&page=2&utm_source=x share one entry.
        proxy_cache_key "$scheme|$proxy_host|$uri|t=$http_x_tenant_id|v=$http_api_version|f=$api_format|page=$arg_page|limit=$arg_limit";

        proxy_cache_bypass $no_tenant;
        proxy_no_cache     $no_tenant;

        # Upstream sends Vary: Accept, API-Version. Both are already in the
        # key, so ignore Vary to stop variants from evicting each other.
        proxy_ignore_headers Vary;

        proxy_cache_valid 200 60s;
        add_header X-Cache-Status $upstream_cache_status always;

        proxy_pass
    }
}

An API cache key that adds tenant, version, and normalized format, names only the query args that matter, bypasses the cache without a tenant, and ignores an upstream Vary already covered by the key.

How freshness is decided and what must never be cached

That same proxy_ignore_headers directive decides who owns freshness. nginx checks X-Accel-Expires first, then Expires and Cache-Control. proxy_cache_valid applies only when none of those set a lifetime. A backend that sends Cache-Control: no-store or private, or Set-Cookie, prevents caching outright unless you ignore that header.

proxy_cache_valid works best as a fallback TTL table keyed by status code. A bare duration covers only 200, 301 and 302. Every other status needs an explicit entry, or any. Caching 404s and 5xx is a deliberate trade-off. Even a few seconds of cached errors shields a struggling backend from retries during an incident. The cost is that those errors keep being served after the upstream has recovered. Keep error TTLs much shorter than success TTLs.

The defaults are narrow but not safe. proxy_cache_methods covers GET and HEAD, and responses with Set-Cookie are skipped. nginx does not skip caching because a request carries Authorization, so one user's response can be served to another. For authenticated endpoints, either set proxy_no_cache and proxy_cache_bypass on the credential, or put the caller's identity in the key. Authenticated APIs have no safe default, and you have to make this choice per endpoint.

Copy# http context
proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:10m
                 max_size=1g inactive=10m;

upstream api_backend {
    server 10.0.0.10:8080;
}

server {
    listen 80;
    server_name api.example.com;

    proxy_cache     api_cache;
    proxy_cache_key "$scheme$host$request_uri";
    # Default proxy_cache_methods is GET HEAD. Stated here so reviewers see it.
    proxy_cache_methods GET HEAD;

    # Fallback TTLs, used only when upstream headers set no lifetime
    # (or when those headers are ignored, as in /v1/catalog/ below).
    proxy_cache_valid 200 301 302 5m;
    proxy_cache_valid 404 30s;              # deliberate: absorbs repeated misses
    proxy_cache_valid 500 502 503 504 5s;   # short: shields a failing backend, recovers fast

    add_header X-Cache-Status $upstream_cache_status always;

    # Public, non-personal data: nginx owns freshness. Upstream Cache-Control,
    # Expires and X-Accel-Expires are ignored. Set-Cookie still blocks caching.
    location /v1/catalog/ {
        proxy_ignore_headers Cache-Control Expires X-Accel-Expires;
        proxy_pass
    }

    # Authenticated: nginx does not skip caching for Authorization on its own.
    # A non-empty header means: don't serve from cache, don't store the response.
    location /v1/account/ {
        proxy_cache_bypass $http_authorization;
        proxy_no_cache     $http_authorization;
        proxy_pass
    }
}

Shows status-code fallback TTLs with short error caching, nginx-owned freshness via proxy_ignore_headers on a public endpoint, and explicit cache bypass on an authenticated endpoint.

Stale-while-revalidate and stampede protection

Continuing from authenticated endpoints, the remaining risk on cacheable endpoints is expiry. When a popular key expires, every concurrent request misses together and hits the upstream at the same moment. Two mechanisms address this.

Serving stale while one request refreshes

proxy_cache_use_stale lists when nginx may return an expired entry instead of waiting on the upstream:

• error and timeout cover connection failures.
• http_500, http_502, http_503 and http_504 cover upstream errors.
• updating covers the case where another request is already refreshing the entry.

Adding proxy_cache_background_update on makes the refresh a background subrequest, so the request that found the expired entry also gets the stale copy and does not wait. Background update depends on updating being in the list.

Upstreams can also control this per response. nginx honors the stale-while-revalidate and stale-if-error Cache-Control extensions, so the application can set stale windows per endpoint instead of one global policy.

Locking cold keys

A key with no cached copy has nothing stale to serve. proxy_cache_lock on sends only one request upstream to fill it. The others wait for that response, up to proxy_cache_lock_timeout. When the timeout passes, the waiting requests go to the upstream themselves, and their responses are not cached.

Trade-offs

• Queued requests wait longer. Their latency includes the leader's full upstream call.
• A slow upstream still produces a burst. If the upstream is slower than the lock timeout, every waiting request reaches it anyway.
• Stale data carries no signal. Clients cannot tell a response is stale unless you expose the cache status or an Age header.

Some data must never be served stale: account balances, permission and role checks, and inventory counts. Keep these endpoints out of the cache entirely rather than shortening their stale window.

Copyproxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:50m
                 max_size=1g inactive=10m use_temp_path=off;

upstream api_backend { server 10.0.0.10:8080; }

server {
    listen 80;
    server_name api.example.internal;

    location /v1/catalog/ {
        proxy_pass
        proxy_cache api_cache;
        proxy_cache_key "$scheme$host$request_uri";
        proxy_cache_valid 200 60s;

        # Serve the expired copy on upstream failure, or while another request refreshes it
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        # Refresh in a background subrequest so no client waits (relies on 'updating' above)
        proxy_cache_background_update on;

        # Cold or evicted keys: one request fills the cache, the rest wait
        proxy_cache_lock on;
        # After 3s, waiters go upstream themselves and their responses are not cached
        proxy_cache_lock_timeout 3s;

        # Let clients see HIT, STALE, UPDATING, MISS, etc.
        add_header X-Cache-Status $upstream_cache_status always;
    }

    # Balances, permissions, inventory: never cached, never served stale
    location ~ ^/v1/(balances|permissions|inventory)/ {
        proxy_pass
        proxy_cache off;
    }
}

A cacheable catalog endpoint that serves stale entries during refreshes and upstream failures, uses a cache lock for cold keys, and exposes cache status to clients, while staleness-sensitive endpoints bypass the cache entirely.

Invalidation without a built-in purge

Open-source nginx has no built-in purge directive, so every invalidation strategy is a workaround with different trade-offs.

Short TTLs limit how long wrong data can live. They require no tooling, but they also cap the hit ratio.

Versioned keys or URLs (a v segment, or a content version folded into proxy_cache_key) make new content get a new key immediately. The old entries are not removed. They sit on disk until inactive expires them or max_size forces eviction.

A controlled refresh path sets proxy_cache_bypass on an internal-only header such as $http_x_cache_refresh. A request carrying that header skips the cache lookup and goes to the upstream. Nginx still stores the response, so it overwrites the existing entry. The header must be stripped or ignored for public traffic, for example by accepting it only on a separate internal listen address. If you skip that step, any client can force cache misses.

Deleting cache files by key hash means computing the MD5 of the key and mapping it to the levels directory layout. It depends on that layout and on reproducing the key exactly, and it only affects one node.

A third-party purge module adds a real purge endpoint. The cost is building nginx with it and maintaining that build.

With several proxies, every node keeps its own cache, so any explicit invalidation has to reach all of them. The same independence lowers hit ratios as you add nodes, because each node warms its own copy of every key. Consistent hashing at the load balancer routes each key to one node and avoids this.

Observability: verifying the key design with $upstream_cache_status

Before trusting the cache, make every decision visible. Add $upstream_cache_status to a response header such as X-Cache-Status with add_header ... always, so error responses carry it too, and to a dedicated log_format. That log line should include the same variables that make up proxy_cache_key, plus $upstream_response_time. Also log a flag derived through map that records whether Authorization or a session cookie was present, without logging the credential itself.

The values mean:

• MISS: the key was not in the cache.
• BYPASS: proxy_cache_bypass matched.
• EXPIRED: the entry existed but was no longer fresh, so nginx refetched it.
• STALE: a stale entry was served because of proxy_cache_use_stale.
• UPDATING: a stale entry was served while another request refreshed it.
• REVALIDATED: a conditional request to the origin returned 304.
• HIT: the response was served fresh from the cache.

Use the logs to check the key design:

• Cardinality: count distinct key tuples per route. If a route has thousands of distinct keys and almost none repeat, the key is fragmenting. Look for unsorted query parameters, tracking parameters, or a header you didn't mean to include.
• Authenticated routes: any HIT where the credential flag is set should be investigated immediately. It means the key does not separate users.
• Eviction versus key design: a MISS on a key that appeared earlier, within its validity window, indicates eviction from max_size or inactive. Fragmentation looks different: most keys never appear twice.

Clients also need to see staleness. Open-source nginx does not compute Age for cached responses. Have the upstream emit a generation timestamp header, which is stored with the cached response, so clients can tell how old the data is.

Worked example: public endpoint vs per-tenant endpoint

The two locations share one cache zone but differ on almost every other directive.

Public catalog location

• proxy_cache_key "$request_method$host$uri$args_normalized": There is no tenant or credential component, because every caller receives the same bytes. $args_normalized is the sorted query string built with a map, as described in section 2.
• proxy_cache_valid 200 60s and proxy_cache_valid 404 30s: A missing SKU is also a stable answer. Caching the 404 stops crawlers from repeatedly sending the same miss to the upstream.
• proxy_cache_use_stale error timeout updating http_502 http_503, proxy_cache_background_update on and proxy_cache_lock on: During an outage or a refresh, clients receive stale data. Only one request per key goes upstream.

Per-tenant location

• proxy_cache_key "$tenant_id$request_method$uri$args_normalized": $tenant_id comes from auth_request_set. It is not taken from a client header that a caller could forge.
• proxy_cache_bypass $tenant_missing and proxy_no_cache $tenant_missing $cookie_session: If identity is unresolved, nginx neither reads from nor writes to the cache.
• proxy_cache_valid 200 10s: Only successful responses are cached. A 404 for one tenant can be an authorization artifact, so it should not be stored.
• proxy_cache_use_stale error timeout: There is no updating here, so tenants do not receive data that is stale beyond the TTL during normal operation.

Illustrative estimate (assumed inputs)

Catalog: Assume 2,000 req/s spread over 5,000 keys. With the lock, the upstream sees at most about one fetch per key every 60 s. That is 5,000 ÷ 60 ≈ 83 req/s, for a hit ratio of roughly 96%.

Tenant endpoint: Assume 200 req/s spread over 8,000 tenant-scoped keys. Each key gets about one request every 40 s, which is longer than its 10 s TTL, so the hit ratio is near zero. Caching this endpoint only pays off for keys that individual tenants request far more often than the TTL.

When to use it, when to avoid it, and alternatives

Proxy caching fits public or tenant-scoped data that is read far more often than it changes and can tolerate staleness up to a known bound. Examples include catalog listings, configuration documents, and aggregate dashboards. The cache key stays small, a stale answer is merely old rather than wrong for the caller, and one layer absorbs load for every service behind it.

Avoid it for per-user data, balances, entitlements, and permission checks. A key mistake there leaks data across users, and a stale answer can grant access that was revoked. Also avoid it where a client writes and then expects to read its own write, or where responses are personalized enough that keys rarely repeat.

Proxy caching can spare each team from building its own caching layer, and it removes backend work without changing application code. That saving is real but not free. Someone has to own the key design, build invalidation across every nginx node, and run leak tests whenever an endpoint adds a header or a new auth scheme. Budget for that as ongoing work, not a one-time setup.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx proxy_cache api, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.