Redis Latency Spikes: Diagnosing Big Keys, Slow Commands and Forks

Why does one slow command cause Redis latency spikes for every client?

A Redis latency spike is a short window where command response times jump well above their normal baseline, often from sub-millisecond to tens or hundreds of milliseconds, and then recover. Most of these come from how Redis runs commands. Redis executes them one at a time on a single main thread. Some builds offload network I/O to helper threads, but command execution itself stays serialized. When one command takes 80 ms, every request queued behind it waits those 80 ms plus its own service time. A single slow call therefore appears as a latency spike for every connected client, including clients sending trivial GETs.

Three causes account for most server-side stalls:

• Big keys. Deleting, serializing or reading a very large hash, set or list in one call blocks the loop for the whole operation.
• O(N) commands. KEYS, SMEMBERS on large sets, or unbounded LRANGE scale with data size instead of returning in constant time.
• Fork and copy-on-write pressure. BGSAVE and AOF rewrite call fork(). Copying the page table blocks the parent, and copy-on-write afterward adds memory and CPU pressure.

Before you blame any of these, confirm the stall is in Redis. Run redis-cli --latency against the server from the application host. Then run redis-cli --intrinsic-latency on the Redis host itself. The intrinsic test measures how long the kernel or hypervisor delays a running process, and it does not involve Redis at all. If the intrinsic latency is already high, the problem is the host. If Redis looks fast from a nearby client but slow from the application, look at the network, pool exhaustion or connection storms. This matters most for backend and platform engineers whose services carry tail-latency SLOs.

Copy#!/usr/bin/env bash
REDIS_HOST=10.0.12.34
REDIS_PORT=6379

timeout 30 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" --latency

timeout 60 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" --latency-history -i 5

redis-cli --intrinsic-latency 60

timeout 30 redis-cli -h 127.0.0.1 -p "$REDIS_PORT" --latency

redis-cli -h 127.0.0.1 -p "$REDIS_PORT" SLOWLOG GET 10
redis-cli -h 127.0.0.1 -p "$REDIS_PORT" INFO persistence | grep latest_fork_usec

A three-step redis-cli check that compares latency from the application host, intrinsic host latency, and local latency on the Redis host, to show whether a spike comes from the host, from Redis or from the network and client path.

Who should use this diagnostic approach?

This workflow is for engineers who own the latency of a service that depends on Redis:

• Backend engineers whose request paths call Redis synchronously. A cache or session lookup that normally takes well under a millisecond becomes the slowest part of a request during a stall, and it shows up in their p99.
• Platform and SRE teams who run Redis instances for other teams. They need to say quickly whether a reported spike is a Redis problem, a host problem or a client problem, and which team's workload caused it.
• On-call engineers who get paged for tail-latency SLO breaches and need a fixed sequence of checks that works under pressure.
• Teams self-hosting Redis on VMs or bare metal. They control THP, swap, disks and persistence placement, so they can act on every finding in this article.

Teams on managed or shared Redis can still use the diagnosis steps (SLOWLOG, LATENCY, INFO, --bigkeys) as long as the provider exposes those commands. Some providers restrict or rename commands such as CONFIG, so check what is available before an incident. The fixes on the data and command side (avoiding O(N) calls, UNLINK, splitting big keys) apply everywhere. The host-level fixes do not.

The approach is less useful if your workload tolerates occasional slow responses, for example a batch job that uses Redis as a queue and has no latency target. In that case, big keys and forks still matter for memory headroom, but chasing individual spikes is rarely worth the effort.

How does the diagnosis work, end to end?

The method narrows a spike down layer by layer, and each layer has its own measurement:

1. Host layer. --intrinsic-latency measures how much the operating system or hypervisor delays a busy process. If this is high, Redis is a victim and nothing inside Redis will fix it.
2. Network and client layer. Comparing --latency from the application host with --latency on the Redis host isolates the network path. Client-side timing and connection-pool wait time cover what neither measurement sees.
3. Event-loop layer. If Redis itself is slow, the slowlog records commands whose execution exceeded a threshold, and the latency monitor records non-command events such as forks and fsync waits. Together they attribute a stall to a command or to an event class.
4. Data-shape layer. If a command is the cause, the question becomes why it was slow. Usually the key it touched was large, or the command scans in proportion to data size. --bigkeys, --memkeys, MEMORY USAGE and the length commands measure that.
5. Persistence layer. If the latency monitor shows fork or fsync events, INFO fields such as latest_fork_usec and the copy-on-write sizes show how expensive persistence is and when it ran.

All of this works because of the single-threaded execution model described above. A stall on the main thread delays every client, so it leaves a trace in server-side data: a slowlog entry, a latency event or a fork timestamp. The diagnosis is mostly a matter of lining those traces up with the time window of the client-side spike. Once a cause is named, the fix follows from the layer it sits in: replace the command, reshape the key, move or reschedule persistence, or fix the host.

How do you read SLOWLOG and the latency monitor to find what stalled the event loop?

Once you have ruled out the client side, check what Redis itself recorded. SLOWLOG GET returns entries with a unique ID, a Unix timestamp, the execution time in microseconds, the command and its arguments (truncated for long calls), and, on most builds, the client address and name. That duration covers only the time the command spent executing. It excludes time queued behind other commands and network round trips, so a 2 ms entry can still mean every client behind it waited 2 ms.

Two settings control the log. slowlog-log-slower-than is the threshold in microseconds. slowlog-max-len caps how many entries the in-memory ring buffer holds. If you set the threshold too low, routine commands fill the buffer and push out the one 400 ms KEYS call you needed to see. The buffer is not persisted, so a restart or failover erases it. Capture it before you take either action.

Commands are only one cause of stalls. Enable the latency monitor with CONFIG SET latency-monitor-threshold (in milliseconds), then use these commands:

Event classes include command, fork and aof-fsync-always. A clean slowlog does not rule out a stall, because forks, fsync waits and swapping happen outside command execution. The monitor records forks and fsync waits. A swap stall can surface only as inflated command latency, so check the host's swap activity as well.

Copy#!/usr/bin/env bash
set -euo pipefail
R="redis-cli -h redis.internal.example -p 6379"
TS=$(date +%Y%m%d-%H%M%S)

$R SLOWLOG LEN
$R SLOWLOG GET 128 > "slowlog-$TS.txt"
$R CONFIG SET slowlog-log-slower-than 10000   # microseconds
$R CONFIG SET slowlog-max-len 1024

$R CONFIG SET latency-monitor-threshold 100   # milliseconds
$R LATENCY LATEST           # event, last timestamp, last ms, max ms
$R LATENCY HISTORY command  # timestamped samples to line up with alerts
$R LATENCY HISTORY fork
$R LATENCY DOCTOR           # human-readable analysis and advice

$R INFO stats | grep latest_fork_usec
vmstat 1 5

A redis-cli triage script that saves the slowlog before it can be lost, tunes its threshold and size, enables the latency monitor to attribute spikes to command, fork or fsync events, and checks fork time and host swap activity, which the slowlog cannot show.

How do you find Redis big keys without making latency worse?

Once the event loop is clearly stalling inside command execution, the next step is to find the keys large enough to cause it. redis-cli --bigkeys iterates the keyspace with SCAN and reports the largest key of each type. It ranks strings by byte length and collections by element count. redis-cli --memkeys uses the same scan but ranks keys by memory footprint, as estimated by MEMORY USAGE. Both print a per-type summary at the end. Treat the names they report as suspects. For each one, run MEMORY USAGE <key> to see its actual size, along with the type-specific length command (HLEN, LLEN, SCARD, ZCARD).

Element count often matters more than bytes. A set of two million short members might use an unremarkable amount of memory, but SMEMBERS, SUNION or a plain DEL on it does work proportional to every member. That work runs on the single command thread, and every other client waits for it to finish. A large string, by contrast, is mostly a cost in network transfer and memory copying. Rank by count to find event-loop risks, and by bytes to find memory and bandwidth risks.

These scans have two trade-offs:

• They add load. The scan walks the entire keyspace and inspects each key. Run it against a replica, or off-peak. The -i option sleeps between SCAN batches, which spreads the cost out at the price of a longer run.
• They report only the top key per type. If you have five oversized hashes, you will see one of them. For a full inventory, write your own SCAN loop that applies a threshold, or analyze an RDB snapshot offline.

Copy#!/usr/bin/env bash
set -euo pipefail

HOST=redis-replica-1.internal   # placeholder: point at a replica, not the primary
PORT=6379

redis-cli -h "$HOST" -p "$PORT" --bigkeys -i 0.1

redis-cli -h "$HOST" -p "$PORT" --memkeys -i 0.1

for key in "session:index" "feed:user:1042" "tags:all"; do
  type=$(redis-cli -h "$HOST" -p "$PORT" TYPE "$key")
  # SAMPLES 0 measures every element instead of estimating from a sample.
  # This is exact, but it is O(N) on the key itself.
  bytes=$(redis-cli -h "$HOST" -p "$PORT" MEMORY USAGE "$key" SAMPLES 0)

  # Element count is what predicts event-loop stalls.
  case "$type" in
    hash) len=$(redis-cli -h "$HOST" -p "$PORT" HLEN "$key") ;;
    list) len=$(redis-cli -h "$HOST" -p "$PORT" LLEN "$key") ;;
    set)  len=$(redis-cli -h "$HOST" -p "$PORT" SCARD "$key") ;;
    zset) len=$(redis-cli -h "$HOST" -p "$PORT" ZCARD "$key") ;;
    string) len=$(redis-cli -h "$HOST" -p "$PORT" STRLEN "$key") ;;
    *)    len="n/a" ;;
  esac

  echo "$key type=$type bytes=$bytes length=$len"
done

The example runs --bigkeys and --memkeys against a replica with throttled SCAN batches, then confirms each suspect with MEMORY USAGE and the matching length command so that byte size and element count can be compared.

Which O(N) commands should you avoid, and what are the alternatives?

Every command in the first column runs to completion on the main thread. Its cost grows with the size of the collection, so a call that returns in microseconds on a 100-element set can stall all clients once the set reaches millions of members.

The *SCAN family returns results in small batches through a cursor. This spreads the work across many short calls instead of one long one. The cursor has two caveats you need to handle in application code:

• Duplicates. The same element can be returned more than once, so callers must deduplicate.
• Concurrent changes. Elements added or removed while the scan runs may or may not appear in the results.

Lazy freeing moves deallocation to a background thread, but the memory is not released immediately. Until the background thread catches up, used_memory stays high. If you are near maxmemory, this can trigger more eviction than you expect, or cause write errors under a noeviction policy.

Splitting a big key into sharded sub-keys fixes the problem at its source, but it has costs:

• Application logic. The client must route each read and write to the correct sub-key.
• Atomicity. You lose atomic whole-object operations.
• Clustered setups. Multi-key commands, transactions and Lua scripts work only if all sub-keys share a hash slot. Forcing them into one slot with hash tags concentrates the load on one node again.

Copyimport redis

r = redis.Redis(host="redis.internal.example", port=6379, decode_responses=True)

seen = set()
for key in r.scan_iter(match="session:*", count=500):
    if key in seen:  # SCAN may return the same key more than once
        continue
    seen.add(key)

fields = {}
for field, value in r.hscan_iter("user:42:prefs", count=200):
    fields[field] = value  # dict assignment also absorbs duplicate fields

theme, lang = r.hmget("user:42:prefs", ["theme", "lang"])

r.unlink("leaderboard:2025")

r.config_set("lazyfree-lazy-expire", "yes")
r.config_set("lazyfree-lazy-eviction", "yes")
r.config_set("lazyfree-lazy-server-del", "yes")

Uses redis-py to replace KEYS and HGETALL with cursor-based SCAN and HSCAN, deduplicating SCAN results, and replaces DEL with UNLINK. It also enables the lazy-free settings so expiry, eviction and implicit deletes free memory in a background thread.

How does fork latency work during BGSAVE and AOF rewrite?

To write an RDB snapshot (BGSAVE) or compact the append-only file (BGREWRITEAOF), Redis calls fork(). The child gets a point-in-time view of memory without a full copy, but the fork call is not free. The kernel must duplicate the parent's page tables, and the main thread is blocked until that finishes. Page-table size grows with resident memory, so a larger dataset means a longer stall on every client.

Once the child is running, parent and child share pages through copy-on-write. Every write to a shared page forces the kernel to copy that page first. Under write-heavy load, the parent's memory grows for as long as the child runs.

Check latest_fork_usec in INFO stats for the duration of the last fork. In INFO persistence, look at rdb_last_cow_size and aof_last_cow_size to see how much memory copy-on-write consumed. Also watch rdb_bgsave_in_progress and aof_rewrite_in_progress to correlate spikes with active children.

Illustrative estimate (assumed numbers, not a benchmark):

• Page-table size ≈ (RSS ÷ page size) × 8 bytes per entry.
• For 24 GB with 4 KB pages: (24 GB ÷ 4 KB) × 8 B ≈ 48 MB of page tables to copy while blocked.
• Copy-on-write growth ≈ distinct pages dirtied × page size.
• If 600,000 distinct pages are written during the child's lifetime, that is about 2.4 GB at 4 KB.

Transparent huge pages make copy-on-write much worse. With 2 MB pages, a one-byte write duplicates 2 MB instead of 4 KB. The same 600,000 writes could dirty nearly the entire dataset, and each fault is far more expensive to service. THP can also add stalls from kernel memory compaction. Redis logs a warning at startup when THP is enabled, and the standard guidance is to disable it for Redis hosts.

When should you change persistence settings to reduce fork and fsync stalls?

Disable transparent huge pages on any host that runs persistence. This has no durability cost, so it is the first change to make. Set it in your boot configuration or init system so it survives reboots, because Redis only warns and does not fix it.

Next, budget memory for copy-on-write. During a fork, every page the parent writes to gets duplicated. A write-heavy instance can approach double its dataset size before the child finishes. Keep enough free RAM for your write rate multiplied by snapshot duration. Never let the host reach swap. A swapped-out Redis page turns a microsecond lookup into a disk read on the event loop, and that is worse than any fork stall.

Then choose an fsync policy:

Moving RDB or AOF to a replica takes fork and fsync off the primary. The cost is a consistency gap. If the primary fails and the persisting replica lags, the replica's lag is lost. A restart from that replica's files can also roll back acknowledged writes. Monitor replication offset and treat replica-only persistence as accepting that loss window. If you keep persistence on the primary, schedule BGSAVE for low-traffic periods.

On managed or shared infrastructure, you usually cannot change THP, swap, or disk settings. In that case, your options are instance sizing, persistence mode, and snapshot timing.

What is a repeatable triage workflow for Redis latency spikes?

Run the same steps in the same order every time, starting with the cheapest and most decisive.

1. Confirm the spike. Check that client-side p99 rose, note the exact time window, and confirm the server's own measurements show a stall in that window.
2. Check the slowlog. Run SLOWLOG GET and look for entries with timestamps inside the window. A matching entry names the command and its arguments.
3. Check latency events. LATENCY LATEST and LATENCY DOCTOR show stalls the slowlog never records, such as fork, aof-fsync-always, or expiry cycles.
4. Check INFO. In INFO persistence, look at latest_fork_usec, rdb_bgsave_in_progress, and aof_rewrite_in_progress. In INFO commandstats, look for commands with high per-call time.
5. Inspect keys. Run redis-cli --bigkeys or targeted MEMORY USAGE calls only after earlier steps point at a data-shape problem.

Illustrative estimate. Assume an HGETALL on a 400,000-field hash takes 200 ms, and assume traffic of 5,000 requests per second. About 1,000 requests arrive while that one call runs. Each of them waits behind it, up to the full 200 ms, and none of them is a slow command itself.

The fixed order matters because each step either names the cause or rules out a whole category. That keeps engineers from adjusting maxmemory-policy, resizing instances, or disabling persistence on a guess. Unnecessary changes cost engineering time and create new risk.

Server-side data has blind spots. It cannot see requests queued in a client's connection pool, network jitter, or CPU stolen by noisy neighbors on shared hosts. If server metrics look clean during a client-reported spike, compare client timing, connection-pool wait time, and host steal time before you conclude Redis is at fault.

How can this approach save engineering time?

Most of the time lost on Redis latency incidents goes to investigating the wrong layer, applying changes that do not address the cause, and repeating the same investigation when the spike comes back. The workflow cuts each of these.

It shortens incidents. Because the steps are ordered from cheapest to most expensive, most investigations end within the first few commands. A slowlog entry or a fork event with a matching timestamp names the cause directly. You do not have to reproduce the problem or read application code to find it.

It avoids wasted changes. Resizing an instance, changing the eviction policy or disabling persistence each takes time to plan, roll out and verify, and none of them helps if the real cause is one KEYS call in a cron job. Naming the cause first means the change you ship is the one that fixes it.

It settles cross-team disputes quickly. When an application team reports slow Redis and the platform team sees a healthy server, the layered checks show who owns the problem: host, network, client pool or Redis itself. That replaces back-and-forth with evidence both sides can read.

It keeps evidence from disappearing. Saving the slowlog before a restart or failover means a single occurrence is enough to diagnose. Without it, teams often have to wait for the spike to recur, sometimes with extra logging added in a hurry.

It turns one-off fixes into prevention. Once the causes are known, you can keep them from coming back:

• Keep slowlog-log-slower-than and latency-monitor-threshold set permanently in redis.conf so the data is already there when the next spike happens.
• Alert on latest_fork_usec and on copy-on-write sizes so growing fork costs show up before they breach an SLO.
• Run --bigkeys or a custom SCAN audit against a replica on a schedule, and treat new oversized keys as bugs.
• Use ACL rules to deny commands such as KEYS to application users, and flag HGETALL, SMEMBERS and unbounded LRANGE in code review when they target collections that can grow without limit.

Prevention costs a little setup time once. In return, a whole class of incidents stops reaching on-call.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to redis latency spikes, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.