How do Redis RDB snapshots and AOF logs work?
Redis offers two persistence mechanisms, and they fail differently.
RDB snapshots are point-in-time dumps of the whole dataset. On BGSAVE, whether triggered by a save <seconds> <changes> rule or run manually, the main process forks. The child serializes memory to a temporary file and renames it over the old dump.rdb. Copy-on-write lets the parent keep serving traffic while the child sees a frozen view. If the server dies, you lose every write since the last completed snapshot. The file is compact, so loading it is fast.
AOF (append-only file) logs every write command. Commands go to an in-memory buffer, get written to the file, and are fsynced according to appendfsync. On startup Redis replays the log to rebuild state. The log grows without bound, so a rewrite (BGREWRITEAOF or automatic) forks a child that emits the smallest command set reproducing the current data. Writes that arrive during the rewrite are preserved and folded into the new file.
writes
│
┌──────▼───────┐ fork (BGSAVE) ┌────────────┐
│ main process │─────────────────▶│ child │──▶ dump.rdb
│ │ └────────────┘
│ AOF buffer ─┼──write──▶ appendonly.aof ──fsync──▶ disk
│ │ fork (rewrite) ┌────────────┐
│ │─────────────────▶│ child │──▶ compacted AOF
└──────────────┘ └────────────┘
You can enable both. When both exist, Redis loads the AOF because it is usually the more complete record. With the RDB preamble option, a rewritten AOF begins with an RDB-format snapshot followed by plain commands. Restarts get faster because the bulk of the data loads as binary. The cost is that you can no longer read or hand-edit the whole file as text.
client writes
│
▼
┌────────────────────────────────────────────┐
│ main process (serves traffic from memory) │
│ ┌────────────┐ │
│ │ AOF buffer │ │
│ └─────┬──────┘ │
└─────────┼──────────────┬──────────────┬────┘
│ write() │ fork │ fork
▼ │ BGSAVE or │ BGREWRITEAOF
┌──────────────────┐ │ save rule │ or auto
│ appendonly.aof │ ▼ ▼
└────────┬─────────┘ ┌────────────┐ ┌────────────────────┐
│ fsync │ child (COW)│ │ child (COW) │
│ per │ frozen view│ │ minimal commands │
│ policy └─────┬──────┘ └─────────┬──────────┘
▼ │ tmp+rename │ + writes made
┌──────────────────┐ ▼ ▼ during rewrite
│ disk │ ┌────────────┐ ┌────────────────────┐
└──────────────────┘ │ dump.rdb │ │ new AOF file │
└────────────┘ │ (opt. RDB preamble)│
└────────────────────┘
Restart: load AOF if enabled, else dump.rdb
The main process forks a copy-on-write child for each RDB snapshot and each AOF rewrite. Separately, it appends write commands from its AOF buffer to the log and fsyncs them according to appendfsync.
What does each appendfsync setting actually guarantee?
appendfsync controls when Redis asks the kernel to flush the AOF to stable storage. Every write is appended to the file. The setting only decides when it becomes durable.
always: Redis fsyncs before acknowledging writes, batched per event-loop iteration. You lose at most the in-flight batch. The price is that every write waits on fsync latency. On local NVMe that may be tolerable. On network or shared block storage, fsync time can dominate response times and cap throughput.everysec: A background thread fsyncs roughly once per second, so normal writes don't wait on the disk. This is a compromise, not a hard one-second bound. If the disk stalls and a previous fsync is still running, Redis first lets the window grow, then blocks writes until the fsync catches up. Expect around a second of loss in normal operation and more during I/O trouble.no: Redis writes to the page cache and leaves flushing to the OS. The loss window is whatever the kernel's writeback policy allows, typically tens of seconds.
| Configuration | Loss window | Throughput impact | Disk use | Restart speed |
|---|---|---|---|---|
| RDB only | Since last snapshot (often minutes) | Low; periodic fork | Compact | Fast |
AOF always |
At most the in-flight batch | High; bound by fsync | Grows until rewrite | Slower |
AOF everysec |
About 1s, wider on stalls | Low to moderate | Grows until rewrite | Slower |
AOF no |
OS-dependent | Lowest of AOF modes | Grows until rewrite | Slower |
| RDB + AOF | Set by the AOF mode | AOF cost plus forks | Both files | Fast with RDB preamble |
How much data could you lose? An illustrative estimate
Illustrative estimate (assumed numbers, not a benchmark). The inputs below are made up so you can see the arithmetic. Replace them with your own figures.
- Job enqueue rate: 200 jobs/s
- Session write rate: 50 writes/s
- RDB snapshot interval: 5 minutes (300 s), with a snapshot that takes 30 s to finish writing
- Disk stall during an
fsync: 8 s
The worst-case loss follows one formula:
lost writes ≈ write rate × unprotected window
With RDB, the unprotected window covers everything since the last completed snapshot. If the server crashes just before the next snapshot finishes, that window is the interval plus the snapshot duration. With AOF everysec, the window is normally about one second. Redis can delay writes while a background fsync is still running, so plan for roughly two seconds. If the disk stalls, the window grows to about the stall length plus that normal second.
| Scenario | Unprotected window | Jobs lost | Session writes lost |
|---|---|---|---|
| RDB every 5 min | ~330 s | ~66,000 | ~16,500 |
AOF everysec, healthy disk |
~1–2 s | ~200–400 | ~50–100 |
AOF everysec, 8 s stall |
~9 s | ~1,800 | ~450 |
Read these numbers with two caveats.
Session writes are an upper bound. Many session writes overwrite the same keys, so the number of distinct sessions affected is usually smaller.
everysec loss mainly applies to host or power failure. If only the Redis process crashes, data it already handed to the kernel normally still reaches disk.
To set your own budget, decide how many lost jobs or sessions you can accept. Divide that number by your write rate to get the largest window you can tolerate, then pick the persistence setting whose window fits.
Which persistence setting should you use for sessions and for queues?
Sessions: usually reconstructable
A lost session costs a re-login or a rebuilt cart, not lost money. RDB snapshots alone, or AOF with appendfsync everysec, cover most session stores. Turning persistence off is also a valid choice if users can tolerate a cold cache after a restart.
One configuration is dangerous: a primary with persistence disabled that a supervisor restarts automatically. It comes back empty, its replicas resynchronize from it, and they discard their copies too. If you run without persistence, either disable auto-restart on the primary or make sure a replica is promoted before the old primary rejoins.
Queues: the only copy of the work
If a list or stream holds jobs that exist nowhere else, a restart that loses data loses work silently. Use AOF and choose the fsync policy on purpose. everysec accepts roughly a second of loss. always trades write latency for a smaller window.
Persistence still does not give you delivery guarantees. It only protects what reached disk. A worker can pop a job and crash, or Redis can restart after a job was processed but before the acknowledgment persisted. The consumer has to handle both cases:
- Assume at-least-once delivery and make handlers idempotent.
- With streams, use consumer groups and reclaim stale entries from the pending entries list.
- With lists, move items atomically into a per-worker processing list and remove them only after the work succeeds.
Getting this mapping right before launch is far cheaper than investigating jobs that vanished without an error later.
# --- sessions.conf: reconstructable data, a cold cache is acceptable ---
# RDB snapshots only: snapshot if >=1 change in 900s or >=100 changes in 300s.
# A crash can lose the writes since the last snapshot. For sessions that
# means some users log in again.
save 900 1
save 300 100
appendonly no
# Alternative: no persistence at all (pure cache).
# save ""
# appendonly no
# If this node is a primary, do not let a supervisor auto-restart it.
# It would come back empty and its replicas would resync to that empty dataset.
# --- queues.conf: lists/streams are the only copy of the work ---
# Log every write to the append-only file.
appendonly yes
appendfilename "appendonly.aof"
# Deliberate fsync choice:
# everysec -> up to about 1s of acknowledged writes can be lost on a crash
# always -> fsync on every write, smaller loss window, higher write latency
appendfsync everysec
# Keep snapshots too. They are useful for backups and faster restarts.
save 900 1
# Rewrites start with an RDB preamble, which keeps restart time down.
aof-use-rdb-preamble yes
# Even with this file, consumers must still be idempotent and must
# reclaim stale pending stream entries (or use a processing list).
# AOF only protects writes that reached disk.
Two redis.conf profiles: RDB-only (or no persistence) for a reconstructable session store, and AOF with an explicit appendfsync policy for a queue whose lists or streams hold the only copy of the work.
When should you avoid relying on Redis persistence for durability?
Persistence protects a single node against a restart. It says nothing about what happens when a different node takes over. Redis replication is asynchronous: the primary acknowledges a write to the client, then streams it to replicas. Suppose the primary crashes before a write reaches any replica, and Sentinel or Cluster promotes one of those replicas. That write is gone, even with appendfsync always. Then the old primary rejoins as a replica, resynchronizes from the new primary, and its own AOF and RDB files are replaced. The data you fsynced to disk does not come back.
WAIT narrows this window but does not close it. It blocks the client until a given number of replicas have acknowledged receiving the preceding writes, or until a timeout expires. It returns how many replicas actually acknowledged, so the client has to check that count. A successful WAIT confirms that the replicas have the write in memory. It does not mean they have persisted it. It also does not guarantee that the replica promoted during a failover is one of the replicas that acknowledged. Redis does not claim that WAIT makes it strongly consistent, and you should not treat it that way.
This matters most when losing or duplicating an acknowledged write has real consequences:
- Payments, balances and ledger entries
- Inventory reservations
- Exactly-once job processing, where a lost dequeue acknowledgement causes reprocessing
For these, make something else the system of record. That can be a database with transactional durability, or a durable log or message broker built for replicated commits. Redis can still sit in front of it as a cache, rate limiter or short-lived buffer. When Redis loses data in that setup, you rebuild it from the system of record.
#!/usr/bin/env bash
# Write a key, then ask Redis how many replicas have received it.
set -euo pipefail
REDIS_HOST="redis-primary.internal" # placeholder hostname
REDIS_PORT=6379
MIN_REPLICAS=1 # replicas that must acknowledge
TIMEOUT_MS=500 # do not block longer than this
# WAIT only counts writes made earlier on the same connection, so both
# commands go through one redis-cli invocation, which uses one connection.
# When stdout is not a terminal, redis-cli prints raw replies:
# "OK" for SET, then the replica count returned by WAIT.
out=$(printf 'SET session:abc123 "user=42"\nWAIT %d %d\n' \
"$MIN_REPLICAS" "$TIMEOUT_MS" \
| redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT")
acked=$(printf '%s\n' "$out" | tail -n 1)
# WAIT returns after the timeout even if too few replicas acknowledged.
# The client has to check the count itself.
if [ "$acked" -lt "$MIN_REPLICAS" ]; then
echo "only $acked replica(s) acknowledged; a failover now could lose this write" >&2
exit 1
fi
# Even when this check passes, the replicas have the write in memory only.
# There is no guarantee that a replica which acknowledged is the one promoted
# during a failover. Treat this as a narrower loss window, not as durability.
echo "write acknowledged by $acked replica(s)"
A redis-cli script that sends a write followed by WAIT on the same connection and checks the replica acknowledgement count, which shrinks the failover loss window but does not make the write durable.
What do fork, memory and recovery time cost you?
Both BGSAVE and AOF rewrite call fork(). The child shares the parent's memory pages through copy-on-write. Every page the parent modifies during the save gets duplicated. Under a write-heavy workload, a queue being pushed and popped constantly can touch most pages, so worst-case memory approaches double the dataset. If the host can't supply it, the kernel's OOM killer may terminate Redis. The fork call itself must copy page tables, which takes longer as the dataset grows and blocks the main thread, so expect latency spikes on large instances. Size memory headroom for the worst case. Also check vm.overcommit_memory: Redis logs a startup warning when it isn't set to 1, because with the default setting the fork can fail outright.
AOF files grow with every write until a rewrite compacts them. That means more disk usage and longer restarts. Rewrites trigger on size thresholds, not on a schedule you choose, so the fork and I/O pressure can land during peak traffic. On restart, an RDB file loads as a compact binary image, while an AOF must be replayed command by command, so a large AOF usually recovers more slowly.
If a crash leaves a truncated AOF, aof-load-truncated lets Redis load what's intact. For corruption elsewhere in the file, back up the file first, then run redis-check-aof --fix. It discards everything from the first bad command onward, so review what it removes before you restart.
Illustrative estimate (assumed numbers, not a benchmark): recovery time ≈ file size ÷ load rate. A 20 GB AOF replayed at an assumed 200 MB/s takes about 100 seconds before Redis serves traffic. Measure your own load rate, because that downtime belongs in your availability budget.
#!/usr/bin/env bash
# Pre-flight checks for fork cost, then a cautious AOF repair.
set -euo pipefail
# 1. Overcommit: Redis warns at startup unless this is 1.
# With the default (0), fork() for BGSAVE or AOF rewrite can fail.
sysctl vm.overcommit_memory
# To change it persistently, add 'vm.overcommit_memory = 1' to /etc/sysctl.conf
# 2. How long did the last fork block the main thread? (microseconds)
redis-cli INFO stats | grep latest_fork_usec
# 3. Dataset size, to size worst-case copy-on-write headroom (up to ~2x)
redis-cli INFO memory | grep -E '^used_memory_human'
# 4. Rewrite triggers: size thresholds, not a schedule you control
redis-cli CONFIG GET auto-aof-rewrite-percentage
redis-cli CONFIG GET auto-aof-rewrite-min-size
redis-cli CONFIG GET aof-load-truncated
# --- Repair after a crash (run with Redis stopped) ---
# Placeholder path: on Redis 7+ the AOF is split into multiple files
# inside appenddirname; pass the manifest file there instead.
AOF=/var/lib/redis/appendonly.aof
# Back up first: --fix truncates the file at the first bad command.
cp -a "$AOF" "$AOF.bak.$(date +%Y%m%d%H%M%S)"
# Check without modifying, so you can see where the damage starts.
redis-check-aof "$AOF" || true
# Repair. It asks for confirmation and discards everything
# from the first invalid command to the end of the file.
redis-check-aof --fix "$AOF"
# Compare sizes to see roughly how much data was dropped before restarting.
ls -l "$AOF" "$AOF".bak.*
A shell checklist that checks overcommit, fork latency, memory size and AOF rewrite triggers, then backs up and repairs a damaged AOF with redis-check-aof.
How do you configure and verify Redis persistence?
the restart budget, not just the disk budget.
Two starting profiles
For sessions, where losing a few minutes means some users log in again, rely on snapshots: save 300 100 plus save 60 10000, with appendonly no. Snapshot frequency scales with write volume, and there is no per-write disk cost.
For queues, set appendonly yes and appendfsync everysec, or always if a lost acknowledged job costs more than the extra write latency. Set aof-use-rdb-preamble yes so rewrites produce a compact RDB base followed by recent commands, which shortens replay. Keep a save rule anyway, because RDB files are what you ship off the host.
Checking state
INFO persistence shows whether persistence is working. Alert when rdb_last_bgsave_status or aof_last_write_status is not ok. Watch rdb_changes_since_last_save to see your current exposure, and check aof_rewrite_in_progress and aof_last_bgrewrite_status to catch rewrites that are stuck or failing.
Measuring loss in staging
Write sequence-numbered entries at production rate. Run kill -9 on the Redis process, restart it, and read back the highest sequence number that survived. The gap between that number and the last write the client saw acknowledged is your measured loss window.
Backups
RDB files are written to a temporary file and renamed into place, so you can safely copy dump.rdb off the host on a schedule. A backup you have never restored is unproven, so test the restore. Remember that a node with appendonly yes loads the AOF at startup, not the RDB. Restore into a scratch instance and compare DBSIZE against the source.
An afternoon running this test costs far less than discovering your real loss window during an incident.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to redis rdb vs aof, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.