π₯ Redis Debugging & Troubleshooting Guide
| β Back to Home | Kafka β | NGINX β | PostgreSQL β |
A comprehensive guide for diagnosing and troubleshooting Redis performance, memory, persistence, and replication issues.
π Table of Contents
- Server Health & Info
- Memory Analysis
- Slow Queries & Latency
- Keyspace Inspection
- Persistence (RDB/AOF)
- Replication
- Cluster & Sentinel
- Client Connections
- Common Issues
π©Ί Server Health & Info
Basic Health Checks
# Ping the server
redis-cli ping
# Full server info
redis-cli info
# Specific info sections
redis-cli info server
redis-cli info memory
redis-cli info stats
redis-cli info replication
redis-cli info persistence
redis-cli info clients
# Server uptime and version
redis-cli info server | grep -E 'redis_version|uptime_in_days'
# Real-time stats (ops/sec, memory, connections)
redis-cli --stat
Key Metrics to Watch
# Hit ratio β below ~0.9 for a cache workload means keys evict/expire too fast
redis-cli info stats | grep -E 'keyspace_hits|keyspace_misses'
# Evicted and expired keys
redis-cli info stats | grep -E 'evicted_keys|expired_keys'
# Rejected connections (maxclients reached)
redis-cli info stats | grep rejected_connections
# Instantaneous ops per second
redis-cli info stats | grep instantaneous_ops_per_sec
π§ Memory Analysis
Memory Overview
# Human-readable memory report with recommendations
redis-cli memory doctor
# Detailed memory stats
redis-cli memory stats
# Key memory numbers
redis-cli info memory | grep -E 'used_memory_human|used_memory_rss_human|used_memory_peak_human|maxmemory_human|mem_fragmentation_ratio|maxmemory_policy'
Interpreting mem_fragmentation_ratio:
| Value | Meaning |
|---|---|
| ~1.0 β 1.5 | Healthy |
| > 1.5 | Fragmentation β RSS much larger than logical usage; consider activedefrag yes or a restart |
| < 1.0 | Redis is swapping β usually a severe problem, add RAM or lower maxmemory |
Finding Big Keys
# Sample the keyspace for the biggest key per type (safe, uses SCAN)
redis-cli --bigkeys
# Sample for keys with most memory usage (Redis 6+)
redis-cli --memkeys
# Memory used by one key
redis-cli memory usage <KEY_NAME>
# Memory usage sampling nested elements
redis-cli memory usage <KEY_NAME> samples 0
Eviction Policy
# Check current policy
redis-cli config get maxmemory-policy
redis-cli config get maxmemory
# Set policy at runtime (persist in redis.conf too)
redis-cli config set maxmemory-policy allkeys-lru
Policy cheat sheet: noeviction (writes fail when full β default), allkeys-lru (classic cache), volatile-lru (only keys with TTL), allkeys-lfu (frequency-based, better for skewed access).
π Slow Queries & Latency
Slowlog
# Show slow commands (default threshold 10ms)
redis-cli slowlog get 25
# How many slow entries recorded
redis-cli slowlog len
# Reset the slowlog
redis-cli slowlog reset
# Lower the threshold to 5ms (microseconds!)
redis-cli config set slowlog-log-slower-than 5000
Latency Monitoring
# Built-in latency doctor (needs latency-monitor-threshold set)
redis-cli config set latency-monitor-threshold 100
redis-cli latency doctor
# Latency history per event
redis-cli latency history command
# Measure round-trip latency from a client
redis-cli --latency
redis-cli --latency-history
# Intrinsic latency of the host itself (run ON the server)
redis-cli --intrinsic-latency 30
Live Command Stream
# Watch every command in real time β EXPENSIVE, use briefly in production
redis-cli monitor
# Per-command call counts and average time
redis-cli info commandstats
π Keyspace Inspection
# Keys per database
redis-cli info keyspace
# NEVER run KEYS * in production β O(N), blocks the server
# Use SCAN instead (cursor-based, non-blocking)
redis-cli --scan --pattern 'session:*'
# Count matching keys without listing them
redis-cli --scan --pattern 'cache:*' | wc -l
# Inspect a key
redis-cli type <KEY_NAME>
redis-cli ttl <KEY_NAME>
redis-cli object encoding <KEY_NAME>
# How stale is a key (seconds since last access)
redis-cli object idletime <KEY_NAME>
πΎ Persistence (RDB/AOF)
Check Persistence Status
# Persistence overview
redis-cli info persistence
# Critical fields
redis-cli info persistence | grep -E 'rdb_last_bgsave_status|rdb_last_save_time|aof_enabled|aof_last_write_status|aof_last_bgrewrite_status'
# When was the last successful save
redis-cli lastsave
RDB Snapshots
# Trigger a background snapshot
redis-cli bgsave
# Check snapshot config
redis-cli config get save
redis-cli config get dir
redis-cli config get dbfilename
# Verify an RDB file offline
redis-check-rdb /var/lib/redis/dump.rdb
AOF
# Check AOF status and rewrite
redis-cli config get appendonly
redis-cli bgrewriteaof
# Repair a truncated/corrupt AOF file (server must be stopped)
redis-check-aof --fix /var/lib/redis/appendonly.aof
If
rdb_last_bgsave_statusiserrandstop-writes-on-bgsave-errorisyes(the default), Redis rejects all writes withMISCONF. Fix the disk/fork problem β donβt just flip the flag.
π Replication
Replication Status
# On the primary: role, connected replicas, replication offsets
redis-cli info replication
# Check replica lag: compare master_repl_offset (primary)
# with slave_repl_offset (replica)
redis-cli -h <REPLICA_HOST> info replication | grep -E 'master_link_status|slave_repl_offset|master_last_io_seconds_ago'
Diagnosing Replication Problems
# Replica can't sync? Check backlog size β too small forces full resyncs
redis-cli config get repl-backlog-size
# Count of full/partial syncs β rising sync_full means repeated full resyncs
redis-cli info stats | grep -E 'sync_full|sync_partial_ok|sync_partial_err'
# Watch primary log for fork/COW memory during full sync
tail -f /var/log/redis/redis-server.log
Common causes of broken replication: network flaps + small repl-backlog-size (full resync loops), replica slower disk during RDB load, client-output-buffer-limit replica too small on the primary (replica disconnected mid-sync β raise it).
πΈοΈ Cluster & Sentinel
Cluster
# Cluster health β state must be "ok", all 16384 slots assigned
redis-cli cluster info
# Node list, roles, slot ranges
redis-cli cluster nodes
# End-to-end check with slot coverage
redis-cli --cluster check <HOST>:6379
# Find which node owns a key's slot
redis-cli cluster keyslot <KEY_NAME>
# Fix slot coverage problems
redis-cli --cluster fix <HOST>:6379
Sentinel
# Ask sentinel about the primary
redis-cli -p 26379 sentinel master <MASTER_NAME>
redis-cli -p 26379 sentinel get-master-addr-by-name <MASTER_NAME>
# List replicas and other sentinels
redis-cli -p 26379 sentinel replicas <MASTER_NAME>
redis-cli -p 26379 sentinel sentinels <MASTER_NAME>
# Force a failover (testing)
redis-cli -p 26379 sentinel failover <MASTER_NAME>
π Client Connections
# Connected client count vs limit
redis-cli info clients
redis-cli config get maxclients
# List every client with age, idle time, last command
redis-cli client list
# Find idle connections (idle > 300s)
redis-cli client list | awk '{for(i=1;i<=NF;i++) if($i ~ /^idle=/){split($i,a,"="); if(a[2]>300) print $0}}'
# Kill a misbehaving client
redis-cli client kill id <CLIENT_ID>
redis-cli client kill addr <IP>:<PORT>
# Check output buffer pressure (omem = output buffer memory)
redis-cli client list | grep -o 'omem=[0-9]*' | sort -t= -k2 -rn | head
π¨ Common Issues
Issue: OOM / βcommand not allowed when used memory > maxmemoryβ
Diagnose:
redis-cli info memory | grep -E 'used_memory_human|maxmemory_human|maxmemory_policy'
redis-cli info stats | grep evicted_keys
redis-cli --bigkeys
Common Causes:
maxmemory-policy noevictionwith a full instance- Missing TTLs on cache keys
- A few huge keys (multi-MB hashes/lists)
- Replication/client output buffers counted toward memory
Solutions:
- Switch to
allkeys-lru/allkeys-lfufor cache workloads - Add TTLs; break up big keys; raise
maxmemoryif RAM allows - Use
UNLINKinstead ofDELfor large keys (async free)
Issue: MISCONF β Redis refuses writes
Diagnose:
redis-cli info persistence | grep rdb_last_bgsave_status
df -h # disk full?
dmesg | grep -i oom # fork killed?
sysctl vm.overcommit_memory # should be 1
Solutions:
- Free disk space in the
dirpath - Set
vm.overcommit_memory=1sobgsavefork succeeds - Verify Redis has write permission to the data directory
Issue: Latency Spikes
Diagnose:
redis-cli slowlog get 25
redis-cli latency doctor
redis-cli info commandstats | sort -t= -k3 -rn | head
Common Causes:
- O(N) commands:
KEYS,SMEMBERS/HGETALL/LRANGE 0 -1on huge keys,SORT - Fork stalls during
BGSAVE/BGREWRITEAOFon hosts with THP enabled appendfsync alwayson slow disks- Swapping (check
mem_fragmentation_ratio < 1)
Solutions:
- Replace
KEYSwithSCAN; paginate range reads - Disable transparent huge pages:
echo never > /sys/kernel/mm/transparent_hugepage/enabled - Use
appendfsync everysec(default, good tradeoff)
Issue: Mass Key Expiry Storm
Diagnose:
redis-cli info stats | grep expired_keys # sudden spike
Solutions:
- Add jitter to TTLs (e.g. TTL + random 0β300s) so keys donβt all expire together
- Redis 6+: tune
active-expire-effortdown if expiry cycles hurt latency
Issue: Replica Keeps Doing Full Resyncs
Diagnose:
redis-cli info stats | grep sync_full
redis-cli config get repl-backlog-size
Solutions:
- Increase
repl-backlog-size(size it to cover your longest expected network blip at your write rate) - Raise
client-output-buffer-limit replica 512mb 128mb 120on the primary
π Best Practices
- Set
maxmemoryand an explicit eviction policy β never rely on defaults for cache workloads - Always use SCAN family, never
KEYS, in application code - Put TTLs on cache keys and jitter them
- Keep keys small β a hash of 100 fields beats 100 string keys, but a 50MB hash beats nothing
- Disable THP on Redis hosts and set
vm.overcommit_memory=1 - Monitor
mem_fragmentation_ratio, hit ratio, andrejected_connections - Use
UNLINKandFLUSHDB ASYNCfor large deletions - Test failover (Sentinel/Cluster) before you need it
- Require AUTH / ACLs and bind to private interfaces β an open Redis port is a shell for an attacker
- Size the replication backlog for real-world network blips
π Useful Tools
- redis-cli β official CLI (
--bigkeys,--latency,--statmodes) - redis-check-rdb / redis-check-aof β offline file verification
- RedisInsight β official GUI with memory analysis
- redis-benchmark β load testing
- rdb-tools / redis-rdb-cli β offline RDB analysis into CSV/JSON
β οΈ Security Notes
- Never expose Redis directly to the internet β bind to private IPs
- Enable
requirepassor Redis 6+ ACLs; disable/renameFLUSHALL,CONFIG,DEBUGfor app users - Use TLS (Redis 6+) for cross-network replication and clients
protected-mode yesmust stay on for instances without auth- Keep placeholders like
<REPLICA_HOST>out of committed configs β use env vars
| β Back to Home | Kafka β | NGINX β | Kubernetes β |