Skip to main content

Caching Patterns

Strategic data caching to reduce database load and improve latency

TL;DR​

Caching strategically trades memory for latency, reducing database load and improving response times. Cache-aside: application checks cache, miss hits database—simplest, best for read-heavy patterns. Write-through: update cache and DB together—consistent, slower. Write-behind: update cache only, async DB—fastest but risky. Choose based on consistency requirements and read/write ratio. Always plan TTL (expiration), cache invalidation, and monitor hit rates to ensure effectiveness.

Learning Objectives​

By the end of this article, you will understand:

  • Three core caching patterns: cache-aside, write-through, write-behind
  • When to use each pattern and their trade-offs
  • Cache invalidation strategies: TTL vs event-driven
  • Cache stampede problem and solutions
  • Monitoring cache effectiveness
  • Operational considerations and best practices
  • Warming, coherence, and failure scenarios

Motivating Scenario​

Your user profile service queries the database 10,000 times per second. Database CPU at 80%; each query takes 5ms. Adding caching to serve profiles from Redis (0.5ms) reduces database load by 90%. But cache must stay synchronized with database writes. If cached profile is stale, users see old data. Strategy: use cache-aside with TTL for freshness, or event-driven invalidation for critical updates.

Core Concepts​

Three Caching Patterns​

Caching Patterns Comparison

1. Cache-Aside (Lazy Loading)

Application is responsible for managing cache:

def get_user(user_id):
    # Check cache first
    cached = redis.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)

    # Cache miss: query database
    user = db.query("SELECT * FROM users WHERE id = ?", user_id)
    if user:
        redis.setex(f"user:{user_id}", 3600, json.dumps(user))  # TTL: 1 hour
    return user

Pros: Simple, handles missing data naturally, flexible Cons: First request slow (cold cache), stale data possible When: Read-heavy workloads, can tolerate eventual consistency

2. Write-Through

Update cache and database atomically:

def update_user(user_id, data):
    # Update cache and DB together
    try:
        db.update("UPDATE users SET ... WHERE id = ?", user_id, **data)
        redis.set(f"user:{user_id}", json.dumps(data))
        return True
    except:
        return False

Pros: Cache always consistent, no stale data Cons: Slower writes, requires transaction support When: Consistency critical, acceptable write latency

3. Write-Behind (Write-Back)

Update cache immediately, DB asynchronously:

def update_user(user_id, data):
    # Update cache immediately (fast)
    redis.set(f"user:{user_id}", json.dumps(data))

    # Queue for async DB write
    queue.enqueue(update_database, user_id, data)
    return True

def update_database(user_id, data):
    # Background worker: write to database
    db.update("UPDATE users SET ... WHERE id = ?", user_id, **data)

Pros: Very fast writes, excellent performance Cons: Data loss if cache fails before DB write, complex When: Performance critical, acceptable data loss risk

Practical Example​

Implementing Cache-Aside with Invalidation​

import redis
import json
import hashlib
from functools import wraps
from datetime import datetime, timedelta

class CachedDatabase:
    def __init__(self, db_conn, redis_host='localhost'):
        self.db = db_conn
        self.cache = redis.Redis(host=redis_host, decode_responses=True)

    def cached_get(self, key, ttl=3600):
        """Cache-aside pattern with TTL"""
        def decorator(func):
            @wraps(func)
            def wrapper(*args, **kwargs):
                # Check cache
                cached_value = self.cache.get(key)
                if cached_value:
                    return json.loads(cached_value)

                # Cache miss: call function
                result = func(*args, **kwargs)

                # Store in cache with TTL
                if result:
                    self.cache.setex(key, ttl, json.dumps(result))
                return result

            return wrapper
        return decorator

    def invalidate(self, pattern):
        """Invalidate cache by key pattern"""
        keys = self.cache.keys(pattern)
        if keys:
            self.cache.delete(*keys)

    def get_user(self, user_id):
        @self.cached_get(f"user:{user_id}", ttl=1800)  # 30 min TTL
        def _fetch():
            cursor = self.db.cursor()
            cursor.execute("SELECT * FROM users WHERE id = ?", (user_id,))
            row = cursor.fetchone()
            return dict(row) if row else None
        return _fetch()

    def update_user(self, user_id, **data):
        # Update database
        cursor = self.db.cursor()
        cursor.execute(
            "UPDATE users SET {} WHERE id = ?".format(
                ", ".join([f"{k} = ?" for k in data.keys()])
            ),
            (*data.values(), user_id)
        )
        self.db.commit()

        # Invalidate cache
        self.invalidate(f"user:{user_id}*")
        return True

    def prevent_cache_stampede(self, key, ttl=3600):
        """Use probabilistic early expiration (XFetch pattern)"""
        cached = self.cache.get(key)
        if not cached:
            return None

        # Get TTL
        remaining_ttl = self.cache.ttl(key)

        # Refresh if TTL < 20% of original
        if remaining_ttl < ttl * 0.2:
            # Background refresh (simplified)
            self.cache.expire(key, ttl)

        return json.loads(cached)

    def warm_cache(self, query, prefix, ttl=3600):
        """Pre-populate cache on startup"""
        cursor = self.db.cursor()
        cursor.execute(query)
        count = 0
        for row in cursor.fetchall():
            key = f"{prefix}:{row[0]}"
            self.cache.setex(key, ttl, json.dumps(dict(row)))
            count += 1
        return count

    def cache_hit_rate(self):
        """Monitor cache effectiveness"""
        info = self.cache.info('stats')
        hits = info.get('keyspace_hits', 0)
        misses = info.get('keyspace_misses', 0)
        total = hits + misses
        rate = (hits / total * 100) if total > 0 else 0
        return {'hits': hits, 'misses': misses, 'hit_rate': rate}

# Usage
import sqlite3
db = sqlite3.connect(':memory:')
cache_db = CachedDatabase(db)

# Warm cache
cache_db.warm_cache("SELECT id, name FROM users", "user")

# Read with caching
user = cache_db.get_user(1)
print(f"User: {user}")

# Monitor
stats = cache_db.cache_hit_rate()
print(f"Cache hit rate: {stats['hit_rate']:.2f}%")

When to Use / When Not to Use​

Use Cache-Aside When:
  1. Read-to-write ratio is high (reads >> writes)
  2. Occasional stale data is acceptable
  3. Application can handle cache misses
  4. Simple to implement and debug
  5. Most web application read patterns
Use Write-Through When:
  1. Strong consistency is critical
  2. Write latency is acceptable
  3. Can support atomic cache+DB updates
  4. Financial transactions, critical data
  5. Cannot tolerate cache-DB divergence
Use Write-Behind When:
  1. Write performance is paramount
  2. Acceptable to lose recent writes if cache fails
  3. Have message queue for reliable async writes
  4. Can monitor queue and ensure DB catches up
  5. Analytics, event logging, non-critical writes

Patterns and Pitfalls​

Cache Stampede
Multiple requests miss cache simultaneously (TTL expired), all query database. Use probabilistic early expiration (refresh before expiry) or locks (only first request refreshes).
Cache Invalidation
TTL-based: simple, eventual staleness. Event-driven: immediate freshness, requires coordination. Hybrid: short TTL + event-driven for critical updates.
Warm Cache on Startup
Pre-populate cache with hot data. Prevents cold start period. Trade-off: slower startup vs faster runtime. Load top N records by popularity/access frequency.
Cache Coherence
Multiple caches (replicas, regions) must stay synchronized. Use consistent invalidation (publish invalidation events). Monitor for divergence.
Monitoring Hit Rate
Track hits vs misses. < 50% hit rate may indicate poor cache strategy or wrong TTL. Aim for > 80% for effective caching.
Memory Management
Monitor cache memory usage. Set eviction policy (LRU, LFU). If memory full, old entries evicted. Plan capacity: 2-3x hot working set.

Operational Considerations​

1. Cache Failure Handling​

# Graceful degradation: if cache fails, fall back to database
def get_user_resilient(user_id):
    try:
        return get_user_cached(user_id)
    except redis.ConnectionError:
        logger.warn(f"Cache unavailable, querying DB for user {user_id}")
        return get_user_from_db(user_id)

2. Cache Invalidation Strategies​

# 1. TTL-based (simple)
redis.setex("user:1", 3600, data)  # Expires after 1 hour

# 2. Event-driven (immediate)
def on_user_updated(user_id):
    redis.delete(f"user:{user_id}")  # Invalidate immediately
    publish_event("user.updated", user_id)

# 3. Hybrid (best of both)
redis.setex("user:1", 300, data)  # 5 min TTL
publish_event("user.updated", 1)  # Also notify immediately

3. Measuring Cache Effectiveness​

# Prometheus metrics
cache_hits = Counter('cache_hits_total', 'Cache hits')
cache_misses = Counter('cache_misses_total', 'Cache misses')

hit_rate = cache_hits / (cache_hits + cache_misses)
if hit_rate < 0.5:
    alert("Low cache hit rate")

Design Review Checklist​

  • Identified high-read-volume queries suitable for caching
  • Measured current latency and target latency with caching
  • Chose appropriate caching pattern (cache-aside, write-through, write-behind)
  • Designed TTL strategy: what staleness is acceptable?
  • Planned cache invalidation: TTL vs event-driven vs hybrid
  • Calculated hot working set size; allocated sufficient memory
  • Implemented cache stampede prevention (early expiration or locks)
  • Set up monitoring: hit rate, memory usage, eviction rate
  • Designed cache warm-up on application startup
  • Planned failure handling: graceful degradation if cache down
  • Documented cache keys, TTL, and invalidation strategy
  • Tested cache behavior under high concurrency and failures

Self-Check Questions​

  1. When would you use write-through vs cache-aside?

    • Write-through: consistency critical. Cache-aside: read-heavy, eventual consistency acceptable.

  2. How do you prevent cache stampede?

    • Probabilistic early expiration: refresh before TTL expires. Locks: only one request refreshes.

  3. What's cache coherence and why does it matter?

    • Multiple caches staying synchronized. Matters for consistency across replicas/regions.

  4. How do you measure cache effectiveness?

    • Track hit rate (hits / total requests). Aim for > 80%. < 50% indicates poor strategy.

Next Steps​

  1. Audit query patterns: Identify slow, frequently-repeated queries
  2. Measure baseline: Latency without caching
  3. Implement cache-aside: Start with simplest pattern
  4. Set TTL conservatively: Start short (5-15 min), extend if staleness acceptable
  5. Monitor hit rate: Set up dashboards and alerts
  6. Optimize: Based on hit rate, adjust TTL or cache strategy
  7. Scale: Add Redis replicas as needed

References​