← All posts

Redis Internals: Data Structures and Use Cases

Explore the core data structures that power Redis, how they’re implemented in memory, and practical scenarios where each structure shines. From fast key‑value lookups to complex real‑time analytics, this guide dives deep into Redis’s architecture and best‑practice use cases.

🚀 The Simple Version (ELI5)

Think of Redis as a super‑fast post‑it board that can hold many different kinds of notes. Each note type—like a single word, a list of words, or a set of unique words—has its own special way of being stored so it can be found and updated instantly. This guide shows how those note types are built and when you’d use each one.

🔍 Understanding Redis Data Structures

Redis supports several native data types, each optimized for specific access patterns. Below is a quick reference:

  • String – Single value (bytes). Best for counters, flags, or simple cache entries.
  • List – Ordered sequence. Ideal for queues, timelines, or LRU caches.
  • Set – Unordered unique values. Great for membership tests and tag collections.
  • Sorted Set (ZSET) – Set with a score for ordering. Used for leaderboards, priority queues.
  • Hash – Key‑value pairs within a key. Perfect for storing object attributes.
  • HyperLogLog – Probabilistic cardinality estimation. Useful for unique visitor counts.
  • Bitmaps – Bit-level manipulation. Handy for tracking binary flags.
  • Streams – Log‑like data structure. Suitable for event sourcing and message brokering.

💾 In‑Memory Storage Mechanics

Redis stores everything in RAM for lightning‑fast access. Each data type has a dedicated C structure that maps directly to the memory layout:

typedef struct RedisObject {
    int type;           // OBJ_STRING, OBJ_LIST, etc.
    int encoding;       // raw, int, ziplist, skiplist, etc.
    void *ptr;          // pointer to the actual data
    int refcount;       // for memory management
} RedisObject;

Redis chooses the most compact representation based on size. For example, small lists use a ziplist, while larger ones switch to a linked list or skiplist for efficiency. This adaptive encoding is a key factor in Redis’s low memory footprint.

🧩 Detailed Structure Implementations

String

Strings are stored as raw byte arrays. When the value is small, Redis uses a simple raw encoding; larger values may be stored as a quicklist (a hybrid of linked list and ziplist) to reduce fragmentation.

List

Lists use a quicklist internally, which is a doubly‑linked list of ziplists. Each ziplist packs up to 512 items in a compact format. When a list grows beyond a threshold, Redis automatically splits it into multiple ziplists to keep push/pop operations O(1).

Set

Sets are implemented either as a hashtable (for large sets) or a ziplist (for small sets). The hash table uses open addressing and a power‑of‑two bucket count for fast lookups.

Sorted Set

Sorted Sets combine a hash table (mapping members to scores) and a skiplist (maintaining order by score). The skiplist provides O(log N) insertion and O(log N) range queries.

Hash

Hashes are stored as ziplists for small objects or as hashtables for larger ones. This allows efficient field lookup and minimal memory usage for sparse objects.

Streams

Streams are built on a deque of entries, each containing a message ID, fields, and values. Internally, Redis uses a combination of a linked list and a hash table for quick ID lookups and consumer group management.

⚙️ Persistence & Replication Considerations

Redis offers two main persistence modes:

  • RDB snapshots – Point‑in‑time dumps written to disk. Ideal for backups.
  • AOF (Append Only File) – Log of every write operation. Provides stronger durability.

Replication is handled via asynchronous master‑replica sync. Each replica maintains its own in‑memory copy and receives updates over a network connection. The data structures remain identical across nodes, ensuring consistency.

🚀 Use Cases by Data Structure

String: Cache and Counters

  • Session storage for web apps.
  • Atomic counters using INCR/DECR.
  • Feature flags toggled with SETNX.

List: Message Queues

  • Task queues with BRPOP/RPOPLPUSH.
  • Real‑time feed pipelines.
  • Rate limiting windows.

Set: Unique Membership

  • User interest tags.
  • Deduplication of event streams.
  • Access control lists.

Sorted Set: Leaderboards & Time‑Series

  • Game leaderboards with ZREVRANGE.
  • Time‑ordered logs with ZADD and ZRANGEBYSCORE.
  • Priority task scheduling.

Hash: Object Storage

  • Storing user profiles with HMSET.
  • Configuration objects with HGETALL.
  • Partial updates to nested data.

HyperLogLog: Approximate Counting

  • Unique visitor analytics.
  • Distinct product views per day.
  • Large‑scale set cardinality estimation.

Bitmaps: Feature Flags & Bloom Filters

  • Tracking daily active users via BITCOUNT.
  • Implementing Bloom filters for fast membership tests.
  • Flagging processed items in data pipelines.

Streams: Event Sourcing & Pub/Sub

  • Real‑time analytics pipelines.
  • Consumer groups for load‑balanced processing.
  • Log aggregation with XREADGROUP.

🔧 Performance Tips

  • Use ziplist encoding for small collections to reduce overhead.
  • Prefer sorted sets over lists when you need ordered data.
  • Keep hash fields under 512 bytes to avoid extra encoding.
  • Set maxmemory-policy to volatile-lru for cache‑like workloads.
  • Batch commands with MSET or MULTI to reduce round‑trips.

📌 Conclusion

Redis’s diverse data structures and efficient in‑memory design make it a versatile tool for many real‑world scenarios—from simple caching to complex event processing. By matching the right structure to your use case and tuning the underlying encoding, you can unlock maximum performance and scalability.