Redis Message Queue: LIST vs STREAM — Differences and Use Cases

Redis offers two fundamentally different approaches to message queues: the classic LIST-based queue and the modern STREAM data type (Redis 5.0+). This guide compares them head-to-head with real commands.

The Problem

Your application needs asynchronous message processing. You could use RabbitMQ or Kafka, but you already have Redis. Can Redis handle it? Yes — but you need to pick the right data structure.

LIST-Based Queue

The simplest message queue pattern in Redis. Producers push, consumers pop.

Basic Producer/Consumer

RPUSH queue:tasks '{"type":"send_email","to":"user@example.com"}'
RPUSH queue:tasks '{"type":"process_payment","order_id":5001}'
RPUSH queue:tasks '{"type":"generate_report","report_id":301}'

LPOP queue:tasks
LPOP queue:tasks
LLEN queue:tasks

RPUSH adds to the tail, LPOP removes from the head — FIFO order.

Blocking Pop (BLPOP)

LPOP returns nil if the queue is empty. BLPOP blocks until a message arrives:

RPUSH queue:tasks "job:1"
BLPOP queue:tasks 5

The 5 is a timeout in seconds. The consumer blocks for up to 5 seconds waiting for a message. This is much more efficient than polling with LPOP in a loop.

Reliable Queue (RPOPLPUSH)

What if a consumer crashes after popping a message but before processing it? The message is lost.

Solution: RPOPLPUSH (or LMOVE in Redis 6.2+) atomically moves the message to a processing list:

RPUSH queue:tasks "job:1" "job:2" "job:3"
RPOPLPUSH queue:tasks queue:processing
LRANGE queue:processing 0 -1
LRANGE queue:tasks 0 -1

After successful processing, remove from the processing list:

LREM queue:processing 1 "job:1"

If the consumer crashes, messages in queue:processing can be re-queued.

LIST Queue Limitations

  • No consumer groups — only one consumer can pop a message.
  • No message acknowledgment built-in.
  • No message replay — once popped, it's gone.
  • No message IDs — you can't track individual messages.

STREAM-Based Queue

Redis Streams (5.0+) are a log-like data structure designed specifically for messaging.

Adding Messages

XADD stream:events * type "user_signup" user_id "1001"
XADD stream:events * type "order_placed" order_id "5001"
XADD stream:events * type "payment_received" order_id "5001"

The * auto-generates a unique ID (timestamp-based).

Reading Messages

XRANGE stream:events - +
XLEN stream:events

XRANGE - + reads all messages from start to end.

Reading New Messages (Tail)

XREAD COUNT 2 STREAMS stream:events 0-0

0-0 reads from the beginning. Use $ to read only new messages (like tail -f).

Consumer Groups

This is where Streams shine. Multiple consumers can share the workload:

XGROUP CREATE stream:events group1 0

XREADGROUP GROUP group1 consumer-a COUNT 1 STREAMS stream:events >
XREADGROUP GROUP group1 consumer-b COUNT 1 STREAMS stream:events >

The > means "give me messages not yet delivered to any consumer in this group."

Message Acknowledgment

After processing, acknowledge the message:

XACK stream:events group1 1739600001-0

Unacknowledged messages can be claimed by other consumers (for crash recovery):

XPENDING stream:events group1 - + 10

Claiming Stale Messages

If a consumer dies, another can claim its pending messages:

XAUTOCLAIM stream:events group1 consumer-b 60000 0-0

This claims messages that have been pending for more than 60 seconds.

Head-to-Head Comparison

FeatureLISTSTREAM
Data modelSimple queueAppend-only log
Consumer groupsNoYes
Message acknowledgmentManual (RPOPLPUSH)Built-in (XACK)
Message replayNo (destructive read)Yes (non-destructive)
Message IDsNoAuto-generated
Blocking readBLPOPXREAD BLOCK
Memory efficiencyHigherLower (stores IDs + fields)
Max throughputHigherSlightly lower
ComplexitySimpleModerate

When to Use LIST

  • Simple task queue with a single consumer.
  • You don't need message replay or acknowledgment.
  • Maximum throughput is critical.
  • You want the simplest possible implementation.
RPUSH jobs "task:1" "task:2" "task:3"
LPOP jobs
LPOP jobs
LLEN jobs

When to Use STREAM

  • Multiple consumers need to share the workload (consumer groups).
  • You need message acknowledgment and retry.
  • You want to replay messages (e.g., for debugging or new consumers).
  • You need message ordering guarantees with IDs.
XADD events * action "click" page "/home"
XADD events * action "purchase" item "widget"
XRANGE events - +
XLEN events

Hybrid Pattern: LIST for Speed, STREAM for Reliability

Some architectures use both:

  1. Hot path: LIST queue for maximum throughput.
  2. Audit trail: STREAM for logging and replay.
RPUSH queue:fast "job:1"
XADD stream:audit * job_id "1" status "queued"

LPOP queue:fast
XADD stream:audit * job_id "1" status "processing"

Pitfalls

  1. LIST memory: Unbounded lists grow forever. Monitor LLEN and set alerts.
  2. STREAM memory: Streams also grow unbounded. Use MAXLEN to cap:
XADD stream:events MAXLEN ~ 10000 * type "event" data "payload"
XLEN stream:events

The ~ allows approximate trimming for better performance.

  1. BLPOP timeout: A timeout of 0 blocks forever. Always set a reasonable timeout.
  2. Consumer group lag: Monitor pending messages with XPENDING. A growing pending list means consumers are falling behind.
  3. Stream ID ordering: Don't use custom IDs unless you understand the implications. Auto-generated IDs guarantee ordering.

Try It in the Editor

Head to the Redis Online Editor and compare both approaches:

RPUSH queue:list "msg:1" "msg:2" "msg:3"
LPOP queue:list
LLEN queue:list

XADD stream:demo * greeting "hello" from "alice"
XADD stream:demo * greeting "world" from "bob"
XRANGE stream:demo - +
XLEN stream:demo

Notice how LPOP removes the message (destructive), while XRANGE reads without removing (non-destructive). That's the fundamental difference.