Need to send an email in 30 minutes? Retry a failed webhook in 5 seconds? Cancel an unpaid order after 30 minutes? A delayed queue is the answer, and Redis ZSET makes it surprisingly elegant.

The Problem

Standard queues (LIST-based) process jobs immediately. But many real-world scenarios need delayed execution:

• Send a reminder email 24 hours after signup.
• Auto-cancel an order if not paid within 30 minutes.
• Retry a failed API call with exponential backoff.
• Schedule a notification for a specific time.

The Pattern: ZSET + Timestamp

Use a Sorted Set where:

• The score is the Unix timestamp when the job should execute.
• The member is the job payload (or job ID).

Scheduling Jobs

ZADD delayed:queue 1739600100 '{"type":"email","to":"user@example.com","template":"welcome"}'
ZADD delayed:queue 1739600200 '{"type":"cancel_order","order_id":5001}'
ZADD delayed:queue 1739600050 '{"type":"webhook_retry","url":"https://api.example.com","attempt":2}'

Checking Pending Jobs

ZRANGEBYSCORE delayed:queue -inf +inf WITHSCORES
ZCARD delayed:queue

Fetching Due Jobs

Poll for jobs whose score (timestamp) is less than or equal to the current time:

ZRANGEBYSCORE delayed:queue 0 1739600150 WITHSCORES

This returns all jobs scheduled to execute before timestamp 1739600150.

Consuming Jobs Atomically

The critical challenge: multiple workers polling the same queue. Without atomic consumption, two workers can grab the same job.

Pattern: ZRANGEBYSCORE + ZREM

ZRANGEBYSCORE delayed:queue 0 1739600150 LIMIT 0 1
ZREM delayed:queue '{"type":"webhook_retry","url":"https://api.example.com","attempt":2}'

But this is NOT atomic. Between ZRANGEBYSCORE and ZREM, another worker might grab the same job.

Better: Lua Script (Atomic Pop)

In production, use a Lua script that atomically fetches and removes:

# Pseudocode:
local jobs = redis.call("ZRANGEBYSCORE", key, 0, now, "LIMIT", 0, 1)
if #jobs > 0 then
    redis.call("ZREM", key, jobs[1])
    return jobs[1]
end
return nil

Alternative: ZPOPMIN (Redis 5.0+)

ZPOPMIN atomically removes and returns the member with the lowest score:

ZADD delayed:queue 1739600050 "job:1" 1739600100 "job:2" 1739600200 "job:3"
ZPOPMIN delayed:queue
ZPOPMIN delayed:queue
ZRANGE delayed:queue 0 -1 WITHSCORES

But ZPOPMIN doesn't check if the job is due — it just pops the lowest score. Your worker must verify the timestamp before processing.

Retry with Backoff

When a job fails, re-schedule it with a later timestamp:

ZADD delayed:queue 1739600050 '{"type":"webhook","attempt":1}'

ZREM delayed:queue '{"type":"webhook","attempt":1}'
ZADD delayed:queue 1739600110 '{"type":"webhook","attempt":2}'

ZREM delayed:queue '{"type":"webhook","attempt":2}'
ZADD delayed:queue 1739600230 '{"type":"webhook","attempt":3}'

Backoff schedule: attempt 1 at T+0, attempt 2 at T+60, attempt 3 at T+180.

Dead Letter Queue

After max retries, move the job to a dead letter queue for manual inspection:

ZADD delayed:dlq 1739600230 '{"type":"webhook","attempt":3,"error":"timeout"}'
ZRANGE delayed:dlq 0 -1 WITHSCORES

Job Deduplication

Prevent duplicate jobs by using a deterministic member value:

ZADD delayed:queue 1739600100 "cancel_order:5001"
ZADD delayed:queue 1739600200 "cancel_order:5001"
ZSCORE delayed:queue "cancel_order:5001"

The second ZADD updates the score (reschedules) instead of creating a duplicate. This is useful for "debouncing" — e.g., resetting the cancel timer when a user takes action.

Cancelling Scheduled Jobs

Simply remove the member:

ZADD delayed:queue 1739600100 "cancel_order:5001"
ZREM delayed:queue "cancel_order:5001"
ZCARD delayed:queue

If the user pays before the 30-minute timeout, remove the cancel job.

Monitoring

Queue Depth

ZCARD delayed:queue

Overdue Jobs (Should Have Executed Already)

ZRANGEBYSCORE delayed:queue 0 1739599000 WITHSCORES
ZCOUNT delayed:queue 0 1739599000

If this returns results, your workers are falling behind.

Upcoming Jobs

ZRANGEBYSCORE delayed:queue 1739600000 +inf WITHSCORES LIMIT 0 10

Pitfalls

1. Polling interval: Too frequent = wasted CPU. Too slow = delayed execution. Start with 1-second polling and adjust.
2. Large payloads: Don't store full job data in the ZSET member. Store a job ID and keep the payload in a separate Hash:

ZADD delayed:queue 1739600100 "job:1001"
HSET job:1001 type "email" to "user@example.com" template "welcome"

3. No persistence guarantee: If Redis restarts without AOF/RDB, queued jobs are lost. Enable AOF with appendonly yes for durability.
4. Single consumer bottleneck: For high throughput, use multiple consumers with the Lua-based atomic pop pattern.
5. Timezone issues: Always use UTC Unix timestamps. Never use local time strings as scores.

Try It in the Editor

Head to the Redis Online Editor and build a delayed queue:

ZADD delayed:queue 100 "job:email:welcome" 200 "job:cancel:order:5001" 50 "job:webhook:retry"
ZRANGEBYSCORE delayed:queue 0 150 WITHSCORES
ZPOPMIN delayed:queue
ZRANGE delayed:queue 0 -1 WITHSCORES
ZADD delayed:queue 300 "job:webhook:retry:2"
ZCARD delayed:queue

Watch how ZPOPMIN grabs the job with the lowest score (earliest deadline) and removes it atomically. That's your delayed queue consumer in action.