What is a NestJS BullMQ queue and when should you use one?

BullMQ is a Node.js job queue that stores jobs, their state and their retry metadata in Redis. Producers add jobs to a named queue, and workers pull them, run a handler, and mark each job completed or failed. The @nestjs/bullmq package wraps this in NestJS idioms. You register queues in a module, inject them into services as producers, and declare workers as processor classes. That way queue setup goes through the same dependency injection and lifecycle hooks as the rest of your app.

This guide is for backend and platform engineers running async work in NestJS services. A job queue in NestJS helps with three problems. It moves slow work, such as email, PDF rendering or third-party syncs, out of the request path. It absorbs traffic spikes by letting a backlog build instead of overloading downstream systems. It retries flaky calls without the client having to wait.

It is the wrong tool when:

  • You need durable messaging between services owned by different teams.
  • You cannot run Redis with maxmemory-policy set to noeviction. With an evicting policy, Redis can silently delete job data.
  • The work is a short, non-critical task that an in-process promise handles fine.
Option Good fit Main trade-off
BullMQ App-owned background jobs Redis becomes critical state
Message broker Cross-service events, fan-out More infrastructure and operations
Managed cloud queue Minimal ops, provider-native setups Provider limits and lock-in
Database queue table Low volume, transactional enqueue Polling and lock contention
Cron Fixed schedules No per-item retries

These settings are not independent. Retry policy, concurrency and observability interact, so you need to design them together. The rest of this article treats them that way.

Who should use NestJS with BullMQ?

BullMQ fits teams that meet most of these conditions:

  • You already run NestJS on Node.js. Queues, producers and processors become ordinary providers. They get dependency injection, configuration and lifecycle hooks the same way as the rest of the codebase, so there is no separate framework to learn.
  • The jobs belong to one application or one team. The same codebase produces and consumes the jobs, and the job payload is an internal contract that can change in a single deploy.
  • You can operate Redis as stateful infrastructure. That means persistence configured to match how much job loss you can tolerate, noeviction, memory monitoring, and a plan for failover. If Redis is currently only a disposable cache in your setup, using it for queues changes its role.
  • You need per-job retries, delays, rate limits or priorities. These are built into BullMQ. With simpler mechanisms you would have to write them yourself.

It is a weaker fit for teams that need a durable event log that many consumers replay independently, for strict cross-service contracts between separately deployed teams, or for organizations that cannot take on another stateful dependency. In those cases, one of the alternatives below is usually the better choice.

What are the alternatives to BullMQ in a NestJS service?

The table above summarizes the options. The deciding factors are these:

  • In-process async work (fire-and-forget promises, event emitters). No infrastructure. Work is lost on crash or deploy, and nothing retries it. Acceptable for best-effort tasks like cache warming. Not acceptable for anything a user or another system depends on.
  • A database-backed queue table. Workers claim rows with SELECT ... FOR UPDATE SKIP LOCKED or a similar locking pattern. The main advantage is transactional enqueue: the job row commits in the same transaction as the business write, so you never enqueue a job for data that rolled back. The costs are polling, lock contention on the table, and building retries, backoff and visibility yourself. This works well at low to moderate volume when you want to avoid adding Redis.
  • A message broker. Better for events consumed by several services, fan-out, and cross-team contracts. You take on more operational work, and per-job features like delayed retries with backoff often need extra configuration or application code.
  • A managed cloud queue. Your provider runs the infrastructure. You accept provider limits on message size, visibility timeouts and delivery semantics, and your code becomes coupled to that provider's API.
  • Cron or a scheduler. Right for fixed-schedule batch work. It does not provide per-item retries, backpressure or parallelism across items. BullMQ also supports repeatable jobs, so it can cover simple schedules if you already run it.

A common pattern combines two of these: commit the business record and an outbox row in one database transaction, then have a relay move outbox rows into BullMQ. You get transactional enqueue and BullMQ's retry and concurrency controls. The cost is one more moving part.

How does BullMQ work inside a NestJS application?

The job lifecycle

A producer calls queue.add(), which writes the job to Redis and places it in the waiting state. Nothing runs in the producer process. A worker atomically moves the job to active and takes a lock with a fixed lease (lockDuration). While your handler runs, the worker renews that lock in the background. This is why CPU-bound code that blocks the Node.js event loop is dangerous: the renewal timer can't fire, the lock expires, and another worker may pick up the same job.

The job then ends in one of four ways:

  • Completed: the handler resolved.
  • Failed: the handler threw and no attempts remain.
  • Delayed: the handler threw, attempts remain, and the job waits out its backoff.
  • Stalled: the lock expired without completion. A stall check returns the job to waiting, or fails it once it exceeds the stall limit.

Wiring it into NestJS

Register the Redis connection once with BullModule.forRoot(), then declare each queue with BullModule.registerQueue() in the module that owns it. Producers receive the queue through @InjectQueue('name'). Consumers are classes decorated with @Processor('name') that extend WorkerHost and implement an async process(job) method. Nest creates the underlying worker when the module initializes.

One deployable or two?

If the processor lives in the API app, every API replica also runs a worker. That is simple to start with. The cost is that queue throughput becomes tied to HTTP scaling, and heavy jobs compete with requests for the same event loop. A separate worker entrypoint lets you scale and tune each side on its own. In exchange, you take on a second build target, its own health checks, and shared queue and connection config that must stay in sync between the two.

import { Injectable, Module } from '@nestjs/common';
import { BullModule, InjectQueue, Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, Queue } from 'bullmq';

// Producer: only writes the job to Redis; nothing runs here.
@Injectable()
export class EmailService {
  constructor(@InjectQueue('emails') private readonly emailQueue: Queue) {}

  async sendWelcome(userId: string) {
    // 3 attempts total; failures go to 'delayed' with exponential backoff.
    await this.emailQueue.add('welcome', { userId }, {
      attempts: 3,
      backoff: { type: 'exponential', delay: 1000 },
    });
  }
}

// Consumer: Nest creates the BullMQ Worker for this class on module init.
// lockDuration is the lease the worker renews while process() runs.
@Processor('emails', { concurrency: 5, lockDuration: 30000 })
export class EmailProcessor extends WorkerHost {
  async process(job: Job<{ userId: string }>): Promise<void> {
    // Resolve -> completed. Throw -> delayed (attempts left) or failed.
    // Keep this async I/O; blocking the event loop stops lock renewal.
    console.log(`job ${job.id}: welcome email for ${job.data.userId}`);
  }
}

@Module({
  imports: [
    // One shared Redis connection for all queues in the app.
    BullModule.forRoot({ connection: { host: 'redis.internal', port: 6379 } }),
    BullModule.registerQueue({ name: 'emails' }),
  ],
  // Monolith: both providers here. For a separate worker deployable,
  // list EmailProcessor only in the worker app's module.
  providers: [EmailService, EmailProcessor],
})
export class EmailModule {}

A minimal @nestjs/bullmq module that registers the Redis connection and an 'emails' queue, enqueues retryable jobs from an injected producer service, and processes them in a WorkerHost subclass.

How do you configure BullMQ retries and backoff safely?

Retries are configured with two job options. attempts is the total number of times the job may run, including the first try. backoff sets the wait between tries. A fixed backoff waits the same delay every time. An exponential backoff doubles the wait after each failure, starting from delay. Set both in defaultJobOptions when you register the queue, so every producer gets the same policy. Override them per job only when one job type genuinely behaves differently, such as a webhook call that needs more patience than an internal recalculation.

Some failures will never succeed on retry: a validation error, a missing record, or a 4xx response from a downstream API. Throw BullMQ's UnrecoverableError for these. The job then goes straight to the failed state and skips its remaining attempts. You can also catch the error, record the outcome on your side, and complete the job normally. Without one of these, every permanent error uses up the full retry budget.

Backoff absorbs brief network blips and short downstream outages. It also has costs:

  • It delays detection: a real bug looks like "still retrying" until the last attempt fails.
  • Every retry sends more traffic to a downstream system that may already be struggling.

Illustrative estimate: assume attempts: 8 and exponential backoff with a 30-second base delay. The seven waits are 30, 60, 120, 240, 480, 960 and 1,920 seconds, a total of 3,810 seconds. That is about 63 minutes per job, plus execution time. A bad deploy can therefore keep thousands of jobs failing repeatedly for over an hour before any of them reaches the failed set.

import { Module, Injectable } from '@nestjs/common';
import { BullModule, InjectQueue, Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, Queue, UnrecoverableError } from 'bullmq';

@Module({
  imports: [
    BullModule.forRoot({ connection: { host: 'redis.internal', port: 6379 } }),
    // Queue-level default: every producer gets the same retry policy.
    BullModule.registerQueue({
      name: 'webhooks',
      defaultJobOptions: {
        attempts: 5, // total runs, including the first try
        backoff: { type: 'exponential', delay: 10_000 }, // 10s, 20s, 40s, 80s
      },
    }),
  ],
})
export class QueueModule {}

@Injectable()
export class WebhookProducer {
  constructor(@InjectQueue('webhooks') private readonly queue: Queue) {}

  // Per-job override only where this job type genuinely needs more patience.
  send(url: string, payload: unknown) {
    return this.queue.add('deliver', { url, payload }, {
      attempts: 8,
      backoff: { type: 'fixed', delay: 60_000 },
    });
  }
}

@Processor('webhooks')
export class WebhookProcessor extends WorkerHost {
  async process(job: Job<{ url: string; payload: unknown }>) {
    const res = await fetch(job.data.url, { method: 'POST', body: JSON.stringify(job.data.payload) });
    // 4xx will never succeed on retry: fail now and skip remaining attempts.
    if (res.status >= 400 && res.status < 500) throw new UnrecoverableError(`Rejected: ${res.status}`);
    // 5xx or network errors throw normally and consume the retry budget.
    if (!res.ok) throw new Error(`Downstream error: ${res.status}`);
  }
}

Queue-level exponential backoff defaults in BullModule.registerQueue, a per-job fixed-backoff override, and a processor that throws UnrecoverableError on 4xx responses so permanent failures skip the remaining attempts.

Why must every BullMQ job handler be idempotent?

BullMQ delivers each job at least once, not exactly once. Retries mean a handler can run more than once for the same job, so it must be safe to repeat. Duplicates come from three sources:

  • Stalls. If a worker's event loop is blocked long enough for its job lock to expire, BullMQ treats the job as stalled and moves it back to the waiting state. Another worker picks it up while the first may still be finishing.
  • Retries after partial success. A handler charges a card, then throws while writing the receipt. The retry charges the card again.
  • Crashes. A process killed between a side effect and job completion leaves the job active. It is later recovered as stalled and runs again.

Deduplicate at two points. When you enqueue, pass a deterministic job ID derived from the business event, such as invoice-${invoiceId}. BullMQ will not add a second job with an ID that already exists. That protection lasts only while the original job is retained. Once removeOnComplete or removeOnFail deletes it, the same ID can be enqueued again.

Inside the handler, check an idempotency key before every side effect. In a database, a unique constraint on the key, inserted in the same transaction as the write, is the most reliable check. In Redis, an atomic SET key value NX with a TTL works when the key only needs to live for a bounded window. For external APIs, forward the key if the provider accepts one.

This has a real cost. Every handler needs a storage lookup and deliberate design work, and no setting gives you exactly-once processing for free. In return, retries become safe to enable aggressively. You will also avoid investigating double charges and duplicate emails months after they happened.

import { InjectQueue, Processor, WorkerHost } from '@nestjs/bullmq';
import { Injectable } from '@nestjs/common';
import { Job, Queue } from 'bullmq';
import Redis from 'ioredis';

type ChargeJob = { invoiceId: string; amountCents: number };
const redis = new Redis('redis://localhost:6379'); // use an injected provider in real code
declare const payments: { charge(a: ChargeJob & { idempotencyKey: string }): Promise<void> };

@Injectable()
export class BillingService {
  constructor(@InjectQueue('billing') private readonly queue: Queue) {}

  // Deterministic ID: re-enqueueing the same invoice is a no-op while the job is retained.
  enqueueCharge(data: ChargeJob) {
    return this.queue.add('charge', data, { jobId: `invoice-${data.invoiceId}` });
  }
}

@Processor('billing')
export class BillingProcessor extends WorkerHost {
  async process(job: Job<ChargeJob>) {
    const key = `idem:charge:${job.data.invoiceId}`;
    // Atomic claim with a TTL: only one run (stalled, retried or duplicate) gets 'OK'.
    const claimed = await redis.set(key, job.id ?? 'unknown', 'EX', 86400, 'NX');
    if (claimed !== 'OK') return { skipped: true }; // side effect already done or in progress

    try {
      // Forward the same key so the provider also dedupes if we crash after charging.
      await payments.charge({ ...job.data, idempotencyKey: key });
    } catch (err) {
      await redis.del(key); // release the claim so a BullMQ retry can try again
      throw err;
    }
    return { charged: true };
  }
}

A deterministic jobId deduplicates enqueues, while an atomic Redis SET NX claim plus a forwarded idempotency key keeps the charge side effect safe across stalls, retries and crashes.

How should you size worker concurrency and rate limits?

The concurrency option on a worker sets how many jobs one worker instance runs at the same time. These jobs share a single Node.js event loop, so concurrency only helps when handlers spend most of their time waiting on I/O. Capacity also multiplies with deployment: total in-flight jobs equal concurrency × workers per process × replicas. As an illustrative estimate, 6 pods with concurrency 20 put up to 120 jobs in flight against your dependencies.

Size against the downstream limit

Begin with the tightest dependency, not with CPU. If each pod has a database pool of 10 connections, a concurrency of 20 leaves jobs waiting for a connection while their timeouts keep running. Once you pass a downstream limit, more concurrency adds queueing instead of throughput. Latency goes up, timeouts fire, and retries push the load higher, which turns a slowdown into a cascade.

Control Scope Protects against
concurrency Per worker instance Overloading local resources such as the connection pool and memory
limiter (max jobs per duration) Global across all workers on the queue, enforced through Redis Exceeding third-party or shared rate limits

Use the limiter when a provider allows N requests per second in total, no matter how many replicas you run.

CPU-bound handlers

Image resizing, PDF rendering or heavy parsing will block the event loop. That stalls every other job on the worker and delays lock renewal, which can make healthy jobs look stalled. Move this work into sandboxed processors or a separate service. Sandboxing has costs:

  • Each child process adds startup overhead.
  • The processor has no direct access to the NestJS DI container.
  • Debugging across process boundaries is harder.
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job } from 'bullmq';

// Assumed limits (placeholders): each pod has a DB pool of 10 connections,
// and the email provider allows 50 requests per second in total.
const DB_POOL_SIZE = 10;

@Processor('email', {
  // Per worker instance. Keep this at or below the pool size so jobs don't
  // wait for a connection while their timeouts keep running.
  // Total in flight = concurrency x workers per process x replicas.
  concurrency: DB_POOL_SIZE,

  // Enforced through Redis across every worker on this queue, so the
  // provider sees at most 50 jobs/second no matter how many pods run.
  limiter: { max: 50, duration: 1000 },
})
export class EmailProcessor extends WorkerHost {
  async process(job: Job<{ to: string; templateId: string }>): Promise<void> {
    // I/O-bound work only: awaiting the DB and HTTP calls frees the event
    // loop for the other concurrent jobs on this worker.
    // Do not do CPU-heavy rendering here. It blocks the event loop and
    // delays lock renewal, so healthy jobs can be marked as stalled.
    await this.sendEmail(job.data.to, job.data.templateId);
  }

  private async sendEmail(to: string, templateId: string): Promise<void> {
    // Call the provider SDK or HTTP client here.
  }
}

A NestJS BullMQ processor that caps per-instance concurrency at the database pool size and uses the queue-wide limiter to stay within a provider's total rate limit.

What causes stalled jobs, and how do you handle failed jobs and shutdowns?

A worker holds a lock on each active job and renews it periodically. If renewal stops, BullMQ's stalled-job checker treats the job as abandoned. It moves the job back to waiting, or fails it once it exceeds maxStalledCount. This mechanism is how BullMQ recovers from crashed workers. It also produces false stalls. A handler that blocks the event loop with synchronous parsing, heavy crypto or a tight loop cannot renew its lock. Another worker then picks up the same job while the first one is still running it, so the job executes twice.

Tuning locks to real runtimes

lockDuration sets a trade-off. A short lock detects crashes quickly but turns slow or blocking handlers into duplicates. A long lock tolerates slow handlers but leaves a crashed worker's jobs unrecovered for longer. Measure actual job runtimes and event-loop delay first. Then set lockDuration and stalledInterval to match, and move CPU-bound work into sandboxed processors rather than stretching the lock indefinitely.

Retention and Redis memory

Completed and failed jobs remain in Redis until removed. Use removeOnComplete and removeOnFail with an age or count limit so job history stays bounded. Set Redis maxmemory-policy to noeviction. Any other policy lets Redis silently delete queue keys under memory pressure, which corrupts queue state.

A deliberate dead-letter workflow

BullMQ has no automatic dead-letter queue. Treat the failed set as one. Inspect failedReason and the stack trace, fix the cause, and then retry or re-enqueue specific jobs on purpose. Avoid blanket retries.

Clean shutdowns

Call app.enableShutdownHooks() so that SIGTERM triggers module teardown and workers close gracefully, finishing in-flight jobs. Without shutdown hooks, every deploy kills active jobs mid-run. Those jobs stall and run again on each rollout.

How do you monitor and test a NestJS BullMQ queue?

You can hook worker events in two ways. In a WorkerHost processor, @OnWorkerEvent('completed' | 'failed' | 'stalled' | 'error') runs inside the worker process and only sees jobs that worker handled. A QueueEvents listener, registered with @QueueEventsListener, subscribes to the Redis event stream and sees every worker's activity. Use it for cross-instance views. In both cases, emit one structured log line per terminal event with job.id, job.name, attemptsMade and duration (finishedOn - processedOn). Those fields let you filter a single job's retry history in your log tool.

Export these metrics to your existing stack:

Metric Source Slow queue Stuck queue
Depth by state getJobCounts() polled waiting grows, active at concurrency waiting grows, active flat or zero
Wait time processedOn - timestamp rising unbounded
Processing latency finishedOn - processedOn rising no samples
Completion and failure rate event counters completions continue completions near zero, stalls appear

Alert on the failure count over a time window, not on single failures, because retries make individual failures expected. Also alert when completions stay at zero while waiting jobs are greater than zero.

Per-job listeners add a Redis round trip or a log write to every job. At high throughput, increment in-memory counters and histograms instead, and log only failures plus a sample of successes.

For testing, keep business logic in a plain service and unit-test it without BullMQ. Then run integration tests against a real Redis instance, for example a disposable container. Force failures in those tests and assert attemptsMade and the spacing between attempts. Mocks cannot verify backoff.

This adds a few days of upfront work. It pays back the first time an incident becomes a dashboard query instead of guesswork.

How can BullMQ save engineering time?

The time savings come from features you would otherwise build and maintain yourself:

  • No homegrown retry machinery. attempts, backoff and UnrecoverableError replace custom retry loops, retry-count columns and scheduler code. Each of those would need its own tests and edge-case fixes.
  • Crash recovery is built in. Lock renewal and stalled-job detection return abandoned work to the queue. Without them, someone has to write a sweeper that finds rows stuck "in progress" and decide when they are safe to reclaim.
  • Rate limiting across replicas. The Redis-backed limiter enforces a global cap. A correct distributed rate limiter is non-trivial to write yourself.
  • Less framework glue. With @nestjs/bullmq, queues and processors are regular NestJS providers. Configuration, DI and shutdown handling follow patterns the team already knows, which shortens onboarding and code review for new job types.
  • Faster incident triage. Job state, attemptsMade, failedReason and timestamps are stored with each job. Combined with the metrics above, "why didn't this email go out?" becomes a lookup instead of a log search across services.
  • Targeted recovery. The failed set lets you retry exactly the jobs affected by a bug after deploying a fix. You avoid writing a one-off backfill script.

Illustrative estimate: assume a team adds four new background job types a year, and that building retries, stall recovery and status tracking by hand would take an assumed two engineer-days per job type, plus an assumed half day per incident to trace a lost job through logs. Under those assumptions, a shared queue library saves roughly eight engineer-days of build work a year before counting incidents. Your actual figures depend on your codebase and failure rate. Treat this as a way to frame the decision, not as a measured result.

The savings are not free. You trade that build time for the operational work described earlier: running Redis as critical state, designing idempotent handlers, and maintaining dashboards and alerts. The trade is favorable when you have several job types and real retry and concurrency requirements. It is not favorable for one or two best-effort tasks that an in-process promise or a cron job already handles.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nestjs bullmq, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.