NestJS BullMQ: Job Retries, Backoff and Concurrency for Workers

Who should use NestJS BullMQ?

NestJS BullMQ is for teams that run a NestJS backend and need to move slow or failure-prone work out of the request path. Typical users fall into three groups:

• Backend engineers on NestJS services. Some handlers call email providers, payment APIs, PDF generators or third-party sync endpoints. Their latency and failures should not reach the user. These engineers want retries and backoff without writing that logic in each handler.
• Platform engineers who own worker fleets. They need to scale background processing apart from the API, cap the request rate sent to shared dependencies, and shut workers down cleanly during deploys.
• Teams that already run Redis or can operate it. BullMQ keeps all queue state in Redis. If nobody on the team can size, persist and monitor a Redis instance, the operational cost covered at the end of this article applies in full.

It is a poor fit for workloads that need strict ordering, exactly-once effects, very low latency or heavy CPU processing in Node.js. Those cases are covered in the final section.

How does the NestJS BullMQ integration work?

@nestjs/bullmq wraps BullMQ, a Redis-backed job queue, in Nest's module and dependency-injection system. It suits backend and platform engineers who need to move slow or failure-prone work out of the request path: sending email, calling a payment provider, generating a PDF, syncing to a third-party API. The HTTP handler records intent and returns quickly. A separate process does the work and can retry it without a user waiting.

The integration has three parts. First, BullModule.forRoot sets the Redis connection once, and BullModule.registerQueue declares a named queue in the module that owns it. Second, any provider in that module can receive the queue through @InjectQueue('name') and call queue.add with a job name, a serializable payload and per-job options. Third, a class decorated with @Processor('name') extends WorkerHost and implements an async process(job) method. Nest starts a BullMQ worker for that class, and the worker pulls jobs from Redis.

A job moves through a small set of states stored in Redis. queue.add puts it in the waiting state, or in the delayed state if it has a delay. A worker takes it and moves it to active, and holds a lock on it while process runs. When process resolves, the job moves to completed. When it throws, BullMQ checks the job's retry options. If attempts remain, the job goes back to waiting, or to delayed when backoff applies. If no attempts remain, it goes to failed. Each later section of this article tunes one of these transitions.

Copy  API replicas                Redis                 Worker replicas
 +-------------+       +----------------+       +-----------------+
 | OrderService| add() |  queue: email  | pull  | EmailProcessor  |
 |  (producer) |------>|  wait / active |<------| process(job)    |
 +-------------+       |  delayed/failed|<------| EmailProcessor  |
                       +----------------+       +-----------------+

Producers and workers share nothing except Redis. That separation lets you scale workers independently, but it also means every failure becomes a decision about the job. This article's thesis is that retries, backoff and concurrency are not three separate settings. Together they form one model of how your downstream dependencies fail. Idempotency is the foundation, so that model only works if a job can safely run twice.

Copyimport { Injectable, Module } from '@nestjs/common';
import { BullModule, InjectQueue, Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, Queue } from 'bullmq';

// Producer: runs in the API process and only records intent.
@Injectable()
export class OrderService {
  constructor(@InjectQueue('email') private readonly emailQueue: Queue) {}

  async confirmOrder(orderId: string, to: string) {
    // A deterministic jobId means a duplicate add() with the same id is ignored.
    await this.emailQueue.add(
      'order-confirmation',
      { orderId, to },
      { jobId: `order-confirmation-${orderId}` },
    );
  }
}

// Consumer: Nest starts a BullMQ worker for this class.
@Processor('email')
export class EmailProcessor extends WorkerHost {
  async process(job: Job<{ orderId: string; to: string }>): Promise<void> {
    // Throwing here marks the attempt as failed; returning marks it completed.
    console.log(`sending ${job.name} for order ${job.data.orderId}`);
  }
}

@Module({
  imports: [
    // One Redis connection for the app; placeholder host and port.
    BullModule.forRoot({ connection: { host: 'redis.internal', port: 6379 } }),
    // Declare the named queue in the module that owns it.
    BullModule.registerQueue({ name: 'email' }),
  ],
  providers: [OrderService, EmailProcessor],
})
export class EmailModule {}

A minimal NestJS module that registers the Redis connection and an 'email' queue, injects the queue into a producer service to add jobs, and handles them in a processor that extends WorkerHost.

Make every job safe to run twice before tuning anything

BullMQ delivers jobs at least once. A handler can run more than once for ordinary reasons. A retry re-runs the whole function after a timeout, even if the payment already went through. A worker that blocks the event loop or loses its connection can miss its lock renewal, and the stalled-job checker hands the job to another worker while the first one is still running. A deploy that kills a process mid-job does the same.

Three techniques

• Deterministic job IDs. Derive the jobId from the business event, for example invoice-${invoiceId}, instead of letting BullMQ generate one. If the job still exists in the queue, adding a job with the same ID is ignored, so double-clicks and producer retries collapse into one job. This only deduplicates enqueueing. It does nothing for repeated execution.
• Idempotency keys on side effects. Pass a stable key, derived from the job ID and the step, to anything external. Most payment APIs accept an idempotency key. For emails and database writes, record the key in your own table under a unique constraint before acting, so a second attempt fails the insert instead of acting twice.
• Checkpoints for multi-step jobs. Persist which steps have completed, either in the job data or in your database. A retry then resumes after "charged" rather than charging again.

The cost

This takes design work for every handler, plus storage and cleanup for keys and checkpoints. That cost is small compared with refunding duplicate charges or explaining a fifth welcome email to a customer. Every retry and concurrency setting covered later assumes this work is done.

Copyimport { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, Queue } from 'bullmq';

type InvoiceJob = { invoiceId: string; customerEmail: string; steps: string[] };
// Placeholder ports: your payment SDK, mailer, and a table with UNIQUE(key).
interface Payments { charge(invoiceId: string, opts: { idempotencyKey: string }): Promise<void> }
interface Mailer { sendReceipt(to: string, invoiceId: string): Promise<void> }
interface SentLog { claim(key: string): Promise<boolean>; release(key: string): Promise<void> }

// Producer: the ID comes from the business event, so double-clicks collapse into one job.
export const enqueue = (q: Queue, d: Omit<InvoiceJob, 'steps'>) =>
  q.add('charge-invoice', { ...d, steps: [] }, { jobId: `invoice-${d.invoiceId}` });

@Processor('billing')
export class InvoiceProcessor extends WorkerHost {
  constructor(private payments: Payments, private mailer: Mailer, private sent: SentLog) { super(); }

  async process(job: Job<InvoiceJob>) {
    const key = (step: string) => `${job.id}-${step}`; // stable across retries
    const done = new Set(job.data.steps);

    if (!done.has('charged')) {
      // The provider dedupes on the key if a previous attempt charged and then crashed.
      await this.payments.charge(job.data.invoiceId, { idempotencyKey: key('charge') });
      done.add('charged');
      await job.updateData({ ...job.data, steps: [...done] }); // checkpoint
    }

    if (!done.has('emailed')) {
      // claim() inserts the key; a unique-constraint violation returns false.
      if (await this.sent.claim(key('receipt'))) {
        try { await this.mailer.sendReceipt(job.data.customerEmail, job.data.invoiceId); }
        catch (err) { await this.sent.release(key('receipt')); throw err; } // allow a retry to send
      }
      done.add('emailed');
      await job.updateData({ ...job.data, steps: [...done] });
    }
  }
}

A NestJS BullMQ processor that combines a deterministic jobId, idempotency keys derived from the job ID for the charge and the email, and step checkpoints saved with job.updateData. A retried or stalled job resumes after completed steps and does not repeat them.

Configuring job retries and exponential backoff

The retry budget is the attempts option, and it counts the first run: attempts: 5 means one try plus four retries. Set a default for the whole queue in defaultJobOptions when you call BullModule.registerQueue, then override it per job in the options argument to queue.add when one job type needs a different policy. A run counts as a failure only when process throws or returns a rejected promise. If an error is caught and logged inside the handler, the job resolves and is marked completed, so the retry policy never runs.

Illustrative estimate. Assume attempts: 5, exponential backoff and a 2-second base delay. The four retries wait 2, 4, 8 and 16 seconds, for 30 seconds of waiting plus the time each run takes. The same settings with fixed backoff wait 8 seconds in total.

Short fixed delays across many workers can cause a retry storm. When a dependency goes down, every in-flight job fails at about the same moment and retries at about the same moment, which adds load to a service that is already struggling. A smaller number of attempts with longer exponential gaps gives the dependency time to recover. Adding jitter helps further because it spreads out jobs that would otherwise retry together. If your BullMQ version has no built-in jitter option, you can add it with a custom backoff strategy. The downside of long backoff is tail latency: a job that succeeds on its fifth attempt finishes minutes late. That is acceptable for a nightly export, but it is the wrong policy for a password-reset email or any request a user is waiting on.

Copyimport { Injectable, Module } from '@nestjs/common';
import { BullModule, InjectQueue, Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, Queue } from 'bullmq';

// Queue-level default: 5 attempts in total (1 run + 4 retries),
// with exponential waits of 2s, 4s, 8s and 16s between them.
@Module({
  imports: [
    BullModule.registerQueue({
      name: 'exports',
      defaultJobOptions: {
        attempts: 5,
        backoff: { type: 'exponential', delay: 2000 },
      },
    }),
  ],
})
export class ExportsModule {}

@Injectable()
export class ExportsService {
  constructor(@InjectQueue('exports') private readonly queue: Queue) {}

  // Per-job override: a time-sensitive job gets fewer, shorter retries.
  enqueueUrgent(reportId: string) {
    return this.queue.add('urgent-export', { reportId }, {
      attempts: 2,
      backoff: { type: 'fixed', delay: 1000 },
    });
  }
}

@Processor('exports')
export class ExportsProcessor extends WorkerHost {
  async process(job: Job<{ reportId: string }>) {
    // A thrown error (or rejected promise) marks this attempt as failed and
    // triggers the retry policy. Catching and swallowing it here would mark
    // the job completed, and no retry would ever run.
    // REPORT_SERVICE_URL is a placeholder for your report service's base address.
    const res = await fetch(`${process.env.REPORT_SERVICE_URL}/reports/${job.data.reportId}`);
    if (!res.ok) throw new Error(`Report service returned ${res.status}`);
    return res.json();
  }
}

This example sets a queue-wide default of 5 attempts with exponential backoff in BullModule.registerQueue and overrides it for one job type in queue.add. It also shows that a retry only runs when process throws or rejects.

Separating retryable from permanent errors

Backoff only helps with failures that can go away on their own. A payload that fails schema validation will fail the same way on every attempt. Retrying it burns the full attempt budget and holds a worker slot during each try. It also keeps the job out of the failed set for the entire backoff window, so nobody notices the problem until minutes later.

BullMQ provides UnrecoverableError for this case. When a processor throws it, the job moves straight to failed and any remaining attempts are skipped. Any other thrown error is treated as retryable. In practice, catch errors at the boundary of each external call, classify them, and rethrow either the original error or an UnrecoverableError that wraps the original message. Keep the classification in one helper so every processor applies the same rules.

The last row shows that classification takes judgment. If you mark a transient error as permanent, you silently drop work that would have succeeded. If you mark a permanent error as transient, you waste attempts and delay the moment the failure becomes visible. When you are unsure, retry with a small attempt cap rather than failing fast.

Copyimport { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job, UnrecoverableError } from 'bullmq';

export class HttpError extends Error {
  constructor(readonly status: number, message: string) { super(message); }
}
export class ValidationError extends Error {}
export class NotFoundError extends Error {}

const NOT_FOUND_RETRY_CAP = 2; // tolerate brief replication lag, then give up

// One shared helper so every processor applies the same retry rules.
export function classify(err: unknown, job: Job): Error {
  if (err instanceof ValidationError) return new UnrecoverableError(err.message);
  if (err instanceof NotFoundError) {
    // attemptsMade counts earlier failed attempts for this job
    return job.attemptsMade < NOT_FOUND_RETRY_CAP ? err : new UnrecoverableError(err.message);
  }
  if (err instanceof HttpError) {
    if (err.status === 429 || err.status >= 500) return err; // transient: use backoff
    // 400/422 bad input, 401/403 credentials: fail fast so the failure shows up now
    if (err.status >= 400) return new UnrecoverableError(`HTTP ${err.status}: ${err.message}`);
  }
  // Timeouts, connection resets and unknown errors: retry, bounded by attempts
  return err instanceof Error ? err : new Error(String(err));
}

@Processor('invoices')
export class InvoiceProcessor extends WorkerHost {
  async process(job: Job<{ invoiceId: string }>): Promise<void> {
    try {
      if (!job.data.invoiceId) throw new ValidationError('invoiceId is required');
      // INVOICE_SERVICE_URL is a placeholder for your invoice service's base address.
      const res = await fetch(`${process.env.INVOICE_SERVICE_URL}/invoices/${job.data.invoiceId}/send`, { method: 'POST' });
      if (res.status === 404) throw new NotFoundError(`invoice ${job.data.invoiceId} not found`);
      if (!res.ok) throw new HttpError(res.status, await res.text());
    } catch (err) {
      throw classify(err, job); // rethrow original (retry) or UnrecoverableError (fail now)
    }
  }
}

A NestJS BullMQ processor that routes every error through one classify helper. Validation errors, 4xx responses and repeated not-found results are rethrown as UnrecoverableError so they fail immediately. Timeouts, 429s and 5xx responses are rethrown unchanged so they retry with backoff.

Queue concurrency, replicas and the shared rate limiter

Worker concurrency sets how many jobs a single worker process holds at once. In NestJS you set it in the @Processor options. It does not create threads. One Node.js event loop interleaves those jobs while they wait on I/O. Replicas multiply that number. Five pods at concurrency 10 can run 50 jobs simultaneously against the same downstream systems, and nothing in a per-worker setting knows about the other four pods.

The queue-level rate limiter does coordinate across workers. When you configure limiter with a max and a duration, BullMQ enforces it through Redis for the whole queue. That makes it the right control for protecting a third-party API with a fixed quota.

Higher concurrency only helps I/O-bound jobs. A CPU-bound job, such as image resizing or parsing a large file, blocks the event loop. While the loop is blocked the worker cannot renew job locks, so BullMQ marks those jobs as stalled and may hand them to another worker, which runs them twice. For this kind of work, use sandboxed processors or more processes with low concurrency.

Before adding replicas, multiply concurrency by instance count and compare the result with your database pool size and your third-party quotas. Limits you set per process stop being limits once you run many processes.

Exhausted jobs, graceful shutdown and observability

A job that exhausts its attempts goes to the failed set. BullMQ does not move it to a dead-letter queue for you. Handle the worker's failed event (in NestJS, @OnWorkerEvent('failed') on your processor), and check job.attemptsMade against job.opts.attempts so you can tell a final failure from one that will be retried.

Keep evidence, but bound it

Use removeOnFail with an age or count limit rather than false. Failed jobs keep their payload, stack trace and attempt history, which is what you need for diagnosis. They also take up Redis memory until something removes them. Retain them long enough to cover your typical time-to-investigate. For replay, write a script that pages through queue.getFailed(), filters by error type, and calls job.retry(). This works because your handlers are already idempotent.

Shut down cleanly

Call app.enableShutdownHooks() so that SIGTERM closes the workers. A closing worker stops taking new jobs and waits for active ones to finish. Set your orchestrator's grace period longer than your slowest job. The worker renews each job's lock while it runs, based on lockDuration. If CPU-bound work blocks the event loop past that window, the stalled-job check treats the job as abandoned and hands it to another worker. Break that work into chunks or offload it rather than raising timeouts without limit.

Observe what matters

Log the job ID, name and attempt number on every failure. Track failure and retry counts by job name. Alert on queue depth and on the age of the oldest waiting job, because that age reflects user-visible delay. During an incident, these signals show which job type is failing, how far behind the queue is, and which jobs to replay.

How NestJS BullMQ can save engineering time

Most of the time saved comes from not building and maintaining queue mechanics yourself. A hand-rolled background job system needs persistence, locking so two workers do not take the same job, stalled-job detection, retry counters, delay scheduling and rate limiting across replicas. BullMQ implements these on Redis, and @nestjs/bullmq exposes them through decorators and modules the team already uses.

• Declarative retry policy instead of retry loops. attempts and backoff sit in defaultJobOptions or in the options passed to queue.add. Engineers do not write or review a custom try/sleep/retry loop in each handler, and changing a policy is a configuration change in one place.
• One error-classification helper for every processor. With UnrecoverableError and a shared classify function, the retry-or-fail decision lives in one tested module. Without it, each team member makes that decision again in each new handler.
• Familiar structure and testability. Producers and processors are ordinary Nest providers. Their dependencies, such as payment clients, mailers and repositories, are injected, so unit tests can replace them with fakes the same way they do for controllers and services.
• Independent scaling without rewrites. Because producers and workers share only Redis, moving work to dedicated worker replicas or adding a queue-wide limiter does not require changes to the API code that enqueues jobs.
• Faster incident handling. Failed jobs keep their payload, stack trace and attempt history. With idempotent handlers, recovering from an outage means running a replay script against queue.getFailed() instead of reconstructing lost work from logs.

These savings depend on the idempotency work described earlier. If handlers are not safe to run twice, retries and replays create new incidents. In that case, the time saved on plumbing goes into cleaning up duplicate side effects.

When to avoid BullMQ, alternatives, and the cost of running it

Those same signals also show when BullMQ is the wrong tool. It fits background I/O work in a Node.js service where downstream failures are transient and retrying is the right response, such as sending email, delivering webhooks or syncing with third-party APIs. Avoid it in these cases:

• Strict ordering. Retries and concurrency reorder jobs.
• Exactly-once effects. Delivery is at-least-once, so the queue cannot guarantee a single execution.
• Very latency-sensitive paths. Every job adds Redis round trips and a worker hop before any work starts.
• Heavy CPU pipelines. A CPU-bound job blocks the Node.js event loop and delays lock renewal, which can get the job marked as stalled and run again.

Redis is the main operational risk. A single instance is a shared failure point and a throughput ceiling for every queue it hosts. Set maxmemory-policy to noeviction. Any policy that evicts keys can delete job data or locks without raising an error, which leaves queue state inconsistent.

BullMQ is cheap to develop with in NestJS because decorators and modules replace most of the plumbing. Most of its cost is operational. Someone has to size, persist, replicate and monitor Redis, and someone has to own failed and stuck jobs.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nestjs bullmq, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.