NestJS Microservices Transport: Redis vs NATS vs Kafka Compared

How the NestJS microservices transport abstraction works

In NestJS, a microservice is an application that listens on a transport (a message broker or socket protocol) instead of an HTTP port. The @nestjs/microservices package hides the transport behind two pieces: a server strategy that receives messages and routes them to handlers, and a ClientProxy that other services use to send messages.

A ClientProxy has two methods with different contracts. send() is request-response. It returns a cold Observable, so nothing goes on the wire until something subscribes. The client then waits for a reply that is correlated to the original request. emit() is fire-and-forget. It publishes an event and does not expect a reply.

Handlers come in matching pairs. A method decorated with @MessagePattern answers send() calls, and its return value becomes the response. A method decorated with @EventPattern consumes emit() events. It returns nothing to the caller, and any number of services can subscribe to the same event.

The transport is selected in configuration. The server side sets it when it creates or connects the microservice. The client side sets it when it registers the client, using the same Transport enum value, such as Transport.REDIS, Transport.NATS or Transport.KAFKA. If you switch that value and its connection options, the decorated handlers usually compile and run unchanged. What changes is how messages move on the wire: Redis pub/sub channels, NATS subjects, or Kafka topics with consumer groups and reply topics.

The shared API does not mean shared behavior. The transports differ on persistence, ordering, delivery guarantees and how scaled-out instances split work. The abstraction also leaks in places. The @Ctx() decorator exposes transport-specific context objects, and Kafka needs extra setup for request-response. For these reasons, choose a transport by its semantics, not by its API.

Copyimport { Controller, Inject } from '@nestjs/common';
import { ClientProxy, EventPattern, MessagePattern, Payload, Transport } from '@nestjs/microservices';
import { firstValueFrom } from 'rxjs';

// Swap this one object to change the wire protocol. Handlers below stay the same.
// Redis: { transport: Transport.REDIS, options: { host: 'localhost', port: 6379 } }
// Kafka: { transport: Transport.KAFKA, options: { client: { brokers: ['localhost:9092'] }, consumer: { groupId: 'orders' } } }
export const transportConfig = {
  transport: Transport.NATS,
  options: { servers: ['nats://localhost:4222'] },
} as const;
// Server: NestFactory.createMicroservice(OrdersModule, transportConfig)
// Client: ClientsModule.register([{ name: 'ORDERS', ...transportConfig }])

@Controller()
export class OrdersController {
  // Answers send(). The return value is sent back as the correlated reply.
  @MessagePattern('orders.get')
  getOrder(@Payload() data: { id: string }) {
    return { id: data.id, status: 'shipped' };
  }

  // Consumes emit(). Nothing is returned to the caller.
  @EventPattern('orders.created')
  onOrderCreated(@Payload() data: { id: string }) {
    console.log(`indexing order ${data.id}`);
  }
}

export class CheckoutService {
  constructor(@Inject('ORDERS') private readonly client: ClientProxy) {}

  async checkout(id: string) {
    // send() is cold: nothing is published until firstValueFrom subscribes.
    // On Kafka, also call client.subscribeToResponseOf('orders.get') before connecting.
    const order = await firstValueFrom(this.client.send('orders.get', { id }));
    // emit() publishes and does not wait for a reply.
    this.client.emit('orders.created', { id });
    return order;
  }
}

The same @MessagePattern and @EventPattern handlers and ClientProxy calls work over NATS, Redis or Kafka. Only the shared transport config object changes, and Kafka needs one extra reply-topic subscription for send().

How does it work? The path of a single message

It helps to follow one message from caller to handler and back.

1. The client builds a packet. When you call send('orders.get', { id }), the ClientProxy wraps the pattern and payload in a packet and attaches a generated correlation ID. emit() builds the same kind of packet without expecting a reply.
2. The packet is serialized and published. The client's serializer (JSON by default) turns the packet into bytes. The transport strategy then publishes it on the broker's native primitive: a Redis channel, a NATS subject, or a Kafka topic named after the pattern.
3. The server strategy receives and routes it. On the receiving service, the server strategy subscribes to the channels, subjects or topics that match the registered patterns. When a message arrives, it deserializes the packet, looks up the handler registered for that pattern, and invokes it. Pipes, guards, interceptors and exception filters run around the handler in the same way they do for HTTP controllers.
4. Events stop here. For an @EventPattern handler, the strategy runs the handler and sends nothing back.
5. Requests get a correlated reply. For a @MessagePattern handler, the strategy serializes the return value (or the error) together with the original correlation ID and publishes it to where the client listens for replies. Depending on the broker, that is a reply channel, a NATS inbox subject, or a Kafka reply topic.
6. The client resolves the Observable. The client matches the reply to the pending request by correlation ID and emits the response, or an error, to the subscriber.

Handlers can return plain values, Promises or Observables. NestJS unwraps them before it builds the reply. Because each step sits behind the strategy and serializer interfaces, you can change the broker or the wire format without changing handler code. That is also why the delivery guarantees discussed below come from the broker and not from NestJS.

Who should use NestJS microservice transports

The microservices package fits best for:

• Teams already building on NestJS that are splitting a monolith or adding services. Teams that need service-to-service messaging but do not want each service to hand-write broker client code, connection handling and message routing.
• Organizations where most services are NestJS. The packet format and pattern-based routing are NestJS conventions. Non-Nest producers and consumers can interoperate, but only if they reproduce the same message shape. That is easy to get wrong without a shared contract.
• Teams that want to defer or revisit the broker choice. Because handlers are transport-agnostic, you can start on the broker you already operate and move later. The rest of this article explains which semantics you then have to re-test.

It is a weaker fit for:

• Workloads that depend on broker-specific features the built-in transports do not expose, such as JetStream durable consumers, Kafka transactions, or fine-grained offset control. You can still reach them through @Ctx(), custom transporters or the broker's own client library, but at that point the abstraction gives you less.
• Heavily polyglot systems where most consumers are not written in NestJS. A broker-native client with an explicit schema format is usually simpler there than imitating NestJS packets in other languages.

Delivery semantics of Redis, NATS and Kafka under the same API

The transport you pick sets the delivery guarantees your handlers get. @MessagePattern and @EventPattern behave the same in code under every transport, but what happens to a message that nobody is ready to receive depends on the broker.

Redis: fire-and-forget pub/sub

The Redis transport is built on Redis pub/sub. Redis does not store a published message. It forwards it to the subscribers connected at that moment and then discards it. There are no consumer groups, no persistence and no replay. If a service is restarting during a deploy, it misses every event published in that window and has no way to recover them. Every connected instance receives every message, so running three replicas means three executions of each event handler.

NATS: core versus JetStream

Core NATS is at-most-once. Latency is low, and request-reply is native: the client publishes with a reply subject and waits on it. Queue groups make one member of the group receive each message, which gives you load balancing without coordination. Like Redis, core NATS keeps nothing for subscribers that are offline. JetStream adds persistence, acknowledgments and durable consumers. The built-in NestJS transport works against core NATS, so durable consumption takes extra work outside the standard transport options, such as a custom transporter or a separate JetStream client.

Kafka: a durable partitioned log

Kafka writes every message to a replicated, partitioned log. Consumer groups divide the partitions among instances. Ordering holds only within a partition. Because consumers track their position with offsets, they can rewind and replay.

Kafka keys, partitions and request-response in NestJS

Kafka guarantees order only within a partition, and the producer chooses the partition by hashing the message key. In NestJS you set the key by emitting an object with key and value fields instead of a bare payload. Key by the entity whose events must stay in sequence, such as orderId. Every event for that order then lands on one partition and is consumed in the order it was written.

A bad key causes one of two failures:

• Too coarse. A key like tenantId funnels a large tenant's traffic onto one partition. That partition and its single consumer become a hot spot, and the other consumers in the group sit mostly idle.
• Missing or too fine. With no key, or a random one, related events spread across partitions. Consumers then process them in parallel, so a shipped event can be handled before its created event.

The key-to-partition mapping holds only while the partition count stays fixed. If you add partitions, existing keys hash to new partitions. Events produced before and after the change can then be consumed out of order. Changing the key scheme breaks ordering in the same way. Settle partition counts and keys before production traffic depends on them.

Kafka has no native request-reply. ClientKafka emulates it in three steps:

1. You call subscribeToResponseOf(pattern) for each pattern, before connect(). This makes the client consume from a reply topic derived from the pattern.
2. Each request carries correlation and reply-topic headers.
3. The responder publishes its answer to the reply topic, and the client matches the response to the request by correlation ID.

You must provision every reply topic yourself. Each response also passes through the broker's write and consumer-poll path. This adds latency and setup that NATS avoids with its built-in inbox replies.

Copyimport { Inject, Injectable, OnModuleInit } from '@nestjs/common';
import { ClientKafka } from '@nestjs/microservices';
import { firstValueFrom } from 'rxjs';

@Injectable()
export class OrdersClient implements OnModuleInit {
  // 'ORDERS_KAFKA' is registered via ClientsModule.register({ transport: Transport.KAFKA, ... })
  constructor(@Inject('ORDERS_KAFKA') private readonly kafka: ClientKafka) {}

  async onModuleInit() {
    // Must run before connect(): the client starts consuming the
    // derived reply topic 'orders.get.reply', which you provision yourself.
    this.kafka.subscribeToResponseOf('orders.get');
    await this.kafka.connect();
  }

  // Fire-and-forget event. Keying by orderId hashes every event for this
  // order to the same partition, so created -> paid -> shipped stay in order.
  publishShipped(orderId: string) {
    return this.kafka.emit('orders.events', {
      key: orderId,
      value: { type: 'shipped', orderId, at: new Date().toISOString() },
    });
  }

  // Emulated request-response: the client adds correlation and reply-topic
  // headers, the responder writes to 'orders.get.reply', and the client
  // matches the answer by correlation ID. Each hop goes through the broker.
  getOrder(orderId: string) {
    return firstValueFrom(
      this.kafka.send('orders.get', { key: orderId, value: { orderId } }),
    );
  }
}

A NestJS ClientKafka service that emits orderId-keyed events for per-order ordering and subscribes to the reply topic before connecting so it can use request-response.

Scaling handlers horizontally and the idempotency it requires

When a service runs as several replicas, the transport decides whether those replicas share the work or each one does all of it.

Redis. The NestJS Redis transporter is built on Redis pub/sub. Every instance subscribed to a channel receives every message published to it. With replicas, each event runs once per instance. Illustrative estimate, assumed inputs, not a measurement: 4 replicas receiving 1,000 events per minute produce 4,000 handler executions per minute. Three of every four are duplicates. For side effects such as charging a card or sending an email, that is a bug.

NATS. Set a queue name in the transporter options, for example options: { servers: [...], queue: 'orders-service' }. NATS then delivers each message to only one member of that queue group. Replicas join the group automatically and split the load without any coordination on your side.

Kafka. Replicas that share a consumer groupId divide the topic's partitions between them. A partition belongs to only one consumer in the group, so parallelism is capped at the partition count. Any replicas beyond that number sit idle.

Work distribution does not guarantee single execution. A Kafka rebalance can redeliver messages whose offsets were not yet committed, and producers on any transport may retry after a timeout. Handlers therefore need to be idempotent. A simple check works like this:

1. Read an idempotency key from the payload, such as an event ID set by the producer.
2. Try to claim it with SET key 1 NX EX <ttl> in Redis.
3. If the command returns nothing, the key was already claimed, so return early.

Set the TTL longer than your maximum redelivery window. Otherwise a late redelivery can arrive after the key has expired and run the handler again.

Copy// main.ts: every replica joins the same NATS queue group
import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.createMicroservice<MicroserviceOptions>(AppModule, {
    transport: Transport.NATS,
    options: {
      servers: ['nats://nats.internal:4222'],
      queue: 'orders-service', // each message is delivered to one replica in this group
    },
  });
  await app.listen();
}
bootstrap();

// orders.controller.ts: claim the event ID before running side effects
import { Controller } from '@nestjs/common';
import { EventPattern, Payload } from '@nestjs/microservices';
import Redis from 'ioredis';

const redis = new Redis('redis://redis.internal:6379');
const DEDUP_TTL_SECONDS = 24 * 60 * 60; // longer than the maximum redelivery window

@Controller()
export class OrdersController {
  @EventPattern('order.created')
  async handleOrderCreated(@Payload() event: { eventId: string; orderId: string }) {
    // SET ... NX returns 'OK' only for the first caller and null if the key already exists
    const claimed = await redis.set(`dedup:order.created:${event.eventId}`, '1', 'EX', DEDUP_TTL_SECONDS, 'NX');
    if (claimed !== 'OK') {
      return; // duplicate delivery: already processed or in progress elsewhere
    }
    // Side effects (charge card, send email) run at most once per eventId within the TTL
  }
}

A NestJS NATS microservice whose replicas share one queue group, plus an event handler that claims each event ID with Redis SET NX EX and returns early on duplicates.

Error handling, acknowledgment, retries and dead letters

On request-response, throw RpcException from a @MessagePattern handler. The built-in exception filter serializes its payload, and the caller's Observable errors with it, so the client sees your error object or message rather than a timeout. Any other thrown error is reduced to a generic "Internal server error". Throw RpcException deliberately with a stable shape, such as { code, message }, so callers can branch on code instead of parsing strings.

Events are harder because nobody is waiting for the result. On Redis pub/sub and core NATS, an exception in an @EventPattern handler is logged and the message is gone. Those transports have no redelivery, so a throw means data loss. On Kafka, redelivery depends on whether the offset was committed. A message that fails on every attempt is a poison message. If you keep retrying it without committing, it stalls every later message in its partition.

None of the built-in transports give you a full dead-letter mechanism, so build one into the handler:

1. Catch the error inside the handler.
2. Track the attempt count in a message header or the payload.
3. Retry with backoff up to a fixed limit.
4. When the limit is reached, publish the original payload, the error and the attempt count to a dedicated destination such as orders.created.dlq.
5. Return normally so the offset advances.

Have a separate process inspect and replay the dead letters.

No built-in transport provides end-to-end exactly-once processing on its own. Retries and dead-letter replays both depend on the idempotency guard from the previous section.

Copyimport { Controller, Inject } from '@nestjs/common';
import { ClientKafka, EventPattern, MessagePattern, Payload, RpcException } from '@nestjs/microservices';
import { lastValueFrom } from 'rxjs';
import { OrdersService, OrderCreated } from './orders.service';

const MAX_ATTEMPTS = 3;
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

@Controller()
export class OrdersController {
  constructor(
    @Inject('KAFKA_CLIENT') private readonly client: ClientKafka,
    private readonly orders: OrdersService,
  ) {}

  @MessagePattern('orders.get')
  async getOrder(@Payload() data: { id: string }) {
    const order = await this.orders.find(data.id);
    // Stable { code, message } shape: the caller's Observable errors with this object
    if (!order) throw new RpcException({ code: 'ORDER_NOT_FOUND', message: `Order ${data.id} not found` });
    return order;
  }

  @EventPattern('orders.created')
  async onOrderCreated(@Payload() event: OrderCreated) {
    for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
      try {
        await this.orders.process(event); // must be idempotent: retries and replays repeat work
        return;
      } catch (err) {
        if (attempt === MAX_ATTEMPTS) {
          // Park the poison message with context so a separate process can inspect and replay it
          const deadLetter = { payload: event, error: (err as Error).message, attempts: attempt };
          await lastValueFrom(this.client.emit('orders.created.dlq', deadLetter));
          return; // return normally so the offset advances and the partition is not stalled
        }
        await sleep(200 * 2 ** attempt); // exponential backoff between attempts
      }
    }
  }
}

A request-response handler that throws a structured RpcException, and a Kafka event handler that retries with backoff and then publishes the failed event to a dead-letter topic before returning normally.

Serialization and message contracts between services

By default, NestJS serializes outgoing packets with JSON.stringify and parses them on the receiving side. This is convenient, but it is lossy. A Date arrives as an ISO string. A class instance arrives as a plain object without its methods. A BigInt throws during serialization. Nothing validates that the payload matches what the handler expects. To change the wire format, for example to a binary encoding, pass custom serializer and deserializer objects in the client and server transport options. Each implements a single serialize or deserialize method. The wire format matters less than the contract it carries.

Schemas matter more than the transport

Treat every pattern and event payload as a public API. Define the payloads in one versioned package or schema definition that producers and consumers both depend on. Include an explicit version field. Restrict changes to additive ones: new optional fields only, no renames or type changes. Validate payloads at the handler boundary so that a malformed message fails loudly and goes to the dead-letter path, instead of corrupting state further downstream.

This discipline pays off when you migrate. If contracts are explicit and validated, moving from Redis to NATS or Kafka mostly means swapping client configuration and serializer wiring. You don't have to rediscover implicit payload shapes service by service.

The contracts do not carry over the rest of the system's behavior, though. Delivery guarantees, ordering scope, redelivery timing and acknowledgment semantics all change with the broker. Re-test idempotency, retry and ordering assumptions under the new transport before you cut traffic over.

Choosing a transport: decision framework, when to avoid each, and cost

Start from the failure you can least afford, not from throughput.

NATS has the lowest development cost for request-response. Its request-reply model matches @MessagePattern directly, so you skip reply-topic plumbing. Core NATS is fire-and-forget, though. If a subscriber is down, the message is gone. Adding JetStream gives you persistence and acknowledgments, but you then manage streams and consumers, which brings back some of the operational work you chose NATS to avoid.

Kafka costs the most engineering time, mostly in operations. You have to manage the cluster, size partitions up front, absorb consumer-group rebalancing pauses, and monitor consumer lag. What you get in return is a replayable log that serves as an audit trail and lets new consumers rebuild state from history. A small team without platform support usually spends that time on operations instead of product work.

Redis costs almost nothing extra if you already run it. The tradeoff is that Pub/Sub delivers only to subscribers connected at that moment, so treat it as best-effort.

If none of these fit, NestJS ships other transports. RabbitMQ gives you per-message acknowledgments and queue-based durability without managing a log, and gRPC gives you typed RPC contracts.

How can it save engineering time?

The time savings come from code you don't write and decisions you can reverse cheaply. They do not come from the broker itself.

• No per-service broker plumbing. Without the abstraction, each service needs its own code for connecting, subscribing, routing messages to functions, correlating replies and handling timeouts. With @MessagePattern, @EventPattern and ClientProxy, that logic lives in the framework, and a new handler is one decorated method.
• One mental model across HTTP and messaging. Dependency injection, pipes, guards, interceptors and exception filters work in microservice handlers as they do in HTTP controllers. Validation, logging and error mapping can be reused instead of rebuilt for the messaging layer.
• Cheaper transport changes. Swapping brokers mostly means changing configuration, as the first code example shows. You still have to re-test delivery, ordering and idempotency, but you don't rewrite handlers.
• Easier testing. Handlers are ordinary class methods, so you can unit-test them by calling them directly with a payload, without a running broker. Integration tests can then focus on the transport-specific behavior that actually differs.
• Less operational work when you match the transport to the workload. Choosing core NATS or an existing Redis for best-effort traffic, and keeping Kafka for streams that really need a durable log, avoids running heavier infrastructure than the workload requires.

The savings disappear if the abstraction is used to hide semantics. A team that assumes Redis behaves like Kafka will spend that time later, debugging lost events and duplicate side effects. The patterns covered above keep the savings real: explicit contracts, idempotent handlers, a dead-letter path and per-transport scaling settings.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nestjs microservices transport, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.