The Transactional Outbox in Practice: Reliable Emails and Side Effects from Next.js and PostgreSQL

Make emails and third-party calls reliable in a Next.js and PostgreSQL app with a transactional outbox: claims, leases, backoff and per-recipient delivery.

Key Takeaways

Sending an email or calling a third-party API inside a business transaction couples your data to someone else's uptime. Sending it after the commit risks losing it. The transactional outbox solves both.

The pattern: write the side effect as a row in an outbox table in the same transaction as the business change, then deliver it separately (microservices.io).

In Next.js, deliver right after the response with after() for speed, and run a poller for retries and recovery.

Make it safe to run on several instances: conditional claims, a lease that expires, exponential backoff and a maximum number of attempts.

For multi-recipient emails, record delivery per recipient so a retry never re-sends to people who already got the message.

We used this pattern in the clinical sample tracking platform we built for a biosciences company, where nine kinds of email notification and every user invitation go through an outbox. The requirements there were simple to state: a registered sample must never fail to save because the email provider is slow, and a coordinator must never silently miss a notification.

This article walks through the design, with the trade-offs we made. It applies to any Node.js application backed by PostgreSQL, but the examples use Next.js.

Why can't you just send the email after saving?

Because there is always a moment between the commit and the send where the process can stop. Consider the two naive options:

Send inside the transaction. If the email API takes ten seconds, the database transaction stays open for ten seconds. If it fails, you either roll back a perfectly valid business change or swallow the error. Your write path now depends on a third party's availability.

Send after the commit. The business change is safe, but if the server restarts, the deploy rolls, or the email call throws between the commit and the send, the notification is gone and nothing records that it was ever owed.

This is the classic dual-write problem: two systems (your database and the email provider) need to change together, and there is no transaction spanning both. The transactional outbox pattern, described by Chris Richardson, avoids it by making the database the only system written during the request.

The pattern in one paragraph

When a business change needs a side effect, the service inserts an outbox event in the same database transaction as the change. The event says what must happen (send this notification, create this invitation) and to whom. If the transaction commits, the event exists; if it rolls back, the event never existed. A separate processor reads pending events, performs the side effect, and marks each event completed, or schedules a retry.

The guarantee you get is at-least-once delivery: every committed change eventually produces its side effect, and a duplicate is only possible in a narrow window we will come back to.

What goes in an outbox row?

A minimal outbox table needs:

Column	Purpose
`type`	What to do (for example, a notification type or "create invitation")
`payload`	The identifiers needed to do it
`status`	Pending, processing, completed or failed
`attemptCount`	How many times delivery has been tried
`availableAt`	When the event may next be picked up (used for backoff)
`lockedBy`, `lockedAt`	Which worker claimed it, and when
`lastError`	The most recent failure, for operators

Two additions made a real difference in practice.

A frozen recipient snapshot. Recipients are resolved when the event is created, not when it is sent, and stored on the row. If a coordinator loses their role five minutes after a sample is registered, the notification about that sample still goes to the people who were entitled to it at the time. It also makes the notification history accurate: you can see exactly who an event was addressed to.

A list of delivered recipients. One event can go to several people. Each successful send is recorded on the row immediately. We come back to why in the section on partial failures.

Writing the event: same transaction, every time

With Prisma, the business change and the outbox insert share one interactive transaction:

await prisma.$transaction(async (tx) => {
    const sample = await tx.sample.create({ data: input });

    await tx.auditEvent.create({ data: auditFor(sample) });

    await tx.outboxEvent.create({
        data: {
            type: "SAMPLE_CREATED",
            payload: { sampleId: sample.id },
            recipientEmails: await resolveRecipients(tx, sample),
        },
    });
});

The rule is that no code path performs the side effect directly. The web form, the spreadsheet import and the machine API all call the same service, so all of them write the same outbox event. If one path sends email inline "because it's simpler", the guarantees disappear for that path.

Delivering it: right away, and again later

An outbox needs a processor. We run two triggers for the same processing function.

Right after the response, with `after()`

Next.js provides after() to schedule work once the response has been sent. In a Server Action or Route Handler, after the transaction commits:

import { after } from "next/server";

after(() => processNotificationOutbox({ limit: 20 }));

The user sees their change saved immediately, and the email usually leaves within a second or two. If this attempt fails or the process dies, nothing is lost: the event is still pending in the database.

A poller for retries and recovery

A small in-process poller, started from Next.js's instrumentation.ts when the server boots, runs the same processor every 60 seconds. It picks up events that failed, events whose backoff has elapsed, and events created while the request-time flush didn't run.

This choice has a consequence worth stating: an in-process poller needs a long-running Node server, such as a Docker container or next start. On a serverless platform there is no process between requests to run it. In that case, disable the poller and call the processor from an external scheduler (a cron job hitting a protected route) instead.

Running safely on more than one instance

The moment you run two app instances, two processors may try to deliver the same event. Three mechanisms prevent that.

Conditional claims

A worker claims an event with an update that only succeeds if the event is still pending:

const claimed = await prisma.outboxEvent.updateMany({
    where: { id: event.id, status: "PENDING" },
    data: { status: "PROCESSING", lockedBy: workerId, lockedAt: new Date() },
});
if (claimed.count !== 1) return; // another worker got it first

Only one worker can move a given row from pending to processing, so each event is handled by one worker at a time.

Leases that expire

If a worker crashes after claiming an event, the row would stay "processing" forever. So the claim is a lease: at the start of every run, the processor releases events whose lock is older than a threshold (we use 15 minutes) back to pending, or marks them failed if they have used up their attempts.

Leases introduce a subtle risk. If one event takes longer than the lease (many recipients, each with a slow provider), another worker might reclaim it while the first is still sending. Two safeguards close that gap: the worker renews its lock after each recipient, and every outcome write (completed, failed, recipient delivered) is conditional on the worker still holding the lock. A worker that has lost its lease stops sending.

Backoff and a maximum number of attempts

A failed delivery goes back to pending with availableAt pushed into the future, doubling each time: in our case 30 seconds, then 1, 2 and 4 minutes. After the fifth failed attempt the event is marked failed and stays visible in the notification history with its last error. Unexpected exceptions follow the same path; an event must never be retried forever.

Partial failures: tracking delivery per recipient

Suppose an event goes to four people and the third address is rejected. Without per-recipient tracking, the retry sends the email to all four again, and the first two get duplicates on every attempt.

Recording each successful recipient on the event as soon as the send succeeds fixes this. On retry, the processor skips anyone already delivered. One bad address no longer blocks the others, and the others don't get spammed while it is being sorted out.

This also defines the one remaining duplicate window precisely: a duplicate email is only possible if the process stops between the provider accepting a message and the app recording it. That is a window of milliseconds, and it is the price of at-least-once delivery. If a side effect absolutely cannot be repeated (charging a card, for example), the receiving side needs an idempotency key as well.

Operating it

An outbox is also an operations tool, because every pending and failed side effect is a queryable row:

sql

-- Backlog and failures by type
SELECT type, status, count(*) FROM "OutboxEvent" GROUP BY 1, 2 ORDER BY 1, 2;

Four habits keep it healthy:

Alert on failed rows. A failed event means someone didn't get a message they were owed.
Show the history in the product. On the sample tracking platform, staff can see every notification, its recipients and its delivery state, without opening a database console.
Prune finished events on a schedule. Completed and failed events are pruned after a retention period (two years by default there), while pending and processing rows are never touched.
Never put secrets or personal data in the payload. Store identifiers and build the message at send time.

When is an outbox overkill?

When nothing important depends on the side effect. A "thanks for your message" auto-reply on a marketing site can be sent inline and forgotten. An outbox earns its place when a missed or duplicated side effect has a cost: a notification a clinical site must act on, an invitation someone is waiting for, a webhook another system depends on.

It also adds a moving part to monitor. For a small application, a single table, one processing function and one poller is the whole of it; there is no message broker to run.

Frequently asked questions

Do I need Kafka or RabbitMQ for a transactional outbox?

No. The outbox pattern is often paired with a message broker in microservice architectures, but the outbox itself is just a table. For an application that sends emails and calls a few APIs, a processor reading the table directly is enough.

Should I use `SELECT ... FOR UPDATE SKIP LOCKED` instead of conditional updates?

Both work in PostgreSQL. SKIP LOCKED is efficient for claiming batches under heavy contention; a conditional UPDATE ... WHERE status = 'PENDING' is simpler with an ORM and plenty fast at notification volumes. Either way, keep the lease and lock checks.

Does `after()` replace the poller?

No. after() makes delivery fast; it doesn't make it reliable. If the process stops before the callback finishes, only the poller (or an external scheduler) will pick the event up again.

How does this relate to FormRoute?

On the sample tracking platform, the outbox processor hands each email to FormRoute, our transactional delivery service. The outbox guarantees the message is owed and retried; FormRoute handles sending it.

The outbox is one of the patterns behind the clinical sample tracking platform we built for a biosciences company. If your application's emails, invitations or webhooks go missing under load or during deploys, talk to us.

All insights →