Key Architectural Principles
- 01.Dual writes to a database and a message broker within one HTTP request will inevitably result in inconsistency.
- 02.The Transactional Outbox persists outbound events into the same SQL transaction as domain state mutations.
- 03.A background worker asynchronously relays outbox rows to Azure Service Bus with exponential Polly retries.
- 04.Message deduplication using deterministic MessageId headers guarantees at-least-once with idempotent consumption.
In distributed microservices, updating a database and publishing an event to a message queue within the same HTTP request is fundamentally vulnerable to partial failure. If the message broker times out after the database commit, events are lost.
The Transactional Outbox pattern guarantees at-least-once messaging by storing outbound domain events in the same database transaction as the domain entity state change. A background hosted service then asynchronously relays the outbox records to Azure Service Bus.
We explore idempotency deduplication with MessageId, dead-letter monitoring, and Polly retry resilience policies in .NET.
1. The Fallacy of the Distributed Dual Write
Attempting to write to SQL and publish to Azure Service Bus in an HTTP controller creates an unavoidable race condition: if the broker connection times out or the app worker crashes between operations, the system drifts into permanent inconsistency.
Distributed transactions (2PC) kill throughput and availability. The Transactional Outbox pattern provides atomic consistency using the local database's ACID transaction guarantee.
2. Designing the Outbox Table & Publisher
An OutboxMessages table contains the message payload, event type, timestamp, correlation IDs, and processed timestamp. When domain entities are saved, outbox records are inserted in the identical DbContext.SaveChangesAsync() call.
A background IHostedService worker polls or reads the outbox table via change-data-capture or ordered batch queries, publishing batches to Service Bus and marking them processed.
3. Idempotent Consumer Protection
Because network hiccups can cause message delivery retries, consumers must be idempotent. Setting the ServiceBusMessage.MessageId to the Outbox Record GUID enables Azure Service Bus duplicate detection window, while consumer-side inbox tables ensure zero duplicate processing.
// Resilient Background Outbox Publisher with Azure Service Bus
public class OutboxPublisherWorker : BackgroundService
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly ServiceBusSender _sender;
private readonly ILogger<OutboxPublisherWorker> _logger;
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
using var scope = _scopeFactory.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
var pendingEvents = await db.OutboxMessages
.Where(m => m.ProcessedAtUtc == null)
.OrderBy(m => m.CreatedAtUtc)
.Take(50)
.ToListAsync(stoppingToken);
foreach (var message in pendingEvents)
{
var busMessage = new ServiceBusMessage(message.Payload)
{
MessageId = message.Id.ToString(), // Azure Service Bus deduplication key
Subject = message.EventType,
ContentType = "application/json"
};
await _sender.SendMessageAsync(busMessage, stoppingToken);
message.ProcessedAtUtc = DateTime.UtcNow;
}
if (pendingEvents.Count > 0)
await db.SaveChangesAsync(stoppingToken);
await Task.Delay(1000, stoppingToken);
}
}
}