One of the easiest assumptions to make when integrating webhooks is that every incoming request represents a new event.
In reality, webhook providers make no such guarantee.
A network timeout, temporary server failure, or delayed acknowledgment may cause the same event to be delivered multiple times. From the provider's perspective, this is expected behavior. From the application's perspective, blindly processing every request can quickly lead to duplicate data and inconsistent conversations.
When implementing inbound SMS, the challenge wasn't receiving messages—it was ensuring every message entered the conversation exactly once.
The Problem
An inbound SMS follows a surprisingly long journey before it appears in an agent's conversation timeline.
The provider receives the message.
It sends an HTTP request to our webhook.
The application validates the request, identifies the tenant, locates the conversation, persists the message, and updates the conversation state.
If any part of that flow fails before the provider receives a successful response, the provider assumes the delivery failed and retries the webhook.
Without safeguards, that retry creates duplicate messages.
Customers see repeated responses.
Agents see duplicated timelines.
Automation may execute twice.
The problem isn't uncommon.
The solution must be intentional.
Accepting That Retries Are Normal
The first design decision was philosophical.
Retries are not exceptional events.
They're part of the delivery protocol.
Instead of trying to prevent providers from retrying, the application assumes every webhook may already have been processed.
That shifts the question from:
"Is this request valid?"
to
"Have I already processed this event?"
Webhook retries are part of a provider's reliability strategy. The application should be designed to safely process repeated deliveries instead of assuming every request is unique.
Every Message Already Has an Identity
Fortunately, inbound SMS messages already arrive with something incredibly useful.
A provider-generated message identifier.
That identifier is globally unique for the lifetime of the message.
Instead of inventing another deduplication mechanism, the platform simply stores that identifier alongside the persisted message.
Conceptually, every inbound message follows this flow.
The identifier becomes the application's definition of message uniqueness.
Idempotency at the Database Layer
Application-level checks alone aren't enough.
Imagine two identical webhook requests arriving almost simultaneously.
Both requests query the database.
Both conclude the message doesn't exist.
Both attempt to insert it.
Without additional protection, duplicates still appear.
To avoid this race condition, uniqueness is enforced where it matters most.
The database.
Instead of relying solely on application logic, message identifiers are protected by a unique constraint.
The application can safely attempt insertion knowing the persistence layer guarantees only one succeeds.
This approach eliminates an entire class of concurrency problems without introducing additional synchronization mechanisms.
Separating Validation from Processing
Another important design decision was keeping webhook validation separate from business logic.
Before the application considers persisting anything, it first establishes that the request genuinely originated from the communication provider.
Only after validation succeeds does the request enter the message processing pipeline.
That separation serves two purposes.
First, it prevents unauthorized requests from reaching the domain layer.
Second, it ensures the rest of the application can assume every inbound event has already passed authenticity checks.
Business logic remains focused on conversations rather than request verification.
Conversation Consistency
Idempotency isn't only about preventing duplicate database rows.
It's about protecting the conversation itself.
Every inbound message potentially updates conversation metadata.
Recent activity.
Unread counts.
Last message preview.
Agent notifications.
If duplicate messages were allowed into the system, every downstream process would execute multiple times.
By ensuring only one message enters the conversation, every dependent operation naturally executes once as well.
The rest of the platform benefits from a simpler assumption:
Every persisted message represents one real customer interaction.
Failure Without Corruption
One of the more subtle benefits of idempotent processing appears during failure recovery.
Suppose persistence succeeds but the HTTP connection drops before the provider receives a successful response.
The provider retries.
Without idempotency, the second delivery creates another message.
With idempotency, the application recognizes the existing identifier and safely ignores the duplicate.
The retry becomes harmless.
Instead of asking whether a request has already been seen, the platform simply asks whether the resulting state already exists.
That distinction makes recovery much more reliable.
Requests may arrive multiple times. What matters is ensuring the resulting conversation state remains correct regardless of how many times the same event is delivered.
Why Not Rely on Memory?
An obvious alternative would be caching recently processed message identifiers.
While faster, that approach introduces new failure modes.
Application restarts lose cached identifiers.
Multiple application instances require distributed synchronization.
Cache expiration risks allowing duplicates after entries disappear.
The database already represents the permanent source of truth.
Allowing it to enforce uniqueness keeps the architecture considerably simpler while remaining reliable across deployments and server restarts.
Beyond SMS
Although this implementation was built around inbound SMS, the underlying pattern is transport-independent.
Any external system capable of retrying event delivery benefits from the same approach.
Payment notifications.
Email providers.
Webhook integrations.
Third-party APIs.
As long as events carry a stable identifier, the application can safely guarantee that processing the same event multiple times produces the same final state.
That's the essence of idempotency.
Lessons Learned
Building webhook integrations changes how you think about incoming requests.
A request is no longer an instruction to execute business logic.
It's evidence that an event may have occurred.
The application's responsibility is to determine whether that event has already been reflected in its own state.
Once that mindset changes, duplicate deliveries become far less intimidating.
Retries stop being a source of bugs and become another expected characteristic of distributed systems.
Future Improvements
The current implementation provides reliable duplicate prevention through provider-generated message identifiers and database-level uniqueness.
Future iterations could strengthen observability around this workflow by introducing:
- Metrics for ignored duplicate deliveries.
- Monitoring for abnormal retry rates.
- Dead-letter handling for malformed webhook payloads.
- Structured event tracing across the webhook lifecycle.
- Operational dashboards for inbound message processing.
These additions wouldn't change the idempotency model itself.
They would simply make the system easier to observe and operate as message volume grows.
Reliable webhook processing isn't about preventing retries.
Retries are inevitable.
The real goal is ensuring that no matter how many times the same event is delivered, the conversation evolves exactly once.
When the system can make that guarantee, webhook delivery becomes predictable, failures become recoverable, and distributed communication becomes significantly easier to reason about.