Overview
One of the platform's core features allows businesses to continue customer conversations through SMS using dedicated phone numbers. Every tenant requires isolated telephony resources before this workflow becomes available.
Provisioning those resources manually was operationally expensive and increasingly difficult to scale as customer onboarding accelerated.
This case study explains how we automated the provisioning lifecycle while ensuring reliability, idempotency, and tenant isolation.
Company-specific implementation details have been generalized. The engineering architecture and design decisions accurately reflect the production system.
The Problem
Provisioning a phone number sounds simple until it becomes part of a production onboarding workflow.
A successful onboarding sequence depended on multiple external operations completing successfully. Failures midway through the process could leave tenants partially configured or require manual intervention.
The provisioning workflow needed to:
- execute automatically
- avoid duplicate resources
- recover safely from failures
- isolate every tenant
- avoid blocking customer onboarding
Architecture
Provisioning executes asynchronously through background workers rather than during the subscription request itself. This keeps onboarding responsive while allowing retries and recovery when external services experience temporary failures.
Engineering Decisions
External APIs should never sit directly in the critical request path. Moving provisioning into background workers improved reliability while making retries straightforward.
Provisioning jobs may execute more than once because of retries, deployments, or worker failures. Every operation was designed so repeated execution produced the same final state.
Challenges
External services introduce uncertainty.
Timeouts, temporary failures, and rate limits meant workers needed retry strategies that balanced resilience without creating duplicate resources.
Monitoring and structured logging became equally important as the provisioning logic itself.
Outcome
Provisioning became a fully automated onboarding process requiring no manual intervention.
The system reliably provisions tenant-specific communication infrastructure while remaining resilient to transient failures and supporting safe retries.
Related Engineering Notes
Designing Multi-Step Telephony Provisioning with Rollback Orchestration
Provisioning communication resources for a tenant required far more than purchasing a phone number. This article explores how we orchestrated a multi-step provisioning workflow while keeping the system consistent in the face of partial failures.
Designing Multi-Tenant Resource Isolation
Supporting multiple companies on a shared platform doesn't mean sharing communication infrastructure. This article explores why every tenant owns its own communication resources and how that decision simplifies security, operations, and future scalability.