TL;DR
Vercel migrated production billing state from ephemeral Redis to durable DynamoDB while handling live traffic, discovering hidden N+1 query bottlenecks in the process.
Key Points
- Redis latency of ~1ms vs DynamoDB's ~5ms at P95 caused supply loop to stall, stretching runs to minutes
- Migration executed in five feature-flagged phases with shadow reads validating every write before cutover
- Concurrent redesign of supply loop eliminated serialization on single reads, removing flaw older than the project
- Redis infrastructure failure in March occurred mid-migration; DynamoDB-migrated state remained operational
Why It Matters
This is a masterclass in production database migrations under load—showing how to validate correctness while discovering that latency assumptions baked into architecture can silently degrade performance. For infrastructure engineers, it demonstrates why measuring actual access patterns matters more than theoretical database selection, and how to redesign systems when store characteristics change rather than chasing performance parity.
Source: vercel.com