TL;DR
Zepto engineered a transformer-based dual-sequence reranker that combines long-term user history with in-session intent to deliver personalized rankings in single-digit milliseconds across millions of daily users.
Key Points
- Dual transformer encoders separately model historical behavior and real-time session intent, preventing either signal from drowning out the other
- Target-aware pooling dynamically rebuilds user profile per candidate SKU, enabling context-aware ranking that adapts instantly to session context switches
- P99 inference latency achieved in low single-digit milliseconds through encode-once optimization and zero-copy tensor expansion across 60+ candidate items
- Hybrid loss function combines in-session listwise ranking with batch-sampled softmax negatives and weighted BCE to prioritize both discovery and replenishment behaviors
Why It Matters
This architecture demonstrates production-grade solutions to hard problems in recommendation systems at scale: handling heterogeneous user intents in real-time, managing cold-start for new inventory, and maintaining sub-millisecond latency under massive throughput. The engineering patterns—encode-once serving, target-aware pooling, and hybrid loss design—are directly applicable to any large-scale ranking system.
Source: blog.zepto.com