TL;DR
Technical analysis of Google's Tensor Processing Unit design evolution, from TPUv1's inference-focused systolic array to TPUv2's training capabilities, revealing how specialization compensates for Moore's Law slowdown.
Key Points
- TPUv1 features 256x256 weight-stationary systolic array with 24MiB unified buffer, achieving 15-30x inference speedup and 30-80x perf/W over K80 GPU and Haswell CPU
- TPU7 (Ironwood) announced April 2025: 9,216 chips per pod, 42.5 Exaflops, 10 MW power consumption across 12-year evolution
- TPUv2 dual-core design introduced BrainFloat16 precision and inter-core interconnects to enable distributed training workloads, replacing TPUv1's fixed activation units
- Over 50% of traditional processor die energy spent on caches and register files; instruction fetch/decode costs 10-4000x more than arithmetic operations
Why It Matters
This architectural deep-dive demonstrates how hardware specialization becomes essential when general-purpose scaling (Moore's Law, Dennard Scaling) plateaus. For infrastructure engineers and chip designers, TPU's co-design approach—optimizing for specific workloads (matrix ops, data movement patterns) rather than generality—provides a proven template for building efficient accelerators in the post-Moore's Law era. Understanding these tradeoffs is critical as organizations evaluate custom silicon vs. commodity GPUs.
Source: considerthebulldog.com