Everything on Hardware Architecture

Inference is a memory-bound problem, not a compute-bound one. The divergence between compute growth (120x) and memory bandwidth growth (17x) creates a bottleneck that limits token generation speed.

Impact: Hardware vendors must prioritize memory capacity and bandwidth over raw FLOPS to remain competitive in the inference market.

— from AI Infrastructure Economics and the Memory Wall · The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch· Sep 19, 2026