Liquid AI raises $250 million to deploy edge-first LLMs in cars

Liquid AI, an MIT spin-off co-founded by Matthias Lechner and Ramin, has secured a $250 million funding round led by AMD, valuing the company at $2.3 billion. The company, which emerged from research at TU Wien and MIT focused on robust neural networks, has deployed its LHM2 foundation models in the Mercedes CLA as an in-car voice assistant. Unlike cloud-based models, Liquid AI’s models are designed for edge devices, achieving GPT-3.5-level instruction following at 40 tokens per second on a Raspberry Pi.

Lechner argues that memory bandwidth and memory size, rather than raw compute, are the primary bottlenecks for local LLM inference. He describes a "triangle" of constraints: compute, memory bandwidth, and memory size. For example, a Raspberry Pi has sufficient RAM but slow bandwidth, limiting it to 40 tokens per second, while phones have decent bandwidth but limited total memory. Lechner predicts future models will leverage Neural Processing Units for low-precision computation and adopt "tiered memory" architectures to manage context efficiently.

The company maintains CEO-level partnerships with Qualcomm and AMD to access unreleased chips and co-design instructions for next-generation hardware. Lechner notes that local inference is essential for privacy-sensitive tasks like 24/7 voice recording, where users prefer on-device processing over cloud transmission. The firm operates a cluster of several thousand GPUs and is currently developing a continual learning platform to allow automotive clients to update their in-car models more frequently than traditional certification cycles permit.