Optimizing LLM Inference with LLMD on Kubernetes
Eduardo Vache discusses the strategic shift toward distributed inference on Kubernetes using LLMD. Learn how to maximize GPU utilization, manage stateful conversations via caching, and leverage smaller models for cost-efficient, private AI deployments.