Modal targets underserved mid-sized AI training workloads

Modal is expanding its product portfolio beyond inference to include training and batch processing, targeting an underserved market of mid-sized training runs, such as those using 64 GPUs. This segment serves researchers, computational biotech, and recommendation systems, rather than competing for the massive 100,000-GPU clusters used by frontier labs. The company is also developing products for post-training, reinforcement learning, and distillation. Modal anticipates that 2027 may be the year non-NVIDIA accelerators, such as AMD and TPUs, become viable alternatives, though it notes bottlenecks at TSMC and ASML. The company is exploring hybrid inference setups using different hardware for pre-fill and decode phases. Modal's strategy focuses on providing general-purpose building blocks for AI development, including storage, data warehousing, and real-time streaming pipelines, to support a wide range of AI-native applications. The firm argues that the market for smaller, iterative training runs is significantly underserved by current infrastructure providers who focus on massive frontier-scale clusters, representing a major growth opportunity.