Mistral releases ML4, a one-trillion-parameter open-weights model
Mistral has released the weights for its ML4 model, which features one trillion total parameters and 49 billion active parameters. The release aims to close the performance gap with leading open-source models while maintaining high inference efficiency.
Mistral released the weights of its ML4 model, nicknamed Le Chong, on Thursday. The model is described as the largest and most advanced from Europe, designed to compete with open-source leaders such as Qwenn, Gemma, and Kimi. Pierre Stock, Vice President of Science at Mistral, explained that the architecture follows a trend of increasing total size while keeping sparsity high. Each token accesses only 50 billion parameters, a design that makes inference extremely efficient and outperforms dense 50 billion parameter models.
The model was trained entirely in Mistral's own French data centers. Stock noted that the pre-training team and cluster team worked together in one room, bringing the model to life in one night, a process that would have taken weeks with a cloud provider. After a security review by research partners and government organizations, the weights were released. Feedback indicated the initial version was too verbose, leading to a final reinforcement learning pass that reduced token consumption by 50% compared to the version from three weeks prior while maintaining intelligence.
On the Artificial Analysis Coding Index, ML4 is the best open-weights model and 50% more efficient than competitors. Quantized to FP4, it can run on four NVIDIA GB300 GPUs, making it accessible to universities and research labs. Mistral argues that owning infrastructure allows for faster debugging and development, and that open weights are crucial for sovereignty and preventing access cuts.