LeashBio releases benchmark showing frontier models excel at chemical decision-making

LeashBio has developed a virtual sandbox environment for AI agents to perform structure-activity relationship (SAR) optimization, a core task in medicinal chemistry. The system uses pre-measured data from DNA-encoded libraries to allow agents to explore millions of compounds against thousands of proteins with instant feedback. The company has conducted approximately 90,000 agent rollouts in this environment and is releasing a preprint, a dataset, and a blog post detailing these results. LeashBio screened about 1,000 proteins against several chemical libraries and aims to scale from two new environments per day to 20 per day by early 2027.

The benchmark reveals that higher-performing models are significantly better at abandoning unproductive chemical neighborhoods, while lower-performing models persist in testing ineffective molecular modifications. In the results, Claude Opus 4 emerged as the top-performing model, with Claude Sonnet 5.5 and Claude Sol 6.1 also performing well. In contrast, Claude Sonnet 5 was described as both expensive and underperformant, with one participant suggesting it would be long considered a mistake by Anthropic. Open-source models were also tested, with DeepSeek v4 Flash noted as performing reasonably well. The results show a clear separation in capability across model families, with the best models demonstrating superior strategic decision-making in navigating the chemical landscape.

LeashBio notes that frontier models previously struggled with SMILES string manipulation, a capability that improved significantly around February, enabling more effective chemical reasoning. The rapid release cycle of new frontier models, occurring roughly every 11 days, complicates the benchmarking process for LeashBio, as they must frequently update their evaluations to include the latest releases. The company argues that pre-measured, counterfactually dense environments provide a scalable method to evaluate and improve the scientific decision-making capabilities of AI agents in drug discovery.