MIT researchers show GPT-6 Astra can edit smaller model weights
A new paper demonstrates that frontier models can directly adjust the weights of smaller networks to improve performance without traditional training. The findings suggest a new optimization technique, though experts debate whether the results indicate general capability or specific bug fixes.
MIT researchers led by PhD student Rukia Takahi published a paper demonstrating that GPT-6 Astra can read and directly adjust the weights of a smaller model to enhance its performance. In experiments using the Qwen 3.5 0.8b model, Astra modified only 0.016% of the weights, resulting in a sixfold improvement in math benchmark scores and a threefold increase in instruction following. The process required no gradients or training; instead, Astra identified reasoning loops in Qwen and edited them out using a budget of 50 forward passes. The researchers noted that frontier large language models are surprisingly capable of natively reading and writing neural network weights, a skill that exceeds expectations for conventional algorithms.
Alex Krensel, a senior researcher at Google, replicated the results and described the capability as a step-change emerging in the latest models. However, he noted that the current application is limited to small models, such as the 20,486-parameter breakout experiment, rather than trillion-parameter frontier systems. Chandan Singh from Voyage AI argued the approach is undersold, claiming that Claude models can write entire functioning neural network weights from scratch. Conversely, Wei A. Zhu from Microsoft Research cautioned that the headline is overhyped, noting that the significant math improvement stemmed from fixing a specific loop bug rather than improving general capability. The findings have sparked debate over whether this represents a significant step toward recursive self-improvement or a niche optimization technique.