Study finds half of cited LLM limitations are obsolete
A Stanford study re-evaluating 50 widely cited findings on language models found that nearly half are obsolete. The researchers urge treating such findings as snapshots of specific model versions rather than general truths.
A study by two Stanford researchers re-evaluated 50 widely cited findings about language models across 20 models from eight developers. The analysis found that nearly half of these findings are obsolete, with the original effects largely vanished. Another third of the findings has significantly weakened, while only about one-sixth remains valid. When a decay date could be estimated, the findings lasted an average of 16 months. Claims about what models cannot do, such as counting letters in specific words or calculating in unusual number systems, decay fastest because developers can specifically train away these errors. However, tendencies like sycophancy, political left-leaning bias, and preference for their own answers persist. In a test on inventing scientific sources, GPT 3.5 invented over 50% of citations in 2023, while GPT 6.1 invented 0% and the latest Claude Sonnet invented 1%. In contrast, Alibaba's QN 3.7 Flash invented more than two-thirds of its sources. The researchers argue that findings about AI capabilities should be treated as momentary snapshots of specific models rather than general truths. They urge that studies must specify the model, version, and date to allow for future verification. This approach is necessary because the rapid iteration of AI models renders static assessments of limitations quickly outdated.