Cursor engineers find LLMs ineffective for asynchronous performance instrumentation

At Cursor, engineers working on reducing time-to-first-token found that LLMs were poor at instrumenting asynchronous systems. While LLMs could generate basic instrumentation, they often failed to understand complex concurrent flows, leading to traces with millions of useless spans or incorrect sampling strategies that broke debugging capabilities. The models also proposed naive optimizations, such as parallelizing tasks that were rarely executed together, which increased load without improving latency. Engineers resorted to manual code analysis, printing large sequence diagrams on blueprint paper to annotate synchronous versus asynchronous chunks. They also found that LLMs wrote unit tests that merely confirmed the code's existing behavior rather than testing edge cases. This necessitated the use of mutation testing to force the models to identify missing coverage. The experience highlights that LLMs currently lack the holistic understanding required for complex performance engineering and should be verified manually, particularly in asynchronous contexts.