Blind test compares human and AI architecture decision records

Eberhard Wolf and Ralf conducted a blind experiment at the Infodays keynote comparing Architecture Decision Records (ADRs) generated by a human versus an AI for a bicycle repair shop system. The AI, identified as Claude Opus 5.5, processed three hours of prior video content, transcribed the audio, and generated an ARC 42 documentation including 17 fine-grained decisions with time-stamped references to the source audio. The experiment covered three specific architectural problems: employee authentication, frontend technology selection, and system availability during webshop outages. For authentication, the human ADR proposed standard software like Keycloak or Auth0 for SaaS extensibility, while the AI ADR chose custom employee accounts to maintain independence from the webshop. For frontend technology, the human ADR selected a Single-Page-Application framework based on team familiarity, whereas the AI ADR chose server-rendered HTML pages to simplify JWT cookie handling and avoid a separate frontend repository. For availability, the human ADR accepted coupled availability with the webshop, while the AI ADR proposed decoupling by validating JWTs locally and copying customer data. Audience voting revealed mixed results: the human ADR was preferred for authentication, with 18 votes to 4, and for availability, while the AI ADR was preferred for frontend technology. The AI was correctly identified as the author in two of the three cases. The presenters argued that AI serves as a useful supporting instrument for architecture decisions, capable of referencing quality scenarios and counter-arguing trade-offs. They emphasized that real-world architecture remains a collaborative process requiring stakeholder buy-in and human review for clarity and domain understanding.