Deterministic checks essential for reliable agentic workflows
Wollmann argues that probabilistic LLMs require hard guardrails and clear human-agent interfaces to prevent costly errors. He advocates for a multi-layered approach combining soft alignment with deterministic rules.
Wollmann describes his personal workflow using VS Code as a central interface for note-taking, slide creation, and prototype building, integrating models like Codex and Copilot via MCP servers. He advocates for using 'skills,' or automated standard operating procedures for LLMs, derived from organizational data warehouses to standardize agent behavior. A critical theme in his analysis is the need for robust guardrails to manage the velocity of coding agents. Wollmann warns that LLMs are probabilistic and can derail, such as randomly refactoring entire codebases during looped tasks, leading to high token costs and broken outputs. He proposes a multi-layered approach combining soft alignment, such as evaluation and fine-tuning, with hard guardrails, including deterministic rules, proxy checks, and judge models. He draws parallels to Domain-Driven Design and Team Topologies, suggesting that clear interfaces between humans and agents are necessary to maintain control. Wollmann notes that while tools like Pi-based agents and OpenClaw are maturing, especially with NVIDIA’s push for sandboxing, enterprises must harden these environments before large-scale deployment to avoid loss of control and liability.