Ingest CEO details infrastructure hurdles for long-running AI agents

Tony Holtstock-Brown, CEO of Ingest, describes long-running agents as the most significant directional shift in artificial intelligence since the introduction of tools. He defines these systems as those that live for months and build their own context, a capability that requires substantial infrastructure support. Holtstock-Brown notes that Ingest’s existing background job infrastructure mapped directly to these agent workflows, though the company had to make low-latency adjustments to handle synchronous chat patterns. The primary engineering hurdles he identifies are context management and statefulness, as large language models are stateless and require external systems to store memory.

To manage asynchronous sub-agent orchestration, Ingest uses a step-based API with a "step.waitForEvent" function, avoiding direct coupling between components. The company recently faced compute constraints, leading to the acquisition of bare metal machines and the implementation of a cell-based multi-tenant architecture to handle high-throughput state storage. Holtstock-Brown argues that tracking production outcomes via events is superior to relying on non-deterministic evaluations. He claims that 35% of teams building AI in production do not use evaluations at all. Furthermore, he notes that 74% of surveyed AI teams experienced a customer-visible incident in the past 90 days, underscoring the need for robust observability. Ingest reports that 93% of customers using agents to interact with the platform activate, compared to a 40% baseline for non-agent interactions.