Demandbase Ripped Out Its Custom AI Orchestrator. The Product Got Better.

Demandbase’s AI team spent months building a custom multi-agent orchestrator for its conversational data assistant — then ripped it out and replaced it with an open-source agent loop, and the product got measurably better. Principal engineer Shea Hawkins’s account of the rebuild is unusually candid for a vendor blog: the team’s first approach, a semantic layer plus an in-house orchestrator, created real bottlenecks — slow responses and an inability to ask clarifying questions, which produced weaker answers even though the architecture was sound on paper. The fix wasn’t a better orchestrator. It was recognizing that “context quality is more important than the technologies you use,” which sent the team into a slower, less glamorous project: enriching data descriptions and metadata across millions of signals. Once that context work was done, testing confirmed their custom infrastructure — not the underlying model — was now the bottleneck. They dropped the semantic layer, swapped their orchestrator for an open-source Strands agent loop, and let the model write more unconstrained code. Their key internal metric, conversation pass rate, moved.

The lesson generalizes further than one vendor’s B2B chat tool. Forrester’s William McKeon-White, evaluating conversational AI platforms for employee services the same week, named “context graphs” as the next competitive battleground precisely because governance and prompt-drafting have become table stakes — vendors now differentiate on the enrichment layer connecting data sources across a workflow, which is the exact layer Demandbase says it had to rebuild by hand. McKeon-White also found deployment time-to-value compressing sharply, from 6-9 months in 2024 to under three weeks in 2026 — but flagged an “ATAKA” gap where most platforms still lack awareness of a customer’s existing agents, tools, and knowledge, forcing rework. Demandbase’s story is what closing that gap actually costs: not a new orchestrator, but a sustained investment in the unglamorous metadata work underneath it.

For teams building or buying conversational AI on top of B2B data, the takeaway cuts against the instinct to solve a quality problem by upgrading the AI framework. Demandbase’s own experience says to check the data layer first — and be willing to delete infrastructure, not just add to it, once a better foundation makes the custom orchestration layer redundant.