How a B2B SaaS Platform Discovered Its AI Agent Orchestrator Was Silently Dropping Tool Outputs During Concurrent Session Spikes (And the Backpressure Queue That Fixed It)

It started with a support ticket that seemed almost too mundane to escalate. A sales operations manager at a mid-market manufacturing firm reported that her AI assistant "occasionally forgets what it just looked at." The support team at Velorix, a B2B SaaS platform providing AI-powered revenue operations tooling, logged it as a UX quirk and moved on.

Three weeks later, that single ticket had spawned forty-seven more. The AI agent powering Velorix's flagship workflow product was not just forgetting things occasionally. It was silently discarding the outputs of tool calls, entire CRM lookups, enrichment API responses, and email thread summaries, during peak concurrent usage windows. No errors were thrown. No alerts fired. The orchestrator simply moved on as if the tool had never been called.

This is the story of how Velorix's engineering team diagnosed a deeply subtle concurrency bug in their AI agent workflow orchestrator, and how they restored full data integrity by implementing a backpressure queue architecture without tearing down and rebuilding their entire pipeline.

The Architecture Before the Incident

Velorix's AI agent system was built on a fairly standard multi-tool orchestration pattern. At its core, a central orchestrator loop received a user task, dispatched it to a large language model (LLM) with a set of registered tools, parsed the model's structured tool-call responses, executed those tools in parallel where possible, and fed the results back into the next LLM context window.

The tool layer included integrations with Salesforce, Apollo.io, a proprietary email intelligence service, a PDF parser, and an internal deal-scoring model. Each tool was wrapped in an async Python function and executed using asyncio.gather() for concurrency. Results were collected into a shared dictionary keyed by tool call ID, then serialized back into the message thread before the next LLM turn.

For most of Velorix's early growth phase, this worked fine. Their agent sessions were largely sequential, one user, one task, one orchestration loop at a time. But as enterprise contracts grew and team-level deployments became common, the number of simultaneous agent sessions running on the same infrastructure jumped from an average of 12 concurrent sessions to over 340 during business hours.

That is when things started breaking silently.

The Silent Drop: What Was Actually Happening

The engineering team's first instinct was to blame the LLM itself. Maybe the model was hallucinating tool results. Maybe context windows were being truncated. They spent nearly a week chasing model-level explanations before a senior backend engineer, Priya Nandakumar, suggested they log every tool execution output at the infrastructure level, not just at the application level.

What they found was alarming. Tool calls were completing successfully at the infrastructure level. The Salesforce lookup would return a full account object. The enrichment API would return a populated contact record. But by the time those results were supposed to be written into the shared session state dictionary, they were gone.

The root cause was a race condition in the shared mutable state layer.

Here is what was happening in precise terms:

• Multiple agent sessions shared a single in-process state manager running on each application server instance.
• When concurrent sessions spiked, multiple orchestrator loops were simultaneously writing tool outputs to the state manager within microseconds of each other.
• The state manager used a simple Python dictionary with no write locking. Under high concurrency, writes from one session would occasionally overwrite or displace writes from another session mid-assignment.
• Because the orchestrator checked for tool output presence using a soft if key in results pattern rather than a strict assertion, a missing key was silently treated as "no result needed" rather than "result was dropped."
• The LLM would then receive an incomplete tool response block, infer a plausible-sounding answer from prior context, and continue. No exception. No log line. Just a quietly wrong answer.

The silent nature of the failure was the most dangerous part. Because the LLM would fill in gaps with confident-sounding hallucinations, end users rarely saw a hard error. They saw subtly wrong data, a deal score that didn't match the CRM, a contact summary that referenced stale information, an email insight that ignored the most recent thread. The kind of errors that erode trust slowly, invisibly, and catastrophically.

Why a Full Rebuild Was Off the Table

The instinctive engineering response to a foundational architecture flaw is often "let's rebuild it properly." Velorix's CTO, Marcus Osei, understood the appeal but ruled it out immediately for three reasons.

First, their enterprise contracts included uptime SLAs with penalty clauses. A full pipeline rebuild would require weeks of parallel running, extensive regression testing across dozens of tool integrations, and a high-risk cutover window. Second, the core orchestration logic, the prompt engineering, tool schemas, retry logic, and context management, represented roughly 14 months of accumulated iteration. Throwing that away was not a neutral act. Third, and most pragmatically, the bug was in the state management and concurrency layer, not in the orchestration logic itself. The orchestrator was not broken. Its environment was.

The goal became targeted: fix the concurrency and data integrity problem without touching the orchestration logic that was already working.

The Solution: A Backpressure Queue Architecture

Priya and the infrastructure team designed a solution around three interlocking components: a per-session isolated state store, a tool output queue with backpressure controls, and a write-ahead log for recovery.

Component 1: Per-Session Isolated State Stores

The shared in-process state dictionary was eliminated entirely. Each agent session was assigned its own isolated state store, backed by a Redis hash keyed to the session ID. Reads and writes were performed through a thin async client with optimistic locking using Redis's WATCH and MULTI/EXEC transaction primitives.

This alone eliminated the cross-session write collision problem. Sessions could no longer stomp on each other's tool outputs because they no longer shared any mutable state.

Component 2: The Tool Output Queue with Backpressure

Rather than writing tool outputs directly to the session state on completion, each tool's async wrapper was refactored to push its output onto a per-session bounded queue. A dedicated consumer coroutine, one per active session, drained that queue and committed results to the Redis state store in strict order.

The backpressure mechanism was the key innovation. The bounded queue had a configurable maximum depth (initially set to 16 items). If a tool attempted to push an output onto a full queue, it did not silently discard the result or block indefinitely. Instead, it triggered a backpressure signal that paused the orchestrator's dispatch of new tool calls for that session until the queue depth fell below a low-water threshold (set to 8 items).

This created a natural flow control loop. Under normal load, the queue drained faster than it filled. Under spike conditions, the orchestrator slowed its own tool dispatch rate proportionally to the consumer's capacity, rather than allowing unbounded concurrent writes to race against each other. The system degraded gracefully under load instead of silently corrupting.

Component 3: Write-Ahead Log for Recovery

To handle edge cases where a session worker crashed mid-drain (due to a pod restart, a network timeout, or an OOM event), the team added a lightweight write-ahead log (WAL) using a Redis sorted set. Every tool output was written to the WAL before being committed to session state. On session worker startup, the worker checked for any uncommitted WAL entries for its session ID and replayed them before accepting new work.

This gave the system crash recovery semantics without requiring a full transactional database. The WAL entries were small (typically under 4KB per tool output) and were pruned automatically after successful commit acknowledgment.

The Rollout: Incremental and Instrumented

Rather than flipping a switch, Velorix deployed the new architecture using a session-level feature flag. New sessions created after the flag was enabled would use the queue-backed architecture. Existing in-flight sessions continued on the old path until they completed naturally.

They instrumented five key metrics to validate the fix in production:

• Tool output commit rate: The percentage of tool executions whose outputs were successfully committed to session state. This was the primary integrity metric.
• Queue depth p95 and p99: To monitor whether backpressure was activating and whether it was effective.
• Orchestrator dispatch pause duration: How long sessions were being held back by backpressure signals, to detect if the fix was creating unacceptable latency.
• WAL replay rate: How frequently the crash recovery path was being triggered, as a proxy for infrastructure stability.
• LLM context completeness score: A custom metric that counted the ratio of expected tool result blocks to actual tool result blocks in each LLM prompt, to catch any remaining silent drops.

The rollout ran across three deployment rings over eight days. By the end of ring two, covering roughly 60% of production traffic, the tool output commit rate had risen from a measured 91.3% (under spike conditions) to 99.97%. The LLM context completeness score reached 99.99% by the end of ring three.

The Results: Numbers That Mattered

Two months after full deployment, Velorix's engineering and customer success teams compiled the impact:

• Support tickets related to "incorrect or missing AI outputs" dropped by 94%.
• The average orchestrator dispatch pause caused by backpressure was 23 milliseconds, well below the threshold of user-perceptible latency.
• Three enterprise customers who had been quietly evaluating alternatives renewed their contracts, with one explicitly citing "improved AI reliability" in their renewal notes.
• The engineering team's on-call burden related to AI pipeline incidents dropped from an average of 6 pages per week to fewer than 1 per month.
• The entire fix, from diagnosis to full production rollout, took 31 days and required no changes to the orchestration logic, prompt templates, or tool schemas.

Lessons Every AI Platform Engineer Should Take From This

The Velorix incident is not unique. As AI agent systems move from experimental deployments to production-grade enterprise infrastructure in 2026, the same class of concurrency and state management bugs is appearing across the industry. A few hard-won lessons from this case study are worth internalizing:

Silent failures are the most dangerous failures in AI systems

A hard crash is easy to detect and fix. An LLM that confidently fills in dropped data with a hallucinated answer is a slow-moving disaster. If your orchestrator does not assert the presence of every expected tool output before constructing the next LLM prompt, you are flying blind.

Shared mutable state and async concurrency do not mix without explicit coordination

This is not a novel insight in distributed systems engineering, but it keeps biting AI platform teams because orchestrators are often prototyped by ML engineers rather than backend engineers. The mental model of "I'll just use a dictionary" works perfectly at one session. It fails in ways that are hard to reproduce at 300 sessions.

Backpressure is a feature, not a performance compromise

Many engineers resist backpressure mechanisms because they feel like artificial throttling. In reality, backpressure is the mechanism that keeps a system honest about its actual capacity. A 23-millisecond dispatch pause is infinitely preferable to a silently wrong answer delivered instantly.

You do not always need to rebuild to fix a foundational problem

Surgical interventions at the right layer can preserve years of accumulated logic while eliminating the specific failure mode. The key is correctly identifying which layer is actually broken. Velorix's team spent a week blaming the wrong layer (the LLM) before finding the right one (the state manager). Invest in deep observability before you invest in a rewrite.

Conclusion

The Velorix case study is a reminder that as AI agent systems scale into real enterprise workloads, the failure modes that emerge are not always glamorous or obvious. They are often the quiet, classical problems of concurrent systems engineering: race conditions, shared mutable state, and the absence of flow control. The novelty of LLMs sitting at the center of these pipelines does not change the physics of concurrency.

What makes this story worth telling is not the sophistication of the fix. The backpressure queue, the per-session state isolation, and the write-ahead log are all well-understood patterns. What makes it worth telling is the discipline it took to find the real problem, resist the impulse to rebuild everything, and apply a targeted solution that preserved the team's existing investment while restoring the data integrity their customers depended on.

If you are running AI agent workflows at scale in 2026 and you have not yet stress-tested your orchestrator's behavior under concurrent session spikes, this case study should be your motivation to do it before your customers do it for you.