betaFeedback

WorksComputer Science

Agent Architecture Cost-Success Trade-offs: A Controlled Evaluation across Enterprise-Style SOPs

Abstract

Enterprises increasingly use large language model agents to execute standard operating procedures (SOPs): long, branching, tool-intensive workflows. Yet there is little controlled evidence for choosing among competing agentic architectures. Holding the model, tasks, and tools fixed and varying only the architecture, we compare four designs (a function-calling single agent, a skill-augmented agent, centralized multi-agent orchestration, and a triage swarm; eight arms) across 13 enterprise SOPs, and introduce a skill-definition quality rubric to build matched high and low-quality skill corpora. Three takeaways emerge. (1)~There is no universal best architecture; the right choice is SOP-dependent. On a given SOP, task success ranges from under 10%10\% on the hardest to near 100%100\% on the easiest; no architecture is statistically separable in aggregate, and added structure only plausibly pays off on complex, branching SOPs (a directional, not yet significant, trend). (2)~Added architecture cost must be justified by a task-success gain. Every coordinated arm (multi-agent or skill-augmented) spends 1.41.4–4.7×4.7\times the tokens and 2.62.6–5.8×5.8\times the latency of the single agent for gains that are usually small and rarely significant, making the single agent the efficient default; high-quality skills do lift success (to 73.1%73.1\% from 67.4%67.4\%) while degraded skills fall below it. (3)~Implementation quality outweighs the single- versus multi-agent choice. A plausible multi-agent implementation loses to the single agent, yet a targeted redesign of it becomes the only arm to significantly beat the baseline (75.6%75.6\% vs.\ 67.4%67.4\%).

Paper details

vlnc:W_rz7sgkej
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC-BY-NC-ND-4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…