Adding two specialists bought no extra accepted answers in this pilot. Solo, forced-team and optional-delegation setups each passed four of four attempts. The team used 2.57× the median reported tokens and took 30% longer at the median than solo.
A November 1 reading of three recent papers adds a useful qualification: delegation can help when it removes a real context or parallel-work bottleneck. The workflow must repay dispatch, repeated setup and the checked merge.
The practical lesson: for a compact task with prepared evidence, start with one capable agent and a clear checklist. Add delegation when you can name the independent work or context problem it will solve. This is a small, dated observation of a CLI orchestration adapter—not a verdict on native subagents or coding agents generally.
What we asked them to do
We tested two decisions where a polished answer can still be wrong. Each setup tried each task twice, on October 30, 2026.
Choose a subscription setup. A fictional US freelancer had a $45 monthly cash cap, including a $5 tax reserve, needed a named model in Codex CLI and wanted a second provider for review. The answer had to respect monthly cash, reject an ineligible student offer and avoid promising capacity from an access listing. An annual plan’s lower effective monthly price did not solve the cash constraint. Dated pricing input; Claude Pro terms. Publication dates were unknown; inputs were retrieved October 30.
Explain a wallet fee discrepancy. Trust’s June 26 support page said no extra platform fee on its Hyperliquid route (retrieved October 29 UTC for the October 30 packet). Under an assumption of compatible export units, preserved partial-day October 22–28 rows yielded a weighted fee/notional ratio of approximately 10 basis points with a third-party builder attribution. Repeated rows were retained; unique executions were unverified. A good answer kept the historical aggregate separate from today’s app quote and net user cost. It also caught two traps: builder parameter f=10 means 1 bp, and the fill’s fee already includes its builder fee. No wallet was connected or trade made.
These were synthesis and arithmetic over curated sources. They offered little context stress or open-ended discovery.
What happened
Small sample: 2 curated tasks × 2 repeats per lane. All lanes accepted 4/4.
Median end-to-end time
Scale: 0–100 seconds
Solo: 64.9 s
Team · 2 specialists: 84.6 s
Hybrid · 0 specialists used: 59.3 s
Median reported tokens
Scale: 0–200,000 input + output tokens
Solo: 62.24k
Team · 2 specialists: 160.24k
Hybrid · 0 specialists used: 64.34k
Bars start at zero. Four observations per median. Tokens are telemetry, not dollars. Exact results and table alternative.
Two fresh, same-model graders scored shuffled final answers without seeing their setup, logs or metrics. Accepted meant at least 85/100, no listed critical error, at most 600 final-answer words and no recorded operational violation. Both graders accepted all twelve and assigned identical scores. This is automated acceptance, without independent human validation.
Team and hybrid mean rubric scores were 97.5/100; solo’s was 96.25. Deductions concerned missing source-date labels in subscription answers. No critical arithmetic, budget or eligibility error was identified. A pass did not mean a perfect answer.
The hybrid chose zero specialists in all four attempts. Its shorter median does not show that optional delegation makes a model faster.
The team’s specialist sessions really overlapped—about 29–35 seconds each run. Yet the complete team workflow was slower than its matched solo run in all four comparisons, by 6.7–42.8 seconds. The total workflow included startup, dispatch, repeated context and merging. Those are plausible overheads, but this timing comparison does not measure each contribution; native harnesses may behave differently.
Before you add a specialist
Use this checklist on your next real task:
- Separate deliverable: Can the child finish a bounded piece without repeatedly asking the parent? Independent service reviews are a better candidate than two people rechecking the same short source packet.
- Context benefit: Will a fresh, discardable context remove a concrete problem—long logs, exploratory branches or conflicting tool instructions? This pilot did not test those benefits.
- Checked return: Specify the evidence, result format and acceptance rule before dispatch. Make someone responsible for checking the merge.
- Accepted-result value: Compare the whole workflow, including failures, retries and cleanup. Keep elapsed time, reported tokens and actual spend in separate columns.
- Continuation plan: Give each worker a checkpoint and a supported resume path. Its saved history, cache lifetime and allowance reset are separate clocks; record each instead of treating every pause as a fresh start.
If you cannot identify either of those first two benefits, try the strong single agent with progressive skills first. Test delegation on your harder cases before turning that into a policy.
Methods, limits and reproducible evidence
The following detail is here so you can judge whether this resembles your workload.
How the three setups were run
All participants requested and reported gpt-6.1-sol, high reasoning, through subscription-authenticated Codex CLI 0.165.0. Source facts, progressive skills, source/decimal tools and final-answer limits matched. Solo could batch tool calls.
The forced team dispatched an evidence checker and an arithmetic checker concurrently in separately logged CLI sessions, then resumed its original coordinator. Hybrid could request either, both or neither. This tests a separate-session adapter, not native subagent features.
The block-randomized, near-balanced schedule, source packets, rubric and amendment were frozen before scored execution, following independent methodological review. Each run started fresh; graders saw anonymized finals only. All twelve first attempts remain preserved, with no task/agent repairs or reruns. Provider-internal transport retries were unobserved.
All lanes shared observed caps of 24 task-tool calls, 600 final-answer words and 1,100 artifact words, plus a 480-second deadline target. Each specialist was allocated five calls and 250 words. These are work proxies, not equal compute. Isolation and tool-use restrictions were procedural and audited, not hard filesystem barriers. Backend revision, serving tier, temperature and seed were not controlled or established.
Exact results and table alternative
Each row has four observations: two curated task clusters, two repeats. These are cases and variability, not a population success-rate estimate.
| Setup | Accepted | Median seconds | Time range, seconds | Median reported tokens |
|---|---|---|---|---|
| Solo | 4/4 | 64.886516 | 57.543085–67.698283 | 62,237.5 |
| Team, two specialists | 4/4 | 84.5527585 | 72.547880–110.478102 | 160,237.5 |
| Hybrid, zero used | 4/4 | 59.2957865 | 56.807325–71.081796 | 64,340 |
The token ratio is 160,237.5 ÷ 62,237.5 = 2.574613×. The time increase is (84.5527585 ÷ 64.886516 − 1) × 100 = 30.308674%. These are ratios of lane medians, not medians of paired ratios.
Reported tokens sum captured input-plus-output response-event counters across participant processes. Cached input is already within input; reasoning is already within output. Small startup events are included. Resumed coordinator counters are cumulative, so they were reconciled rather than added twice.
No paid API-key requests or purchases were made; incremental paid API spend was $0. Subscription allowance consumption, full economic cost and dollar cost per accepted result are unknown. Human diagnosis and cleanup time were not measured. The token ratio is not a subscription-cost ratio.
Preflight failures, amendments and evidence download
The original controlled protocol remained unexecuted because its hard controls were not demonstrated. The first CLI preflight encountered a runtime write restriction; a bounded supported invocation succeeded. A native-delegation rehearsal did not expose child actions adequately, and its collector needed correction. A separately captured adapter fixture and independent review qualified observational V2 before the twelve scored attempts. These limitations and the superseded protocol remain in the record.
Start with the V2 amendment, frozen rubric and per-case results. V1 remains unexecuted.
Download the sanitized evidence ZIP (474,918 bytes). It contains the protocols, task/source packets, prompts, rubric, schedule, first attempts, observable action receipts, reported counters, blind scores, per-case results and review records. It excludes credentials, account metadata, hidden reasoning and raw sensitive diagnostics. Exported host paths/session handles are redacted; the manifest maps original and sanitized hashes. Historical runner paths need local configuration before reuse.
SHA-256: ea455788848cd97a78b4d9163bbeee804316c50b66d72c1606bd5d88ada1c2a4.
This public derivative retains the fixed source-fact packets and capture provenance but omits complete third-party page captures. The original private ZIP and its Library identity remain unchanged. Historical “no publication” notes describe their original recording time.
How this fits the wider debate
Anthropic’s October 22, 2026 announcement donated Skills to AAIF; it was not a comparative result. The progressive-skills article dates to October 16, 2025.
November 1 research update: benefits depend on the workload
The October 16, 2026 MS-Eval preprint studied 166 MCP-intensive tasks in modified Kimi-Code with built-in skills disabled. Its four-design comparison used DeepSeek-V4-Flash-0731; the specialized design was then evaluated across eight models. Specialization led the practical designs, but delegation raised estimated cost for most models. The two closed-source models lost performance and efficiency; several open-source models improved performance, with mixed efficiency.
Its case analysis points to useful context offloading and independent parallel work as benefits, and unnecessary delegation or unchecked returns as failure modes. Its efficiency measure is critical-path steps, not elapsed seconds. Appendix B.1 explicitly excludes inputCacheCreation billing. It does not test approval pauses, turn-limit recovery or restarts, so it cannot settle their cache economics.
The October 20 user-serving multi-agent study found the team worse in 12 of 13 model/domain cells passing mechanical validity checks, tied in the other. Its planner never saw tool schemas, and its preregistered execution-fidelity criterion failed in every cell. That makes the result a warning about the tested architecture, not proof that teams always lose.
SquidAgent supplies a counterexample: on nine author-designed tasks using Sonnet 4.6, its authors report 2.6× mean wall-time speedup over their Claude Code baseline. It targets repeated discovery and post-hoc reconciliation with inherited worker context and shared conventions. One model, a small task suite and model-judged quality limit how far that result travels.
Our inference: start with a strong single-agent baseline, then test a named benefit. A literature sweep across independent sources or separate modules with agreed interfaces may repay coordination. Splitting a short prepared source packet may not. Verify every return and include merging and failed attempts in the comparison.
Memdoor: the bug was in the manager’s objection
The AI manager found a bug. The bug was in its own objection. A separate coding experiment supplies a concrete reason to check a reviewer’s work before accepting its correction.
Memdoor’s archived Qwen3.8-27B run tackled a string puzzle: each operation replaces every occurrence of one letter with another. The first worker solution passed all four public samples, as the message shown to the manager records. The manager demanded a rewrite: it said transforming abcd into bcaa should take five operations.
Four suffice:
- Replace
c→d:abcd → abdd. - Replace
b→c:abdd → acdd. - Replace
a→b:acdd → bcdd. - Replace
d→a:bcdd → bcaa.
This is our demonstration, not transcript dialogue. Each of the four original letters must change, so at least four operations are necessary. The sequence reaches that lower bound.
The worker nevertheless changed its formula, and the manager signed off. The final program passed the public samples but received a recorded full-suite grade of 32/44. The grader includes public and hidden tests, stops at the first failure and marks the remaining cases unsuccessful. The authors’ notes say the first program was correct; we have checked this counterexample, not independently regraded that entire program.
The broader result matters too. The scored run table records 49/50 successes across ten problems repeated five times, using 345 calls. Those are fifty scored attempts, not best-of-five selection. The archive contains a manager/worker loop with repeated calls. It is a different setting from MS-Eval’s MCP tasks and our twelve-run pilot. Different serving setups, possible benchmark contamination and the absence of an equal-budget best-of-N control prevent a general model ranking.
Builder takeaway: require a checkable witness for a claimed counterexample. For a small case, use an independent oracle or an explicit construction before changing the algorithm. Preserve the previous version and rerun regressions; passing samples alone is not proof of correctness. A second opinion is useful when its evidence survives inspection.
Cache economics are a separate experiment
This pilot measured neither cache expiry nor API bills. Its 2.57× token ratio cannot establish a dollar ratio. A team’s effective cost also depends on which prefixes are reused, which worker resumes, the gap between requests and the provider’s write/read prices.
For example, Claude Code’s included-subscription main conversation defaults to one hour, while workers normally get five minutes. A worker can retain its history across an eight-minute approval wait and still need a new cache write. Our parent/worker timeline and 100K-token API comparison show why saved progress and discounted input are different. The cost illustration is not this pilot’s observed billing.
The next useful test would isolate independent deliverables or growing context, retain a checked merge, and log cache reads, writes and pause/resume events alongside acceptance and time. Neither benefit nor those cache effects were established by the twelve attempts here.
Related reading
- Agents running agents — the earlier runner investigation; connectivity and accepted-result evidence are separate.
- Prompt caching, subagent lifetimes and approval costs — compare warm reads, new writes and saved-state resume.
- LLM cost-saving playbook — the reusable spending method, including retries and review.
- How to read AI benchmarks — match an evaluation to your work.
AIHackers-owned pilot, tested October 30, 2026. Original context sources checked October 30; scoped research, Memdoor logs and cache links checked November 1. Two task clusters, two repeats and same-model judging do not establish universal superiority.