A measurement here is rarely one agent. A normal job for me is 12 things to look at and 3 things to do with each one: read the source, then pull the numbers out and check them back against it. That comes to 36 agents and the thing that runs them is a JavaScript file with a loop in it. I describe that setup as deterministic whenever somebody asks how it works. This week I sat down to find out which part of the run the word covers.
What I want out of it is resume. A run of 36 costs money and about 20 minutes of waiting. So when I fix a sentence in the third stage I want to pay for the third stage and take the other 24 agents out of the journal of the previous run. That is the whole promise. It holds only if the second run asks for the same things in the same order and I had never checked that it does.
The bench
The orchestrator is 118 lines. There is agent(), parallel() which puts a barrier between stages, pipeline() which does not, a FIFO semaphore for the concurrency cap and a journal that records every call. Nothing in it talks to a model. An agent there waits a given number of milliseconds and returns a string built from its own name.
The milliseconds are real. I recorded 30 calls through the claude CLI 2.1.235 on claude-opus-5, from an empty directory. A CLAUDE.md sitting in the working folder goes into the prompt silently and moves both the answer and the time. Single word answers averaged 6742 ms. The 700 word ones averaged 40164 ms, with a middle weight at 11043. The whole set ran from 5790 to 54520 ms and I replay those durations divided by 20 so that a grid of 36 finishes in seconds. Node is 22.23.1 on 4 cores.
While I was there I put a number on the process boundary, since I had never done that either. The same run with real child processes instead of timers took 5674 ms against 5497, which is 4.9 ms per agent. A repeat gave 5730 and 6.5 ms. Next to an agent that needs 6 to 55 seconds either number is nothing. It does not price my scheduler though. The semaphore and the journal run in both arms and cancel out, so the loop around the agents stays unpriced.
Barrier against pipeline
At a cap of 4 the barrier version takes 8514 ms and the pipeline version 6986. Lift the cap so all 36 can run at once and it becomes 6644 against 5053, a gap of 23.9 percent. Both shapes are within 14 ms of their own floor at that point. A barrier cannot finish sooner than the sum of the slowest agent in each stage, which is 6630 ms on this grid. The pipeline cannot finish sooner than the slowest single chain of 3, which is 5044.
I wanted to be sure that the spread is what pays for this, so I ran the same grid again with every duration set to the mean. The barrier then costs nothing at caps of 4, 16 and 36. At a cap of 8 it still costs 16.7 percent, because 12 items do not divide by 8 and the last wave of each stage runs 4 wide while 4 slots sit idle.
The measurement that lied to me
Then I ran the same grid 30 times at a cap of 4 and hashed the order in which the calls were made. Both shapes came back with 1 distinct order out of 30. Completion order was 2 out of 30 for the barrier and 1 for the pipeline. A repeat of the whole bench gave 1 for the barrier too, so even that 2 was timer noise. Perfectly reproducible. I believed it for about 10 minutes, until I noticed that my fake agents always took exactly the same time. Real ones never do. I sent the same prompt 24 times across 2 rounds and it came back between 7347 and 10586 ms.
So I put that spread back in. Every replayed duration now gets multiplied by one of those 24 measured ratios. With the jitter in place the barrier still gives 1 distinct call order in 30 runs and the pipeline gives 26. Completion order is 30 out of 30 for both.
The reason is where the ordinal comes from. In the barrier shape the 12 calls of a stage are all made in one synchronous pass, so call number 0 is item 0 of stage 0 in every run there will ever be. In the pipeline shape a call is made when the previous stage of that item returns, so the number goes to whoever finished first.
The journal
My journal keys its entries by ordinal and replays the longest prefix that still matches, which is the obvious rule when the script is a loop. With the barrier that rule behaves exactly as advertised. Nothing edited gives 36 hits out of 36, an edit to the last stage gives 24, an edit to the middle stage gives 12 and an edit to the first stage gives 0.
The pipeline with the same key recovers 12 of 36 with nothing edited at all and nothing changed in the inputs. 24 agents are bought a second time. The first 12 always match because those are the 12 stage 1 calls the pipeline makes up front, before anything can return. From call 12 onwards the journal and the resume run disagree. 2 fresh runs of the same script first disagree at call 13. The resume run disagrees 1 call earlier. The reason is sharper than noise. A replayed agent returns at once, so every stage 2 call goes out in item order and number 12 lands on item 0. I tried that on 60 grids and it held on all 60. In the journal number 12 belongs to whoever won the race, which on this grid was item 1.
That 12 is not a law though. Of those 60 grids 47 broke at exactly 12, 12 broke at 13 and 1 at 14, because now and then item 0 does win its own race.
Keying by the task instead of by its position fixes it. Both shapes then recover 36 of 36 when nothing changed and 24 of 36 after an edit to any single stage. That is the honest number for a 3 stage job, since a changed stage is a third of the work. It only holds while the prompt of an agent does not carry the text of an earlier one. Half of my real stages do exactly that, so their key moves whenever an answer above them moves.
The agents themselves do not repeat
That same prompt, 24 times, gave 24 distinct answers, between 375 and 473 characters long. A resumed run therefore puts old answers next to new ones and it is never the run I would have got by starting over. Nothing about a result is reproducible here, so the only things a journal can hold still are the plan and the key.
What I did not check
The orchestrator never calls a model, so all of this is about scheduling and none of it about quality. One machine, 4 cores, no network and no rate limits. No agent fails anywhere in the bench, which is the case where resume matters most and I have no number for it. One grid, one seed, 3 repeats per point in the timing bench against 30 in the ordering one and 10 in the resume one. Those 10 resume runs all read one and the same journal, so they could not have disagreed with each other. The scheduler itself is never priced apart from the process it spawns. My content key is also just the task name, where a real one would be a hash of the prompt and the options.
I have moved my own journal to the content key and left the pipeline where it was. The next thing worth measuring is what a failure in the middle does to all of this.