Benchmarks

This page gives what the gateway costs, measured on the current build: how many tool calls one gateway serves and at what latency, what a fleet adds, what sessions, notifications, watches, limits and MCP Events cost, what the coordinator store and the plugin boundary cost, and how memory and threads follow the load. The commands that reproduce every table are at the end of the page.

At a glance

¹ The load generator's limit: each gateway used 1.3 of its 2 cores. ² On the isolated host, in the same layout (see How the figures were measured).

• One gateway on two cores serves about 4 800 tool calls a second with the default settings and about 7 700 with three settings that are off by default. Its own work is 250–430 µs of CPU a call; the rest of the latency is the queue of 64 connections.
• A fleet adds stateless capacity gateway by gateway. A stateless (2026-07-28) call needs nothing from the coordinator. A legacy (2025-11-25) call with the default settings writes its session to the coordinator: three NATS nodes on 2 cores carry about 2 750 of them a second, one Redis server about 7 800. unary_json_fast_path takes that write away.
• MCP Events reach every stream once. The inbound route sets the rate a deployment takes: about 1 600 events a second on one gateway alone, all 1 000 offered on four gateways over Redis, and about 400 with every delivery within 3 s on NATS R3 with 2 cores, where the coordinator is the limit: past it the gateways refuse what the inbox cannot hold and the streams fall behind.
• Limits and watches hold on a fleet: every fleet-wide concurrency cap, rate limit and budget held, on NATS and on Redis (see Limits). No watched change was lost when a replica was killed.
• Memory follows the load. A burst's threads exit about a second after it; one gateway on NATS R3 is back at 136–186 MiB within seconds (see Memory and threads).

How the figures were measured

Everything runs on loopback on the one host, so the gateways, the coordinator and the load generator share its 16 cores (on the isolated host, its 32 vCPU). Two measures carry over to other hosts:

• CPU per call — the gateway's user and system CPU time over a run, divided by the calls it served. One core serves about 1 000 000 / (CPU µs per call) calls a second.
• Coordinator messages per call — what the NATS nodes received per call, replication included. It sets the coordinator's share of the work.

The VM shares its physical host with other guests. When the hypervisor gives part of the CPU time to them (steal), every figure falls: one gateway alone with the tuned settings served 7 718 stateless calls a second at 0 % steal and 2 817 at 25 %. Every tool-call, limit and MCP Events result records the steal of its run; those on this page ran at 0–5 %, except where a note says otherwise.

Most figures of Inside one process come from the nightly benchmark workflow on the CI runners, a different host; that section names which.

The path of a tool call

flowchart LR
  C["MCP client"] -->|"POST /mcp"| T["HTTP transport<br/>validation, access log"]
  T --> I["Identity<br/>API key, OIDC, mTLS"]
  I --> G["Policy, guards,<br/>limits"]
  G --> D["Dispatcher"]
  D --> P["Backend plugin<br/>cdylib or built-in"]
  P --> U[("Upstream")]
  D -. "awaited: allowed and<br/>completed events" .-> A["Audit sink"]
  G -. "fleet counters:<br/>concurrency, rate, budget" .-> K[("Coordinator<br/>NATS or Redis")]
  T -. "legacy wire only:<br/>session row, replay" .-> K

A stateless (2026-07-28) call sends the coordinator nothing unless a fleet-wide limit applies to the tool. A legacy (2025-11-25) call reads its session and, when it answers with an SSE stream, writes the session row with the stream's replay: that is the coordinator traffic of the legacy rows below. Each call waits for its two audit events (mcpg.tool.call.allowed before the backend, mcpg.tool.call.completed after it) to be durable. The local file sink commits every batch of queued events with one write and one fsync, so a call waits for about two fsyncs, whatever the number of calls in flight.

Tool calls on one gateway

SCENARIOS=scale: tools/call of a mock tool from 64 connections, default settings. Memory is the gateway's resident set right after a run, which is its working set at full speed (see Memory and threads).

The same load with the three settings of Settings that change throughput on:

• A stateless call costs the same alone and in a fleet: it reads and writes nothing in the coordinator (the 0.02–0.04 messages per call are the fleet's own leases and heartbeats, spread over the calls).
• A legacy call with the default settings is dearer in a fleet: it answers with a one-frame SSE stream and writes the session row with that stream's replay, about 8 NATS messages per call with replication. Alone, the same write goes to the gateway's memory.
• With the tuned settings both wires cost the same, alone or in a fleet: the gateway uses its 2 cores, so CPU sets the rate.
• Latency is queueing. 64 connections each wait for their answer before the next call, so the latency is about 64 / (calls/s): a gateway's own work per call is the CPU column.

Settings that change throughput

Three settings are off by default because each one gives something up. The "tuned" figures have all three on.

What the three do together, one gateway on NATS R3:

unary_json_fast_path alone accounts for the legacy wire's coordinator traffic: it is what lets legacy calls scale with the fleet (see below). inline_dispatch takes the last thread hand-off out of a call: a gateway then needs about 10 threads instead of about 100 at the same concurrency.

A fleet on NATS

SCENARIOS=scale on one, two and four gateways over the three NATS nodes, three runs of each load, medians. Each gateway has 2 cores of its own; the coordinator has 2 for its three nodes.

---
config:
  themeVariables:
    xyChart:
      plotColorPalette: "#3987e5, #d95926, #199e70"
---
xychart-beta
  title "Calls per second by fleet size"
  x-axis ["1 gateway", "2 gateways", "4 gateways"]
  y-axis "calls/s" 0 --> 16000
  line [4695, 7798, 9773]
  line [2153, 2756, 2745]
  line [7537, 13772, 14872]

² Each call on the next replica, so every call reads its session from the coordinator; the runs with each connection's calls on one replica had 13 % steal (2 116 calls/s). The other legacy rows keep each connection's calls on one replica; with each call on the next replica, two and four tuned gateways served 11 949 and 14 114 calls a second.

The lines, from the top at four gateways: legacy tuned (green), stateless default (blue), legacy default (orange).

• Stateless calls and tuned legacy calls scale with the gateways until the load generator (4 cores) sets the limit: at four gateways each gateway uses 1.3 of its 2 cores. The coordinator does almost nothing for them.
• Legacy calls with the default settings do not scale past two gateways: the coordinator is the limit. At four gateways the three NATS nodes use 1.67 of their 2 cores while each gateway uses 0.65. Each such call writes its session's replay, 6–8 NATS messages with replication; how many depends on whether the gateway's own NATS node leads the stream.
• The tuned settings make the legacy wire as cheap as the stateless one, because a unary call answers with one JSON body and writes nothing to the coordinator.
• CPU per call rises a little with the fleet (256 → 354 µs tuned) at the same load: each gateway serves fewer calls for the same fixed work. For sizing, use the four-gateway figures.

A fleet on Redis

COORDINATOR=redis: the same loads over one redis-server without persistence on the coordinator's 2 cores, one and four gateways, three runs of each load, medians. NATS R3 keeps three copies of every write; this Redis keeps one.

• On Redis the legacy wire scales with the gateways with the default settings too: the session writes that hold three NATS nodes to about 2 750 calls a second take one Redis server 0.49 of a core at 7 800. The gateways' CPU is the limit there (800 µs a legacy call).
• A stateless call costs the same on either coordinator: the coordinator sees none of it.
• With the tuned settings the two coordinators give the same fleet: about 14 800 calls a second on four gateways, the load generator's limit, with the coordinator idle.

Sessions and notifications

These belong to the legacy (2025-11-25) wire, whose sessions live in the coordinator on a fleet.

SCENARIOS=churn: 16 connections each open a session on one replica, open its stream on the next replica (with one gateway, the same one), and end the session on the first; the stream must end.

SCENARIOS=listen: 64 sessions on every replica subscribe to 20 resources, and a webhook changes them 20 times a second through the replicas in turn; every subscriber must hear every change once.

• A session costs coordinator round trips on a fleet: its creation, the read on the replica that serves its stream, and its end, which every replica hears. A gateway alone keeps all of it in memory.
• A notification is kept for a stream that connects later: it is written to the session's buffer in the coordinator before it is sent, which is most of its latency on a fleet.

Watches and failover

SCENARIOS="watch failover": 32 sessions subscribe to 50 resources that the gateway polls every 10 s from an upstream whose versions move every 25 s.

• A fleet polls each resource once: the replicas split the resources into shards, each held by one replica under a lease, so four gateways poll no more than one.
• The delay is the poll interval: a change waits on average half of it.
• After a kill, the survivors take the dead replica's shards when its leases expire: on NATS the last shard moved 15.7 s after the kill, on Redis 19.9 s, and no change was lost; the longest wait (33.4 s and 39.4 s) is that hand-over plus a poll interval.

Limits

A fleet-wide limit (scope: global) is decided in the coordinator: a concurrency cap of N is N slot keys that the replicas take and give back, a rate limit and a budget are counters every replica charges. Two things keep that cheap:

flowchart LR
  A["A call under a fleet cap of N"] --> L{"A local slot free?<br/>N slots per replica"}
  L -- "no" --> R1["Refused here,<br/>no store operation"]
  L -- "yes" --> I["Take a free slot key<br/>in the coordinator"]
  I --> S{"A slot key free<br/>of the N?"}
  S -- "yes" --> OK["Admitted; the key and<br/>the local slot go back at the end"]
  S -- "no" --> R2["Refused; for 20 ms the replica<br/>refuses at once, without a store operation"]

• A replica never holds more than the cap at once, so it asks the coordinator only for the calls that hold one of its N local slots; the others are refused at once, without a store operation.
• A refused rate limit is remembered on the replica until a token fits, and a refused budget for up to one second (never past its window), so a hammered limit costs the coordinator nothing until its answer can change.

SCENARIOS=governed: 64 connections call one capped tool as fast as they can for 30 s; a refused call is sent again at once.

• Every limit held on every fleet. A rate bucket is one counter that every replica moves: the time up to which its calls are paid for. A call raises it to the present when it lies in the past and adds one interval, in one atomic step of the coordinator, so two replicas never pay for the same interval.
• Most refusals answer from the replica's memory: the rate limit judged 12 500 calls a second on one gateway on NATS with the coordinator at 0.08 cores.
• The concurrency cap admits under its ceiling on a fleet. An admitted call holds a slot key in the coordinator for its 100 ms plus the round trips that take and free the key, and a replica that finds every key taken refuses for 20 ms: 67–70 calls a second of the 80 the cap allows, against 75 on a gateway alone.

MCP Events

sequenceDiagram
  autonumber
  participant P as Producer
  participant R as Inbound route (any replica)
  participant K as Coordinator KV
  participant O as Events owner
  participant S as Push streams (each replica)
  P->>R: POST /webhooks/events/{source}
  R->>K: reserve an inbox slot, write the row
  R-->>P: 202 Accepted
  R--)O: hint: the inbox has rows
  O->>K: read up to 256 inbox rows, 16 at once
  O->>K: dedup markers, rows, the head after every 32 rows
  O--)S: hint: the head moved
  S->>K: the bounds and each new row, once per replica
  S-->>S: every stream of the replica reads them from memory

The owner (one replica, under a lease) moves the inbox rows into the log in batches, and every replica reads each new row from the store once for all of its streams. The posts a replica holds at once share their slots: one read of the inbox's bounds and one counter step reserve a slot for each of them.

SCENARIOS=events on the isolated host, small shape: 64 push streams on events/stream (one principal), one event type fed by an inbound webhook source, one producer per 100 events a second offered, each posting its next event when the last one is answered. A post is refused when the answer is not 202. A delivery counts when its stream heard the event within 3 s after the last post. The gateways also load the Prometheus plugin (FLEET_METRICS=1), whose /metrics gives the coordinator calls behind each event.

³ nats-server 2.14.8. Two runs, each rate after the one before it on the same fleet; a range where they differ, and a share of the deliveries where not all arrived within 3 s. Steal 0–1 %.

---
config:
  themeVariables:
    xyChart:
      plotColorPalette: "#3987e5, #d95926, #199e70"
---
xychart-beta
  title "Deliveries a second to 64 streams"
  x-axis ["100 offered", "400 offered", "1000 offered", "1600 offered", "3200 offered"]
  y-axis "deliveries/s" 0 --> 110000
  bar [6408, 25608, 64021, 102210, 107534]
  line [6408, 25608, 49043, 35858, 25088]
  line [6408, 25446, 47017, 32101, 21196]

The bars (blue) are one gateway alone; the lines are one gateway (orange) and four gateways (green) on NATS R3, one run each (deliveries within 3 s a second, over the 30 s run).

• Every accepted event reached every stream once in every run: no duplicate, and no event a stream did not expect.
• The inbound route sets the rate. An accepted event is a row write before its 202, and the posts a replica holds at once share one counter step for their slots. One gateway alone took about 1 600 events a second into the file-backed inbox, at 1.8 of its 2 cores. Four gateways over one Redis server took all 1 000 offered with Redis at 0.39 of a core. On NATS R3 with 2 cores the coordinator is the limit: one gateway took all 400 a second offered and four gateways 398, every delivery within 3 s, with the three nodes at 1.3–1.7 of their cores.
• Past that rate, the inbox fills and the gateways refuse. The owner moves the events into the log slower than they arrive; once the inbox holds inbox_max_rows (10 000 by default), a post gets 503 with a Retry-After at once, and the streams fall behind. At 1 000 offered four gateways took 829–865 a second and refused none (each post was answered later), and 89–91 % of the deliveries arrived within 3 s; at 1 600, 752–849 and 67–72 %. A producer that gets 503 or 429 should retry after the Retry-After it is given. Size the coordinator for the event rate, or use Redis; more coordinator cores raise the rate less than their count (see A larger machine).
• Fan-out is cheap. A delivery costs 17–37 µs of gateway CPU on one gateway alone, 36–98 µs on one gateway on NATS, 64–207 µs over four gateways on NATS and 71–117 µs over four on Redis: the streams of a replica share each read of the log, so 64 streams cost about what a few do.
• The coordinator's share is 52–108 NATS messages per event on one gateway and 60–152 on four, replication included: the inbox write, the owner's drain (dedup marker, row, head every 32 rows, inbox delete), and each replica's read of the new rows. It is lowest at 1 000–1 600 a second, where more posts share each counter step, and rises again past that with the refused posts.

Memory and threads

One gateway alone, a 30 s burst of stateless calls from 64 connections, sampled every 2 s, then 60 s without load:

SCENARIOS=memtime: three bursts of 30 s on one gateway, with 120, 60 and 120 s without load after them:

---
config:
  themeVariables:
    xyChart:
      plotColorPalette: "#3987e5, #d95926, #199e70"
---
xychart-beta
  title "Resident memory through a 30 s burst (MiB)"
  x-axis "seconds from the burst's start" [0, 4, 10, 16, 22, 28, 32, 34, 40, 50, 60, 70, 80, 88]
  y-axis "MiB" 0 --> 600
  line [63, 498, 571, 543, 543, 552, 543, 543, 543, 543, 543, 543, 543, 543]
  line [61, 552, 531, 529, 527, 534, 495, 137, 137, 137, 137, 137, 137, 136]
  line [61, 104, 104, 104, 104, 103, 84, 84, 84, 84, 82, 82, 82, 82]

Blue and orange are two runs with the default settings, green is inline_dispatch on the backend.

• Memory follows the work in flight, not the history. With the default settings, every plugin call in flight holds a blocking-pool thread (the spawn_blocking hand-off that bounds a plugin's time) and its allocator cache: about 100 threads and 500–580 MiB at 64 connections.
• It comes back. The threads exit about a second after the burst. Their memory goes back to the system once other allocations of the gateway reclaim the pages they left: on a gateway alone that idles, 4 s to more than a minute after the burst (the two default runs above; 27–31 s in the memtime runs), on one gateway on NATS within seconds.
• Inline plugins keep a gateway small: a call runs on the worker that serves its request, so the same burst needs 8 threads and 105 MiB, at 41–63 % more calls.
• On a fleet, a gateway gives a burst back within seconds. The coordinator's store calls run on the same blocking pool and keep a few of its threads busy between bursts: one gateway on NATS went back to 7–9 threads and 136–186 MiB after every burst, in four runs of three bursts each. With inline_dispatch a burst needs no blocking threads for its plugin calls, and the same gateway stays at 80–94 MiB.
• Sessions that start together need threads too. A legacy session's store writes run on the blocking pool: with the tuned settings, 512 sessions that started at once took one gateway on NATS from 12–13 to 241–244 threads and from about 170 MiB to 850–865 MiB; alone, the same 512 sessions cost 4 MiB. Each such thread holds about 3 MiB.
• Plan a gateway's memory for its peak at the concurrency it serves.

The coordinator store

tools/bench/cluster/kv.sh: each coordinator plugin's own store, without the gateway and without the plugin boundary, on fresh servers for each backend and keyspace. A seed of 100 000 keys with 256-byte values (1 000 of them under one prefix), then each operation alone for 5 s from 1 and from 64 tasks. NATS R1 is one node, NATS R3 three nodes on one host, Redis one server without persistence.

• The keyspace hardly matters. From 10 000 to 1 000 000 keys every operation stays within 25 % of its rate; the p99 of a full pass over a prefix grows 1.8× on NATS and 2.2× on Redis at one task, and not at all with 16 passes at once.
• On NATS an incr and a delete cost two round trips: a delete reads the key, then writes its delete marker; an incr reads the counter and writes it back on the condition that nothing else wrote it first, and retries when something did. Many writers of one counter wait for one another: under 64 tasks its p99 is 23 ms on R1 and 31 ms on R3.
• Replication costs writes, not reads. On three nodes a read takes twice as long as on one node at one task, and a server serves as many at 64 tasks; a write takes 2.5–3 times as long.
• Storage: 553–557 bytes a key on NATS (the value and an index record) and 749–815 on Redis. The default NATS state bucket of 256 MiB (jetstream.state_bucket_max_bytes) holds about 480 000 such keys.

Inside one process

The nightly benchmark workflow (benchmark.yml) measures the pieces of a call alone, on the CI runners: self-hosted x86_64 pods that other jobs share. Instruction and allocation counts do not depend on the host; the times do, so compare them only with the same lane.

The plugin boundary

libs/plugin-host/benches: a call to the no-op bench-noop plugin through the real dlopened cdylib vtable, against the same call in process.

A plugin call costs about 30 µs through the blocking pool and 1–2 µs inline: the hand-off to a blocking-pool thread, not the boundary's marshaling, is most of a ferried call. That is what inline_dispatch saves on every call, and why the tuned figures need about 10 threads.

Request costs

apps/gateway/benches and the SQL backend's benches, criterion medians:

End to end in one process

apps/gateway/tests/bench_*.rs (#[ignore]d), run on the benchmark host, not the CI runners: the gateway and a reqwest driver in one process pinned to 12 cores, a mock binding, the default settings.

The gateway against the backend's latency (bench_backend_latency):

With a backend of 50 ms or more, the gateway adds 2–3 ms at p50, and a deployment serves its sessions over the backend's time: 100 sessions on a 200 ms backend, 500 calls a second.

Sustained load (bench_load), each session calling tools/call as fast as it can for 10 s:

From 250 sessions on, the kernel's listen queue overflowed: 1 615 times in one 250-session run. A connection the queue drops is tried again after 1, 3, 7 and 15 s, which is the p99 and the longest calls of those rows. The fleet benchmark's load generator keeps its connections open, and with 512 of them on one gateway the p99 stayed under 100 ms.

Plugins, the debug tools and the number of bindings (bench_plugins, 100 sessions, 10 s each):

In-process plugins, the debug tools and the number of bindings change the rate by less than 10 %.

Sessions (bench_gateway): one client runs a session's lifecycle (initialize, tools/call, delete) in 0.67 ms at p50 and 0.85 ms at p99, 1 457 a second; tools/list answers in 0.16 ms at p50, 5 690 a second. With 10 to 500 clients each running full lifecycles (initialize, tools/list, ten tools/call, delete), the gateway serves 3 200–3 800 requests a second.

Memory per session (bench_backends bench_memory, nightly on the CI runners): 56 MiB at idle, 113 MiB with 256 sessions, about 230 KiB a session with the warm-up; 5 MiB above the start after the sessions drained.

The proxy lane

tools/bench/proxy-load on a 48-core CI runner, nightly: the legacy wire with the default settings, 32 sessions, 1 KiB results, the gateway on the runner's cores without pinning.

A larger machine

On the isolated host, the small shape is the layout of the tables above. The large shape gives each gateway 4 cores (one gateway alone 8), the three NATS nodes 8 shared cores for tool calls and 12 for MCP Events, and the load generator 8 cores for tool calls and 4 for MCP Events. The cores are of one type in both shapes: the large shape adds cores, and its 32 vCPU run on 16 physical cores, but no core is faster. Tool calls: 20 s runs, two a figure; MCP Events as in MCP Events, two runs; the range of the runs.

⁵ Both shapes on nats-server 2.10.24, whose subject-filtered purges hold up the bucket's writes during a sweep. On 2.14.8 the small shape takes 829–865 a second at 1 000 offered, 89–91 % within 3 s (see MCP Events).

• One gateway alone more than doubles on 8 cores: 11 800–12 000 stateless calls a second with the default settings, at p50 5.3 ms, while it used 5.0 of its 8 cores. The 64 connections of the load generator set that figure, not the gateway.
• A coordinator with more cores carries more sessions. A legacy call on NATS with the default settings writes its session to the coordinator: the NATS nodes on 8 cores carried 4 760–4 853 a second, 1.6 times what they carried on 2.
• Tuned, four gateways serve 14 600–16 800 calls a second in both shapes: the 64 connections of the load generator are the limit there.
• MCP Events on NATS 2.10.24 take a quarter to a half more in the large shape, while the NATS nodes used 3.6–5.1 of their 12 cores: more coordinator cores raise the events rate less than their count.

Sizing

The figures above give five relations to size a deployment with. Take the four-gateway figures: a gateway in a fleet serves fewer calls for the same fixed work than one alone.

1. Gateway cores = calls a second × CPU per call. One core serves about 2 800 calls a second with the tuned settings (350 µs a call), about 2 300 stateless calls with the default settings (440 µs), and about 1 000 legacy calls with the default settings (950 µs). Keep a gateway under about 70 % of its cores at peak.
2. Coordinator load = calls a second × messages per call. Stateless calls and tuned legacy calls send it about nothing. Legacy calls with the default settings send 6–8 messages each with replication: three NATS nodes on 2 cores carried about 2 700 of them a second (about 17 000 messages a second) at 1.67 cores. Size NATS for that traffic, or turn on unary_json_fast_path. The store's own ceiling, three NATS nodes on one host: about 30 000 reads and 11 000–20 000 writes a second; one Redis server: 140 000 reads and 65 000–110 000 writes.
3. Memory = idle base + working set at the concurrency served. About 60–190 MiB at idle; at 64 concurrent calls 520–690 MiB with the default settings (the legacy wire on NATS at the top) and 105–160 MiB with inline_dispatch. A blocking-pool thread holds about 3 MiB while it lives: one per plugin call in flight with the default settings, and one per store call in flight when many sessions start together. Plan for the peak: a burst's memory goes back after it, but a gateway alone that idles can hold it for a minute or more.
4. MCP Events ingest is per deployment, delivery is per gateway. One events owner takes the inbound events: about 1 600 a second on one gateway alone with the file store, all 1 000 offered on four gateways over Redis, and about 400 with every delivery within 3 s on NATS R3 with 2 coordinator cores (about 850 accepted, with the streams behind); keep the offered rate under that, as past it the gateways refuse posts and the streams fall behind. Deliveries cost 17–207 µs of gateway CPU each, wherever the streams are. A NATS coordinator for MCP Events runs nats-server 2.14 or later.
5. Limits are cheap when they refuse. A refusal answers from the replica's memory; only an admitted call under a fleet-wide cap costs coordinator operations.

A worked example: 20 000 tool calls a second on the stateless wire with the default settings is about 20 000 × 441 µs ≈ 8.8 cores of gateway CPU, so about seven gateways of 2 cores at 70 %, and a small coordinator. The same load on the legacy wire needs unary_json_fast_path, or a coordinator sized for about 125 000 messages a second.

Reproduce

# One gateway alone, then one, two and four on three NATS nodes (prebuilt binaries optional):
COORDINATOR=none GATEWAYS=1 SCENARIOS="scale governed churn listen watch memtime" DUR=30 \
  tools/bench/cluster/fleet.sh
COORDINATOR=nats GATEWAYS="1 2 4" SCENARIOS=scale DUR=30 tools/bench/cluster/fleet.sh
COORDINATOR=nats GATEWAYS=4 SCENARIOS="governed churn listen watch failover" DUR=30 \
  tools/bench/cluster/fleet.sh
# The tuned settings; Redis:
FLEET_TUNED=1 SCENARIOS=scale DUR=30 tools/bench/cluster/fleet.sh
COORDINATOR=redis GATEWAYS="1 4" SCENARIOS=scale DUR=30 tools/bench/cluster/fleet.sh
# MCP Events at several rates, with the coordinator calls of each event (the Prometheus plugin
# in PLUGIN_DIR); COORDINATOR=none, nats or redis, GATEWAYS=1 or 4:
FLEET_METRICS=1 SCENARIOS=events EVENT_RATES="100 400 1000 1600 3200" DUR=30 \
  tools/bench/cluster/fleet.sh
# The large shape of a larger machine, for tool calls and for MCP Events:
GATEWAY_CPUS_EACH=4 COORDINATOR_CPUS=16-23 LOADGEN_CPUS=24-31 GATEWAYS="1 4" SCENARIOS=scale \
  SCALE_REPS=2 DUR=20 tools/bench/cluster/fleet.sh
GATEWAY_CPUS_EACH=4 COORDINATOR_CPUS=16-27 LOADGEN_CPUS=28-31 GATEWAYS=4 FLEET_METRICS=1 \
  SCENARIOS=events EVENT_RATES="100 400 1000 1600 3200" DUR=30 tools/bench/cluster/fleet.sh
# The tables of a run:
python3 tools/bench/cluster/fleet_report.py "$OUT_DIR/fleet.jsonl"
# The coordinator store:
tools/bench/cluster/kv.sh
# Inside one process, as the nightly workflow runs them:
mkdir -p bench-results
for lane in ffi-iai criterion integration budgets proxy; do
  tools/bench/run-bench.sh "$lane" 2>&1 | tee "bench-results/$lane.log"
done
node tools/bench/gen-report.mjs bench-results
# The end-to-end benches on one host, as in the tables above (also bench_plugins,
# bench_backend_latency, bench_gateway):
taskset -c 0-11 cargo test -p mcpg --release --test bench_load -- \
  --ignored --nocapture --test-threads=1

tools/bench/cluster/README.md names every variable of the fleet and store benchmarks, and docs/ci/BENCHMARKING.md the nightly lanes. Compare runs on one host and at similar steal; the CPU per call and the coordinator messages per call are what carry over.