Fleet sizing: e2 baseline against c3 (A vs B)

2026-09-23 · started 07:08 UTC · ended 09:11 UTC · merged mixed window 49 min 49 s

telcoin-network e85bd7a3f · main · commit inferred

No full node logs were saved for this run. It ran the mutable tag adiri:v0.1.0-devnet, which was not pinned to a digest at the time. Six days later the 29 September run resolved the same tag to digest sha256:f6fcef5db5f14fec4011a7d86ad8c85d01d8efc26635b597531fa16491e093d2, whose node logs verified commit e85bd7a3f07f510c548cac9d783fb907fdf7fcca on main ("fix(network): refresh published node records (#1371)", 17 September 2026). A filtered log excerpt kept from one node, bench-validator-03, shows telcoin-network 0.1.0 (e85bd7a3f07f510c548cac9d783fb907fdf7fcca) starting at 07:39 UTC during this run's deploy. The other ten nodes' logs were not kept, so the commit is not verified fleet-wide.

Ten validators and one observer on e2-custom-4-8192 (4 vCPU, 8 GB) with 100 GB pd-ssd disks, run under a 40-minute mixed workload and compared with the same test on c3-highcpu-8 (run r20260923-0515). Over the window in which all ten generators ran, fleet A averaged 2,346.7 tx/s with a best 60 seconds of 3,569.4 tx/s, against 2,557.5 and 3,518.4 tx/s for fleet B over its own all-generators window. Fleet A's validators crossed the report's memory and disk lines, and the same pd-ssd volume limited both fleets, so on these disks the volume, not the machine shape, was the binding resource.

Key figures

2,346.7tx/s
Average chain throughput, all load generators running
all generators active
3,569.4tx/s
Best 60 seconds, all load generators running
all generators active
42.04s
Time to land, p95
whole run
26–30s
Commit latency, p95
merged mixed window

How it was run

Software

Upstream committelcoin-network e85bd7a3f07f510c548cac9d783fb907fdf7fcca · main · commit inferred
Node imageadiri:v0.1.0-devnet
Node image digestnot recorded
Node version0.1.0
Fork schedulenot recorded
Load generator commit3644adc
Load generator image digestsha256:811ab3a78637b436d69868ab10bdfeab5e50780bd5906d4643f1acc9de1425ca
Analysis commit1046acb
Harness commitnot recorded
Harness submodulesnot recorded

Fleet

Fleete2-custom-4-8192 + pd-ssd (fleet A)
Validators10
Observers1
Machine typee2-custom-4-8192
vCPUs per node4
Memory per node8 GB
Diskpd-ssd, 100 GB
Regionsaustralia-southeast1, europe-west2, europe-west4, northamerica-northeast1, us-west2 (5)
Load generators10 generators × e2-standard-2 × 600 senders each
Load placementone blaster per validator in the validator's zone

Chain

Chain ID32286
Networkbench
Epoch length1,200 s (20 min)

Workload

Warm-up120 s (2 min)
Mixed phase2,400 s (40 min)
Cool-down180 s (3 min)
Transaction typesbatched_self_send, contract_deploy, faucet_drip, native_transfer, stablecoin_approve, stablecoin_transfer, uniswap_v2_add_liquidity, uniswap_v2_remove_liquidity, uniswap_v2_swap, uniswap_v3_swap, uniswap_v4_swap, wtel_unwrap, wtel_wrap

Chaos events

MarkerTime (UTC)NodeActionDetail
C12026-09-23 08:30:12bench-validator-05docker kill -s KILL telcoin5 min outage then restart (replay worst case); systemctl stop holds it down (unit has Restart=on-failure). bench-blaster-05 targets ONLY this node, so its sends fail for the outage by design; the generator resends on connection errors
C22026-09-23 08:35:11bench-validator-05systemctl start telcoindocker start is impossible: the unit runs docker run --rm
C32026-09-23 08:40:16bench-observer-11systemctl restart telcoinobserver restart

Steps

  1. Deployed a throwaway fleet of ten validators and one observer on its own chain (id 32286, 20-minute epochs), separate from the live devnet. Two validators ran in each of five regions (us-west2, australia-southeast1, northamerica-northeast1, europe-west4 and europe-west2), and the observer ran in us-west2.
  2. Ran every node on e2-custom-4-8192 (4 vCPU, 8 GB RAM) with a 100 GB pd-ssd boot disk. This is fleet A.
  3. Started the nodes from the image tag adiri:v0.1.0-devnet with the devnet's default consensus timing. The harness is the one later used for the 29 September hyperdisk run, which also ran the devnet's default consensus timing; this run predates the harness's image digest pinning and the hyperdisk volumes.
  4. Placed one load generator (e2-standard-2) in each validator's zone. Each ran tn-transaction-generator blast with 600 funded senders and sent only to its own validator.
  5. Each generator ran a 120-second warm-up, a 2,400-second mixed phase over 13 transaction types (native and stablecoin transfers, stablecoin approvals, wrapping and unwrapping TEL, Uniswap v2, v3 and v4 swaps, v2 liquidity changes, contract deploys, faucet drips and batched self-sends) and a cooldown of up to 180 seconds.
  6. Fifteen minutes into the planned mixed phase, killed bench-validator-05 with SIGKILL, kept it down for five minutes and restarted it. Five minutes later, restarted the observer. The generator paired with validator-05 could not submit during the outage, by design.
  7. Measured throughput and time to land from the blocks the chain produced. Each generator read every block, and the merged result counts each block once.
  8. Collected CPU, memory, disk and network figures from Cloud Monitoring and Prometheus, and pressure-stall and memory figures from a sampler on every node.
  9. Recomputed the KPIs on this page from the run's data. Each KPI names its window: all ten generators in their mixed phase, the merged mixed phase, or the whole run.
  10. Rebuilt the report with build-report.py from the run's data, comparing fleet A with run r20260923-0515 (fleet B). Fleet B ran the same test about two hours earlier on c3-highcpu-8 (8 vCPU, 16 GB) with the same disks, regions and load.
  11. The report's headroom section checks every node against fixed lines, for example CPU p95 above 70 % of vCPUs, memory used p95 above 75 % of RAM and IO pressure above 20 %, and rates each node class as at the limit, tight or with headroom.

The harness scripts live in a private repository; the steps above describe what they do.

Results

MetricValueWindowSourceMethod
Average chain throughput, all load generators running2,346.7 tx/sall generators activecombined.series.per_second[chain_tx]Chain transactions per block-timestamp second, summed over the 1811-s window when every load generator was in its mixed phase and divided by its length.
Best 60 seconds, all load generators running3,569.4 tx/sall generators activecombined.series.per_second[chain_tx]Highest mean chain tx/s over any 60 consecutive seconds inside the all-generators window (aggregate.py best_window_mean).
Transactions landed over the whole blast7,416,315 txwhole runcombined.totals.landed_chainSum of tx_count over every block the load generators saw during the blast, own and foreign transactions alike.
Blocks per second7.02 blocks/smerged mixed windowcombined.blocks.rows[timestamp]Chain blocks whose timestamp falls in the merged mixed window, divided by its length (aggregate.py landed.capacity.blocks_per_sec over the same window).
Time to land, median7.6 swhole runcombined.latency.ts_ms.p50Block timestamp minus submit time over every landed own transaction in the blast, from the load generators' merged latency histograms (aggregate.py merge_latency), in seconds.
Time to land, p9542.04 swhole runcombined.latency.ts_ms.p95Block timestamp minus submit time over every landed own transaction in the blast, from the load generators' merged latency histograms (aggregate.py merge_latency), in seconds.
Commit latency, p9526–30 smerged mixed windowmetrics.prometheus.series.tn_primary_commit_latency_seconds.by_node[healthy validators].p95For each healthy validator, the nearest-rank p95 over the merged mixed window of its 15-s histogram_quantile(0.95) commit-latency series; reported as the range over validators.
Execution time per consensus output3.27–5.57 smerged mixed windowmetrics.prometheus.series.tn_engine_execution_duration_seconds.by_node[healthy validators].meanFor each healthy validator, the nearest-rank median over the merged mixed window of its 15-s mean execution-duration series; reported as the range over validators.
Write throughput, p95, busiest validator130.5 MiB/smerged mixed windowmetrics.gcp[bench-validator-09].disk_write_bytes_per_sec p95Nearest-rank p95 of the 60-s mean write-byte rate over the merged mixed window for each healthy validator; the highest (bench-validator-09) is reported.
Write operations, p95, busiest validator5,119 IOPSmerged mixed windowmetrics.gcp[bench-validator-04].disk_write_ops_per_sec p95Nearest-rank p95 of the 60-s mean write-operation rate over the merged mixed window for each healthy validator; the highest (bench-validator-04) is reported.
CPU, p95, busiest validator55.2 %merged mixed windowmetrics.gcp[bench-validator-03].cpu_utilization p95Nearest-rank p95 of the 60-s CPU utilisation over the merged mixed window for each healthy validator; the highest (bench-validator-03) is reported.
Node resident memory, p95, busiest validator6.66 GiBmerged mixed windowmetrics.gcp[bench-validator-04].telcoin_rss_bytes p95Nearest-rank p95 of the 60-s telcoin process resident memory over the merged mixed window for each healthy validator; the highest (bench-validator-04) is reported.
IO pressure, highest reading, range over validators15.1–37.2 %merged mixed windowsampler[healthy validators].psi_io_some_avg10 maxFor each healthy validator, the highest 10-s 'some' IO pressure reading the on-node sampler logged inside the merged mixed window; reported as the range over validators.

Recovery

Nodebench-validator-05
Killed2026-09-23 08:30:12 UTC
Restarted2026-09-23 08:35:11 UTC
Rejoinednot recorded
Catch-up timenot recorded
Failed batch reportsnot recorded

Chain TPS per 5-minute interval

Chain TPS per 5-minute interval 10 intervals from 08:13 to 09:03 UTC; average 2,285 TPS, highest 2,578 TPS. The merged mixed window ran from 08:13 to 09:03 UTC. TPS 0 500 1,000 1,500 2,000 2,500 3,000 08:13–08:18 UTC: 2,578 TPS, 5.8 generators active 08:18–08:23 UTC: 2,502 TPS, 6.1 generators active 08:23–08:28 UTC: 2,539 TPS, 10.0 generators active 08:28–08:33 UTC: 2,413 TPS, 10.0 generators active 08:33–08:38 UTC: 2,394 TPS, 10.0 generators active 08:38–08:43 UTC: 2,466 TPS, 10.0 generators active 08:43–08:48 UTC: 2,208 TPS, 10.0 generators active 08:48–08:53 UTC: 2,094 TPS, 10.0 generators active 08:53–08:58 UTC: 1,609 TPS, 4.2 generators active 08:58–09:03 UTC: 2,044 TPS, 4.0 generators active avg 2,285 08:13 UTC 08:23 UTC 08:33 UTC 08:43 UTC 08:53 UTC C1 2026-09-23 08:30:12 UTC: docker kill -s KILL telcoin on bench-validator-05 (5 min outage then restart (replay worst case); systemctl stop holds it down (unit has Restart=on-failure). bench-blaster-05 targets ONLY this node, so its sends fail for the outage by design; the generator resends on connection errors) C1 C2 2026-09-23 08:35:11 UTC: systemctl start telcoin on bench-validator-05 (docker start is impossible: the unit runs docker run --rm) C2 C3 2026-09-23 08:40:16 UTC: systemctl restart telcoin on bench-observer-11 (observer restart) C3

Each bar is the average chain TPS over one 5-minute interval of the merged mixed window (08:13–09:03 UTC); fainter bars had fewer generators active than the maximum, the dashed line is the average of all bars and amber lines mark the chaos events listed below.

  • C1 2026-09-23 08:30:12 UTC: docker kill -s KILL telcoin on bench-validator-05 (5 min outage then restart (replay worst case); systemctl stop holds it down (unit has Restart=on-failure). bench-blaster-05 targets ONLY this node, so its sends fail for the outage by design; the generator resends on connection errors)
  • C2 2026-09-23 08:35:11 UTC: systemctl start telcoin on bench-validator-05 (docker start is impossible: the unit runs docker run --rm)
  • C3 2026-09-23 08:40:16 UTC: systemctl restart telcoin on bench-observer-11 (observer restart)
5-minute data (10 rows)
start_utcend_utcgenerators_activechain_tps
2026-09-23T08:13:48Z2026-09-23T08:18:48Z5.82,578.5
2026-09-23T08:18:48Z2026-09-23T08:23:48Z6.12,502.2
2026-09-23T08:23:48Z2026-09-23T08:28:48Z10.02,538.9
2026-09-23T08:28:48Z2026-09-23T08:33:48Z10.02,412.9
2026-09-23T08:33:48Z2026-09-23T08:38:48Z10.02,394.5
2026-09-23T08:38:48Z2026-09-23T08:43:48Z10.02,465.6
2026-09-23T08:43:48Z2026-09-23T08:48:48Z10.02,208.5
2026-09-23T08:48:48Z2026-09-23T08:53:48Z10.02,094.5
2026-09-23T08:53:48Z2026-09-23T08:58:48Z4.21,609.1
2026-09-23T08:58:48Z2026-09-23T09:03:37Z4.02,043.7

The fine-grained chain throughput series (its step_s field gives the seconds per point) is published as series/tps-1s.json.

Other networks

MeasureTelcoin Network (this run)Tempo
Throughput2,346.7 tx/s (Avg chain TPS)16,455 TPS, shown as "Settled TPS" for the nightly run of 5 Oct 2026 (commit de5f5d3). perf.tempo.xyz describes it as "Median settled throughput across 130 measured blocks."
How throughput is measuredChain transactions per block-timestamp second, summed over the 1811-s window when every load generator was in its mixed phase and divided by its length.The methodology page defines TPS as "Transactions per second (derived from gas / avg tx gas)": gas used divided by the average gas of one transaction, reported as the median over the measured blocks.
Run length49.8 min merged mixed window; each generator was set to a 40 min mixed phase2m 39s for the 5 Oct 2026 nightly run (130 measured blocks at a 687 ms average block time), per perf.tempo.xyz.
Validators and placement10 validators and 1 observer across 5 regions: australia-southeast1, europe-west2, europe-west4, northamerica-northeast1, us-west2; load generators: one blaster per validator in the validator's zoneperf.tempo.xyz labels the scenario "Public mix · multi-region". The methodology page says "Benchmarks run two local validators in consensus on a dedicated bare-metal server with 100 GiB pre-bloated state." The public tempoxyz/tempo repo has nightly workflows for both kinds of setup: bench-e2e runs on one bare-metal runner, and bench-e2e-multi-region defaults to 10 validators on cloud VMs (its regions input reads "Empty uses the selected cloud's five defaults."). The run page does not say which setup produced this number.
Hardwaree2-custom-4-8192, 4 vCPUs, 8 GB memory; disk: pd-ssd, 100 GBThe methodology page lists a dedicated bare-metal server: "AMD EPYC 4585PX (16C/32T, 4.3/5.7 GHz)", "128 GB 3600 MHz" memory and "2×960 GB NVMe + 2×1.92 TB NVMe via schelk". The repo's multi-region workflow uses cloud VMs instead (c4d-standard-48-lssd on GCP, c6id.8xlarge on AWS).
Workloadbatched_self_send, contract_deploy, faucet_drip, native_transfer, stablecoin_approve, stablecoin_transfer, uniswap_v2_add_liquidity, uniswap_v2_remove_liquidity, uniswap_v2_swap, uniswap_v3_swap, uniswap_v4_swap, wtel_unwrap, wtel_wrap; 10 generators × 600 senders eachperf.tempo.xyz labels the scenario "Public mix"; the methodology page says "100% TIP-20 token transfers (ERC-20 equivalent)". The run used 1,000 accounts and 4 TIP-20 tokens.
StateFresh chain from genesis for every runThe methodology page says "100 GiB pre-bloated state".
Latency (as each source defines it)Time to land, median: 7.6 s; Time to land, p95: 42.04 s; Commit latency, p95: 26–30 sBlock time of 687 ms, which the methodology page defines as "Average wall-clock time between blocks". The run page gives no per-transaction latency. Separately, tempo.xyz says "Blocks finalize in ~0.6 seconds with no re-orgs." (a network claim, not a figure from this benchmark).
Source and fetch dateThis page; every figure names its source field.perf.tempo.xyz nightly run of 5 Oct 2026 (commit de5f5d3), the perf.tempo.xyz methodology page, tempo.xyz, and the benchmark workflows in the tempoxyz/tempo repo. All fetched on 5 Oct 2026.

Sources for Tempo (fetched 2026-10-05):

Tempo's setup may not match ours: its methodology page describes two validators on one bare-metal server, while our runs use ten validators across five regions, so read this table as context, not a ranking.

This table is context, not a ranking.

Caveats

  • This report is a regeneration: it was rebuilt from the run's data and replaces an internal version of the same analysis. Its verdicts and headline figures match that version.
  • Full node logs were not kept, so the upstream commit is inferred, not verified. The provenance note explains the inference.
  • The devnet's default consensus timing was in force: 3 s maximum header delay, 1 s minimum header delay and 1 s maximum batch delay. Neither the manifest nor the node logs recorded it for this run.
  • Four of the ten generators (those for validators 01, 03, 07 and 08) reached their 600-second preamble timeout and began their mixed phase about ten minutes after the other six. The merged mixed window therefore spans 2,989 seconds, while all ten generators were in their mixed phase together for 1,811 seconds (08:23:37 to 08:53:48 UTC).
  • The all-generators window includes the five-minute validator-05 outage, when the generator paired with that node could not submit, so the average throughput includes that period.
  • The report's "Sustained mixed TPS" is each fleet's best 60-second mean over its whole mixed phase, so its A-versus-B table sets 3,569.4 tx/s against 5,383.5 tx/s (1.51×). Fleet B's best minute was the first minute of its mixed phase, before its last generator had started. Over each fleet's all-generators window the best 60 seconds are 3,569.4 tx/s (A) and 3,518.4 tx/s (B), and the averages 2,346.7 tx/s (A) and 2,557.5 tx/s (B).
  • The report's "Landed transactions per second" chart counts transactions per block-timestamp second and shows isolated spikes, the highest 65,169 tx in a single second, next to seconds with no blocks. These are bursts of blocks sharing a timestamp second, not sustained rates.
  • Disk headroom is judged against Google's published pd-ssd limits (9,000 IOPS and 240 MiB/s for a 100 GB volume on these machine types, as fetched on 22 September 2026), not against measured limits.
  • Fleet A and fleet B are one run each on separate deployments. The B ÷ A ratios in the report compare two single runs, not repeated trials.
  • Per-validator rows cover 9 validators; bench-validator-05 excluded as chaos target(s).
  • Recovery: no tn_node_mode{mode="cvv_inactive"} series for bench-validator-05; demoted / rejoined / catch-up unavailable.
  • Recovery: no tn_batch_builder_seal_failures_total rate series for bench-validator-05; failed_reports unavailable.
  • The consensus parameters were not recorded for this run, so none are published.

Full report

Full report HTML · 1.0 MB

Changes from the original report:

  • The site adds a banner at the top of the report that links back to this page.

SHA-256 of the committed file, before the banner is added: 084efe134daef866fa7e4778be4ad719557cfd521a724ee39f125db606e51a05