This is the full report of the run “The main branch with lazy protocol negotiation and vote-path instrumentation”. Back to the run page.
r20261105-0803 · Fleet c3-hdb · c3-highcpu-8

Telcoin Network benchmark: mixed workload

Throughput is measured from blocks the chain produced (own senders vs every sender), resource use from Cloud Monitoring, Prometheus and the on-node sampler. Summaries cover the mixed phase. Hover any number for the JSON field it came from.

10 blaster(s), 6K senders, chain 32286, schema tn-blast-combined/v2 · mixed phase 2026-11-05 09:11–09:51 UTC · 1 observers, 10 validators

Preflight: WTEL from genesis (0x0000…37e1), v3 fee tier 3000, v4 available, Multicall3 present · Cooldown ended: all_landed ×2, idle_30s ×8

Headline numbers

Sustained mixed TPS (own)
5,524.8 tx/s
best 60-s mean of our own landed tx in the mixed phase
Sustained mixed TPS (chain)
5,524.8 tx/s
all tx in blocks, same window
Landed / accepted
99.9 %
12.2M of 12.2M accepted
Reverted (own)
0
0.00 % of landed
Latency p50
3,303 ms
submit → block timestamp; seen p50 12,918 ms
Latency p95
12,837 ms
seen p95 29,778 ms
Gas / s (own)
231M gas/s
chain 231M gas/s
Capacity at mix
9,431.4 tx/s
gas-limit basis 6,116.6 tx/s; block gas limit 30M

Per transaction type

Landed counts only our own senders; latency is submit → block timestamp.

TypeSubmittedAcceptedLandedRevertedUnlandedSkipsGas limitGas used avgp50p95
native_transfer4M3.57M3.57M01.61K024.1K21K3,291 ms12,779 ms
batched_self_send2.19M1.84M1.84M01.21K024.1K21K3,375 ms13,061 ms
uniswap_v2_swap1.98M1.66M1.66M01.22K0180K106K3,299 ms12,748 ms
stablecoin_transfer1.64M1.38M1.37M0978075K43.9K3,294 ms12,748 ms
wtel_wrap1.32M1.11M1.11M0814050K27.9K3,294 ms12,757 ms
wtel_unwrap874K735K735K0527045K35.2K3,298 ms12,779 ms
stablecoin_approve656K552K552K0397060K31.8K3,299 ms12,755 ms
uniswap_v2_add_liquidity435K365K365K02680250K128K3,295 ms12,698 ms
contract_deploy326K274K274K02020140K74.2K3,297 ms12,765 ms
uniswap_v4_swap325K273K273K01940250K124K3,294 ms12,763 ms
uniswap_v3_swap221K186K185K01260220K120K3,303 ms12,750 ms
uniswap_v2_remove_liquidity220K185K185K01270220K129K3,303 ms12,744 ms
faucet_drip126K119K119K01423K150K107K2,780 ms15,269 ms

Submission by RPC node

NodeSubmittedAcceptedAccepted %Top errorsSend p50Send p95
bench-validator-01
bench-validator-01:8545
1.7M1.69M99.5 %known 3.56K6 ms19 ms
bench-validator-02
bench-validator-02:8545
1.74M1.74M99.6 %known 3.17K7 ms24 ms
bench-validator-03
bench-validator-03:8545
1.53M1.51M98.6 %known 12.5K8 ms24 ms
bench-validator-04
bench-validator-04:8545
1.57M1.57M99.9 %known 1949 ms30 ms
bench-validator-05
bench-validator-05:8545
1.7M1.25M73.7 %other 447K, known 3906 ms24 ms
bench-validator-06
bench-validator-06:8545
1.31M1.28M97.8 %known 13.2K7 ms25 ms
bench-validator-07
bench-validator-07:8545
3.02M532K17.6 %other 2.49M, known 631 ms11 ms
bench-validator-08
bench-validator-08:8545
3.41M163K4.8 %other 3.24M, known 1.31K1 ms3 ms
bench-validator-09
bench-validator-09:8545
1.36M1.34M98.2 %known 12.4K9 ms30 ms
bench-validator-10
bench-validator-10:8545
1.28M1.25M97.9 %known 13.1K8 ms24 ms

Throughput over time

Shaded: phases outside the mixed window. Hairlines: epoch closes. Dashed orange: chaos events.

Landed transactions per second

own senders vs chain total, per block-timestamp second

Gas per second

own vs chain gas used, per block-timestamp second

CPU and memory per node

CPU utilization

compute.googleapis.com/instance/cpu/utilization, 60-s mean, % of all vCPUs; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMax
bench-observer-11observer25.2 %27.1 %27.5 %27.5 %
bench-validator-01validator34.9 %36.2 %36.7 %36.7 %
bench-validator-02validator34.4 %35.9 %37.4 %37.4 %
bench-validator-03validator34.6 %36.5 %37.2 %37.2 %
bench-validator-04validator34.8 %36.3 %36.7 %36.7 %
bench-validator-05validator28.7 %35.9 %38.0 %38.0 %
bench-validator-06validator34.5 %36.3 %36.9 %36.9 %
bench-validator-07validator34.1 %36.0 %36.8 %36.8 %
bench-validator-08validator31.9 %34.1 %34.8 %34.8 %
bench-validator-09validator34.5 %36.5 %37.0 %37.0 %
bench-validator-10validator34.4 %36.4 %37.3 %37.3 %

CPU: mean vs peak 60-s mean

One row per node, over the mixed phase.

telcoin resident memory

agent processes/rss_usage summed over telcoin pids; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMax
bench-observer-11observer2.50 GiB3.22 GiB3.40 GiB3.40 GiB
bench-validator-01validator6.39 GiB8.05 GiB8.25 GiB8.25 GiB
bench-validator-02validator5.87 GiB7.40 GiB7.50 GiB7.50 GiB
bench-validator-03validator5.97 GiB8.37 GiB8.72 GiB8.72 GiB
bench-validator-04validator5.76 GiB7.25 GiB7.55 GiB7.55 GiB
bench-validator-05validator4.63 GiB6.37 GiB6.53 GiB6.53 GiB
bench-validator-06validator6.74 GiB8.85 GiB8.96 GiB8.96 GiB
bench-validator-07validator6.14 GiB7.77 GiB8.06 GiB8.06 GiB
bench-validator-08validator6.70 GiB8.49 GiB8.79 GiB8.79 GiB
bench-validator-09validator6.86 GiB8.70 GiB8.91 GiB8.91 GiB
bench-validator-10validator6.77 GiB8.80 GiB9.03 GiB9.03 GiB

telcoin RSS: mean vs peak 60-s mean

One row per node, over the mixed phase.

Memory used

agent memory/percent_used, state=used; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMax
bench-observer-11observer12.4 %13.5 %13.7 %13.7 %
bench-validator-01validator35.0 %39.1 %39.3 %39.3 %
bench-validator-02validator31.8 %34.8 %35.4 %35.4 %
bench-validator-03validator32.5 %40.7 %41.6 %41.6 %
bench-validator-04validator31.2 %34.1 %34.3 %34.3 %
bench-validator-05validator23.6 %30.2 %32.6 %32.6 %
bench-validator-06validator37.5 %43.9 %44.0 %44.0 %
bench-validator-07validator33.6 %37.5 %37.9 %37.9 %
bench-validator-08validator36.9 %41.8 %41.8 %41.8 %
bench-validator-09validator37.9 %43.0 %44.4 %44.4 %
bench-validator-10validator37.3 %43.4 %43.6 %43.6 %

Memory used: mean vs peak 60-s mean

One row per node, over the mixed phase.

Disk

Disk throttling (validator)
not collected –
metric type unavailable in project
Disk performance degraded (validator)
none –
any sample

Disk read + write throughput

instance/disk/{read,write}_bytes_count rate, all devices; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMaxLimit
bench-observer-11observer225.1 MiB/s309.8 MiB/s328.6 MiB/s328.6 MiB/s800.0 MiB/s
bench-validator-01validator179.8 MiB/s231.8 MiB/s244.3 MiB/s244.3 MiB/s800.0 MiB/s
bench-validator-02validator167.4 MiB/s217.0 MiB/s242.4 MiB/s242.4 MiB/s800.0 MiB/s
bench-validator-03validator166.8 MiB/s222.7 MiB/s246.7 MiB/s246.7 MiB/s800.0 MiB/s
bench-validator-04validator170.1 MiB/s224.6 MiB/s243.5 MiB/s243.5 MiB/s800.0 MiB/s
bench-validator-05validator134.7 MiB/s219.0 MiB/s241.5 MiB/s241.5 MiB/s800.0 MiB/s
bench-validator-06validator165.8 MiB/s223.5 MiB/s243.7 MiB/s243.7 MiB/s800.0 MiB/s
bench-validator-07validator187.8 MiB/s250.3 MiB/s273.1 MiB/s273.1 MiB/s800.0 MiB/s
bench-validator-08validator253.7 MiB/s310.1 MiB/s320.2 MiB/s320.2 MiB/s800.0 MiB/s
bench-validator-09validator164.6 MiB/s208.6 MiB/s220.7 MiB/s220.7 MiB/s800.0 MiB/s
bench-validator-10validator166.7 MiB/s217.4 MiB/s237.0 MiB/s237.0 MiB/s800.0 MiB/s

Disk read + write IOPS

instance/disk/{read,write}_ops_count rate, all devices; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMaxLimit
bench-observer-11observer6.23K IOPS9.19K IOPS9.54K IOPS9.54K IOPS20K IOPS
bench-validator-01validator7.35K IOPS9.32K IOPS10.3K IOPS10.3K IOPS20K IOPS
bench-validator-02validator7.65K IOPS9.67K IOPS9.86K IOPS9.86K IOPS20K IOPS
bench-validator-03validator7.68K IOPS9.67K IOPS9.83K IOPS9.83K IOPS20K IOPS
bench-validator-04validator7.57K IOPS9.58K IOPS10K IOPS10K IOPS20K IOPS
bench-validator-05validator6.13K IOPS9.4K IOPS9.96K IOPS9.96K IOPS20K IOPS
bench-validator-06validator7.7K IOPS9.76K IOPS9.97K IOPS9.97K IOPS20K IOPS
bench-validator-07validator6.98K IOPS9.51K IOPS9.58K IOPS9.58K IOPS20K IOPS
bench-validator-08validator5.09K IOPS8.01K IOPS8.84K IOPS8.84K IOPS20K IOPS
bench-validator-09validator7.67K IOPS9.43K IOPS9.73K IOPS9.73K IOPS20K IOPS
bench-validator-10validator7.67K IOPS9.92K IOPS10.1K IOPS10.1K IOPS20K IOPS
Disk limits used
NodeMachinevCPUDiskIOPS limitMiB/s limitBasis
bench-observer-11c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-01c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-02c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-03c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-04c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-05c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-06c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-07c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-08c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-09c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget
bench-validator-10c3-highcpu-88hyperdisk-balanced 100 GB20K IOPS800.0 MiB/shyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget

Network

Network sent

instance/network/sent_bytes_count rate; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMax
bench-observer-11observer0.9 Mbps1.2 Mbps1.3 Mbps1.3 Mbps
bench-validator-01validator81.8 Mbps85.8 Mbps89.0 Mbps89.0 Mbps
bench-validator-02validator81.8 Mbps86.9 Mbps88.5 Mbps88.5 Mbps
bench-validator-03validator83.5 Mbps99.1 Mbps126.2 Mbps126.2 Mbps
bench-validator-04validator81.3 Mbps87.5 Mbps96.6 Mbps96.6 Mbps
bench-validator-05validator66.0 Mbps85.3 Mbps88.9 Mbps88.9 Mbps
bench-validator-06validator80.0 Mbps86.4 Mbps88.2 Mbps88.2 Mbps
bench-validator-07validator77.5 Mbps82.7 Mbps85.7 Mbps85.7 Mbps
bench-validator-08validator73.0 Mbps79.2 Mbps83.3 Mbps83.3 Mbps
bench-validator-09validator85.5 Mbps103.5 Mbps119.2 Mbps119.2 Mbps
bench-validator-10validator86.6 Mbps99.1 Mbps155.0 Mbps155.0 Mbps

Network received

instance/network/received_bytes_count rate; summaries over the mixed phase

Table view
NodeClassMeanp95Peak 60-s meanMax
bench-observer-11observer5.9 Mbps13.3 Mbps40.0 Mbps40.0 Mbps
bench-validator-01validator8.1 Mbps9.0 Mbps9.4 Mbps9.4 Mbps
bench-validator-02validator8.1 Mbps8.7 Mbps9.7 Mbps9.7 Mbps
bench-validator-03validator7.8 Mbps8.9 Mbps9.0 Mbps9.0 Mbps
bench-validator-04validator7.8 Mbps8.9 Mbps9.7 Mbps9.7 Mbps
bench-validator-05validator10.8 Mbps19.3 Mbps62.8 Mbps62.8 Mbps
bench-validator-06validator7.4 Mbps8.6 Mbps8.8 Mbps8.8 Mbps
bench-validator-07validator16.2 Mbps33.2 Mbps55.0 Mbps55.0 Mbps
bench-validator-08validator17.4 Mbps30.9 Mbps54.9 Mbps54.9 Mbps
bench-validator-09validator7.5 Mbps8.8 Mbps9.0 Mbps9.0 Mbps
bench-validator-10validator7.5 Mbps8.7 Mbps8.7 Mbps8.7 Mbps

Consensus health

MetricQueryClassNodesMedian of node meansWorst peak 60-s meanMax
tn_primary_commit_latency_secondsp95validator94,343 ms30.0 s30.0 s
tn_engine_execution_duration_secondsp95validator105,781 ms23.9 s26.5 s
tn_engine_execution_duration_secondsp95observer15,611 ms9,372 ms9,550 ms
tn_worker_batch_size_bytesp95validator9241.8 KiB246.4 KiB246.4 KiB
tn_worker_batch_transactionsp95validator94841.48K1.48K
tn_executor_output_transactionsp95validator1029.1K46.2K47.2K
tn_primary_consensus_output_bytesmaxvalidator101016.0 KiB2.1 MiB3.0 MiB
tn_primary_consensus_output_bytesmaxobserver11013.8 KiB2.0 MiB2.6 MiB
tn_engine_queued_outputsmaxvalidator103.3188
tn_engine_queued_outputsmaxobserver14.688
tn_primary_roundmaxvalidator107152.94K2.96K
tn_primary_roundmaxobserver17152.94K2.95K
tn_primary_committed_roundmaxvalidator107132.94K2.95K
tn_primary_committed_roundmaxobserver17152.94K2.95K
tn_epoch_currentmaxvalidator103.3544
tn_epoch_currentmaxobserver13.3444
tn_batch_builder_pending_pool_transactionsmaxvalidator105159.62K9.62K
tn_batch_builder_pending_pool_transactionsmaxobserver1000
reth_process_resident_memory_bytesmaxvalidator106.39 GiB9.04 GiB9.11 GiB
reth_process_resident_memory_bytesmaxobserver12.50 GiB3.39 GiB3.40 GiB
tn_primary_subdags_committed_totalratevalidator100.4320.91.09
tn_primary_subdags_committed_totalrateobserver1000
tn_primary_round_laglagvalidator102.0529.838
tn_primary_round_laglagobserver1000
tn_batch_builder_seal_failures_totalratevalidator400.01670.0222
tn_node_modemaxvalidator10011
tn_node_modemaxobserver1000
tn_primary_vote_requests_totalratevalidator106.7519.321.4
tn_primary_vote_requests_totalrateobserver1000
tn_primary_votes_created_totalratevalidator106.6916.719.3
tn_primary_votes_created_totalrateobserver1000
tn_primary_votes_recast_totalratevalidator10000
tn_primary_votes_recast_totalrateobserver1000
tn_primary_vote_requests_too_new_totalratevalidator10011.313.4
tn_primary_vote_requests_too_new_totalrateobserver1000
tn_primary_vote_timeouts_total{stage="exec_wait"}ratevalidator1000.5390.822
tn_primary_vote_timeouts_total{stage="exec_wait"}rateobserver1000
tn_primary_vote_timeouts_total{stage="lead_sleep"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="lead_sleep"}rateobserver1000
tn_primary_vote_timeouts_total{stage="parents"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="parents"}rateobserver1000
tn_primary_vote_timeouts_total{stage="persist"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="persist"}rateobserver1000
tn_primary_vote_timeouts_total{stage="sync_batches"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="sync_batches"}rateobserver1000
tn_primary_vote_timeouts_total{stage="validate"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="validate"}rateobserver1000
tn_primary_vote_timeouts_total{stage="vote_store"}ratevalidator10000
tn_primary_vote_timeouts_total{stage="vote_store"}rateobserver1000
tn_primary_vote_stage_seconds{stage="exec_wait"}p95validator81,764 ms9,458 ms9,583 ms
tn_primary_vote_stage_seconds{stage="lead_sleep"}p95validator80 ms0 ms0 ms
tn_primary_vote_stage_seconds{stage="lock"}p95validator8298 ms9,750 ms9,750 ms
tn_primary_vote_stage_seconds{stage="parents"}p95validator81 ms3 ms4 ms
tn_primary_vote_stage_seconds{stage="persist"}p95validator845 ms175 ms186 ms
tn_primary_vote_stage_seconds{stage="sync_batches"}p95validator80 ms1 ms2 ms
tn_primary_vote_stage_seconds{stage="validate"}p95validator83 ms5 ms5 ms
tn_primary_vote_stage_seconds{stage="vote_store"}p95validator81 ms12 ms15 ms
tn_primary_leader_rounds_total{leader="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2",swapped="false",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2",swapped="false",found="true"}ratevalidator100.02850.07780.111
tn_primary_leader_rounds_total{leader="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2",swapped="true",found="true"}ratevalidator100.007990.06670.0889
tn_primary_leader_rounds_total{leader="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB",swapped="false",found="false"}ratevalidator100.004710.05560.0667
tn_primary_leader_rounds_total{leader="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB",swapped="false",found="true"}ratevalidator100.03830.08890.111
tn_primary_leader_rounds_total{leader="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB",swapped="true",found="false"}ratevalidator100.00250.06670.0889
tn_primary_leader_rounds_total{leader="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB",swapped="true",found="true"}ratevalidator100.01180.06110.0889
tn_primary_leader_rounds_total{leader="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD",swapped="false",found="false"}ratevalidator100.00240.05560.0667
tn_primary_leader_rounds_total{leader="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD",swapped="false",found="true"}ratevalidator100.02830.09440.111
tn_primary_leader_rounds_total{leader="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD",swapped="true",found="false"}ratevalidator100.001150.0440.066
tn_primary_leader_rounds_total{leader="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD",swapped="true",found="true"}ratevalidator100.010.06110.0889
tn_primary_leader_rounds_total{leader="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn",swapped="false",found="false"}ratevalidator100.002080.03330.0444
tn_primary_leader_rounds_total{leader="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn",swapped="false",found="true"}ratevalidator100.04110.10.111
tn_primary_leader_rounds_total{leader="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn",swapped="true",found="true"}ratevalidator100.01310.06670.0889
tn_primary_leader_rounds_total{leader="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq",swapped="false",found="false"}ratevalidator100.004170.06670.0889
tn_primary_leader_rounds_total{leader="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq",swapped="false",found="true"}ratevalidator100.008330.09440.111
tn_primary_leader_rounds_total{leader="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq",swapped="true",found="false"}ratevalidator100.001630.04440.0667
tn_primary_leader_rounds_total{leader="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq",swapped="true",found="true"}ratevalidator100.003330.05560.0667
tn_primary_leader_rounds_total{leader="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ",swapped="false",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ",swapped="false",found="true"}ratevalidator100.04570.10.111
tn_primary_leader_rounds_total{leader="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ",swapped="true",found="true"}ratevalidator100.01510.04440.0667
tn_primary_leader_rounds_total{leader="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye",swapped="false",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye",swapped="false",found="true"}ratevalidator100.04530.09440.111
tn_primary_leader_rounds_total{leader="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye",swapped="true",found="true"}ratevalidator100.01710.06670.0667
tn_primary_leader_rounds_total{leader="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR",swapped="false",found="false"}ratevalidator100.002830.06090.0667
tn_primary_leader_rounds_total{leader="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR",swapped="false",found="true"}ratevalidator100.02130.09440.111
tn_primary_leader_rounds_total{leader="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR",swapped="true",found="true"}ratevalidator100.01120.08330.111
tn_primary_leader_rounds_total{leader="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn",swapped="false",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn",swapped="false",found="true"}ratevalidator100.04280.08330.111
tn_primary_leader_rounds_total{leader="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn",swapped="true",found="true"}ratevalidator100.009830.050.0667
tn_primary_leader_rounds_total{leader="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs",swapped="false",found="false"}ratevalidator100.0007730.0330.044
tn_primary_leader_rounds_total{leader="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs",swapped="false",found="true"}ratevalidator100.02530.08330.111
tn_primary_leader_rounds_total{leader="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs",swapped="true",found="false"}ratevalidator10000
tn_primary_leader_rounds_total{leader="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs",swapped="true",found="true"}ratevalidator100.008060.06110.0889
tn_primary_leader_commits_total{leader="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2"}ratevalidator100.03670.1110.156
tn_primary_leader_commits_total{leader="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB"}ratevalidator100.05010.1110.133
tn_primary_leader_commits_total{leader="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD"}ratevalidator100.03830.1330.178
tn_primary_leader_commits_total{leader="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn"}ratevalidator100.05350.1330.2
tn_primary_leader_commits_total{leader="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq"}ratevalidator100.01170.1390.156
tn_primary_leader_commits_total{leader="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ"}ratevalidator100.06080.1330.156
tn_primary_leader_commits_total{leader="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye"}ratevalidator100.06240.1440.156
tn_primary_leader_commits_total{leader="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR"}ratevalidator100.03250.1390.178
tn_primary_leader_commits_total{leader="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn"}ratevalidator100.05260.1220.156
tn_primary_leader_commits_total{leader="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs"}ratevalidator100.03330.1440.178
tn_engine_queue_wait_secondsp95validator1015.8 s29.7 s30.0 s
tn_engine_queue_wait_secondsp95observer126.0 s29.0 s29.0 s
tn_engine_persist_duration_secondsp95validator81,148 ms8,469 ms9,125 ms
tn_engine_persist_duration_secondsp95observer11,699 ms4,365 ms4,583 ms
tn_engine_busy_milliseconds_totalratevalidator109651.12K1.21K
tn_engine_busy_milliseconds_totalrateobserver19811.05K1.1K
tn_reth_persist_save_blocks_secondsp95validator9413 ms8,469 ms9,125 ms
tn_reth_persist_save_blocks_secondsp95observer1615 ms4,083 ms4,583 ms
tn_reth_persist_commit_secondsp95validator9780 ms1,988 ms2,250 ms
tn_reth_persist_commit_secondsp95observer1934 ms1,065 ms1,300 ms
tn_reth_persisted_accounts_totalratevalidator102K2.36K2.53K
tn_reth_persisted_accounts_totalrateobserver11.98K2.34K2.43K
tn_reth_persisted_storage_slots_totalratevalidator102.76K3.28K3.39K
tn_reth_persisted_storage_slots_totalrateobserver12.73K3.18K3.27K
tn_node_mode_transitions_total{to="cvv_inactive",reason="epoch_behind"}ratevalidator1000
tn_node_mode_transitions_total{to="cvv_inactive",reason="outside_gc_window"}ratevalidator2000
tn_executor_rejoin_blocked_total{reason="epoch_closed"}ratevalidator30.02420.3830.4
tn_executor_rejoin_blocked_total{reason="epoch_mismatch"}ratevalidator3000
tn_executor_rejoin_blocked_total{reason="no_known_header"}ratevalidator3000
tn_executor_rejoin_blocked_total{reason="round_gap"}ratevalidator30.2240.4281.11
tn_executor_rejoin_blocked_total{reason="stale_head"}ratevalidator30.1120.2940.356
tn_worker_sync_serves_denied_total{worker="0",tier="committee"}ratevalidator10000
tn_worker_sync_serves_denied_total{worker="0",tier="committee"}rateobserver1000
tn_worker_sync_serves_denied_total{worker="0",tier="public"}ratevalidator10000
tn_worker_sync_serves_denied_total{worker="0",tier="public"}rateobserver1000
tn_worker_batch_requests_denied_total{worker="0",peer="2144FHRwrcvmEkUTAChHeoVdYn6vt1mQLCRLaSfVuieBgpKo9aMinwYFiqG8M3A61AEcbHZciXpvFFVYf9WipKxLTBUeZuDohtvK7FAtbmQBuc26DnJrRCY4tkqAhpZJkTGB"}ratevalidator20.006440.3660.366
tn_worker_batch_requests_denied_total{worker="0",peer="21hNc4w4BkZF6F6jQvFr1E36NpvJ2veVAzh1Zyy7C4kNgBfFCu423Ncw8t18vgbLt4PdtKeW8fZUJJA7emBzDbPnE8dDyY59bw64hftwyb8bNnXFDRXx1RUYabsqUNbBShYT"}ratevalidator300.5070.507
tn_worker_batch_requests_denied_total{worker="0",peer="nE5uq6wTBpXvRMnqjwm51T7ysmhfHshFJpXwuTpqcUBqqLyvgPVKF2i1vWik1vk1avXzV9Tu9bcrGJkSd2Yn2YhYMsZapcduTXK3Da2s9mrHJmqUA2ZyXXfuMd2nH3y1mty"}ratevalidator20.006390.3380.338
tn_worker_batch_requests_denied_total{worker="0",peer="nZtVqRH4VuyHtF3ASV74DMPQ93mDi1vcntRBsVoBwKWaz1tkbSLyGtkZD6wJq21M4znSfATuS9uVkgsb1TkxepdgKm9b6Rvug4aW2NpW1iKFNcHv7dBcmoz3ik4nT63PbzU"}ratevalidator10.001490.08440.0844
tn_worker_batch_requests_denied_total{worker="0",peer="omnUG8kmgM38zGPxu8w39WpDqPtgjinaJV6pUPRtV81Phho5JxSaczk6PNrFC24bcWgL6Sucw2hBNrTFMMg94DAFfsPpYLgabACTpoHfq2kEocssSxyxjBmjDkrNCBtacp2"}ratevalidator20.01140.6470.647
tn_worker_batch_requests_denied_total{worker="0",peer="sHv24SnwtmMr3yzEHBqH5tFoNWDkzVdGLLMr8oaHDpcGSdiZV5SrK2jD5Fcp8J5zZVDkiZ1PGGaB7DYLJKCgtE1tvhexUZuR4Shz6gE2KTWb4xrGdPPySgX4YQXFzG4rMd8"}ratevalidator2000
tn_worker_batch_requests_denied_total{worker="0",peer="sUDaca6agCTkShRNgnS31xLNdovgGqqCSn1hxgvSd6aD29FJhnhubtkQikashEAecU5xJEJfuWERN9CMhzD17HJYbHpbeK9n4oMLeDwjg5uexrvpVexNCQ1j3TvRu9Ccigk"}ratevalidator10.000990.05630.0563
tn_worker_batch_requests_denied_total{worker="0",peer="xDya61TkRgwqXhL621p5f5sqN4SxRG9RmTwLMdk2vAzqdXsqCtp5FTkUrPbw9zZTSm8s5b9TaDHvduCgFJgsSAVkavhmaj6gQmp6SVvogR2k9g9UaJuR7MWZBnfe7x5HgUP"}ratevalidator20.01140.6470.647
tn_worker_batch_requests_denied_total{worker="0",peer="xQAX3CxFYar64P2zTGvebVPT5JqMQe9pTyAnvLJmCHm7gxKWQ4SVcYho956GYuwVRUN5LzAhoV2VZodvzT2ePSFzS7iQMmb4xmAjvfwCDZa7V8TvcrqJaCxtGaQzBcZhyzD"}ratevalidator20.01390.7880.788
tn_worker_batch_requests_denied_total{worker="0",peer="zXUFxNGREuf8atZat5FZBk1NG5beET3h5KEfUFEnkXvnnMBJXPXaPvAYzj9Ny5Hur4ASkkDhwFgheryjVsqyU7oab9f5D5oZrAPSUVkbDFXW3GEoBzKbKGAr2TQBq84oriY"}ratevalidator300.2810.281
tn_worker_batch_requests_denied_total{worker="0",peer="zZTwQkGwmXD9pUpeNa5ehNm8GmdG85N3s8nsv6tZ4jpt5rHMZU881cXpvJUrvZYERZtkLb6QVnxKht9XhxjFeMV8ahr8GtG2b5nz5Ee75iVSqx6249xRKifoWcwWrvpP6TV"}ratevalidator20.001980.1130.113
tn_worker_batch_requests_failed_total{worker="0",peer="2144FHRwrcvmEkUTAChHeoVdYn6vt1mQLCRLaSfVuieBgpKo9aMinwYFiqG8M3A61AEcbHZciXpvFFVYf9WipKxLTBUeZuDohtvK7FAtbmQBuc26DnJrRCY4tkqAhpZJkTGB"}ratevalidator20.002970.1690.169
tn_worker_batch_requests_failed_total{worker="0",peer="21hNc4w4BkZF6F6jQvFr1E36NpvJ2veVAzh1Zyy7C4kNgBfFCu423Ncw8t18vgbLt4PdtKeW8fZUJJA7emBzDbPnE8dDyY59bw64hftwyb8bNnXFDRXx1RUYabsqUNbBShYT"}ratevalidator300.02820.0282
tn_worker_batch_requests_failed_total{worker="0",peer="nE5uq6wTBpXvRMnqjwm51T7ysmhfHshFJpXwuTpqcUBqqLyvgPVKF2i1vWik1vk1avXzV9Tu9bcrGJkSd2Yn2YhYMsZapcduTXK3Da2s9mrHJmqUA2ZyXXfuMd2nH3y1mty"}ratevalidator20.003470.1970.197
tn_worker_batch_requests_failed_total{worker="0",peer="nZtVqRH4VuyHtF3ASV74DMPQ93mDi1vcntRBsVoBwKWaz1tkbSLyGtkZD6wJq21M4znSfATuS9uVkgsb1TkxepdgKm9b6Rvug4aW2NpW1iKFNcHv7dBcmoz3ik4nT63PbzU"}ratevalidator1000
tn_worker_batch_requests_failed_total{worker="0",peer="omnUG8kmgM38zGPxu8w39WpDqPtgjinaJV6pUPRtV81Phho5JxSaczk6PNrFC24bcWgL6Sucw2hBNrTFMMg94DAFfsPpYLgabACTpoHfq2kEocssSxyxjBmjDkrNCBtacp2"}ratevalidator20.0004950.02820.0282
tn_worker_batch_requests_failed_total{worker="0",peer="sHv24SnwtmMr3yzEHBqH5tFoNWDkzVdGLLMr8oaHDpcGSdiZV5SrK2jD5Fcp8J5zZVDkiZ1PGGaB7DYLJKCgtE1tvhexUZuR4Shz6gE2KTWb4xrGdPPySgX4YQXFzG4rMd8"}ratevalidator2000
tn_worker_batch_requests_failed_total{worker="0",peer="sUDaca6agCTkShRNgnS31xLNdovgGqqCSn1hxgvSd6aD29FJhnhubtkQikashEAecU5xJEJfuWERN9CMhzD17HJYbHpbeK9n4oMLeDwjg5uexrvpVexNCQ1j3TvRu9Ccigk"}ratevalidator10.0004950.02820.0282
tn_worker_batch_requests_failed_total{worker="0",peer="xDya61TkRgwqXhL621p5f5sqN4SxRG9RmTwLMdk2vAzqdXsqCtp5FTkUrPbw9zZTSm8s5b9TaDHvduCgFJgsSAVkavhmaj6gQmp6SVvogR2k9g9UaJuR7MWZBnfe7x5HgUP"}ratevalidator20.005450.310.31
tn_worker_batch_requests_failed_total{worker="0",peer="xQAX3CxFYar64P2zTGvebVPT5JqMQe9pTyAnvLJmCHm7gxKWQ4SVcYho956GYuwVRUN5LzAhoV2VZodvzT2ePSFzS7iQMmb4xmAjvfwCDZa7V8TvcrqJaCxtGaQzBcZhyzD"}ratevalidator20.006930.3940.394
tn_worker_batch_requests_failed_total{worker="0",peer="zXUFxNGREuf8atZat5FZBk1NG5beET3h5KEfUFEnkXvnnMBJXPXaPvAYzj9Ny5Hur4ASkkDhwFgheryjVsqyU7oab9f5D5oZrAPSUVkbDFXW3GEoBzKbKGAr2TQBq84oriY"}ratevalidator300.02820.0282
tn_worker_batch_requests_failed_total{worker="0",peer="zZTwQkGwmXD9pUpeNa5ehNm8GmdG85N3s8nsv6tZ4jpt5rHMZU881cXpvJUrvZYERZtkLb6QVnxKht9XhxjFeMV8ahr8GtG2b5nz5Ee75iVSqx6249xRKifoWcwWrvpP6TV"}ratevalidator20.000990.05630.0563
tn_node_epoch_record_signersmaxvalidator107.741010
tn_node_epoch_record_signersmaxobserver18.191010
tn_node_epoch_record_missing_signers_total{signer="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2"}ratevalidator10000
tn_node_epoch_record_missing_signers_total{signer="2UYxCFESCqw61C7pEFpzxAzcdu1aeTFTRC6pcgZivJG2"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB"}ratevalidator1000.01520.0203
tn_node_epoch_record_missing_signers_total{signer="3GBP8A9iAAinoV4Cme8WzPoXfYRkErZ8GHx78FgaZazB"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD"}ratevalidator100.0003650.01650.022
tn_node_epoch_record_missing_signers_total{signer="3VrEiVQncxBCKqm5AN8KTtvrSShmeCT6bBQswYYtAutD"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn"}ratevalidator10000
tn_node_epoch_record_missing_signers_total{signer="7CgjP1y53obfnVnshMfKtLaaDuMWD7Eys21AQjYjjNsn"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq"}ratevalidator100.0003650.01650.022
tn_node_epoch_record_missing_signers_total{signer="AhKq47odXwQXCiQJqJRtvrpufzSh2twSUUijT6Jw4bWq"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ"}ratevalidator10000
tn_node_epoch_record_missing_signers_total{signer="CsBXvWKVeeTEDzk4kqHeRd7gjHs9VSTU68Z5Zsrrw2AJ"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye"}ratevalidator100.0003650.01580.021
tn_node_epoch_record_missing_signers_total{signer="G38nSkpHUZfYUQEeXNzD5SuoKjyapZU5WEgwe5tVddye"}rateobserver10.0004240.01370.0182
tn_node_epoch_record_missing_signers_total{signer="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR"}ratevalidator100.0007790.01670.0222
tn_node_epoch_record_missing_signers_total{signer="GYZ77es5DWi4ChjbRqcGQBFdM63pQHLLWBX4jhhNznBR"}rateobserver10.0004240.01370.0182
tn_node_epoch_record_missing_signers_total{signer="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn"}ratevalidator10000
tn_node_epoch_record_missing_signers_total{signer="HDKF9cCXBezTpvjiz14LQiEZCfn7MY2SY3pBoNuRauMn"}rateobserver1000
tn_node_epoch_record_missing_signers_total{signer="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs"}ratevalidator10000
tn_node_epoch_record_missing_signers_total{signer="vp782ooaY6XgWBo6vH5D844aXbcVpYdaxwJTHi7CEWs"}rateobserver1000
tn_network_outbound_request_duration_seconds{outcome="failure",class="batch"}p95validator815.9 s29.0 s29.0 s
tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"}p95validator514.7 s29.0 s29.0 s
tn_network_outbound_request_duration_seconds{outcome="failure",class="vote"}p95validator628.2 s29.0 s29.0 s
tn_network_outbound_request_duration_seconds{outcome="response",class="batch"}p95validator6444 ms477 ms477 ms
tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"}p95validator6459 ms494 ms499 ms
tn_network_outbound_request_duration_seconds{outcome="response",class="vote"}p95validator53,910 ms8,543 ms8,718 ms
tn_network_closed_admission_denied_total{network="bench",direction="in"}ratevalidator10000
tn_network_closed_admission_denied_total{network="bench",direction="in"}rateobserver1000
tn_network_closed_admission_denied_total{network="bench",direction="out"}ratevalidator10000
tn_network_closed_admission_denied_total{network="bench",direction="out"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"}rateobserver1000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"}ratevalidator10000
tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"}rateobserver1000
tn_network_admission_modemaxvalidator10222
tn_network_admission_modemaxobserver1222
tn_network_admission_fallbackmaxvalidator10000
tn_network_admission_fallbackmaxobserver1000
tn_network_outbound_request_failures_total{kind="connection"}ratevalidator10000
tn_network_outbound_request_failures_total{kind="connection"}rateobserver1000
tn_network_outbound_request_failures_total{kind="dial"}ratevalidator109.9667.583.9
tn_network_outbound_request_failures_total{kind="dial"}rateobserver1000
tn_network_outbound_request_failures_total{kind="io"}ratevalidator101.847.819.91
tn_network_outbound_request_failures_total{kind="io"}rateobserver10.3120.7630.948
tn_network_outbound_request_failures_total{kind="timeout"}ratevalidator1017.665.986.1
tn_network_outbound_request_failures_total{kind="timeout"}rateobserver10.003320.02090.0222
tn_network_outbound_request_failures_total{kind="unsupported"}ratevalidator10000
tn_network_outbound_request_failures_total{kind="unsupported"}rateobserver1000
tn_network_inbound_requests_shed_total{class="batch",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="batch",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"}ratevalidator100.11733.841.6
tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"}rateobserver1000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"}rateobserver1000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"}ratevalidator100.1971.11.42
tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"}rateobserver10.004330.04040.0444
tn_network_inbound_requests_shed_total{class="gossip",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="gossip",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"}ratevalidator100.03181.171.63
tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"}rateobserver1000
tn_network_inbound_requests_shed_total{class="other",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="other",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="other",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="other",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"}ratevalidator100.02820.1120.135
tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"}rateobserver1000
tn_network_inbound_requests_shed_total{class="vote",reason="admission"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="vote",reason="admission"}rateobserver1000
tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"}rateobserver1000
tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"}ratevalidator10000
tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"}rateobserver1000
tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"}ratevalidator100.4984.586.67
tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"}rateobserver1000
tn_network_inbound_request_service_seconds{class="batch"}p95validator317 ms21 ms21 ms
tn_network_inbound_request_service_seconds{class="epoch_record"}p95validator47 ms155 ms228 ms
tn_network_inbound_request_service_seconds{class="epoch_record"}p95observer16 ms36 ms80 ms
tn_network_inbound_request_service_seconds{class="vote"}p95validator71,233 ms9,747 ms9,750 ms
tn_network_established_connectionsmaxvalidator1019.62121
tn_network_established_connectionsmaxobserver119.82121
tn_worker_sync_streams_shed_total{reason="budget_exhausted"}ratevalidator2000
tn_worker_sync_streams_shed_total{reason="denied"}ratevalidator100.006923.023.02
tn_worker_sync_streams_shed_total{reason="denied"}rateobserver1000
tn_worker_batch_fetch_duration_secondsp95validator99 ms1,437 ms2,312 ms
tn_executor_batch_fetch_failures_totalratevalidator10000
tn_executor_batch_fetch_failures_totalrateobserver1000
tn_primary_votes_received_totalratevalidator105.3411.313.1
tn_primary_votes_received_totalrateobserver1000
tn_primary_vote_request_failures_totalratevalidator100.7953.614.73
tn_primary_vote_request_failures_totalrateobserver1000
tn_primary_certificates_formed_totalratevalidator100.8641.882.18
tn_primary_certificates_formed_totalrateobserver1000
tn_primary_certificate_form_duration_secondsp95validator101,001 ms4,930 ms9,250 ms
tn_engine_outputs_executed_totalratevalidator100.4330.8941.2
tn_engine_outputs_executed_totalrateobserver10.4270.7671.36
tn_engine_persist_failures_totalratevalidator10000
tn_engine_persist_failures_totalrateobserver1000
tn_node_consensus_sync_distancemaxvalidator103.88173190
tn_node_consensus_sync_distancemaxobserver111.32128
reth_db_table_size{table="AccountChangeSets"}maxvalidator10220 MiB396 MiB396 MiB
reth_db_table_size{table="AccountChangeSets"}maxobserver1217 MiB395 MiB396 MiB
reth_db_table_size{table="AccountsHistory"}maxvalidator1025 MiB52 MiB52 MiB
reth_db_table_size{table="AccountsHistory"}maxobserver125 MiB52 MiB52 MiB
reth_db_table_size{table="HashedAccounts"}maxvalidator1016 MiB31 MiB31 MiB
reth_db_table_size{table="HashedAccounts"}maxobserver116 MiB31 MiB31 MiB
reth_db_table_size{table="HashedStorages"}maxvalidator1022 MiB24 MiB24 MiB
reth_db_table_size{table="HashedStorages"}maxobserver122 MiB24 MiB24 MiB
reth_db_table_size{table="PlainAccountState"}maxvalidator1014 MiB26 MiB26 MiB
reth_db_table_size{table="PlainAccountState"}maxobserver113 MiB26 MiB26 MiB
reth_db_table_size{table="PlainStorageState"}maxvalidator1021 MiB24 MiB24 MiB
reth_db_table_size{table="PlainStorageState"}maxobserver121 MiB24 MiB24 MiB
reth_db_table_size{table="StorageChangeSets"}maxvalidator10268 MiB515 MiB515 MiB
reth_db_table_size{table="StorageChangeSets"}maxobserver1265 MiB513 MiB515 MiB
reth_db_table_size{table="StoragesHistory"}maxvalidator1053 MiB69 MiB69 MiB
reth_db_table_size{table="StoragesHistory"}maxobserver153 MiB69 MiB69 MiB
reth_db_table_size{table="TransactionHashNumbers"}maxvalidator10498 MiB876 MiB876 MiB
reth_db_table_size{table="TransactionHashNumbers"}maxobserver1492 MiB874 MiB876 MiB
reth_db_table_size{table="TransactionSenders"}maxvalidator10256 MiB451 MiB451 MiB
reth_db_table_size{table="TransactionSenders"}maxobserver1254 MiB450 MiB451 MiB
reth_db_freelistmaxvalidator1039K54.8K58.5K
reth_db_freelistmaxobserver139.1K52.9K57.6K
reth_database_transaction_commit_write_duration_secondsp95validator105 ms5 ms5 ms
reth_database_transaction_commit_write_duration_secondsp95observer15 ms5 ms5 ms
reth_database_transaction_commit_sync_duration_secondsp95validator9711 ms1,825 ms2,162 ms
reth_database_transaction_commit_sync_duration_secondsp95observer1930 ms1,023 ms1,150 ms
reth_database_transaction_commit_whole_duration_secondsp95validator9711 ms1,825 ms2,162 ms
reth_database_transaction_commit_whole_duration_secondsp95observer1930 ms1,023 ms1,150 ms

Commit latency p95

tn_primary_commit_latency_seconds

Round lag

tn_primary_round − tn_primary_committed_round

Engine queued outputs

tn_engine_queued_outputs, max by node

Certification by node

The share of each validator's own headers that formed a certificate, per window (tools/cert-rate.py on the node logs). bench-validator-08, bench-validator-09 and bench-validator-10 certified fewer than 90 % of their own headers in a window where they proposed at least 50. Own leader % is the validator's share of the committed leaders and not committed % the share of its certificates no committed leader reached, both over the run window; the fair share is 100 / 10 validators = 10.0 %. Greyed: fewer than 50 headers proposed, not judged.

ValidatorRegionphase preamblephase warmupphase mixedphase cooldownrunOwn leader %Not committed %
bench-validator-01us-west2 (Los Angeles)95.897.199.7100.099.314.10.2
bench-validator-02europe-west2 (London)100.0100.099.699.799.711.51.1
bench-validator-03australia-southeast1 (Sydney)94.092.495.399.795.612.92.5
bench-validator-04asia-northeast1 (Tokyo)100.0100.699.999.799.914.40.2
bench-validator-05us-east4 (Ashburn)100.0100.699.9–100.013.90.3
bench-validator-06europe-west4 (Netherlands)100.0100.093.999.795.48.02.0
bench-validator-07asia-southeast1 (Singapore)100.0100.699.9–100.016.33.1
bench-validator-08 below 90 %asia-east2 (Hong Kong)58.680.296.770.085.38.46.8
bench-validator-09 below 90 %asia-northeast3 (Seoul)97.780.284.799.787.212.21.2
bench-validator-10 below 90 %asia-south1 (Mumbai)84.791.394.7100.094.48.65.9

On-node sampler

NodeClassPeak RssAnonPeak RssFilePeak cachePSI memPSI ioPSI cpudb growthstatic_files growthRoot usedOOM Δ
bench-observer-11observer1.14 GiB2.45 GiB– 0.0 %12.6 %1.7 %0 B2.83 GiB64.0 %0
bench-validator-01validator5.37 GiB2.95 GiB– 0.0 %11.0 %4.4 %0 B2.83 GiB65.0 %0
bench-validator-02validator4.72 GiB2.87 GiB– 0.0 %10.5 %4.3 %0 B2.83 GiB65.0 %0
bench-validator-03validator5.78 GiB2.95 GiB– 0.0 %11.6 %4.1 %0 B2.83 GiB65.0 %0
bench-validator-04validator4.60 GiB2.97 GiB– 0.0 %11.9 %4.3 %0 B2.83 GiB65.0 %0
bench-validator-05validator4.40 GiB2.31 GiB– 0.0 %11.5 %4.4 %0 B2.69 GiB57.0 %0
bench-validator-06validator6.12 GiB2.92 GiB– 0.0 %11.0 %4.3 %0 B2.83 GiB65.0 %0
bench-validator-07validator5.19 GiB2.95 GiB– 0.0 %12.6 %4.0 %0 B2.83 GiB65.0 %0
bench-validator-08validator5.79 GiB3.14 GiB– 0.1 %17.4 %4.5 %0 B2.83 GiB63.0 %0
bench-validator-09validator6.29 GiB2.96 GiB– 0.0 %10.5 %4.6 %0 B2.83 GiB65.0 %0
bench-validator-10validator6.08 GiB2.91 GiB– 0.0 %10.2 %4.5 %0 B2.83 GiB65.0 %0

RssAnon vs RssFile

/proc/<pid>/status of the telcoin process on the node with the largest RSS

consensus-db/cache size

Not collected: consensus-db/cache size.

Pressure stall (some, avg10)

/proc/pressure on the node with the highest memory PSI

Headroom

Per role and per node, from sizing.json (p95 over the landed window; PSI and RssAnon max over the whole sampler run). The lines follow the hardware doc's tier rules: CPU p95 over 70 % of vCPUs; memory used p95 over 75 % of RAM, or telcoin RSS p95 over 85 % of RAM; disk write p95 over 60 % of the PD MiB/s limit, or write IOPS p95 over 60 % of the IOPS limit; IO pressure over 20 % or memory pressure over 5 % (PSI some avg10, max); network p95 over 50 % of 1 Gbps; root disk over 80 %. A line crossed on any node puts the role at the limit; a nonzero reading within 15 points of its line is tight; otherwise the role has headroom. OOM kills, disk throttling and degraded disk performance are flagged too.

SignalClassWorst nodeValueShare of capacityLineNodes over / tightStatus
CPU p95validatorbench-validator-0936.7 %36.7 %> 70 %0 / 0 headroom
Memory used p95validatorbench-validator-0643.5 %43.5 %> 75 %0 / 0 headroom
telcoin RSS p95validatorbench-validator-109.28 GB54.0 %> 85 %0 / 0 headroom
PSI memory maxvalidatorbench-validator-080.1 %0.1 %> 5 %0 / 1 tight
RssAnon peakvalidatorbench-validator-096.75 GB39.3 %info– info
Disk write p95validatorbench-validator-08325 MB/s38.7 %> 60 %0 / 0 headroom
Write IOPS p95validatorbench-validator-109,88249.4 %> 60 %0 / 8 tight
PSI io maxvalidatorbench-validator-0817.4 %17.4 %> 20 %0 / 10 tight
Root disk maxvalidatorbench-validator-0163.9 %63.9 %> 80 %0 / 0 headroom
Network p95validatorbench-validator-09103 Mbps10.3 %> 50 %0 / 0 headroom
Consensus cache peakvalidator–––info– not collected
OOM killsvalidator––anyany– none
Disk throttlingvalidator––metric type unavailable in projectany– not collected
Disk performance degradedvalidator––any sampleany– none
CPU p95observerbench-observer-1127.4 %27.4 %> 70 %0 / 0 headroom
Memory used p95observerbench-observer-1113.5 %13.5 %> 75 %0 / 0 headroom
telcoin RSS p95observerbench-observer-113.45 GB20.1 %> 85 %0 / 0 headroom
PSI memory maxobserverbench-observer-110.0 %0.0 %> 5 %0 / 0 headroom
RssAnon peakobserverbench-observer-111.22 GB7.1 %info– info
Disk write p95observerbench-observer-11325 MB/s38.7 %> 60 %0 / 0 headroom
Write IOPS p95observerbench-observer-118,83044.1 %> 60 %0 / 0 headroom
PSI io maxobserverbench-observer-1112.6 %12.6 %> 20 %0 / 1 tight
Root disk maxobserverbench-observer-1162.1 %62.1 %> 80 %0 / 0 headroom
Network p95observerbench-observer-1113 Mbps1.3 %> 50 %0 / 0 headroom
Consensus cache peakobserver–––info– not collected
OOM killsobserver––anyany– none
Disk throttlingobserver––metric type unavailable in projectany– not collected
Disk performance degradedobserver––any sampleany– none

Per node

NodeClassCPU p95Memory used p95telcoin RSS p95PSI memory maxRssAnon peakDisk write p95Write IOPS p95PSI io maxRoot disk maxNetwork p95Consensus cache peak
Line> 70 %> 75 %> 85 %> 5 %info> 60 %> 60 %> 20 %> 80 %> 50 %info
bench-validator-01validator36.4 %38.9 %8.62 GB 50 %0.0 %5.76 GB 34 %243 MB/s 29 %9,282 46 %11.0 %63.9 %86 Mbps 9 %–
bench-validator-02validator35.9 %34.7 %7.88 GB 46 %0.0 %5.07 GB 30 %227 MB/s 27 %9,621 48 %10.5 %63.6 %87 Mbps 9 %–
bench-validator-03validator36.6 %40.3 %8.76 GB 51 %0.0 %6.20 GB 36 %233 MB/s 28 %9,629 48 %11.6 %62.8 %99 Mbps 10 %–
bench-validator-04validator36.4 %34.1 %7.69 GB 45 %0.0 %4.94 GB 29 %235 MB/s 28 %9,552 48 %11.9 %62.7 %90 Mbps 9 %–
bench-validator-05validator36.3 %29.9 %6.04 GB 35 %0.0 %5.12 GB 30 %230 MB/s 27 %8,916 45 %11.5 %55.5 %86 Mbps 9 %–
bench-validator-06validator36.4 %43.5 %9.27 GB 54 %0.0 %6.58 GB 38 %234 MB/s 28 %9,697 48 %11.0 %62.8 %88 Mbps 9 %–
bench-validator-07validator36.1 %37.5 %8.24 GB 48 %0.0 %5.57 GB 32 %262 MB/s 31 %9,262 46 %12.6 %61.5 %83 Mbps 8 %–
bench-validator-08validator34.2 %41.6 %9.07 GB 53 %0.1 %6.21 GB 36 %325 MB/s 39 %7,956 40 %17.4 %59.4 %80 Mbps 8 %–
bench-validator-09validator36.7 %43.0 %9.18 GB 53 %0.0 %6.75 GB 39 %220 MB/s 26 %9,346 47 %10.5 %62.9 %103 Mbps 10 %–
bench-validator-10validator36.5 %43.4 %9.28 GB 54 %0.0 %6.53 GB 38 %228 MB/s 27 %9,882 49 %10.2 %63.0 %99 Mbps 10 %–
bench-observer-11observer27.4 %13.5 %3.45 GB 20 %0.0 %1.22 GB 7 %325 MB/s 39 %8,830 44 %12.6 %62.1 %13 Mbps 1 %–

Small figures are the share of capacity: RAM for RSS, the PD limit for disk write (VM cap applied), 1 Gbps for network. Hover a cell for its JSON path.

Verdict

Fleet A vs Fleet B

A = Fleet c3-hdb · c3-highcpu-8 · r20261105-0803
B = Fleet c3-hdb · c3-highcpu-8 · r20261103-1922

Verdict

tight c3-highcpu-8 validators at 5,525 sustained TPS: tight on memory and disk. Memory tight: memory pressure (PSI some avg10) max 0.1 % (bench-validator-08), tight against the 5 % line. Disk tight: write IOPS p95 9,882, 49.4 % of the 20,000 IOPS limit (bench-validator-10), tight against the 60 % line; IO pressure (PSI some avg10) max 17.4 % (bench-validator-08), tight against the 20 % line. CPU has headroom: CPU p95 36.7 % of 8 vCPUs (bench-validator-09). Network has headroom: network p95 103 Mbps, 10.3 % of 1 Gbps (bench-validator-09).

tight c3-highcpu-8 observers at 5,525 sustained TPS: tight on disk. Disk tight: IO pressure (PSI some avg10) max 12.6 % (bench-observer-11), tight against the 20 % line. CPU has headroom: CPU p95 27.4 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 13.5 % of RAM (bench-observer-11); telcoin RSS p95 3.45 GB, 20.1 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 13 Mbps, 1.3 % of 1 Gbps (bench-observer-11).

at the limit c3-highcpu-8 validators at 7,599 sustained TPS: at the limit on disk. Disk at the limit: write p95 602–613 MB/s, up to 73.1 % of the 800 MiB/s limit, over the 60 % line on 9 of 10 validators (bench-validator-01, 02, 03, 04, 06, 07, 08, 09, 10); IO pressure (PSI some avg10) max 25.8–33.2 %, over the 20 % line on all 10 validators (bench-validator-01, 02, 03, 04, 05, 06, 07, 08, 09, 10); write IOPS p95 10,292, 51.5 % of the 20,000 IOPS limit (bench-validator-05), tight against the 60 % line. CPU has headroom: CPU p95 27.4 % of 8 vCPUs (bench-validator-09). Memory has headroom: memory used p95 34.8 % of RAM (bench-validator-06); telcoin RSS p95 7.83 GB, 45.6 % of 16 GiB RAM (bench-validator-06); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 113 Mbps, 11.3 % of 1 Gbps (bench-validator-07). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).

at the limit c3-highcpu-8 observers at 7,599 sustained TPS: at the limit on disk. Disk at the limit: write p95 526 MB/s, 62.8 % of the 800 MiB/s limit on bench-observer-11, over the 60 % line; IO pressure (PSI some avg10) max 32.0 % on bench-observer-11, over the 20 % line. CPU has headroom: CPU p95 14.3 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 11.0 % of RAM (bench-observer-11); telcoin RSS p95 3.45 GB, 20.1 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 5 Mbps, 0.5 % of 1 Gbps (bench-observer-11). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).

Validators: Fleet A (c3-highcpu-8) is tight on memory and disk, with headroom on CPU and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. Disk is at the limit on Fleet B (c3-highcpu-8) only. Fleet A (c3-highcpu-8) stays under the disk lines that Fleet B (c3-highcpu-8) crosses.

Observers: Fleet A (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. Disk is at the limit on Fleet B (c3-highcpu-8) only. Fleet A (c3-highcpu-8) stays under the disk lines that Fleet B (c3-highcpu-8) crosses.

KPIFleet AFleet BB ÷ A
Sustained mixed TPS (own)5,524.8 tx/s7,599.3 tx/s1.38×
Sustained mixed TPS (chain)5,524.8 tx/s7,599.3 tx/s1.38×
Landed / accepted99.9 %99.6 %1.00×
Reverted (own)00–
Latency p503,303 ms3,494 ms1.06×
Latency p9512,837 ms14,173 ms1.10×
Gas / s (own)231M gas/s198M gas/s0.86×
Capacity at mix9,431.4 tx/s8,037.3 tx/s0.85×

Headroom, worst node per class

SignalClassFleet AFleet BLine
CPU p95validator36.7 % headroom27.4 % headroom> 70 %
Memory used p95validator43.5 % headroom34.8 % headroom> 75 %
telcoin RSS p95validator9.28 GB 54.0 % headroom7.83 GB 45.6 % headroom> 85 %
PSI memory maxvalidator0.1 % tight0.0 % headroom> 5 %
Disk write p95validator325 MB/s 38.7 % headroom613 MB/s 73.1 % at the limit> 60 %
Write IOPS p95validator9,882 49.4 % tight10,292 51.5 % tight> 60 %
PSI io maxvalidator17.4 % tight33.2 % at the limit> 20 %
Root disk maxvalidator63.9 % headroom13.3 % headroom> 80 %
Network p95validator103 Mbps 10.3 % headroom113 Mbps 11.3 % headroom> 50 %
CPU p95observer27.4 % headroom14.3 % headroom> 70 %
Memory used p95observer13.5 % headroom11.0 % headroom> 75 %
telcoin RSS p95observer3.45 GB 20.1 % headroom3.45 GB 20.1 % headroom> 85 %
PSI memory maxobserver0.0 % headroom0.0 % headroom> 5 %
Disk write p95observer325 MB/s 38.7 % headroom526 MB/s 62.8 % at the limit> 60 %
Write IOPS p95observer8,830 44.1 % headroom5,493 27.5 % headroom> 60 %
PSI io maxobserver12.6 % tight32.0 % at the limit> 20 %
Root disk maxobserver62.1 % headroom12.1 % headroom> 80 %
Network p95observer13 Mbps 1.3 % headroom5 Mbps 0.5 % headroom> 50 %

Own TPS, trailing 60-s mean

aligned on each fleet's mixed-phase start

CPU, hottest validator

aligned on each fleet's mixed-phase start

Memory used, hottest validator

aligned on each fleet's mixed-phase start

telcoin RSS, hottest validator

aligned on each fleet's mixed-phase start

Disk MiB/s, hottest validator

aligned on each fleet's mixed-phase start

Network sent, hottest validator

aligned on each fleet's mixed-phase start

Reference chains

Not collected: reference chains.