Throughput is measured from blocks the chain produced (own senders vs every sender), resource use from Cloud Monitoring, Prometheus and the on-node sampler. Summaries cover the mixed phase. Hover any number for the JSON field it came from.
Landed counts only our own senders; latency is submit → block timestamp.
| Type | Submitted | Accepted | Landed | Reverted | Unlanded | Skips | Gas limit | Gas used avg | p50 | p95 |
|---|---|---|---|---|---|---|---|---|---|---|
| native_transfer | 4.48M | 4.4M | 4.4M | 0 | 37 | 0 | 24.1K | 21K | 674 ms | 14,431 ms |
| batched_self_send | 2.22M | 2.16M | 2.16M | 0 | 54 | 0 | 24.1K | 21K | 698 ms | 14,586 ms |
| uniswap_v2_swap | 2M | 1.94M | 1.94M | 0 | 24 | 0 | 180K | 106K | 675 ms | 14,418 ms |
| stablecoin_transfer | 1.66M | 1.61M | 1.61M | 0 | 20 | 0 | 75K | 43.9K | 673 ms | 14,360 ms |
| wtel_wrap | 1.33M | 1.29M | 1.29M | 0 | 15 | 0 | 50K | 27.9K | 673 ms | 14,404 ms |
| wtel_unwrap | 888K | 863K | 863K | 0 | 10 | 0 | 45K | 35.2K | 673 ms | 14,394 ms |
| stablecoin_approve | 668K | 649K | 649K | 0 | 5 | 0 | 60K | 31.8K | 673 ms | 14,415 ms |
| uniswap_v2_add_liquidity | 443K | 431K | 431K | 0 | 6 | 0 | 250K | 128K | 674 ms | 14,360 ms |
| contract_deploy | 333K | 324K | 324K | 0 | 2 | 0 | 140K | 74.2K | 673 ms | 14,528 ms |
| uniswap_v4_swap | 332K | 323K | 323K | 0 | 3 | 0 | 250K | 124K | 676 ms | 14,555 ms |
| uniswap_v2_remove_liquidity | 223K | 216K | 216K | 0 | 6 | 0 | 220K | 129K | 673 ms | 14,421 ms |
| uniswap_v3_swap | 222K | 215K | 215K | 0 | 1 | 0 | 220K | 120K | 674 ms | 14,328 ms |
| faucet_drip | 126K | 126K | 126K | 0 | 0 | 430K | 150K | 107K | 876 ms | 40,498 ms |
| Node | Submitted | Accepted | Accepted % | Top errors | Send p50 | Send p95 |
|---|---|---|---|---|---|---|
| bench-validator-01 bench-validator-01:8545 | 1.8M | 1.8M | 100.0 % | none | 5 ms | 19 ms |
| bench-validator-02 bench-validator-02:8545 | 549K | 534K | 97.2 % | known 8.29K | 14 ms | 47 ms |
| bench-validator-03 bench-validator-03:8545 | 1.88M | 1.88M | 100.0 % | none | 4 ms | 16 ms |
| bench-validator-04 bench-validator-04:8545 | 1.82M | 1.82M | 100.0 % | none | 5 ms | 19 ms |
| bench-validator-05 bench-validator-05:8545 | 1.89M | 779K | 41.2 % | other 1.11M, known 81 | 1 ms | 23 ms |
| bench-validator-06 bench-validator-06:8545 | 474K | 460K | 97.1 % | known 8K | 15 ms | 48 ms |
| bench-validator-07 bench-validator-07:8545 | 1.88M | 1.88M | 100.0 % | none | 5 ms | 21 ms |
| bench-validator-08 bench-validator-08:8545 | 1.88M | 1.88M | 100.0 % | none | 5 ms | 18 ms |
| bench-validator-09 bench-validator-09:8545 | 1.91M | 1.91M | 100.0 % | none | 3 ms | 14 ms |
| bench-validator-10 bench-validator-10:8545 | 1.68M | 1.68M | 100.0 % | known 138 | 4 ms | 17 ms |
Shaded: phases outside the mixed window. Hairlines: epoch closes. Dashed orange: chaos events.
own senders vs chain total, per block-timestamp second
own vs chain gas used, per block-timestamp second
compute.googleapis.com/instance/cpu/utilization, 60-s mean, % of all vCPUs; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 17.9 % | 20.4 % | 26.6 % | 26.6 % |
| bench-validator-01 | validator | 32.2 % | 35.3 % | 51.0 % | 51.0 % |
| bench-validator-02 | validator | 29.9 % | 33.6 % | 46.6 % | 46.6 % |
| bench-validator-03 | validator | 32.2 % | 36.0 % | 52.4 % | 52.4 % |
| bench-validator-04 | validator | 32.0 % | 35.3 % | 51.9 % | 51.9 % |
| bench-validator-05 | validator | 27.9 % | 34.9 % | 49.0 % | 49.0 % |
| bench-validator-06 | validator | 30.4 % | 34.8 % | 48.0 % | 48.0 % |
| bench-validator-07 | validator | 31.8 % | 35.1 % | 51.2 % | 51.2 % |
| bench-validator-08 | validator | 31.8 % | 35.0 % | 51.3 % | 51.3 % |
| bench-validator-09 | validator | 31.5 % | 34.1 % | 50.9 % | 50.9 % |
| bench-validator-10 | validator | 32.1 % | 35.4 % | 51.1 % | 51.1 % |
One row per node, over the mixed phase.
agent processes/rss_usage summed over telcoin pids; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 3.20 GiB | 4.23 GiB | 4.38 GiB | 4.38 GiB |
| bench-validator-01 | validator | 4.79 GiB | 5.48 GiB | 5.61 GiB | 5.61 GiB |
| bench-validator-02 | validator | 4.98 GiB | 5.91 GiB | 6.09 GiB | 6.09 GiB |
| bench-validator-03 | validator | 5.06 GiB | 6.07 GiB | 6.29 GiB | 6.29 GiB |
| bench-validator-04 | validator | 5.01 GiB | 5.83 GiB | 6.06 GiB | 6.06 GiB |
| bench-validator-05 | validator | 4.45 GiB | 5.29 GiB | 5.29 GiB | 5.29 GiB |
| bench-validator-06 | validator | 5.21 GiB | 6.40 GiB | 6.43 GiB | 6.43 GiB |
| bench-validator-07 | validator | 4.56 GiB | 5.52 GiB | 5.80 GiB | 5.80 GiB |
| bench-validator-08 | validator | 5.03 GiB | 5.92 GiB | 6.18 GiB | 6.18 GiB |
| bench-validator-09 | validator | 4.61 GiB | 5.55 GiB | 5.79 GiB | 5.79 GiB |
| bench-validator-10 | validator | 5.11 GiB | 6.26 GiB | 6.43 GiB | 6.43 GiB |
One row per node, over the mixed phase.
agent memory/percent_used, state=used; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 10.5 % | 12.4 % | 12.5 % | 12.5 % |
| bench-validator-01 | validator | 20.1 % | 20.7 % | 20.8 % | 20.8 % |
| bench-validator-02 | validator | 20.6 % | 21.3 % | 21.4 % | 21.4 % |
| bench-validator-03 | validator | 20.8 % | 21.3 % | 21.4 % | 21.4 % |
| bench-validator-04 | validator | 21.0 % | 21.7 % | 21.9 % | 21.9 % |
| bench-validator-05 | validator | 16.5 % | 20.2 % | 20.3 % | 20.3 % |
| bench-validator-06 | validator | 21.8 % | 22.7 % | 22.9 % | 22.9 % |
| bench-validator-07 | validator | 17.9 % | 18.3 % | 18.4 % | 18.4 % |
| bench-validator-08 | validator | 20.8 % | 21.6 % | 21.7 % | 21.7 % |
| bench-validator-09 | validator | 18.1 % | 18.5 % | 18.7 % | 18.7 % |
| bench-validator-10 | validator | 20.9 % | 21.6 % | 21.6 % | 21.6 % |
One row per node, over the mixed phase.
instance/disk/{read,write}_bytes_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 87.5 MiB/s | 92.3 MiB/s | 96.3 MiB/s | 96.3 MiB/s | 800.0 MiB/s |
| bench-validator-01 | validator | 98.7 MiB/s | 104.3 MiB/s | 110.5 MiB/s | 110.5 MiB/s | 800.0 MiB/s |
| bench-validator-02 | validator | 98.4 MiB/s | 103.7 MiB/s | 109.9 MiB/s | 109.9 MiB/s | 800.0 MiB/s |
| bench-validator-03 | validator | 98.5 MiB/s | 104.5 MiB/s | 108.9 MiB/s | 108.9 MiB/s | 800.0 MiB/s |
| bench-validator-04 | validator | 98.6 MiB/s | 104.2 MiB/s | 109.0 MiB/s | 109.0 MiB/s | 800.0 MiB/s |
| bench-validator-05 | validator | 84.5 MiB/s | 103.0 MiB/s | 109.1 MiB/s | 109.1 MiB/s | 800.0 MiB/s |
| bench-validator-06 | validator | 98.2 MiB/s | 103.6 MiB/s | 110.3 MiB/s | 110.3 MiB/s | 800.0 MiB/s |
| bench-validator-07 | validator | 98.3 MiB/s | 102.7 MiB/s | 109.6 MiB/s | 109.6 MiB/s | 800.0 MiB/s |
| bench-validator-08 | validator | 98.4 MiB/s | 102.7 MiB/s | 109.4 MiB/s | 109.4 MiB/s | 800.0 MiB/s |
| bench-validator-09 | validator | 98.5 MiB/s | 103.2 MiB/s | 108.8 MiB/s | 108.8 MiB/s | 800.0 MiB/s |
| bench-validator-10 | validator | 98.6 MiB/s | 103.2 MiB/s | 108.7 MiB/s | 108.7 MiB/s | 800.0 MiB/s |
instance/disk/{read,write}_ops_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 19.7K IOPS | 20.1K IOPS | 20.1K IOPS | 20.1K IOPS | 20K IOPS |
| bench-validator-01 | validator | 19.9K IOPS | 20.1K IOPS | 20.2K IOPS | 20.2K IOPS | 20K IOPS |
| bench-validator-02 | validator | 19.9K IOPS | 20.1K IOPS | 20.3K IOPS | 20.3K IOPS | 20K IOPS |
| bench-validator-03 | validator | 19.9K IOPS | 20.1K IOPS | 20.4K IOPS | 20.4K IOPS | 20K IOPS |
| bench-validator-04 | validator | 19.9K IOPS | 20.1K IOPS | 20.1K IOPS | 20.1K IOPS | 20K IOPS |
| bench-validator-05 | validator | 17K IOPS | 20.1K IOPS | 20.2K IOPS | 20.2K IOPS | 20K IOPS |
| bench-validator-06 | validator | 19.9K IOPS | 20.1K IOPS | 20.4K IOPS | 20.4K IOPS | 20K IOPS |
| bench-validator-07 | validator | 19.8K IOPS | 20.1K IOPS | 20.1K IOPS | 20.1K IOPS | 20K IOPS |
| bench-validator-08 | validator | 19.8K IOPS | 20.1K IOPS | 20.2K IOPS | 20.2K IOPS | 20K IOPS |
| bench-validator-09 | validator | 19.9K IOPS | 20.1K IOPS | 20.1K IOPS | 20.1K IOPS | 20K IOPS |
| bench-validator-10 | validator | 19.8K IOPS | 20.1K IOPS | 20.1K IOPS | 20.1K IOPS | 20K IOPS |
| Node | Machine | vCPU | Disk | IOPS limit | MiB/s limit | Basis |
|---|---|---|---|---|---|---|
| bench-observer-11 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-01 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-02 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-03 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-04 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-05 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-06 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-07 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-08 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-09 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-10 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 20K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 20,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
instance/network/sent_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 2.1 Mbps | 2.2 Mbps | 2.4 Mbps | 2.4 Mbps |
| bench-validator-01 | validator | 97.8 Mbps | 110.9 Mbps | 135.8 Mbps | 135.8 Mbps |
| bench-validator-02 | validator | 92.8 Mbps | 101.7 Mbps | 126.8 Mbps | 126.8 Mbps |
| bench-validator-03 | validator | 98.1 Mbps | 109.2 Mbps | 140.3 Mbps | 140.3 Mbps |
| bench-validator-04 | validator | 97.8 Mbps | 107.5 Mbps | 133.0 Mbps | 133.0 Mbps |
| bench-validator-05 | validator | 83.4 Mbps | 107.1 Mbps | 124.1 Mbps | 124.1 Mbps |
| bench-validator-06 | validator | 92.7 Mbps | 103.6 Mbps | 124.4 Mbps | 124.4 Mbps |
| bench-validator-07 | validator | 100.1 Mbps | 130.4 Mbps | 152.4 Mbps | 152.4 Mbps |
| bench-validator-08 | validator | 99.4 Mbps | 111.7 Mbps | 137.3 Mbps | 137.3 Mbps |
| bench-validator-09 | validator | 98.1 Mbps | 111.3 Mbps | 137.1 Mbps | 137.1 Mbps |
| bench-validator-10 | validator | 97.1 Mbps | 110.7 Mbps | 133.4 Mbps | 133.4 Mbps |
instance/network/received_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 6.9 Mbps | 11.5 Mbps | 27.1 Mbps | 27.1 Mbps |
| bench-validator-01 | validator | 10.5 Mbps | 11.6 Mbps | 19.0 Mbps | 19.0 Mbps |
| bench-validator-02 | validator | 8.2 Mbps | 9.5 Mbps | 14.3 Mbps | 14.3 Mbps |
| bench-validator-03 | validator | 10.7 Mbps | 12.0 Mbps | 20.7 Mbps | 20.7 Mbps |
| bench-validator-04 | validator | 10.7 Mbps | 11.5 Mbps | 19.3 Mbps | 19.3 Mbps |
| bench-validator-05 | validator | 13.9 Mbps | 22.1 Mbps | 96.6 Mbps | 96.6 Mbps |
| bench-validator-06 | validator | 8.3 Mbps | 10.0 Mbps | 15.1 Mbps | 15.1 Mbps |
| bench-validator-07 | validator | 10.7 Mbps | 12.1 Mbps | 20.3 Mbps | 20.3 Mbps |
| bench-validator-08 | validator | 10.7 Mbps | 12.2 Mbps | 20.5 Mbps | 20.5 Mbps |
| bench-validator-09 | validator | 10.7 Mbps | 12.5 Mbps | 21.5 Mbps | 21.5 Mbps |
| bench-validator-10 | validator | 10.4 Mbps | 11.9 Mbps | 19.9 Mbps | 19.9 Mbps |
| Metric | Query | Class | Nodes | Median of node means | Worst peak 60-s mean | Max |
|---|---|---|---|---|---|---|
| tn_primary_commit_latency_seconds | p95 | validator | 10 | 975 ms | 1,312 ms | 1,600 ms |
| tn_engine_execution_duration_seconds | p95 | validator | 10 | 1,250 ms | 2,778 ms | 3,250 ms |
| tn_engine_execution_duration_seconds | p95 | observer | 1 | 1,219 ms | 2,358 ms | 2,442 ms |
| tn_worker_batch_size_bytes | p95 | validator | 10 | 232.7 KiB | 246.4 KiB | 246.4 KiB |
| tn_worker_batch_transactions | p95 | validator | 10 | 493 | 1.48K | 1.48K |
| tn_executor_output_transactions | p95 | validator | 10 | 8.44K | 23K | 26.9K |
| tn_primary_consensus_output_bytes | max | validator | 10 | 335.3 KiB | 1.8 MiB | 2.3 MiB |
| tn_primary_consensus_output_bytes | max | observer | 1 | 318.4 KiB | 670.9 KiB | 995.2 KiB |
| tn_engine_queued_outputs | max | validator | 10 | 7.88 | 8 | 8 |
| tn_engine_queued_outputs | max | observer | 1 | 7.75 | 8 | 8 |
| tn_primary_round | max | validator | 10 | 1.69K | 3.39K | 3.42K |
| tn_primary_round | max | observer | 1 | 1.65K | 3.34K | 3.41K |
| tn_primary_committed_round | max | validator | 10 | 1.69K | 3.38K | 3.42K |
| tn_primary_committed_round | max | observer | 1 | 1.65K | 3.34K | 3.41K |
| tn_epoch_current | max | validator | 10 | 4.21 | 5 | 5 |
| tn_epoch_current | max | observer | 1 | 4.22 | 5 | 5 |
| tn_batch_builder_pending_pool_transactions | max | validator | 10 | 308 | 4.98K | 9.84K |
| tn_batch_builder_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_process_resident_memory_bytes | max | validator | 10 | 5.03 GiB | 6.48 GiB | 6.56 GiB |
| reth_process_resident_memory_bytes | max | observer | 1 | 3.21 GiB | 4.42 GiB | 4.47 GiB |
| tn_primary_subdags_committed_total | rate | validator | 10 | 1.38 | 1.44 | 1.44 |
| tn_primary_subdags_committed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_round_lag | lag | validator | 10 | 2.09 | 3 | 3 |
| tn_primary_round_lag | lag | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_seal_failures_total | rate | validator | 9 | 0.000412 | 0.0167 | 0.0222 |
| tn_node_mode | max | validator | 10 | 0 | 1 | 1 |
| tn_node_mode | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_total | rate | validator | 10 | 23.8 | 33.3 | 33.4 |
| tn_primary_vote_requests_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_created_total | rate | validator | 10 | 23.7 | 25.8 | 25.8 |
| tn_primary_votes_created_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_recast_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_votes_recast_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_stage_seconds{stage="exec_wait"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lead_sleep"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lock"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="parents"} | p95 | validator | 10 | 1 ms | 2 ms | 2 ms |
| tn_primary_vote_stage_seconds{stage="persist"} | p95 | validator | 10 | 36 ms | 45 ms | 46 ms |
| tn_primary_vote_stage_seconds{stage="prelock_wait"} | p95 | validator | 10 | 0 ms | 97 ms | 211 ms |
| tn_primary_vote_stage_seconds{stage="sync_batches"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="validate"} | p95 | validator | 10 | 3 ms | 4 ms | 4 ms |
| tn_primary_vote_stage_seconds{stage="vote_store"} | p95 | validator | 10 | 1 ms | 3 ms | 4 ms |
| tn_primary_leader_rounds_total{leader="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf",swapped="false",found="true"} | rate | validator | 10 | 0.124 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf",swapped="true",found="true"} | rate | validator | 10 | 0.0357 | 0.122 | 0.156 |
| tn_primary_leader_rounds_total{leader="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau",swapped="false",found="true"} | rate | validator | 10 | 0.00947 | 0.15 | 0.156 |
| tn_primary_leader_rounds_total{leader="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau",swapped="true",found="true"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk",swapped="true",found="true"} | rate | validator | 10 | 0.0479 | 0.133 | 0.178 |
| tn_primary_leader_rounds_total{leader="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX",swapped="true",found="true"} | rate | validator | 10 | 0.0513 | 0.128 | 0.156 |
| tn_primary_leader_rounds_total{leader="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg",swapped="true",found="true"} | rate | validator | 10 | 0.0483 | 0.122 | 0.156 |
| tn_primary_leader_rounds_total{leader="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU",swapped="false",found="false"} | rate | validator | 10 | 0.0037 | 0.106 | 0.133 |
| tn_primary_leader_rounds_total{leader="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU",swapped="false",found="true"} | rate | validator | 10 | 0.065 | 0.15 | 0.156 |
| tn_primary_leader_rounds_total{leader="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU",swapped="true",found="true"} | rate | validator | 10 | 0.00864 | 0.111 | 0.133 |
| tn_primary_leader_rounds_total{leader="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62",swapped="true",found="true"} | rate | validator | 10 | 0.0492 | 0.122 | 0.156 |
| tn_primary_leader_rounds_total{leader="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH",swapped="false",found="true"} | rate | validator | 10 | 0.00412 | 0.117 | 0.156 |
| tn_primary_leader_rounds_total{leader="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH",swapped="true",found="true"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG",swapped="true",found="true"} | rate | validator | 10 | 0.0471 | 0.111 | 0.133 |
| tn_primary_leader_rounds_total{leader="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T",swapped="false",found="true"} | rate | validator | 10 | 0.138 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T",swapped="true",found="true"} | rate | validator | 10 | 0.0579 | 0.156 | 0.2 |
| tn_primary_leader_commits_total{leader="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf"} | rate | validator | 10 | 0.159 | 0.261 | 0.311 |
| tn_primary_leader_commits_total{leader="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau"} | rate | validator | 10 | 0.00947 | 0.15 | 0.156 |
| tn_primary_leader_commits_total{leader="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk"} | rate | validator | 10 | 0.186 | 0.283 | 0.333 |
| tn_primary_leader_commits_total{leader="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX"} | rate | validator | 10 | 0.189 | 0.272 | 0.289 |
| tn_primary_leader_commits_total{leader="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg"} | rate | validator | 10 | 0.186 | 0.267 | 0.311 |
| tn_primary_leader_commits_total{leader="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU"} | rate | validator | 10 | 0.0737 | 0.261 | 0.289 |
| tn_primary_leader_commits_total{leader="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62"} | rate | validator | 10 | 0.187 | 0.272 | 0.311 |
| tn_primary_leader_commits_total{leader="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH"} | rate | validator | 10 | 0.00412 | 0.117 | 0.156 |
| tn_primary_leader_commits_total{leader="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG"} | rate | validator | 10 | 0.184 | 0.256 | 0.289 |
| tn_primary_leader_commits_total{leader="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T"} | rate | validator | 10 | 0.195 | 0.306 | 0.333 |
| tn_engine_queue_wait_seconds | p95 | validator | 10 | 9,681 ms | 27.2 s | 27.2 s |
| tn_engine_queue_wait_seconds | p95 | observer | 1 | 9,590 ms | 9,819 ms | 9,848 ms |
| tn_engine_persist_duration_seconds | p95 | validator | 10 | 572 ms | 973 ms | 1,000 ms |
| tn_engine_persist_duration_seconds | p95 | observer | 1 | 680 ms | 956 ms | 962 ms |
| tn_engine_busy_milliseconds_total | rate | validator | 10 | 996 | 1.02K | 1.06K |
| tn_engine_busy_milliseconds_total | rate | observer | 1 | 980 | 1.01K | 1.03K |
| tn_reth_persist_save_blocks_seconds | p95 | validator | 10 | 194 ms | 467 ms | 488 ms |
| tn_reth_persist_save_blocks_seconds | p95 | observer | 1 | 141 ms | 245 ms | 267 ms |
| tn_reth_persist_commit_seconds | p95 | validator | 10 | 478 ms | 547 ms | 683 ms |
| tn_reth_persist_commit_seconds | p95 | observer | 1 | 492 ms | 860 ms | 918 ms |
| tn_reth_persisted_accounts_total | rate | validator | 10 | 2.27K | 7.78K | 7.78K |
| tn_reth_persisted_accounts_total | rate | observer | 1 | 2.24K | 7.46K | 7.46K |
| tn_reth_persisted_storage_slots_total | rate | validator | 10 | 2.95K | 3.88K | 4.16K |
| tn_reth_persisted_storage_slots_total | rate | observer | 1 | 2.94K | 3.82K | 3.96K |
| tn_executor_rejoin_blocked_total{reason="attempts_exhausted"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="epoch_closed"} | rate | validator | 1 | 0.103 | 0.888 | 1.2 |
| tn_executor_rejoin_blocked_total{reason="epoch_mismatch"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="no_known_header"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="round_gap"} | rate | validator | 1 | 1.1 | 1.39 | 1.42 |
| tn_executor_rejoin_blocked_total{reason="stale_head"} | rate | validator | 1 | 0.0186 | 0.191 | 0.255 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="21Xm7N2ETof1EiczCkTxnvAT7RqncJynW7oDG9LdfXbftTGyimPKJScjFEtgu8jbz3zQvZqeVDSfuBUzZJsqVhbf7Gp48EjVd82BfYWUqaTq6Q4pxHFrH4cvX8rt6bPCJXj8"} | rate | validator | 2 | 0.0099 | 0.25 | 0.333 |
| tn_worker_batch_requests_denied_total{worker="0",peer="other"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="p9mw9r81Ez8WtYtDpGY2hisE8d2bSdbsBKxsLfXP14FJvQFx9FNSQTZ3QwVj5WFqhoCW8hWM1wi3CWnzBMToFSCmvj9AxjJGK9C9NVhSz1Mrqk4cPuTsZo2S2Z4mPSonatK"} | rate | validator | 3 | 0 | 0.433 | 0.578 |
| tn_worker_batch_requests_denied_total{worker="0",peer="pTvhwCdG6MJvMRXWZCCkXpnBRNnbTEMapA5Yb3r2uMaAbeyBbCMho1oGshFMR6HSzDSrvFFVJ9aDLXDXmXsbDVRJXmhVe1an8tatLUx22JEZ48oxWQbzi3kM7VSetHhmXiv"} | rate | validator | 3 | 0 | 0.15 | 0.2 |
| tn_worker_batch_requests_denied_total{worker="0",peer="pyGb28dcEZzWZ1SvkNNBYWAHoKMahzr9HPfWirMkfTKiZ9EqMrUCj24FQnuzzXy3pbUEvr7CVCwdy56ctVseVfB4Spspgc8YTAGzihVVWxsQXan3z7tEx4t2mXpcGR68RsS"} | rate | validator | 3 | 0 | 0.35 | 0.467 |
| tn_worker_batch_requests_denied_total{worker="0",peer="rJNJunJgoDxtiNRocRMQRemyibjCubR4DyXDyFrMi1ua3cjripqHz64w7ovUwmNnnqA16FWkYCbZNHfkxdRgdeC3H2mFaZeffCqJuZTJncWfC7mr8ne6AFRhPn4va2wj2uv"} | rate | validator | 3 | 0 | 0.256 | 0.356 |
| tn_worker_batch_requests_denied_total{worker="0",peer="u6Ar9c81k4sdSWY82JbijYWSVKkR6xLfywxgd4yxZEkG2EBZPdeM6HKUX61c7iFz2fosBvn9RgsCFFEUqVMnfpd1dwkAr6P8xK3hXG33vv2xNrQn8cmePd51rGeZBqAzkge"} | rate | validator | 2 | 0.0119 | 0.3 | 0.4 |
| tn_worker_batch_requests_denied_total{worker="0",peer="u7ZWUHcAbH1aoouEbxvd9ofiRExo5kdcpZXN5kCyehc7Px9GJWaiefgLuNnHuBvEqWBCDbzhNXRjwVPv7Po19hZ8aGrrU4dhLJYE4PDXWV64nFtDtbSa44fWuX8qzdAyMin"} | rate | validator | 2 | 0.0123 | 0.417 | 0.556 |
| tn_worker_batch_requests_denied_total{worker="0",peer="xBZnXmyYqFi1mCwSH6KCF4f1W5xtdsXALBknGRb669vULDvaXqq89RXcW9hTfCi1SVABCSjavn3BWgLoeERA7SzM22Y7Lf74EvcRGa9X69mxReSJR9Lw4pX5VcM8Stdp1nz"} | rate | validator | 2 | 0.0145 | 0.367 | 0.489 |
| tn_worker_batch_requests_denied_total{worker="0",peer="ziGAo3Fw5vYzBuWHw8u8FUD72W3hE8p5kdW2Zd3NHmBWugD5UARi6jCW6aBGBo3UbMPgQwgCpBQ2nZWt52yk4PdQJtx7fBVkNTVqvjz8Lv2Y2iR8CcEaA59Au8atMzNJmks"} | rate | validator | 5 | 0 | 0.0167 | 0.0222 |
| tn_worker_batch_requests_denied_total{worker="0",peer="zixBdNoyd77oeFw5LFYX7YADGxbYGj1shubF8Xbt2F6pdVrdNmV2G4BLVaiaZ7Y4NGqjFSLFQyTV6U5YG9JivwgVutT71ouveXcP9oPe3srBzb4zL25KLhCGj4HybzSMGFt"} | rate | validator | 2 | 0.0104 | 0.35 | 0.467 |
| tn_worker_batch_requests_failed_total{worker="0",peer="21Xm7N2ETof1EiczCkTxnvAT7RqncJynW7oDG9LdfXbftTGyimPKJScjFEtgu8jbz3zQvZqeVDSfuBUzZJsqVhbf7Gp48EjVd82BfYWUqaTq6Q4pxHFrH4cvX8rt6bPCJXj8"} | rate | validator | 2 | 0.0066 | 0.167 | 0.222 |
| tn_worker_batch_requests_failed_total{worker="0",peer="other"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="p9mw9r81Ez8WtYtDpGY2hisE8d2bSdbsBKxsLfXP14FJvQFx9FNSQTZ3QwVj5WFqhoCW8hWM1wi3CWnzBMToFSCmvj9AxjJGK9C9NVhSz1Mrqk4cPuTsZo2S2Z4mPSonatK"} | rate | validator | 3 | 0 | 0.0833 | 0.111 |
| tn_worker_batch_requests_failed_total{worker="0",peer="pTvhwCdG6MJvMRXWZCCkXpnBRNnbTEMapA5Yb3r2uMaAbeyBbCMho1oGshFMR6HSzDSrvFFVJ9aDLXDXmXsbDVRJXmhVe1an8tatLUx22JEZ48oxWQbzi3kM7VSetHhmXiv"} | rate | validator | 3 | 0 | 0.133 | 0.178 |
| tn_worker_batch_requests_failed_total{worker="0",peer="pyGb28dcEZzWZ1SvkNNBYWAHoKMahzr9HPfWirMkfTKiZ9EqMrUCj24FQnuzzXy3pbUEvr7CVCwdy56ctVseVfB4Spspgc8YTAGzihVVWxsQXan3z7tEx4t2mXpcGR68RsS"} | rate | validator | 3 | 0 | 0.1 | 0.133 |
| tn_worker_batch_requests_failed_total{worker="0",peer="rJNJunJgoDxtiNRocRMQRemyibjCubR4DyXDyFrMi1ua3cjripqHz64w7ovUwmNnnqA16FWkYCbZNHfkxdRgdeC3H2mFaZeffCqJuZTJncWfC7mr8ne6AFRhPn4va2wj2uv"} | rate | validator | 3 | 0 | 0.15 | 0.2 |
| tn_worker_batch_requests_failed_total{worker="0",peer="u6Ar9c81k4sdSWY82JbijYWSVKkR6xLfywxgd4yxZEkG2EBZPdeM6HKUX61c7iFz2fosBvn9RgsCFFEUqVMnfpd1dwkAr6P8xK3hXG33vv2xNrQn8cmePd51rGeZBqAzkge"} | rate | validator | 2 | 0.0066 | 0.167 | 0.222 |
| tn_worker_batch_requests_failed_total{worker="0",peer="u7ZWUHcAbH1aoouEbxvd9ofiRExo5kdcpZXN5kCyehc7Px9GJWaiefgLuNnHuBvEqWBCDbzhNXRjwVPv7Po19hZ8aGrrU4dhLJYE4PDXWV64nFtDtbSa44fWuX8qzdAyMin"} | rate | validator | 2 | 0.00296 | 0.1 | 0.133 |
| tn_worker_batch_requests_failed_total{worker="0",peer="xBZnXmyYqFi1mCwSH6KCF4f1W5xtdsXALBknGRb669vULDvaXqq89RXcW9hTfCi1SVABCSjavn3BWgLoeERA7SzM22Y7Lf74EvcRGa9X69mxReSJR9Lw4pX5VcM8Stdp1nz"} | rate | validator | 2 | 0.00396 | 0.1 | 0.133 |
| tn_worker_batch_requests_failed_total{worker="0",peer="ziGAo3Fw5vYzBuWHw8u8FUD72W3hE8p5kdW2Zd3NHmBWugD5UARi6jCW6aBGBo3UbMPgQwgCpBQ2nZWt52yk4PdQJtx7fBVkNTVqvjz8Lv2Y2iR8CcEaA59Au8atMzNJmks"} | rate | validator | 5 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="zixBdNoyd77oeFw5LFYX7YADGxbYGj1shubF8Xbt2F6pdVrdNmV2G4BLVaiaZ7Y4NGqjFSLFQyTV6U5YG9JivwgVutT71ouveXcP9oPe3srBzb4zL25KLhCGj4HybzSMGFt"} | rate | validator | 2 | 0.000988 | 0.0333 | 0.0444 |
| tn_node_epoch_record_signers | max | validator | 10 | 8.21 | 10 | 10 |
| tn_node_epoch_record_signers | max | observer | 1 | 8.65 | 9 | 9 |
| tn_node_epoch_record_missing_signers_total{signer="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="3j3KF4X3yr9q8fFenDb8U8pBoYzCRdcZnwSEo6EqCXbf"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau"} | rate | validator | 10 | 0 | 0.016 | 0.0214 |
| tn_node_epoch_record_missing_signers_total{signer="3rgB5qYVyKYbKTe9H9yChbYHCFzrNRAP8ne9VHYgoRau"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk"} | rate | validator | 10 | 0 | 0.016 | 0.0214 |
| tn_node_epoch_record_missing_signers_total{signer="4BzeDizv6RBRPHFiABmgio6TzJrgo61kQimHbVULGFrk"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="Db9g4C5zk7Ce5huvb1JzXZ7AVZphwyuaEXakuXRmHosX"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="ETvZruGgDgN4adQCn6WzZEf3KbWDa18x1ZxecCnwGBjg"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU"} | rate | validator | 10 | 0.000823 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="GQE9Et1BLw4UTXmadgkuV5WAR5fzYctMx5muaPvn7tEU"} | rate | observer | 1 | 0.000946 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="GhBcDwgtgFxi4EGQggUgQgT9T6Evq6AxXmXM7dUB7w62"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="Gq3UnvwZr5F4uXqKr2jwMFJF2b1Qp8BeN5UWeujxnnQH"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG"} | rate | validator | 10 | 0 | 0.0126 | 0.0168 |
| tn_node_epoch_record_missing_signers_total{signer="GyiPPV2w5XNa3c7a9JHCNqCybtGAiWh7uKJ9BNVEnFtG"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="HLXJoW6Dh4yCaobZkphz23F98wB43CHrVq9AUFDu8G3T"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="batch"} | p95 | validator | 4 | 18.4 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"} | p95 | validator | 7 | 14.5 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="vote"} | p95 | validator | 3 | 9,693 ms | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="response",class="batch"} | p95 | validator | 6 | 446 ms | 871 ms | 871 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"} | p95 | validator | 5 | 437 ms | 479 ms | 481 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"} | p95 | observer | 1 | 240 ms | 247 ms | 248 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="vote"} | p95 | validator | 5 | 407 ms | 454 ms | 459 ms |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_admission_mode | max | validator | 10 | 2 | 2 | 2 |
| tn_network_admission_mode | max | observer | 1 | 1.99 | 2 | 2 |
| tn_network_admission_fallback | max | validator | 10 | 0 | 1 | 1 |
| tn_network_admission_fallback | max | observer | 1 | 0.00617 | 0.25 | 1 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | validator | 10 | 11.8 | 82 | 98.6 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | validator | 10 | 2.29 | 14.1 | 15.6 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | observer | 1 | 0.48 | 1.01 | 1.45 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | validator | 10 | 0.517 | 3.07 | 3.4 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | observer | 1 | 0.00332 | 0.026 | 0.0372 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | validator | 10 | 0.0259 | 68.8 | 85.5 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | validator | 10 | 0.404 | 1.11 | 1.42 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | observer | 1 | 0.249 | 0.505 | 0.632 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | validator | 10 | 0.00957 | 0.2 | 0.222 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | observer | 1 | 0.0746 | 0.149 | 0.186 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | validator | 10 | 0.0442 | 0.227 | 0.276 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | validator | 10 | 0.22 | 38.6 | 48 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_request_service_seconds{class="batch"} | p95 | validator | 2 | 32 ms | 181 ms | 184 ms |
| tn_network_inbound_request_service_seconds{class="epoch_record"} | p95 | validator | 3 | 5 ms | 52 ms | 69 ms |
| tn_network_inbound_request_service_seconds{class="epoch_record"} | p95 | observer | 1 | 5 ms | 17 ms | 52 ms |
| tn_network_inbound_request_service_seconds{class="vote"} | p95 | validator | 6 | 40 ms | 47 ms | 47 ms |
| tn_network_established_connections | max | validator | 10 | 19.2 | 20 | 20 |
| tn_network_established_connections | max | observer | 1 | 19.7 | 20.5 | 21 |
| tn_worker_sync_streams_shed_total{reason="denied"} | rate | validator | 9 | 0.0115 | 0.533 | 0.711 |
| tn_worker_sync_streams_shed_total{reason="denied"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_fetch_duration_seconds | p95 | validator | 10 | 5 ms | 492 ms | 712 ms |
| tn_executor_batch_fetch_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_executor_batch_fetch_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_received_total | rate | validator | 10 | 16.5 | 17.3 | 17.3 |
| tn_primary_votes_received_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_request_failures_total | rate | validator | 10 | 0 | 0.15 | 0.2 |
| tn_primary_vote_request_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificates_formed_total | rate | validator | 10 | 2.76 | 2.89 | 2.89 |
| tn_primary_certificates_formed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificate_form_duration_seconds | p95 | validator | 10 | 243 ms | 490 ms | 490 ms |
| tn_engine_outputs_executed_total | rate | validator | 10 | 1.37 | 1.48 | 1.67 |
| tn_engine_outputs_executed_total | rate | observer | 1 | 1.36 | 1.49 | 1.53 |
| tn_engine_persist_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_engine_persist_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_consensus_sync_distance | max | validator | 10 | 17.6 | 506 | 507 |
| tn_node_consensus_sync_distance | max | observer | 1 | 16.7 | 29.2 | 30 |
| reth_db_table_size{table="AccountChangeSets"} | max | validator | 10 | 382 MiB | 568 MiB | 571 MiB |
| reth_db_table_size{table="AccountChangeSets"} | max | observer | 1 | 383 MiB | 566 MiB | 569 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | validator | 10 | 42 MiB | 76 MiB | 77 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | observer | 1 | 42 MiB | 76 MiB | 77 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | validator | 10 | 27 MiB | 44 MiB | 45 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | observer | 1 | 27 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="HashedStorages"} | max | validator | 10 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="HashedStorages"} | max | observer | 1 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | validator | 10 | 22 MiB | 37 MiB | 37 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | observer | 1 | 22 MiB | 37 MiB | 37 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | validator | 10 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | observer | 1 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | validator | 10 | 458 MiB | 689 MiB | 694 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | observer | 1 | 459 MiB | 687 MiB | 691 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | validator | 10 | 100 MiB | 116 MiB | 116 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | observer | 1 | 100 MiB | 116 MiB | 116 MiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | validator | 10 | 826 MiB | 1.23 GiB | 1.24 GiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | observer | 1 | 827 MiB | 1.23 GiB | 1.23 GiB |
| reth_db_table_size{table="TransactionSenders"} | max | validator | 10 | 425 MiB | 651 MiB | 655 MiB |
| reth_db_table_size{table="TransactionSenders"} | max | observer | 1 | 426 MiB | 649 MiB | 653 MiB |
| reth_db_freelist | max | validator | 10 | 28.1K | 53.9K | 58K |
| reth_db_freelist | max | observer | 1 | 28.1K | 53K | 56.5K |
| reth_database_transaction_commit_write_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_write_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | validator | 10 | 476 ms | 543 ms | 683 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | observer | 1 | 491 ms | 860 ms | 918 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | validator | 10 | 476 ms | 543 ms | 683 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | observer | 1 | 491 ms | 860 ms | 918 ms |
| tn_engine_recover_seconds | p95 | validator | 10 | 116 ms | 367 ms | 488 ms |
| tn_engine_recover_seconds | p95 | observer | 1 | 128 ms | 452 ms | 467 ms |
| tn_engine_evm_seconds | p95 | validator | 10 | 253 ms | 725 ms | 725 ms |
| tn_engine_evm_seconds | p95 | observer | 1 | 244 ms | 444 ms | 456 ms |
| tn_engine_root_seconds | p95 | validator | 10 | 358 ms | 901 ms | 1,038 ms |
| tn_engine_root_seconds | p95 | observer | 1 | 338 ms | 826 ms | 867 ms |
| tn_engine_finish_seconds | p95 | validator | 10 | 165 ms | 440 ms | 440 ms |
| tn_engine_finish_seconds | p95 | observer | 1 | 127 ms | 305 ms | 350 ms |
| tn_engine_forward_wait_seconds | p95 | validator | 10 | 9,865 ms | 29.0 s | 29.0 s |
| tn_engine_forward_wait_seconds | p95 | observer | 1 | 9,605 ms | 29.0 s | 29.0 s |
| tn_engine_channel_depth | max | validator | 10 | 8.31 | 20.2 | 21 |
| tn_engine_channel_depth | max | observer | 1 | 6.9 | 18.5 | 19 |
| tn_engine_executed_transactions_total | rate | validator | 10 | 5.48K | 13.8K | 13.8K |
| tn_engine_executed_transactions_total | rate | observer | 1 | 5.43K | 13.1K | 13.1K |
| tn_executor_execution_wait_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| tn_primary_commit_run_ahead_blocked_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_commit_run_ahead_blocked_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_consensus_exec_lag_rounds | max | validator | 10 | 35.5 | 60 | 62 |
| tn_primary_consensus_exec_lag_rounds | max | observer | 1 | 32.2 | 56.5 | 58 |
| tn_primary_commit_lag_rounds | max | validator | 10 | 2.09 | 3 | 3 |
| tn_primary_commit_lag_rounds | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_rejoin_grace_holds_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_rejoin_grace_holds_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_inactive_total | rate | validator | 10 | 0 | 33.3 | 33.4 |
| tn_primary_votes_refused_inactive_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_execution_behind_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_votes_refused_execution_behind_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_pool_age_seconds | p95 | validator | 10 | 5,413 ms | 30.0 s | 30.0 s |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="drained"} | rate | validator | 10 | 2.05 | 3.09 | 3.09 |
| tn_batch_builder_builds_total{reason="drained"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | validator | 10 | 1.07 | 2.83 | 3.49 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_report_seconds | p95 | validator | 10 | 0 ms | 1 ms | 4 ms |
| tn_primary_digest_wait_seconds | p95 | validator | 10 | 478 ms | 30.0 s | 30.0 s |
| tn_primary_headers_proposed_total{reason="max_delay"} | rate | validator | 10 | 0.00453 | 0.117 | 0.133 |
| tn_primary_headers_proposed_total{reason="min_delay"} | rate | validator | 10 | 2.75 | 2.87 | 2.87 |
| tn_primary_headers_proposed_total{reason="parents_and_threshold"} | rate | validator | 10 | 0 | 0.1 | 0.133 |
| tn_primary_digests_requeued_total{cause="canceled"} | rate | validator | 10 | 0 | 0.167 | 0.222 |
| tn_primary_digests_requeued_total{cause="committed"} | rate | validator | 10 | 0 | 0.928 | 1.38 |
| tn_executor_consensus_pack_persist_seconds | p95 | validator | 10 | 48 ms | 83 ms | 86 ms |
| tn_batch_builder_seal_duration_seconds | p95 | validator | 10 | 252 ms | 900 ms | 900 ms |
| tn_worker_quorum_wait_seconds | p95 | validator | 10 | 251 ms | 800 ms | 867 ms |
| tn_primary_proposer_pending_digests | max | validator | 10 | 0.549 | 6.25 | 24 |
| tn_primary_proposer_pending_digests | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_transactions | max | validator | 10 | 308 | 4.98K | 9.84K |
| reth_transaction_pool_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_pool_transactions | max | validator | 10 | 174 | 4.41K | 6.4K |
| reth_transaction_pool_queued_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_all_transactions_by_hash | max | validator | 10 | 839 | 9.9K | 10.2K |
| reth_transaction_pool_all_transactions_by_hash | max | observer | 1 | 0 | 0 | 0 |
| reth_storage_providers_database_save_blocks_total | p95 | validator | 10 | 120 ms | 320 ms | 374 ms |
| reth_storage_providers_database_save_blocks_total | p95 | observer | 1 | 109 ms | 215 ms | 240 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | validator | 10 | 120 ms | 320 ms | 374 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | observer | 1 | 109 ms | 215 ms | 240 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | validator | 10 | 28 ms | 60 ms | 69 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | observer | 1 | 31 ms | 66 ms | 78 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | validator | 10 | 363 ms | 493 ms | 525 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | observer | 1 | 403 ms | 551 ms | 565 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | validator | 10 | 10 ms | 16 ms | 16 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | observer | 1 | 8 ms | 9 ms | 10 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | validator | 10 | 21 ms | 46 ms | 51 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | observer | 1 | 19 ms | 38 ms | 44 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | validator | 10 | 14 ms | 26 ms | 29 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | observer | 1 | 13 ms | 23 ms | 24 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | validator | 10 | 19 ms | 35 ms | 44 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | observer | 1 | 17 ms | 30 ms | 33 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| tn_node_switch{name="digest_dedupe"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="digest_dedupe"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_record_state_check"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="epoch_record_state_check"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="parent_grace"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="parent_grace"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="quorum_exec_block"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="recovery_threads"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="recovery_threads"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="vote_queue_fix"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="vote_queue_fix"} | max | observer | 1 | 1 | 1 | 1 |
| tn_primary_epoch_entry_rounds_behind | max | validator | 10 | 3 | 29 | 29 |
| tn_primary_epoch_entry_rounds_behind | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_random_advice_bytes | max | validator | 10 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_random_advice_bytes | max | observer | 1 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_used_bytes | max | validator | 10 | 2.44 GiB | 3.65 GiB | 3.65 GiB |
| tn_reth_mdbx_used_bytes | max | observer | 1 | 2.45 GiB | 3.62 GiB | 3.65 GiB |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | validator | 10 | 36.8M | 61.2M | 61.6M |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | observer | 1 | 29.1M | 54.3M | 54.7M |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | validator | 10 | 6.49K | 9.76K | 9.83K |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | observer | 1 | 5.24K | 8.8K | 8.86K |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | validator | 10 | 637K | 1.21M | 1.24M |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | observer | 1 | 515K | 850K | 864K |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | validator | 10 | 640K | 956K | 957K |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | observer | 1 | 518K | 857K | 867K |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | validator | 10 | 1 | 1 | 1 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | observer | 1 | 1 | 1 | 1 |
| tn_reth_sealed_txs_marked | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_marked | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_size_bytes | max | validator | 10 | 110.0 KiB | 1.7 MiB | 3.4 MiB |
| reth_transaction_pool_pending_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_queued_pool_size_bytes | max | validator | 10 | 61.9 KiB | 1.5 MiB | 1.8 MiB |
| reth_transaction_pool_queued_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_pending_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_pending_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_removed_transactions | max | validator | 10 | 1.4M | 2.39M | 2.4M |
| reth_transaction_pool_removed_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | observer | 1 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_close_backlog_outputs | max | validator | 10 | 15 | 26 | 26 |
| tn_node_epoch_close_backlog_outputs | max | observer | 1 | 7.68 | 9 | 9 |
| tn_node_epoch_close_detect_lag_seconds | max | validator | 10 | 637 ms | 343.4 s | 343.4 s |
| tn_node_epoch_close_detect_lag_seconds | max | observer | 1 | 1,128 ms | 1,204 ms | 1,204 ms |
| tn_node_epoch_close_drain_seconds | max | validator | 10 | 10.2 s | 20.5 s | 20.5 s |
| tn_node_epoch_close_drain_seconds | max | observer | 1 | 5,097 ms | 6,386 ms | 6,386 ms |
| tn_node_epoch_record_vote_skipped_total{reason="certified_before_close"} | max | validator | 1 | 1 | 1 | 1 |
| tn_node_epoch_record_vote_skipped_total{reason="not_in_committee"} | max | observer | 1 | 4.09 | 5 | 5 |
| tn_primary_parent_grace_total{outcome="complete"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="expired"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="preempted"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_late_parents_total{origin="older_round"} | max | validator | 10 | 5.28K | 10.3K | 10.4K |
| tn_primary_late_parents_total{origin="own_round_behind"} | max | validator | 10 | 31.2K | 41K | 41K |
| tn_node_execution_backlog_outputs | max | validator | 10 | 18.2 | 30.5 | 31 |
| tn_node_execution_backlog_outputs | max | observer | 1 | 16.8 | 28.5 | 29 |
| tn_engine_unpersisted_outputs | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_unpersisted_outputs | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_built_height | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_built_height | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | observer | 1 | 0 | 0 | 0 |
tn_primary_commit_latency_seconds
tn_primary_round − tn_primary_committed_round
tn_engine_queued_outputs, max by node
The share of each validator's own headers that formed a certificate, per window (tools/cert-rate.py on the node logs). Every validator certified at least 90 % of its own headers in every window where it proposed at least 50. Own leader % is the validator's share of the committed leaders and not committed % the share of its certificates no committed leader reached, both over the run window; the fair share is 100 / 10 validators = 10.0 %. Greyed: fewer than 50 headers proposed, not judged.
| Validator | Region | blast 0 preamble | blast 0 warmup | blast 0 mixed | blast 0 cooldown | blast 1 preamble | blast 1 warmup | blast 1 mixed | blast 1 cooldown | blast 2 preamble | blast 2 warmup | blast 2 mixed | blast 2 cooldown | run | Own leader % | Not committed % |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-validator-01 | us-west2 (Los Angeles) | 100.0 | 100.4 | 100.1 | 100.0 | 100.0 | 100.0 | 99.9 | 99.6 | 101.0 | 100.0 | 100.0 | 100.0 | 100.0 | 13.6 | 0.0 |
| bench-validator-02 | europe-west2 (London) | 100.0 | 100.0 | 99.3 | 99.4 | 100.6 | 99.7 | 99.8 | 100.4 | 100.0 | 100.0 | 99.9 | 99.5 | 99.7 | 0.5 | 72.0 |
| bench-validator-03 | australia-southeast1 (Sydney) | 100.0 | 100.0 | 100.1 | 100.6 | 99.4 | 100.0 | 100.1 | 100.0 | 99.0 | 100.0 | 100.0 | 100.0 | 100.0 | – | – |
| bench-validator-04 | asia-northeast1 (Tokyo) | 100.0 | 100.0 | 100.1 | 100.6 | 99.4 | 100.0 | 100.0 | 99.6 | 100.0 | 100.0 | 100.0 | 99.5 | 100.0 | – | – |
| bench-validator-05 | us-east4 (Ashburn) | 100.0 | 100.0 | 99.8 | 100.0 | 100.0 | 100.0 | 99.6 | 100.0 | 99.0 | 100.0 | 99.9 | – | 99.8 | 8.5 | 12.1 |
| bench-validator-06 | europe-west4 (Netherlands) | 98.6 | 99.2 | 99.2 | 99.4 | 100.0 | 100.0 | 99.4 | 100.0 | 100.0 | 100.0 | 99.7 | 99.5 | 99.6 | 0.3 | 75.0 |
| bench-validator-07 | asia-southeast1 (Singapore) | 100.0 | 100.4 | 100.1 | 99.4 | 100.0 | 99.7 | 100.0 | 100.0 | 100.0 | 100.2 | 100.0 | 99.5 | 100.0 | 13.7 | 0.0 |
| bench-validator-08 | asia-east2 (Hong Kong) | 100.0 | 99.6 | 100.1 | 100.6 | 99.4 | 100.3 | 100.0 | 100.0 | 100.0 | 100.2 | 100.0 | 99.5 | 100.0 | – | – |
| bench-validator-09 | asia-northeast3 (Seoul) | 100.0 | 99.6 | 100.1 | 100.6 | 100.0 | 100.0 | 99.9 | 100.0 | 101.0 | 100.2 | 100.0 | 99.5 | 100.0 | – | – |
| bench-validator-10 | asia-south1 (Mumbai) | 98.6 | 99.6 | 100.0 | 100.0 | 99.4 | 99.7 | 99.9 | 100.0 | 100.0 | 100.0 | 99.9 | 99.5 | 99.9 | 11.7 | 4.8 |
| Node | Class | Peak RssAnon | Peak RssFile | Peak cache | PSI mem | PSI io | PSI cpu | db growth | static_files growth | Root used | OOM Δ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 1010 MiB | 3.55 GiB | – | 0.0 % | 13.7 % | 1.0 % | 0 B | 4.08 GiB | 16.0 % | 0 |
| bench-validator-01 | validator | 2.36 GiB | 3.23 GiB | – | 0.0 % | 13.6 % | 8.7 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-02 | validator | 2.47 GiB | 3.65 GiB | – | 0.0 % | 14.3 % | 8.1 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-03 | validator | 2.47 GiB | 3.82 GiB | – | 0.0 % | 12.9 % | 9.3 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-04 | validator | 2.53 GiB | 3.51 GiB | – | 0.0 % | 14.2 % | 8.8 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-05 | validator | 2.27 GiB | 3.93 GiB | – | 0.0 % | 11.9 % | 6.0 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-06 | validator | 2.70 GiB | 3.80 GiB | – | 0.0 % | 14.8 % | 6.1 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-07 | validator | 1.98 GiB | 3.82 GiB | – | 0.0 % | 14.4 % | 6.4 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-08 | validator | 2.54 GiB | 3.63 GiB | – | 0.0 % | 14.9 % | 6.9 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-09 | validator | 2.00 GiB | 3.78 GiB | – | 0.0 % | 14.3 % | 5.6 % | 0 B | 4.08 GiB | 18.0 % | 0 |
| bench-validator-10 | validator | 2.53 GiB | 3.90 GiB | – | 0.0 % | 13.4 % | 8.3 % | 0 B | 4.08 GiB | 18.0 % | 0 |
/proc/<pid>/status of the telcoin process on the node with the largest RSS
Not collected: consensus-db/cache size.
/proc/pressure on the node with the highest memory PSI
Per role and per node, from sizing.json (p95 over the landed window; PSI and RssAnon max over the whole sampler run). The lines follow the hardware doc's tier rules: CPU p95 over 70 % of vCPUs; memory used p95 over 75 % of RAM, or telcoin RSS p95 over 85 % of RAM; disk write p95 over 60 % of the PD MiB/s limit, or write IOPS p95 over 60 % of the IOPS limit; IO pressure over 20 % or memory pressure over 5 % (PSI some avg10, max); network p95 over 50 % of 1 Gbps; root disk over 80 %. A line crossed on any node puts the role at the limit; a nonzero reading within 15 points of its line is tight; otherwise the role has headroom. OOM kills, disk throttling and degraded disk performance are flagged too.
| Signal | Class | Worst node | Value | Share of capacity | Line | Nodes over / tight | Status |
|---|---|---|---|---|---|---|---|
| CPU p95 | validator | bench-validator-03 | 36.0 % | 36.0 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | validator | bench-validator-06 | 22.7 % | 22.7 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | validator | bench-validator-06 | 6.87 GB | 40.0 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | validator | bench-validator-01 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | validator | bench-validator-06 | 2.90 GB | 16.9 % | info | – | info |
| Disk write p95 | validator | bench-validator-03 | 110 MB/s | 13.1 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | validator | bench-validator-06 | 20,146 | 100.7 % | > 60 % | 10 / 0 | at the limit |
| PSI io max | validator | bench-validator-08 | 14.9 % | 14.9 % | > 20 % | 0 / 10 | tight |
| Root disk max | validator | bench-validator-03 | 17.5 % | 17.5 % | > 80 % | 0 / 0 | headroom |
| Network p95 | validator | bench-validator-07 | 130 Mbps | 13.0 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | validator | – | – | – | info | – | not collected |
| OOM kills | validator | – | – | any | any | – | none |
| Disk throttling | validator | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | validator | – | – | any sample | any | – | none |
| CPU p95 | observer | bench-observer-11 | 20.4 % | 20.4 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | observer | bench-observer-11 | 12.4 % | 12.4 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | observer | bench-observer-11 | 4.54 GB | 26.4 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | observer | bench-observer-11 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | observer | bench-observer-11 | 1.06 GB | 6.2 % | info | – | info |
| Disk write p95 | observer | bench-observer-11 | 97 MB/s | 11.5 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | observer | bench-observer-11 | 20,071 | 100.4 % | > 60 % | 1 / 0 | at the limit |
| PSI io max | observer | bench-observer-11 | 13.7 % | 13.7 % | > 20 % | 0 / 1 | tight |
| Root disk max | observer | bench-observer-11 | 15.9 % | 15.9 % | > 80 % | 0 / 0 | headroom |
| Network p95 | observer | bench-observer-11 | 12 Mbps | 1.2 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | observer | – | – | – | info | – | not collected |
| OOM kills | observer | – | – | any | any | – | none |
| Disk throttling | observer | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | observer | – | – | any sample | any | – | none |
| Node | Class | CPU p95 | Memory used p95 | telcoin RSS p95 | PSI memory max | RssAnon peak | Disk write p95 | Write IOPS p95 | PSI io max | Root disk max | Network p95 | Consensus cache peak |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Line | > 70 % | > 75 % | > 85 % | > 5 % | info | > 60 % | > 60 % | > 20 % | > 80 % | > 50 % | info | |
| bench-validator-01 | validator | 35.3 % | 20.7 % | 5.89 GB 34 % | 0.0 % | 2.53 GB 15 % | 109 MB/s 13 % | 20,103 101 % | 13.6 % | 17.5 % | 111 Mbps 11 % | – |
| bench-validator-02 | validator | 33.6 % | 21.3 % | 6.35 GB 37 % | 0.0 % | 2.66 GB 15 % | 109 MB/s 13 % | 20,123 101 % | 14.3 % | 17.5 % | 102 Mbps 10 % | – |
| bench-validator-03 | validator | 36.0 % | 21.3 % | 6.52 GB 38 % | 0.0 % | 2.65 GB 15 % | 110 MB/s 13 % | 20,144 101 % | 12.9 % | 17.5 % | 109 Mbps 11 % | – |
| bench-validator-04 | validator | 35.3 % | 21.7 % | 6.26 GB 36 % | 0.0 % | 2.72 GB 16 % | 109 MB/s 13 % | 20,093 100 % | 14.2 % | 17.5 % | 107 Mbps 11 % | – |
| bench-validator-05 | validator | 34.9 % | 20.2 % | 5.67 GB 33 % | 0.0 % | 2.43 GB 14 % | 108 MB/s 13 % | 20,113 101 % | 11.9 % | 16.9 % | 107 Mbps 11 % | – |
| bench-validator-06 | validator | 34.8 % | 22.7 % | 6.87 GB 40 % | 0.0 % | 2.90 GB 17 % | 109 MB/s 13 % | 20,146 101 % | 14.8 % | 17.3 % | 104 Mbps 10 % | – |
| bench-validator-07 | validator | 35.1 % | 18.3 % | 5.92 GB 34 % | 0.0 % | 2.13 GB 12 % | 108 MB/s 13 % | 20,101 101 % | 14.4 % | 17.5 % | 130 Mbps 13 % | – |
| bench-validator-08 | validator | 35.0 % | 21.6 % | 6.35 GB 37 % | 0.0 % | 2.73 GB 16 % | 108 MB/s 13 % | 20,084 100 % | 14.9 % | 17.5 % | 112 Mbps 11 % | – |
| bench-validator-09 | validator | 34.1 % | 18.5 % | 5.96 GB 35 % | 0.0 % | 2.15 GB 12 % | 108 MB/s 13 % | 20,050 100 % | 14.3 % | 17.5 % | 111 Mbps 11 % | – |
| bench-validator-10 | validator | 35.4 % | 21.6 % | 6.72 GB 39 % | 0.0 % | 2.72 GB 16 % | 108 MB/s 13 % | 20,096 100 % | 13.4 % | 17.5 % | 111 Mbps 11 % | – |
| bench-observer-11 | observer | 20.4 % | 12.4 % | 4.54 GB 26 % | 0.0 % | 1.06 GB 6 % | 97 MB/s 12 % | 20,071 100 % | 13.7 % | 15.9 % | 12 Mbps 1 % | – |
Small figures are the share of capacity: RAM for RSS, the PD limit for disk write (VM cap applied), 1 Gbps for network. Hover a cell for its JSON path.
A = Fleet c3-hdb · c3-highcpu-8 · r20261113-1351
B = Fleet c3-hdb · c3-highcpu-8 · r20261107-1504
at the limit c3-highcpu-8 validators at 8,065 sustained TPS: at the limit on disk. Disk at the limit: write IOPS p95 20,050–20,146, up to 100.7 % of the 20,000 IOPS limit, over the 60 % line on all 10 validators (bench-validator-01, 02, 03, 04, 05, 06, 07, 08, 09, 10); IO pressure (PSI some avg10) max 14.9 % (bench-validator-08), tight against the 20 % line. CPU has headroom: CPU p95 36.0 % of 8 vCPUs (bench-validator-03). Memory has headroom: memory used p95 22.7 % of RAM (bench-validator-06); telcoin RSS p95 6.87 GB, 40.0 % of 16 GiB RAM (bench-validator-06); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 130 Mbps, 13.0 % of 1 Gbps (bench-validator-07). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
at the limit c3-highcpu-8 observers at 8,065 sustained TPS: at the limit on disk. Disk at the limit: write IOPS p95 20,071, 100.4 % of the 20,000 IOPS limit on bench-observer-11, over the 60 % line; IO pressure (PSI some avg10) max 13.7 % (bench-observer-11), tight against the 20 % line. CPU has headroom: CPU p95 20.4 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 12.4 % of RAM (bench-observer-11); telcoin RSS p95 4.54 GB, 26.4 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 12 Mbps, 1.2 % of 1 Gbps (bench-observer-11). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
at the limit c3-highcpu-8 validators at 4,939 sustained TPS: at the limit on disk. Disk at the limit: write p95 547–595 MB/s, up to 70.9 % of the 800 MiB/s limit, over the 60 % line on all 10 validators (bench-validator-01, 02, 03, 04, 05, 06, 07, 08, 09, 10); IO pressure (PSI some avg10) max 29.4–34.2 %, over the 20 % line on all 10 validators (bench-validator-01, 02, 03, 04, 05, 06, 07, 08, 09, 10). CPU has headroom: CPU p95 22.2 % of 8 vCPUs (bench-validator-08). Memory has headroom: memory used p95 34.2 % of RAM (bench-validator-02); telcoin RSS p95 8.11 GB, 47.2 % of 16 GiB RAM (bench-validator-02); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 82 Mbps, 8.2 % of 1 Gbps (bench-validator-08). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
at the limit c3-highcpu-8 observers at 4,939 sustained TPS: at the limit on disk. Disk at the limit: write p95 588 MB/s, 70.1 % of the 800 MiB/s limit on bench-observer-11, over the 60 % line; IO pressure (PSI some avg10) max 32.8 % on bench-observer-11, over the 20 % line. CPU has headroom: CPU p95 13.2 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 11.7 % of RAM (bench-observer-11); telcoin RSS p95 3.79 GB, 22.1 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 4 Mbps, 0.4 % of 1 Gbps (bench-observer-11). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
Validators: Fleet A (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. The same disk limit binds both fleets: every validator runs a 100 GB hyperdisk-balanced, which allows 20,000 IOPS on both; its throughput limit is 800 MiB/s on both. Disk is at the limit on both (Fleet A (c3-highcpu-8): write IOPS p95 20,146, 100.7 % of the 20,000 IOPS limit; Fleet B (c3-highcpu-8): write p95 595 MB/s, 70.9 % of the 800 MiB/s limit, IO pressure (PSI some avg10) max 34.2 %). Changing the machine shape did not clear it; the volume is the binding resource.
Observers: Fleet A (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. The same disk limit binds both fleets: every observer runs a 100 GB hyperdisk-balanced, which allows 20,000 IOPS on both; its throughput limit is 800 MiB/s on both. Disk is at the limit on both (Fleet A (c3-highcpu-8): write IOPS p95 20,071, 100.4 % of the 20,000 IOPS limit; Fleet B (c3-highcpu-8): write p95 588 MB/s, 70.1 % of the 800 MiB/s limit, IO pressure (PSI some avg10) max 32.8 %). Changing the machine shape did not clear it; the volume is the binding resource.
| KPI | Fleet A | Fleet B | B ÷ A |
|---|---|---|---|
| Sustained mixed TPS (own) | 8,065.4 tx/s | 4,939.2 tx/s | 0.61× |
| Sustained mixed TPS (chain) | 8,065.4 tx/s | 4,939.2 tx/s | 0.61× |
| Landed / accepted | 100.0 % | 99.6 % | 1.00× |
| Reverted (own) | 0 | 0 | – |
| Latency p50 | 679 ms | 2,809 ms | 4.13× |
| Latency p95 | 14,662 ms | 14,404 ms | 0.98× |
| Gas / s (own) | 269M gas/s | 174M gas/s | 0.65× |
| Capacity at mix | 14,103.0 tx/s | 7,705.4 tx/s | 0.55× |
| Signal | Class | Fleet A | Fleet B | Line |
|---|---|---|---|---|
| CPU p95 | validator | 36.0 % headroom | 22.2 % headroom | > 70 % |
| Memory used p95 | validator | 22.7 % headroom | 34.2 % headroom | > 75 % |
| telcoin RSS p95 | validator | 6.87 GB 40.0 % headroom | 8.11 GB 47.2 % headroom | > 85 % |
| PSI memory max | validator | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | validator | 110 MB/s 13.1 % headroom | 595 MB/s 70.9 % at the limit | > 60 % |
| Write IOPS p95 | validator | 20,146 100.7 % at the limit | 8,909 44.5 % headroom | > 60 % |
| PSI io max | validator | 14.9 % tight | 34.2 % at the limit | > 20 % |
| Root disk max | validator | 17.5 % headroom | 15.6 % headroom | > 80 % |
| Network p95 | validator | 130 Mbps 13.0 % headroom | 82 Mbps 8.2 % headroom | > 50 % |
| CPU p95 | observer | 20.4 % headroom | 13.2 % headroom | > 70 % |
| Memory used p95 | observer | 12.4 % headroom | 11.7 % headroom | > 75 % |
| telcoin RSS p95 | observer | 4.54 GB 26.4 % headroom | 3.79 GB 22.1 % headroom | > 85 % |
| PSI memory max | observer | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | observer | 97 MB/s 11.5 % headroom | 588 MB/s 70.1 % at the limit | > 60 % |
| Write IOPS p95 | observer | 20,071 100.4 % at the limit | 5,206 26.0 % headroom | > 60 % |
| PSI io max | observer | 13.7 % tight | 32.8 % at the limit | > 20 % |
| Root disk max | observer | 15.9 % headroom | 13.2 % headroom | > 80 % |
| Network p95 | observer | 12 Mbps 1.2 % headroom | 4 Mbps 0.4 % headroom | > 50 % |
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
Not collected: reference chains.