Throughput is measured from blocks the chain produced (own senders vs every sender), resource use from Cloud Monitoring, Prometheus and the on-node sampler. Summaries cover the mixed phase. Hover any number for the JSON field it came from.
Landed counts only our own senders; latency is submit → block timestamp.
| Type | Submitted | Accepted | Landed | Reverted | Unlanded | Skips | Gas limit | Gas used avg | p50 | p95 |
|---|---|---|---|---|---|---|---|---|---|---|
| native_transfer | 5.52M | 5.39M | 5.38M | 0 | 5.67K | 0 | 24.1K | 21K | 984 ms | 3,867 ms |
| batched_self_send | 3.06M | 2.96M | 2.95M | 0 | 4.89K | 0 | 24.1K | 21K | 1,028 ms | 3,912 ms |
| uniswap_v2_swap | 2.75M | 2.65M | 2.65M | 0 | 4.12K | 0 | 180K | 106K | 985 ms | 3,870 ms |
| stablecoin_transfer | 2.28M | 2.21M | 2.2M | 0 | 3.35K | 0 | 75K | 43.9K | 984 ms | 3,865 ms |
| wtel_wrap | 1.83M | 1.76M | 1.76M | 0 | 2.71K | 0 | 50K | 27.9K | 984 ms | 3,863 ms |
| wtel_unwrap | 1.22M | 1.18M | 1.18M | 0 | 1.84K | 0 | 45K | 35.2K | 983 ms | 3,866 ms |
| stablecoin_approve | 912K | 880K | 879K | 0 | 1.31K | 0 | 60K | 31.8K | 984 ms | 3,862 ms |
| uniswap_v2_add_liquidity | 608K | 587K | 587K | 0 | 851 | 0 | 250K | 128K | 983 ms | 3,866 ms |
| uniswap_v4_swap | 457K | 442K | 441K | 0 | 719 | 0 | 250K | 124K | 985 ms | 3,862 ms |
| contract_deploy | 454K | 439K | 438K | 0 | 686 | 0 | 140K | 74.2K | 984 ms | 3,861 ms |
| uniswap_v2_remove_liquidity | 305K | 295K | 294K | 0 | 432 | 0 | 220K | 129K | 986 ms | 3,876 ms |
| uniswap_v3_swap | 304K | 294K | 293K | 0 | 459 | 0 | 220K | 120K | 985 ms | 3,866 ms |
| faucet_drip | 126K | 126K | 126K | 0 | 14 | 634K | 150K | 107K | 994 ms | 7,610 ms |
| Node | Submitted | Accepted | Accepted % | Top errors | Send p50 | Send p95 |
|---|---|---|---|---|---|---|
| bench-validator-01 bench-validator-01:8545 | 2.79M | 2.78M | 99.6 % | known 6.3K | 6 ms | 20 ms |
| bench-validator-02 bench-validator-02:8545 | 1.01M | 954K | 94.8 % | other 36.4K, known 7.53K | 5 ms | 28 ms |
| bench-validator-03 bench-validator-03:8545 | 2.77M | 2.76M | 99.5 % | known 6.96K | 6 ms | 21 ms |
| bench-validator-04 bench-validator-04:8545 | 2.11M | 2.11M | 99.9 % | known 621 | 6 ms | 22 ms |
| bench-validator-05 bench-validator-05:8545 | 2.37M | 548K | 23.1 % | other 1.82M, known 836 | 1 ms | 11 ms |
| bench-validator-06 bench-validator-06:8545 | 1.05M | 994K | 95.0 % | known 26K | 6 ms | 29 ms |
| bench-validator-07 bench-validator-07:8545 | 3.02M | 3M | 99.3 % | known 12K | 7 ms | 24 ms |
| bench-validator-08 bench-validator-08:8545 | 2.6M | 2.57M | 98.5 % | known 21.1K | 6 ms | 19 ms |
| bench-validator-09 bench-validator-09:8545 | 2.06M | 2.01M | 97.6 % | known 26.9K | 4 ms | 16 ms |
| bench-validator-10 bench-validator-10:8545 | 1.6M | 1.55M | 97.1 % | known 21.5K | 6 ms | 22 ms |
Shaded: phases outside the mixed window. Hairlines: epoch closes. Dashed orange: chaos events.
own senders vs chain total, per block-timestamp second
own vs chain gas used, per block-timestamp second
compute.googleapis.com/instance/cpu/utilization, 60-s mean, % of all vCPUs; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 22.5 % | 24.2 % | 27.0 % | 27.0 % |
| bench-validator-01 | validator | 39.6 % | 41.9 % | 51.5 % | 51.5 % |
| bench-validator-02 | validator | 35.7 % | 38.4 % | 48.7 % | 48.7 % |
| bench-validator-03 | validator | 39.8 % | 43.3 % | 50.4 % | 50.4 % |
| bench-validator-04 | validator | 38.3 % | 39.8 % | 51.5 % | 51.5 % |
| bench-validator-05 | validator | 31.9 % | 39.4 % | 49.4 % | 49.4 % |
| bench-validator-06 | validator | 37.4 % | 39.4 % | 48.8 % | 48.8 % |
| bench-validator-07 | validator | 40.0 % | 42.7 % | 50.2 % | 50.2 % |
| bench-validator-08 | validator | 39.3 % | 42.7 % | 49.7 % | 49.7 % |
| bench-validator-09 | validator | 38.5 % | 42.5 % | 49.6 % | 49.6 % |
| bench-validator-10 | validator | 38.0 % | 40.6 % | 50.6 % | 50.6 % |
One row per node, over the mixed phase.
agent processes/rss_usage summed over telcoin pids; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 3.42 GiB | 4.86 GiB | 5.14 GiB | 5.14 GiB |
| bench-validator-01 | validator | 5.19 GiB | 6.14 GiB | 6.17 GiB | 6.17 GiB |
| bench-validator-02 | validator | 5.44 GiB | 6.39 GiB | 6.47 GiB | 6.47 GiB |
| bench-validator-03 | validator | 5.13 GiB | 6.13 GiB | 6.19 GiB | 6.19 GiB |
| bench-validator-04 | validator | 5.09 GiB | 6.11 GiB | 6.19 GiB | 6.19 GiB |
| bench-validator-05 | validator | 4.82 GiB | 7.12 GiB | 7.27 GiB | 7.27 GiB |
| bench-validator-06 | validator | 5.33 GiB | 6.49 GiB | 6.54 GiB | 6.54 GiB |
| bench-validator-07 | validator | 5.07 GiB | 6.03 GiB | 6.16 GiB | 6.16 GiB |
| bench-validator-08 | validator | 5.38 GiB | 6.40 GiB | 6.46 GiB | 6.46 GiB |
| bench-validator-09 | validator | 5.26 GiB | 6.24 GiB | 6.29 GiB | 6.29 GiB |
| bench-validator-10 | validator | 5.16 GiB | 6.07 GiB | 6.15 GiB | 6.15 GiB |
One row per node, over the mixed phase.
agent memory/percent_used, state=used; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 10.0 % | 11.1 % | 12.2 % | 12.2 % |
| bench-validator-01 | validator | 20.1 % | 21.0 % | 22.0 % | 22.0 % |
| bench-validator-02 | validator | 21.6 % | 22.6 % | 22.6 % | 22.6 % |
| bench-validator-03 | validator | 20.2 % | 21.2 % | 21.8 % | 21.8 % |
| bench-validator-04 | validator | 19.7 % | 20.6 % | 21.1 % | 21.1 % |
| bench-validator-05 | validator | 16.0 % | 19.6 % | 19.7 % | 19.7 % |
| bench-validator-06 | validator | 22.0 % | 25.1 % | 25.5 % | 25.5 % |
| bench-validator-07 | validator | 19.6 % | 20.5 % | 21.1 % | 21.1 % |
| bench-validator-08 | validator | 21.7 % | 22.9 % | 23.4 % | 23.4 % |
| bench-validator-09 | validator | 20.9 % | 22.2 % | 23.0 % | 23.0 % |
| bench-validator-10 | validator | 20.5 % | 21.5 % | 23.0 % | 23.0 % |
One row per node, over the mixed phase.
instance/disk/{read,write}_bytes_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 107.7 MiB/s | 116.2 MiB/s | 119.4 MiB/s | 119.4 MiB/s | 800.0 MiB/s |
| bench-validator-01 | validator | 118.3 MiB/s | 125.9 MiB/s | 128.3 MiB/s | 128.3 MiB/s | 800.0 MiB/s |
| bench-validator-02 | validator | 114.0 MiB/s | 121.8 MiB/s | 123.3 MiB/s | 123.3 MiB/s | 800.0 MiB/s |
| bench-validator-03 | validator | 118.5 MiB/s | 125.8 MiB/s | 127.0 MiB/s | 127.0 MiB/s | 800.0 MiB/s |
| bench-validator-04 | validator | 117.5 MiB/s | 124.4 MiB/s | 126.7 MiB/s | 126.7 MiB/s | 800.0 MiB/s |
| bench-validator-05 | validator | 98.1 MiB/s | 122.7 MiB/s | 126.1 MiB/s | 126.1 MiB/s | 800.0 MiB/s |
| bench-validator-06 | validator | 118.5 MiB/s | 127.0 MiB/s | 128.8 MiB/s | 128.8 MiB/s | 800.0 MiB/s |
| bench-validator-07 | validator | 118.7 MiB/s | 126.6 MiB/s | 127.4 MiB/s | 127.4 MiB/s | 800.0 MiB/s |
| bench-validator-08 | validator | 118.8 MiB/s | 127.3 MiB/s | 129.0 MiB/s | 129.0 MiB/s | 800.0 MiB/s |
| bench-validator-09 | validator | 118.7 MiB/s | 125.8 MiB/s | 130.7 MiB/s | 130.7 MiB/s | 800.0 MiB/s |
| bench-validator-10 | validator | 118.6 MiB/s | 126.5 MiB/s | 127.8 MiB/s | 127.8 MiB/s | 800.0 MiB/s |
instance/disk/{read,write}_ops_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 23.8K IOPS | 26.1K IOPS | 27K IOPS | 27K IOPS | 50K IOPS |
| bench-validator-01 | validator | 23.9K IOPS | 26.2K IOPS | 26.8K IOPS | 26.8K IOPS | 50K IOPS |
| bench-validator-02 | validator | 23.1K IOPS | 25.6K IOPS | 26K IOPS | 26K IOPS | 50K IOPS |
| bench-validator-03 | validator | 23.9K IOPS | 26.1K IOPS | 26.4K IOPS | 26.4K IOPS | 50K IOPS |
| bench-validator-04 | validator | 23.8K IOPS | 26K IOPS | 26.1K IOPS | 26.1K IOPS | 50K IOPS |
| bench-validator-05 | validator | 19.7K IOPS | 25K IOPS | 25.4K IOPS | 25.4K IOPS | 50K IOPS |
| bench-validator-06 | validator | 23.9K IOPS | 26.3K IOPS | 26.9K IOPS | 26.9K IOPS | 50K IOPS |
| bench-validator-07 | validator | 23.9K IOPS | 26.1K IOPS | 26.5K IOPS | 26.5K IOPS | 50K IOPS |
| bench-validator-08 | validator | 23.9K IOPS | 26.4K IOPS | 26.7K IOPS | 26.7K IOPS | 50K IOPS |
| bench-validator-09 | validator | 23.9K IOPS | 26K IOPS | 27.1K IOPS | 27.1K IOPS | 50K IOPS |
| bench-validator-10 | validator | 23.9K IOPS | 26K IOPS | 26.5K IOPS | 26.5K IOPS | 50K IOPS |
| Node | Machine | vCPU | Disk | IOPS limit | MiB/s limit | Basis |
|---|---|---|---|---|---|---|
| bench-observer-11 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-01 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-02 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-03 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-04 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-05 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-06 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-07 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-08 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-09 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-10 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
instance/network/sent_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 4.7 Mbps | 20.8 Mbps | 39.7 Mbps | 39.7 Mbps |
| bench-validator-01 | validator | 132.0 Mbps | 138.3 Mbps | 142.7 Mbps | 142.7 Mbps |
| bench-validator-02 | validator | 120.2 Mbps | 129.2 Mbps | 134.2 Mbps | 134.2 Mbps |
| bench-validator-03 | validator | 133.9 Mbps | 143.6 Mbps | 175.5 Mbps | 175.5 Mbps |
| bench-validator-04 | validator | 128.9 Mbps | 133.6 Mbps | 140.0 Mbps | 140.0 Mbps |
| bench-validator-05 | validator | 104.2 Mbps | 126.0 Mbps | 130.9 Mbps | 130.9 Mbps |
| bench-validator-06 | validator | 125.4 Mbps | 131.6 Mbps | 134.4 Mbps | 134.4 Mbps |
| bench-validator-07 | validator | 136.8 Mbps | 149.2 Mbps | 184.0 Mbps | 184.0 Mbps |
| bench-validator-08 | validator | 132.2 Mbps | 141.8 Mbps | 151.8 Mbps | 151.8 Mbps |
| bench-validator-09 | validator | 129.2 Mbps | 137.8 Mbps | 142.3 Mbps | 142.3 Mbps |
| bench-validator-10 | validator | 127.6 Mbps | 138.5 Mbps | 142.1 Mbps | 142.1 Mbps |
instance/network/received_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 8.4 Mbps | 16.0 Mbps | 29.0 Mbps | 29.0 Mbps |
| bench-validator-01 | validator | 13.8 Mbps | 17.5 Mbps | 20.8 Mbps | 20.8 Mbps |
| bench-validator-02 | validator | 12.2 Mbps | 17.6 Mbps | 67.1 Mbps | 67.1 Mbps |
| bench-validator-03 | validator | 14.0 Mbps | 17.7 Mbps | 19.4 Mbps | 19.4 Mbps |
| bench-validator-04 | validator | 12.6 Mbps | 13.7 Mbps | 20.3 Mbps | 20.3 Mbps |
| bench-validator-05 | validator | 21.1 Mbps | 54.3 Mbps | 85.7 Mbps | 85.7 Mbps |
| bench-validator-06 | validator | 10.8 Mbps | 12.6 Mbps | 17.7 Mbps | 17.7 Mbps |
| bench-validator-07 | validator | 14.4 Mbps | 16.6 Mbps | 20.1 Mbps | 20.1 Mbps |
| bench-validator-08 | validator | 13.7 Mbps | 17.7 Mbps | 19.5 Mbps | 19.5 Mbps |
| bench-validator-09 | validator | 12.6 Mbps | 16.3 Mbps | 18.8 Mbps | 18.8 Mbps |
| bench-validator-10 | validator | 11.7 Mbps | 15.3 Mbps | 20.4 Mbps | 20.4 Mbps |
| Metric | Query | Class | Nodes | Median of node means | Worst peak 60-s mean | Max |
|---|---|---|---|---|---|---|
| tn_primary_commit_latency_seconds | p95 | validator | 8 | 1,164 ms | 4,056 ms | 5,250 ms |
| tn_engine_execution_duration_seconds | p95 | validator | 9 | 2,011 ms | 4,190 ms | 4,417 ms |
| tn_engine_execution_duration_seconds | p95 | observer | 1 | 1,771 ms | 3,253 ms | 3,667 ms |
| tn_worker_batch_size_bytes | p95 | validator | 8 | 234.1 KiB | 246.4 KiB | 246.4 KiB |
| tn_worker_batch_transactions | p95 | validator | 9 | 496 | 1.48K | 1.48K |
| tn_executor_output_transactions | p95 | validator | 8 | 13.1K | 24.2K | 24.2K |
| tn_primary_consensus_output_bytes | max | validator | 10 | 554.5 KiB | 1.4 MiB | 2.6 MiB |
| tn_primary_consensus_output_bytes | max | observer | 1 | 546.0 KiB | 1.1 MiB | 1.4 MiB |
| tn_engine_queued_outputs | max | validator | 10 | 5.44 | 8 | 8 |
| tn_engine_queued_outputs | max | observer | 1 | 1.33 | 4.5 | 6 |
| tn_primary_round | max | validator | 10 | 1.46K | 2.95K | 3K |
| tn_primary_round | max | observer | 1 | 1.47K | 2.94K | 3K |
| tn_primary_committed_round | max | validator | 10 | 1.46K | 2.95K | 3K |
| tn_primary_committed_round | max | observer | 1 | 1.47K | 2.94K | 3K |
| tn_epoch_current | max | validator | 10 | 6.6 | 8 | 8 |
| tn_epoch_current | max | observer | 1 | 6.6 | 8 | 8 |
| tn_batch_builder_pending_pool_transactions | max | validator | 10 | 531 | 8.11K | 8.15K |
| tn_batch_builder_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_process_resident_memory_bytes | max | validator | 10 | 5.23 GiB | 7.27 GiB | 7.32 GiB |
| reth_process_resident_memory_bytes | max | observer | 1 | 3.53 GiB | 5.14 GiB | 5.15 GiB |
| tn_primary_subdags_committed_total | rate | validator | 10 | 1.17 | 1.42 | 1.42 |
| tn_primary_subdags_committed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_round_lag | lag | validator | 10 | 2.23 | 3.5 | 5 |
| tn_primary_round_lag | lag | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_seal_failures_total | rate | validator | 8 | 0.000409 | 0.0167 | 0.0222 |
| tn_node_mode | max | validator | 10 | 0 | 1 | 1 |
| tn_node_mode | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_total | rate | validator | 10 | 20 | 32.9 | 33.1 |
| tn_primary_vote_requests_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_created_total | rate | validator | 10 | 19.9 | 25.4 | 25.6 |
| tn_primary_votes_created_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_recast_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_votes_recast_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | validator | 10 | 0 | 0.0856 | 0.114 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_stage_seconds{stage="exec_wait"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lead_sleep"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lock"} | p95 | validator | 9 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="parents"} | p95 | validator | 10 | 1 ms | 2 ms | 2 ms |
| tn_primary_vote_stage_seconds{stage="persist"} | p95 | validator | 10 | 3 ms | 4 ms | 4 ms |
| tn_primary_vote_stage_seconds{stage="prelock_wait"} | p95 | validator | 10 | 368 ms | 2,425 ms | 2,425 ms |
| tn_primary_vote_stage_seconds{stage="sync_batches"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="validate"} | p95 | validator | 10 | 3 ms | 5 ms | 5 ms |
| tn_primary_vote_stage_seconds{stage="vote_store"} | p95 | validator | 10 | 1 ms | 1 ms | 1 ms |
| tn_primary_leader_rounds_total{leader="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY",swapped="false",found="false"} | rate | validator | 10 | 0.000765 | 0.0167 | 0.0222 |
| tn_primary_leader_rounds_total{leader="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY",swapped="false",found="true"} | rate | validator | 10 | 0.118 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY",swapped="true",found="false"} | rate | validator | 10 | 0.000356 | 0.0165 | 0.0221 |
| tn_primary_leader_rounds_total{leader="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY",swapped="true",found="true"} | rate | validator | 10 | 0.0369 | 0.0833 | 0.111 |
| tn_primary_leader_rounds_total{leader="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12",swapped="false",found="false"} | rate | validator | 10 | 0.00117 | 0.0222 | 0.0222 |
| tn_primary_leader_rounds_total{leader="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12",swapped="false",found="true"} | rate | validator | 10 | 0.117 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12",swapped="true",found="false"} | rate | validator | 10 | 0.00106 | 0.0221 | 0.0222 |
| tn_primary_leader_rounds_total{leader="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12",swapped="true",found="true"} | rate | validator | 10 | 0.0349 | 0.117 | 0.156 |
| tn_primary_leader_rounds_total{leader="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc",swapped="false",found="false"} | rate | validator | 10 | 0.0028 | 0.0778 | 0.0889 |
| tn_primary_leader_rounds_total{leader="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc",swapped="false",found="true"} | rate | validator | 10 | 0.0781 | 0.144 | 0.156 |
| tn_primary_leader_rounds_total{leader="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc",swapped="true",found="true"} | rate | validator | 10 | 0.00277 | 0.0444 | 0.0667 |
| tn_primary_leader_rounds_total{leader="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT",swapped="false",found="false"} | rate | validator | 10 | 0.00281 | 0.0389 | 0.0444 |
| tn_primary_leader_rounds_total{leader="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT",swapped="false",found="true"} | rate | validator | 10 | 0.116 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT",swapped="true",found="false"} | rate | validator | 10 | 0.00035 | 0.0165 | 0.0221 |
| tn_primary_leader_rounds_total{leader="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT",swapped="true",found="true"} | rate | validator | 10 | 0.0274 | 0.0944 | 0.133 |
| tn_primary_leader_rounds_total{leader="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr",swapped="false",found="true"} | rate | validator | 10 | 0.0769 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr",swapped="true",found="false"} | rate | validator | 10 | 0.000356 | 0.0165 | 0.0221 |
| tn_primary_leader_rounds_total{leader="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr",swapped="true",found="true"} | rate | validator | 10 | 0.00982 | 0.0778 | 0.0889 |
| tn_primary_leader_rounds_total{leader="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76",swapped="false",found="false"} | rate | validator | 10 | 0.00713 | 0.114 | 0.133 |
| tn_primary_leader_rounds_total{leader="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76",swapped="false",found="true"} | rate | validator | 10 | 0.0541 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76",swapped="true",found="false"} | rate | validator | 10 | 0.000712 | 0.0331 | 0.0441 |
| tn_primary_leader_rounds_total{leader="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76",swapped="true",found="true"} | rate | validator | 10 | 0.0157 | 0.1 | 0.133 |
| tn_primary_leader_rounds_total{leader="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr",swapped="false",found="true"} | rate | validator | 10 | 0.119 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr",swapped="true",found="false"} | rate | validator | 10 | 0.00118 | 0.0167 | 0.0222 |
| tn_primary_leader_rounds_total{leader="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr",swapped="true",found="true"} | rate | validator | 10 | 0.0357 | 0.0833 | 0.111 |
| tn_primary_leader_rounds_total{leader="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE",swapped="false",found="true"} | rate | validator | 10 | 0.119 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE",swapped="true",found="true"} | rate | validator | 10 | 0.0439 | 0.128 | 0.156 |
| tn_primary_leader_rounds_total{leader="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp",swapped="false",found="true"} | rate | validator | 10 | 0.00982 | 0.128 | 0.133 |
| tn_primary_leader_rounds_total{leader="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp",swapped="true",found="true"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT",swapped="false",found="true"} | rate | validator | 10 | 0.118 | 0.15 | 0.156 |
| tn_primary_leader_rounds_total{leader="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT",swapped="true",found="true"} | rate | validator | 10 | 0.0334 | 0.0833 | 0.111 |
| tn_primary_leader_commits_total{leader="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY"} | rate | validator | 10 | 0.155 | 0.222 | 0.244 |
| tn_primary_leader_commits_total{leader="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12"} | rate | validator | 10 | 0.152 | 0.222 | 0.267 |
| tn_primary_leader_commits_total{leader="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc"} | rate | validator | 10 | 0.081 | 0.15 | 0.178 |
| tn_primary_leader_commits_total{leader="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT"} | rate | validator | 10 | 0.142 | 0.239 | 0.289 |
| tn_primary_leader_commits_total{leader="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr"} | rate | validator | 10 | 0.0867 | 0.206 | 0.222 |
| tn_primary_leader_commits_total{leader="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76"} | rate | validator | 10 | 0.0698 | 0.233 | 0.267 |
| tn_primary_leader_commits_total{leader="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr"} | rate | validator | 10 | 0.154 | 0.215 | 0.222 |
| tn_primary_leader_commits_total{leader="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE"} | rate | validator | 10 | 0.163 | 0.261 | 0.289 |
| tn_primary_leader_commits_total{leader="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp"} | rate | validator | 10 | 0.00982 | 0.128 | 0.133 |
| tn_primary_leader_commits_total{leader="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT"} | rate | validator | 10 | 0.151 | 0.206 | 0.222 |
| tn_engine_queue_wait_seconds | p95 | validator | 9 | 6,943 ms | 28.0 s | 28.9 s |
| tn_engine_queue_wait_seconds | p95 | observer | 1 | 2,246 ms | 5,887 ms | 7,062 ms |
| tn_engine_persist_duration_seconds | p95 | validator | 8 | 502 ms | 1,025 ms | 1,225 ms |
| tn_engine_persist_duration_seconds | p95 | observer | 1 | 483 ms | 886 ms | 900 ms |
| tn_engine_busy_milliseconds_total | rate | validator | 10 | 967 | 1.02K | 1.07K |
| tn_engine_busy_milliseconds_total | rate | observer | 1 | 898 | 998 | 1.02K |
| tn_reth_persist_save_blocks_seconds | p95 | validator | 8 | 270 ms | 941 ms | 982 ms |
| tn_reth_persist_save_blocks_seconds | p95 | observer | 1 | 247 ms | 443 ms | 450 ms |
| tn_reth_persist_commit_seconds | p95 | validator | 7 | 256 ms | 524 ms | 608 ms |
| tn_reth_persist_commit_seconds | p95 | observer | 1 | 254 ms | 443 ms | 450 ms |
| tn_reth_persisted_accounts_total | rate | validator | 10 | 3.01K | 7.59K | 7.59K |
| tn_reth_persisted_accounts_total | rate | observer | 1 | 3.02K | 7.15K | 7.15K |
| tn_reth_persisted_storage_slots_total | rate | validator | 10 | 3.99K | 4.63K | 4.79K |
| tn_reth_persisted_storage_slots_total | rate | observer | 1 | 3.99K | 4.6K | 4.62K |
| tn_node_mode_transitions_total{to="cvv_inactive",reason="epoch_behind"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="attempts_exhausted"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="epoch_closed"} | rate | validator | 2 | 0.182 | 1.1 | 1.2 |
| tn_executor_rejoin_blocked_total{reason="epoch_mismatch"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="no_known_header"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="round_gap"} | rate | validator | 2 | 0.865 | 1.26 | 1.31 |
| tn_executor_rejoin_blocked_total{reason="stale_head"} | rate | validator | 2 | 0.0119 | 0.117 | 0.156 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="22sZN3dfzGaFxmXDGRmsRAfGVdDaQL5QTF3LucJjK7TCoCjs5KWDh65fySh3as2EY3prEnEvhAUr9jWMq1do9K8Yqb6JuiVdJeKkgaamMCVtKEjEsv2SKkhsznyjCC7B9K6f"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="246yKXSoqMieNxok98Gc4YK1jvcyXB2aZGnCUZVzdd4u7xnreRoG33zjTx2j39eEoWqxrNfrArkRZgdjzrv3uiVWKkFst3EPDDFK1tpD6d8GELQYfoksz5dWTrfWw8XSc23C"} | rate | validator | 3 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="248YoQPU4YeF32nHAFy4Bpqi6D82pGtcsLeBiTVWkY9d7wZP1jJAigfSYTcz9uLCxYX3P9WCXx4kcmjc9LR8wsPKweKxLJmGkSwvBd1L2tmnUHc7GAR2Lg9DKg7y7qhmkhyU"} | rate | validator | 4 | 0 | 0.0444 | 0.0444 |
| tn_worker_batch_requests_denied_total{worker="0",peer="24FLp4x96H8a5uLAxPoR7dXZwYdjVgXtAeuZ33AC78exiwYZiw1nGQRFp8FSgVKNHKJ91BwPt3xMTpHuPibwkeSrUjzgVRuBrzeiwg9NxKEs9mqkkfnfQbTo3xRUcGhbaNDy"} | rate | validator | 3 | 0 | 0.0222 | 0.0222 |
| tn_worker_batch_requests_denied_total{worker="0",peer="mmP1kjNxDQkDKpQtLWQuHL359GeEpJ5GJ92pEzuvpgDYZDyijtJuvdLfKFuBco19CsAtTQ91ySVPASDQpf17rMHdKoiqWfSzPKYnzcS171G1ECrD3fPWSPZUMe8qpsGN4FF"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="o87qNkBiNtFRU21RPZct6PNWZKyErCNs5SE8jKWc2U1p17MceZWVy8TCNoTQzWr1mw46WAJYbQu8jxPG2Mr9cSpR6THfTnjFFh7P4cdQjhQ2ppSCnDwWeyR88maKC7VuW5s"} | rate | validator | 2 | 0.00649 | 0.143 | 0.19 |
| tn_worker_batch_requests_denied_total{worker="0",peer="ofFnex8dHPBxBsLF1cMwxkeeGaczu6bZbit1C1rrGifGULALhjyT3NmkXu7yKm7x68XG7PtknmESg64VUDx6tUYqn1xvE15TDY66FSStJ51SLkC16trU3YKUgSgFwfo7usQ"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="oxeGgwHXVvJ7zysJxhdnaTBDBmSwdFF6rTUhzsu2LMYMX6rAjTwdbq3MknqA5dicHehf8fiMSUqirLJrpdouru82oQGjJgwZvi2QNX3QW1DM9xrp5TRNcqRzd1Ljnz57scS"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="ufF6X1ErVVMivUGoNP7Uh8JSxAmghNxnwDd3Z5ovw8yrh89qCbrmuvrNcJPufFQeLbiuCeh5L913BZERvjBeyj4EzfYzaAHYnzx7ZH7W6detPnFTZfSPAYojJCZoNak9PVM"} | rate | validator | 2 | 0.000917 | 0.0334 | 0.0334 |
| tn_worker_batch_requests_denied_total{worker="0",peer="xAM3H8UEq3dtRNVRF5cvLSXojnMskoSnjAyMHeuRVc6JdgTaLNNgF3hbgMsJn284GKXQjF5rccMRnJaGHRYCYd1ULPuGnuv7m2Lt1Se3heDfhTb39oeLiTDLdiTqwkhw9qe"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="22sZN3dfzGaFxmXDGRmsRAfGVdDaQL5QTF3LucJjK7TCoCjs5KWDh65fySh3as2EY3prEnEvhAUr9jWMq1do9K8Yqb6JuiVdJeKkgaamMCVtKEjEsv2SKkhsznyjCC7B9K6f"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="246yKXSoqMieNxok98Gc4YK1jvcyXB2aZGnCUZVzdd4u7xnreRoG33zjTx2j39eEoWqxrNfrArkRZgdjzrv3uiVWKkFst3EPDDFK1tpD6d8GELQYfoksz5dWTrfWw8XSc23C"} | rate | validator | 3 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="248YoQPU4YeF32nHAFy4Bpqi6D82pGtcsLeBiTVWkY9d7wZP1jJAigfSYTcz9uLCxYX3P9WCXx4kcmjc9LR8wsPKweKxLJmGkSwvBd1L2tmnUHc7GAR2Lg9DKg7y7qhmkhyU"} | rate | validator | 4 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="24FLp4x96H8a5uLAxPoR7dXZwYdjVgXtAeuZ33AC78exiwYZiw1nGQRFp8FSgVKNHKJ91BwPt3xMTpHuPibwkeSrUjzgVRuBrzeiwg9NxKEs9mqkkfnfQbTo3xRUcGhbaNDy"} | rate | validator | 3 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="mmP1kjNxDQkDKpQtLWQuHL359GeEpJ5GJ92pEzuvpgDYZDyijtJuvdLfKFuBco19CsAtTQ91ySVPASDQpf17rMHdKoiqWfSzPKYnzcS171G1ECrD3fPWSPZUMe8qpsGN4FF"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="o87qNkBiNtFRU21RPZct6PNWZKyErCNs5SE8jKWc2U1p17MceZWVy8TCNoTQzWr1mw46WAJYbQu8jxPG2Mr9cSpR6THfTnjFFh7P4cdQjhQ2ppSCnDwWeyR88maKC7VuW5s"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="ofFnex8dHPBxBsLF1cMwxkeeGaczu6bZbit1C1rrGifGULALhjyT3NmkXu7yKm7x68XG7PtknmESg64VUDx6tUYqn1xvE15TDY66FSStJ51SLkC16trU3YKUgSgFwfo7usQ"} | rate | validator | 2 | 0.00499 | 0.0886 | 0.132 |
| tn_worker_batch_requests_failed_total{worker="0",peer="oxeGgwHXVvJ7zysJxhdnaTBDBmSwdFF6rTUhzsu2LMYMX6rAjTwdbq3MknqA5dicHehf8fiMSUqirLJrpdouru82oQGjJgwZvi2QNX3QW1DM9xrp5TRNcqRzd1Ljnz57scS"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="ufF6X1ErVVMivUGoNP7Uh8JSxAmghNxnwDd3Z5ovw8yrh89qCbrmuvrNcJPufFQeLbiuCeh5L913BZERvjBeyj4EzfYzaAHYnzx7ZH7W6detPnFTZfSPAYojJCZoNak9PVM"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="xAM3H8UEq3dtRNVRF5cvLSXojnMskoSnjAyMHeuRVc6JdgTaLNNgF3hbgMsJn284GKXQjF5rccMRnJaGHRYCYd1ULPuGnuv7m2Lt1Se3heDfhTb39oeLiTDLdiTqwkhw9qe"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_node_epoch_record_signers | max | validator | 10 | 8.36 | 10 | 10 |
| tn_node_epoch_record_signers | max | observer | 1 | 8.87 | 10 | 10 |
| tn_node_epoch_record_missing_signers_total{signer="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="1Vge17NWtME42tFv13ygpke1RnwtfRfQ3bB8f9LhvJY"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="45NSoQM2SZZt2dus3jPGygK9UREEpqDjwPxJC3Krnp12"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc"} | rate | validator | 10 | 0.000741 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="4T2tfJZmemUhL2MbHGS6w4V83kjeXFZrNXhQqRHr7zVc"} | rate | observer | 1 | 0.00052 | 0.0156 | 0.0208 |
| tn_node_epoch_record_missing_signers_total{signer="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="4kX5oKXKBi2yPeW88mrJbQMACeKoA4AGbJ3D5vyRVvTT"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="96CMnptik7z4MPDnp4oax9bHUUvZGCSaYzsMcg33CEcr"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76"} | rate | validator | 10 | 0.000741 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="BhMgY6bDVD8P8eChCuUFK66GUAEFZQDGtg8AcVk3us76"} | rate | observer | 1 | 0.00052 | 0.0156 | 0.0208 |
| tn_node_epoch_record_missing_signers_total{signer="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr"} | rate | validator | 10 | 0 | 0.013 | 0.0173 |
| tn_node_epoch_record_missing_signers_total{signer="CoyGfRch9FC9Kp8HYjyyY8NWQM3pw94E13Jc9qBM9tRr"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE"} | rate | validator | 10 | 0.000733 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="EqPrjKthssuTFThDFhA1UFVvQfH2YCryuvJLa6hqEDLE"} | rate | observer | 1 | 0.00052 | 0.0156 | 0.0208 |
| tn_node_epoch_record_missing_signers_total{signer="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="FgN9BDskVaZSgk5fEeXFVzKT1GeZko5tJbPjx5ayVFAp"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="J5rtTGKhcBys9w1R4FgwK1QL4Ci7R7GUGRRRBUy5oLgT"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="batch"} | p95 | validator | 2 | 15.2 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"} | p95 | validator | 5 | 29.0 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"} | p95 | observer | 1 | 29.0 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="vote"} | p95 | validator | 5 | 11.1 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="response",class="batch"} | p95 | validator | 1 | 483 ms | 654 ms | 743 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"} | p95 | validator | 5 | 436 ms | 450 ms | 452 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"} | p95 | observer | 1 | 240 ms | 241 ms | 241 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="vote"} | p95 | validator | 3 | 1,065 ms | 2,238 ms | 2,252 ms |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_admission_mode | max | validator | 10 | 2 | 2 | 2 |
| tn_network_admission_mode | max | observer | 1 | 2 | 2 | 2 |
| tn_network_admission_fallback | max | validator | 10 | 0 | 1 | 1 |
| tn_network_admission_fallback | max | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | validator | 10 | 0 | 1.57 | 2.03 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | validator | 10 | 24 | 102 | 118 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | validator | 10 | 3.21 | 14 | 14.5 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | observer | 1 | 0.361 | 0.989 | 1.54 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | validator | 10 | 0.968 | 4.3 | 4.7 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | observer | 1 | 0.00351 | 0.0271 | 0.0416 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | validator | 10 | 0.0143 | 55.4 | 84.2 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | validator | 10 | 0.401 | 1.62 | 1.62 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | observer | 1 | 0.16 | 0.543 | 0.708 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | validator | 10 | 0.0202 | 0.106 | 0.123 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | observer | 1 | 0.0492 | 0.167 | 0.167 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | validator | 10 | 0.0374 | 0.204 | 0.282 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | validator | 10 | 0.228 | 19 | 28.9 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_request_service_seconds{class="batch"} | p95 | validator | 4 | 62 ms | 241 ms | 241 ms |
| tn_network_inbound_request_service_seconds{class="epoch_record"} | p95 | validator | 6 | 5 ms | 63 ms | 85 ms |
| tn_network_inbound_request_service_seconds{class="vote"} | p95 | validator | 5 | 34 ms | 2,425 ms | 2,425 ms |
| tn_network_established_connections | max | validator | 10 | 19.5 | 20 | 20 |
| tn_network_established_connections | max | observer | 1 | 19 | 20 | 20 |
| tn_worker_sync_streams_shed_total{reason="denied"} | rate | validator | 10 | 0.00286 | 0.117 | 0.167 |
| tn_worker_batch_fetch_duration_seconds | p95 | validator | 8 | 5 ms | 508 ms | 683 ms |
| tn_executor_batch_fetch_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_executor_batch_fetch_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_received_total | rate | validator | 10 | 14.1 | 16.9 | 17.1 |
| tn_primary_votes_received_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_request_failures_total | rate | validator | 10 | 0.0135 | 3.64 | 5.16 |
| tn_primary_vote_request_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificates_formed_total | rate | validator | 10 | 2.25 | 2.82 | 2.84 |
| tn_primary_certificates_formed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificate_form_duration_seconds | p95 | validator | 8 | 488 ms | 957 ms | 1,131 ms |
| tn_engine_outputs_executed_total | rate | validator | 10 | 1.17 | 1.47 | 1.56 |
| tn_engine_outputs_executed_total | rate | observer | 1 | 1.17 | 1.43 | 1.44 |
| tn_engine_persist_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_engine_persist_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_consensus_sync_distance | max | validator | 10 | 6.91 | 459 | 460 |
| tn_node_consensus_sync_distance | max | observer | 1 | 2.1 | 5 | 7 |
| reth_db_table_size{table="AccountChangeSets"} | max | validator | 10 | 454 MiB | 723 MiB | 725 MiB |
| reth_db_table_size{table="AccountChangeSets"} | max | observer | 1 | 454 MiB | 723 MiB | 725 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | validator | 10 | 51 MiB | 83 MiB | 83 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | observer | 1 | 51 MiB | 83 MiB | 83 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | validator | 10 | 33 MiB | 57 MiB | 57 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | observer | 1 | 33 MiB | 57 MiB | 57 MiB |
| reth_db_table_size{table="HashedStorages"} | max | validator | 10 | 40 MiB | 43 MiB | 43 MiB |
| reth_db_table_size{table="HashedStorages"} | max | observer | 1 | 40 MiB | 43 MiB | 43 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | validator | 10 | 27 MiB | 48 MiB | 48 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | observer | 1 | 27 MiB | 48 MiB | 48 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | validator | 10 | 40 MiB | 43 MiB | 43 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | observer | 1 | 40 MiB | 43 MiB | 43 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | validator | 10 | 542 MiB | 869 MiB | 871 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | observer | 1 | 542 MiB | 869 MiB | 871 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | validator | 10 | 101 MiB | 124 MiB | 124 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | observer | 1 | 101 MiB | 124 MiB | 124 MiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | validator | 10 | 987 MiB | 1.56 GiB | 1.57 GiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | observer | 1 | 987 MiB | 1.56 GiB | 1.57 GiB |
| reth_db_table_size{table="TransactionSenders"} | max | validator | 10 | 508 MiB | 824 MiB | 826 MiB |
| reth_db_table_size{table="TransactionSenders"} | max | observer | 1 | 509 MiB | 823 MiB | 826 MiB |
| reth_db_freelist | max | validator | 10 | 34.6K | 59.2K | 64.7K |
| reth_db_freelist | max | observer | 1 | 34.3K | 56.7K | 61.5K |
| reth_database_transaction_commit_write_duration_seconds | p95 | validator | 8 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_write_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | validator | 8 | 258 ms | 460 ms | 489 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | observer | 1 | 251 ms | 435 ms | 443 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | validator | 8 | 255 ms | 460 ms | 489 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | observer | 1 | 252 ms | 435 ms | 443 ms |
| tn_engine_recover_seconds | p95 | validator | 9 | 234 ms | 964 ms | 964 ms |
| tn_engine_recover_seconds | p95 | observer | 1 | 233 ms | 666 ms | 829 ms |
| tn_engine_evm_seconds | p95 | validator | 10 | 343 ms | 844 ms | 883 ms |
| tn_engine_evm_seconds | p95 | observer | 1 | 307 ms | 721 ms | 800 ms |
| tn_engine_root_seconds | p95 | validator | 8 | 573 ms | 1,816 ms | 1,975 ms |
| tn_engine_root_seconds | p95 | observer | 1 | 558 ms | 1,458 ms | 1,700 ms |
| tn_engine_finish_seconds | p95 | validator | 9 | 260 ms | 766 ms | 860 ms |
| tn_engine_finish_seconds | p95 | observer | 1 | 206 ms | 406 ms | 433 ms |
| tn_engine_forward_wait_seconds | p95 | validator | 9 | 1,616 ms | 9,547 ms | 9,618 ms |
| tn_engine_forward_wait_seconds | p95 | observer | 1 | 0 ms | 0 ms | 0 ms |
| tn_engine_channel_depth | max | validator | 10 | 0.534 | 6.75 | 8 |
| tn_engine_channel_depth | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_executed_transactions_total | rate | validator | 10 | 7.41K | 14.1K | 14.1K |
| tn_engine_executed_transactions_total | rate | observer | 1 | 7.42K | 13.5K | 13.5K |
| tn_executor_execution_wait_seconds | p95 | validator | 9 | 189 ms | 30.0 s | 30.0 s |
| tn_primary_commit_run_ahead_blocked_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_commit_run_ahead_blocked_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_consensus_exec_lag_rounds | max | validator | 10 | 14.3 | 99 | 102 |
| tn_primary_consensus_exec_lag_rounds | max | observer | 1 | 3.31 | 9.5 | 14 |
| tn_primary_commit_lag_rounds | max | validator | 10 | 2.23 | 3.5 | 5 |
| tn_primary_commit_lag_rounds | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_rejoin_grace_holds_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_rejoin_grace_holds_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_inactive_total | rate | validator | 10 | 0 | 32.9 | 33.1 |
| tn_primary_votes_refused_inactive_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_execution_behind_total | rate | validator | 10 | 0.0198 | 1.01 | 1.38 |
| tn_primary_votes_refused_execution_behind_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_pool_age_seconds | p95 | validator | 8 | 5,863 ms | 30.0 s | 30.0 s |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="drained"} | rate | validator | 10 | 1.6 | 3.16 | 3.24 |
| tn_batch_builder_builds_total{reason="drained"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | validator | 10 | 1.51 | 5.44 | 5.62 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_report_seconds | p95 | validator | 9 | 0 ms | 3 ms | 4 ms |
| tn_primary_digest_wait_seconds | p95 | validator | 8 | 1,373 ms | 30.0 s | 30.0 s |
| tn_primary_headers_proposed_total{reason="max_delay"} | rate | validator | 10 | 0.0219 | 0.167 | 0.2 |
| tn_primary_headers_proposed_total{reason="min_delay"} | rate | validator | 10 | 2.32 | 2.83 | 2.84 |
| tn_primary_headers_proposed_total{reason="parents_and_threshold"} | rate | validator | 10 | 0.0298 | 0.456 | 0.556 |
| tn_primary_digests_requeued_total{cause="canceled"} | rate | validator | 10 | 0.0815 | 3.19 | 4.11 |
| tn_primary_digests_requeued_total{cause="committed"} | rate | validator | 10 | 0.0035 | 1.41 | 1.56 |
| tn_executor_consensus_pack_persist_seconds | p95 | validator | 8 | 7 ms | 10 ms | 12 ms |
| tn_batch_builder_seal_duration_seconds | p95 | validator | 7 | 260 ms | 719 ms | 800 ms |
| tn_worker_quorum_wait_seconds | p95 | validator | 9 | 255 ms | 719 ms | 800 ms |
| tn_primary_proposer_pending_digests | max | validator | 10 | 0.908 | 5.25 | 10 |
| tn_primary_proposer_pending_digests | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_transactions | max | validator | 10 | 531 | 8.11K | 8.15K |
| reth_transaction_pool_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_pool_transactions | max | validator | 10 | 1.88K | 5.65K | 6.09K |
| reth_transaction_pool_queued_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_all_transactions_by_hash | max | validator | 10 | 3.76K | 8.51K | 9.53K |
| reth_transaction_pool_all_transactions_by_hash | max | observer | 1 | 0 | 0 | 0 |
| reth_storage_providers_database_save_blocks_total | p95 | validator | 7 | 195 ms | 747 ms | 790 ms |
| reth_storage_providers_database_save_blocks_total | p95 | observer | 1 | 172 ms | 357 ms | 376 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | validator | 7 | 195 ms | 747 ms | 790 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | observer | 1 | 172 ms | 357 ms | 376 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | validator | 7 | 42 ms | 101 ms | 112 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | observer | 1 | 42 ms | 106 ms | 107 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | validator | 8 | 183 ms | 439 ms | 469 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | observer | 1 | 175 ms | 308 ms | 376 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | validator | 8 | 10 ms | 13 ms | 15 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | observer | 1 | 8 ms | 11 ms | 12 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | validator | 8 | 31 ms | 62 ms | 65 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | observer | 1 | 30 ms | 57 ms | 64 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | validator | 7 | 20 ms | 33 ms | 35 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | observer | 1 | 19 ms | 28 ms | 29 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | validator | 8 | 26 ms | 41 ms | 44 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | observer | 1 | 25 ms | 38 ms | 40 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | validator | 8 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | validator | 8 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | validator | 7 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | validator | 8 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | validator | 8 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| tn_node_switch{name="digest_dedupe"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="digest_dedupe"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_record_state_check"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="epoch_record_state_check"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="parent_grace"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="parent_grace"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="quorum_exec_block"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | validator | 10 | 4 | 4 | 4 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | observer | 1 | 4 | 4 | 4 |
| tn_node_switch{name="recovery_threads"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="recovery_threads"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="vote_queue_fix"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="vote_queue_fix"} | max | observer | 1 | 1 | 1 | 1 |
| tn_primary_epoch_entry_rounds_behind | max | validator | 10 | 3 | 18.2 | 22 |
| tn_primary_epoch_entry_rounds_behind | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_random_advice_bytes | max | validator | 10 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_random_advice_bytes | max | observer | 1 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_used_bytes | max | validator | 10 | 2.88 GiB | 4.54 GiB | 4.54 GiB |
| tn_reth_mdbx_used_bytes | max | observer | 1 | 2.88 GiB | 4.54 GiB | 4.54 GiB |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | validator | 10 | 42M | 72M | 72.2M |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | observer | 1 | 24.9M | 52.2M | 52.7M |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | validator | 10 | 6.18K | 8.84K | 8.86K |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | observer | 1 | 3.84K | 7.15K | 7.2K |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | validator | 10 | 991K | 2.3M | 2.31M |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | observer | 1 | 603K | 1.16M | 1.19M |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | validator | 10 | 755K | 1.2M | 1.2M |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | observer | 1 | 460K | 900K | 907K |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | validator | 10 | 1 | 1 | 1 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | observer | 1 | 1 | 1 | 1 |
| tn_reth_sealed_txs_marked | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_marked | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_size_bytes | max | validator | 10 | 189.2 KiB | 2.8 MiB | 2.9 MiB |
| reth_transaction_pool_pending_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_queued_pool_size_bytes | max | validator | 10 | 657.8 KiB | 2.0 MiB | 2.1 MiB |
| reth_transaction_pool_queued_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_pending_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_pending_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_removed_transactions | max | validator | 10 | 1.67M | 3.44M | 3.44M |
| reth_transaction_pool_removed_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | observer | 1 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_close_backlog_outputs | max | validator | 10 | 5.35 | 12 | 12 |
| tn_node_epoch_close_backlog_outputs | max | observer | 1 | 1.7 | 3 | 3 |
| tn_node_epoch_close_detect_lag_seconds | max | validator | 10 | 692 ms | 372.6 s | 372.6 s |
| tn_node_epoch_close_detect_lag_seconds | max | observer | 1 | 1,179 ms | 1,330 ms | 1,330 ms |
| tn_node_epoch_close_drain_seconds | max | validator | 10 | 3,900 ms | 11.5 s | 11.5 s |
| tn_node_epoch_close_drain_seconds | max | observer | 1 | 1,204 ms | 2,740 ms | 2,740 ms |
| tn_node_epoch_record_vote_skipped_total{reason="certified_before_close"} | max | validator | 2 | 1 | 1 | 1 |
| tn_node_epoch_record_vote_skipped_total{reason="not_in_committee"} | max | observer | 1 | 5.92 | 7 | 7 |
| tn_primary_parent_grace_total{outcome="complete"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="expired"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="preempted"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_late_parents_total{origin="older_round"} | max | validator | 10 | 8.29K | 11.8K | 11.8K |
| tn_primary_late_parents_total{origin="own_round_behind"} | max | validator | 10 | 46.6K | 56.2K | 56.2K |
| tn_node_execution_backlog_outputs | max | validator | 10 | 7.54 | 17.8 | 19 |
| tn_node_execution_backlog_outputs | max | observer | 1 | 2.2 | 5.5 | 7 |
| tn_engine_unpersisted_outputs | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_unpersisted_outputs | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_built_height | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_built_height | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | observer | 1 | 0 | 0 | 0 |
tn_primary_commit_latency_seconds
tn_primary_round − tn_primary_committed_round
tn_engine_queued_outputs, max by node
The share of each validator's own headers that formed a certificate, per window (tools/cert-rate.py on the node logs). bench-validator-06 and bench-validator-10 certified fewer than 90 % of their own headers in a window where they proposed at least 50. Own leader % is the validator's share of the committed leaders and not committed % the share of its certificates no committed leader reached, both over the run window; the fair share is 100 / 10 validators = 10.0 %. Greyed: fewer than 50 headers proposed, not judged.
| Validator | Region | blast 0 preamble | blast 0 warmup | blast 0 mixed | blast 0 cooldown | blast 1 preamble | blast 1 warmup | blast 1 mixed | blast 1 cooldown | blast 2 preamble | blast 2 warmup | blast 2 mixed | blast 2 cooldown | run | Own leader % | Not committed % |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-validator-01 | us-west2 (Los Angeles) | 101.5 | 100.4 | 100.0 | 100.0 | 99.6 | 100.0 | 100.0 | 100.4 | 99.2 | 99.8 | 99.2 | 100.0 | 99.6 | 12.7 | 0.1 |
| bench-validator-02 | europe-west2 (London) | 100.0 | 100.0 | 100.0 | 100.0 | 99.6 | 98.4 | 99.3 | 100.0 | 96.8 | 97.7 | 99.6 | – | 99.7 | 3.7 | 34.7 |
| bench-validator-03 | australia-southeast1 (Sydney) | 101.5 | 100.4 | 100.0 | 101.1 | 100.4 | 99.7 | 99.5 | 100.4 | 100.0 | 97.9 | 99.1 | 100.5 | 99.4 | 13.0 | 0.1 |
| bench-validator-04 | asia-northeast1 (Tokyo) | 100.0 | 100.4 | 100.0 | 101.1 | 100.0 | 100.3 | 100.0 | 100.0 | 99.2 | 99.8 | 99.9 | 100.5 | 100.0 | 13.6 | 0.1 |
| bench-validator-05 | us-east4 (Ashburn) | 101.5 | 99.6 | 100.0 | 100.0 | 99.6 | 99.7 | 100.0 | 100.4 | 100.0 | 99.8 | 100.0 | – | 100.0 | 11.3 | 0.1 |
| bench-validator-06 below 90 % | europe-west4 (Netherlands) | 100.0 | 100.4 | 99.5 | 100.0 | 99.6 | 98.2 | 99.1 | 99.5 | 100.0 | 97.7 | 84.1 | 99.0 | 91.7 | 0.6 | 42.8 |
| bench-validator-07 | asia-southeast1 (Singapore) | 100.0 | 100.4 | 99.9 | 100.0 | 99.6 | 99.2 | 99.6 | 100.4 | 95.2 | 97.9 | 98.3 | 100.0 | 99.1 | 13.3 | 0.1 |
| bench-validator-08 | asia-east2 (Hong Kong) | 101.5 | 100.0 | 100.0 | 101.1 | 99.6 | 99.0 | 99.1 | 100.4 | 97.6 | 97.0 | 94.8 | 97.1 | 97.2 | 13.1 | 0.1 |
| bench-validator-09 | asia-northeast3 (Seoul) | 100.0 | 100.0 | 100.0 | 100.0 | 99.6 | 99.5 | 99.5 | 100.4 | 97.6 | 96.8 | 90.6 | 100.0 | 95.1 | 12.4 | 0.3 |
| bench-validator-10 below 90 % | asia-south1 (Mumbai) | 100.0 | 99.6 | 100.0 | 100.0 | 99.2 | 97.7 | 98.9 | 100.0 | 99.2 | 97.2 | 85.7 | 100.0 | 92.5 | 9.6 | 10.5 |
| Node | Class | Peak RssAnon | Peak RssFile | Peak cache | PSI mem | PSI io | PSI cpu | db growth | static_files growth | Root used | OOM Δ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 939 MiB | 4.23 GiB | – | 0.0 % | 2.2 % | 2.4 % | 4.00 GiB | 5.20 GiB | 19.0 % | 0 |
| bench-validator-01 | validator | 2.59 GiB | 3.84 GiB | – | 0.0 % | 2.6 % | 10.0 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-02 | validator | 2.68 GiB | 3.83 GiB | – | 0.0 % | 4.0 % | 4.8 % | 4.00 GiB | 5.20 GiB | 22.0 % | 0 |
| bench-validator-03 | validator | 2.56 GiB | 3.78 GiB | – | 0.0 % | 2.9 % | 7.4 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-04 | validator | 2.46 GiB | 3.86 GiB | – | 0.0 % | 3.0 % | 8.9 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-05 | validator | 2.14 GiB | 5.42 GiB | – | 0.0 % | 4.9 % | 6.5 % | 4.00 GiB | 5.14 GiB | 22.0 % | 0 |
| bench-validator-06 | validator | 3.19 GiB | 3.56 GiB | – | 0.0 % | 3.4 % | 5.6 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-07 | validator | 2.45 GiB | 3.88 GiB | – | 0.0 % | 2.9 % | 8.2 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-08 | validator | 2.84 GiB | 3.78 GiB | – | 0.0 % | 2.9 % | 8.0 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-09 | validator | 2.74 GiB | 3.77 GiB | – | 0.0 % | 2.8 % | 6.9 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
| bench-validator-10 | validator | 2.77 GiB | 3.80 GiB | – | 0.0 % | 3.5 % | 5.8 % | 4.00 GiB | 5.20 GiB | 21.0 % | 0 |
/proc/<pid>/status of the telcoin process on the node with the largest RSS
Not collected: consensus-db/cache size.
/proc/pressure on the node with the highest memory PSI
Per role and per node, from sizing.json (p95 over the landed window; PSI and RssAnon max over the whole sampler run). The lines follow the hardware doc's tier rules: CPU p95 over 70 % of vCPUs; memory used p95 over 75 % of RAM, or telcoin RSS p95 over 85 % of RAM; disk write p95 over 60 % of the PD MiB/s limit, or write IOPS p95 over 60 % of the IOPS limit; IO pressure over 20 % or memory pressure over 5 % (PSI some avg10, max); network p95 over 50 % of 1 Gbps; root disk over 80 %. A line crossed on any node puts the role at the limit; a nonzero reading within 15 points of its line is tight; otherwise the role has headroom. OOM kills, disk throttling and degraded disk performance are flagged too.
| Signal | Class | Worst node | Value | Share of capacity | Line | Nodes over / tight | Status |
|---|---|---|---|---|---|---|---|
| CPU p95 | validator | bench-validator-03 | 43.3 % | 43.3 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | validator | bench-validator-06 | 25.1 % | 25.1 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | validator | bench-validator-05 | 7.56 GB | 44.0 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | validator | bench-validator-01 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | validator | bench-validator-06 | 3.43 GB | 19.9 % | info | – | info |
| Disk write p95 | validator | bench-validator-08 | 133 MB/s | 15.9 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | validator | bench-validator-08 | 26,414 | 52.8 % | > 60 % | 0 / 10 | tight |
| PSI io max | validator | bench-validator-05 | 4.9 % | 4.9 % | > 20 % | 0 / 0 | headroom |
| Root disk max | validator | bench-validator-05 | 21.3 % | 21.3 % | > 80 % | 0 / 0 | headroom |
| Network p95 | validator | bench-validator-07 | 149 Mbps | 14.9 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | validator | – | – | – | info | – | not collected |
| OOM kills | validator | – | – | any | any | – | none |
| Disk throttling | validator | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | validator | – | – | any sample | any | – | none |
| CPU p95 | observer | bench-observer-11 | 24.2 % | 24.2 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | observer | bench-observer-11 | 11.1 % | 11.1 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | observer | bench-observer-11 | 5.22 GB | 30.4 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | observer | bench-observer-11 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | observer | bench-observer-11 | 0.98 GB | 5.7 % | info | – | info |
| Disk write p95 | observer | bench-observer-11 | 122 MB/s | 14.5 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | observer | bench-observer-11 | 26,139 | 52.3 % | > 60 % | 0 / 1 | tight |
| PSI io max | observer | bench-observer-11 | 2.2 % | 2.2 % | > 20 % | 0 / 0 | headroom |
| Root disk max | observer | bench-observer-11 | 18.2 % | 18.2 % | > 80 % | 0 / 0 | headroom |
| Network p95 | observer | bench-observer-11 | 21 Mbps | 2.1 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | observer | – | – | – | info | – | not collected |
| OOM kills | observer | – | – | any | any | – | none |
| Disk throttling | observer | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | observer | – | – | any sample | any | – | none |
| Node | Class | CPU p95 | Memory used p95 | telcoin RSS p95 | PSI memory max | RssAnon peak | Disk write p95 | Write IOPS p95 | PSI io max | Root disk max | Network p95 | Consensus cache peak |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Line | > 70 % | > 75 % | > 85 % | > 5 % | info | > 60 % | > 60 % | > 20 % | > 80 % | > 50 % | info | |
| bench-validator-01 | validator | 41.9 % | 21.0 % | 6.60 GB 38 % | 0.0 % | 2.78 GB 16 % | 132 MB/s 16 % | 26,160 52 % | 2.6 % | 20.7 % | 138 Mbps 14 % | – |
| bench-validator-02 | validator | 38.4 % | 22.6 % | 6.87 GB 40 % | 0.0 % | 3.04 GB 18 % | 128 MB/s 15 % | 25,602 51 % | 4.0 % | 20.9 % | 129 Mbps 13 % | – |
| bench-validator-03 | validator | 43.3 % | 21.2 % | 6.58 GB 38 % | 0.0 % | 2.75 GB 16 % | 132 MB/s 16 % | 26,116 52 % | 2.9 % | 20.6 % | 144 Mbps 14 % | – |
| bench-validator-04 | validator | 39.8 % | 20.6 % | 6.56 GB 38 % | 0.0 % | 2.64 GB 15 % | 130 MB/s 16 % | 25,953 52 % | 3.0 % | 20.5 % | 134 Mbps 13 % | – |
| bench-validator-05 | validator | 39.4 % | 19.6 % | 7.56 GB 44 % | 0.0 % | 2.30 GB 13 % | 129 MB/s 15 % | 24,984 50 % | 4.9 % | 21.3 % | 126 Mbps 13 % | – |
| bench-validator-06 | validator | 39.4 % | 25.1 % | 6.97 GB 41 % | 0.0 % | 3.43 GB 20 % | 133 MB/s 16 % | 26,277 53 % | 3.4 % | 20.4 % | 132 Mbps 13 % | – |
| bench-validator-07 | validator | 42.7 % | 20.5 % | 6.47 GB 38 % | 0.0 % | 2.63 GB 15 % | 133 MB/s 16 % | 26,113 52 % | 3.0 % | 20.4 % | 149 Mbps 15 % | – |
| bench-validator-08 | validator | 42.7 % | 22.9 % | 6.87 GB 40 % | 0.0 % | 3.04 GB 18 % | 133 MB/s 16 % | 26,414 53 % | 2.9 % | 20.5 % | 142 Mbps 14 % | – |
| bench-validator-09 | validator | 42.5 % | 22.2 % | 6.70 GB 39 % | 0.0 % | 2.95 GB 17 % | 132 MB/s 16 % | 25,990 52 % | 2.8 % | 20.5 % | 138 Mbps 14 % | – |
| bench-validator-10 | validator | 40.6 % | 21.5 % | 6.52 GB 38 % | 0.0 % | 2.97 GB 17 % | 133 MB/s 16 % | 25,953 52 % | 3.5 % | 20.5 % | 139 Mbps 14 % | – |
| bench-observer-11 | observer | 24.2 % | 11.1 % | 5.22 GB 30 % | 0.0 % | 0.98 GB 6 % | 122 MB/s 15 % | 26,139 52 % | 2.2 % | 18.2 % | 21 Mbps 2 % | – |
Small figures are the share of capacity: RAM for RSS, the PD limit for disk write (VM cap applied), 1 Gbps for network. Hover a cell for its JSON path.
A = Fleet c3-hdb · c3-highcpu-8 · r20261114-0000
B = Fleet c3-hdb · c3-highcpu-8 · r20261113-1630
tight c3-highcpu-8 validators at 8,337 sustained TPS: tight on disk. Disk tight: write IOPS p95 26,414, 52.8 % of the 50,000 IOPS limit (bench-validator-08), tight against the 60 % line. CPU has headroom: CPU p95 43.3 % of 8 vCPUs (bench-validator-03). Memory has headroom: memory used p95 25.1 % of RAM (bench-validator-06); telcoin RSS p95 7.56 GB, 44.0 % of 16 GiB RAM (bench-validator-05); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 149 Mbps, 14.9 % of 1 Gbps (bench-validator-07).
tight c3-highcpu-8 observers at 8,337 sustained TPS: tight on disk. Disk tight: write IOPS p95 26,139, 52.3 % of the 50,000 IOPS limit (bench-observer-11), tight against the 60 % line. CPU has headroom: CPU p95 24.2 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 11.1 % of RAM (bench-observer-11); telcoin RSS p95 5.22 GB, 30.4 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 21 Mbps, 2.1 % of 1 Gbps (bench-observer-11).
tight c3-highcpu-8 validators at 8,539 sustained TPS: tight on disk. Disk tight: write IOPS p95 27,058, 54.1 % of the 50,000 IOPS limit (bench-validator-09), tight against the 60 % line. CPU has headroom: CPU p95 43.6 % of 8 vCPUs (bench-validator-09). Memory has headroom: memory used p95 27.3 % of RAM (bench-validator-05); telcoin RSS p95 7.04 GB, 41.0 % of 16 GiB RAM (bench-validator-10); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 153 Mbps, 15.3 % of 1 Gbps (bench-validator-07).
tight c3-highcpu-8 observers at 8,539 sustained TPS: tight on disk. Disk tight: write IOPS p95 27,047, 54.1 % of the 50,000 IOPS limit (bench-observer-11), tight against the 60 % line. CPU has headroom: CPU p95 23.9 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 10.4 % of RAM (bench-observer-11); telcoin RSS p95 5.11 GB, 29.7 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 11 Mbps, 1.1 % of 1 Gbps (bench-observer-11).
Validators: Fleet A (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network.
Observers: Fleet A (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network.
| KPI | Fleet A | Fleet B | B ÷ A |
|---|---|---|---|
| Sustained mixed TPS (own) | 8,337.1 tx/s | 8,539.0 tx/s | 1.02× |
| Sustained mixed TPS (chain) | 8,337.1 tx/s | 8,539.0 tx/s | 1.02× |
| Landed / accepted | 99.9 % | 99.9 % | 1.00× |
| Reverted (own) | 0 | 0 | – |
| Latency p50 | 990 ms | 858 ms | 0.87× |
| Latency p95 | 3,879 ms | 3,528 ms | 0.91× |
| Gas / s (own) | 363M gas/s | 365M gas/s | 1.01× |
| Capacity at mix | 16,723.6 tx/s | 16,530.3 tx/s | 0.99× |
| Signal | Class | Fleet A | Fleet B | Line |
|---|---|---|---|---|
| CPU p95 | validator | 43.3 % headroom | 43.6 % headroom | > 70 % |
| Memory used p95 | validator | 25.1 % headroom | 27.3 % headroom | > 75 % |
| telcoin RSS p95 | validator | 7.56 GB 44.0 % headroom | 7.04 GB 41.0 % headroom | > 85 % |
| PSI memory max | validator | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | validator | 133 MB/s 15.9 % headroom | 136 MB/s 16.2 % headroom | > 60 % |
| Write IOPS p95 | validator | 26,414 52.8 % tight | 27,058 54.1 % tight | > 60 % |
| PSI io max | validator | 4.9 % headroom | 4.1 % headroom | > 20 % |
| Root disk max | validator | 21.3 % headroom | 20.1 % headroom | > 80 % |
| Network p95 | validator | 149 Mbps 14.9 % headroom | 153 Mbps 15.3 % headroom | > 50 % |
| CPU p95 | observer | 24.2 % headroom | 23.9 % headroom | > 70 % |
| Memory used p95 | observer | 11.1 % headroom | 10.4 % headroom | > 75 % |
| telcoin RSS p95 | observer | 5.22 GB 30.4 % headroom | 5.11 GB 29.7 % headroom | > 85 % |
| PSI memory max | observer | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | observer | 122 MB/s 14.5 % headroom | 125 MB/s 14.9 % headroom | > 60 % |
| Write IOPS p95 | observer | 26,139 52.3 % tight | 27,047 54.1 % tight | > 60 % |
| PSI io max | observer | 2.2 % headroom | 2.5 % headroom | > 20 % |
| Root disk max | observer | 18.2 % headroom | 18.6 % headroom | > 80 % |
| Network p95 | observer | 21 Mbps 2.1 % headroom | 11 Mbps 1.1 % headroom | > 50 % |
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
Not collected: reference chains.