Throughput is measured from blocks the chain produced (own senders vs every sender), resource use from Cloud Monitoring, Prometheus and the on-node sampler. Summaries cover the mixed phase. Hover any number for the JSON field it came from.
Landed counts only our own senders; latency is submit → block timestamp.
| Type | Submitted | Accepted | Landed | Reverted | Unlanded | Skips | Gas limit | Gas used avg | p50 | p95 |
|---|---|---|---|---|---|---|---|---|---|---|
| native_transfer | 5.56M | 5.49M | 5.48M | 0 | 3.15K | 0 | 24.1K | 21K | 853 ms | 3,507 ms |
| batched_self_send | 3.03M | 2.98M | 2.97M | 0 | 2.74K | 0 | 24.1K | 21K | 881 ms | 3,577 ms |
| uniswap_v2_swap | 2.72M | 2.68M | 2.67M | 0 | 2.19K | 0 | 180K | 106K | 854 ms | 3,504 ms |
| stablecoin_transfer | 2.27M | 2.23M | 2.23M | 0 | 1.85K | 0 | 75K | 43.9K | 853 ms | 3,500 ms |
| wtel_wrap | 1.81M | 1.78M | 1.78M | 0 | 1.45K | 0 | 50K | 27.9K | 853 ms | 3,506 ms |
| wtel_unwrap | 1.21M | 1.19M | 1.18M | 0 | 955 | 0 | 45K | 35.2K | 853 ms | 3,503 ms |
| stablecoin_approve | 908K | 892K | 891K | 0 | 731 | 0 | 60K | 31.8K | 853 ms | 3,502 ms |
| uniswap_v2_add_liquidity | 606K | 595K | 595K | 0 | 502 | 0 | 250K | 128K | 854 ms | 3,510 ms |
| uniswap_v4_swap | 454K | 446K | 446K | 0 | 397 | 0 | 250K | 124K | 855 ms | 3,507 ms |
| contract_deploy | 453K | 445K | 445K | 0 | 381 | 0 | 140K | 74.2K | 853 ms | 3,503 ms |
| uniswap_v3_swap | 303K | 298K | 298K | 0 | 238 | 0 | 220K | 120K | 855 ms | 3,499 ms |
| uniswap_v2_remove_liquidity | 302K | 296K | 296K | 0 | 225 | 0 | 220K | 129K | 854 ms | 3,494 ms |
| faucet_drip | 124K | 124K | 124K | 0 | 114 | 633K | 150K | 107K | 832 ms | 13,729 ms |
| Node | Submitted | Accepted | Accepted % | Top errors | Send p50 | Send p95 |
|---|---|---|---|---|---|---|
| bench-validator-01 bench-validator-01:8545 | 1.44M | 1.44M | 100.0 % | known 109 | 3 ms | 17 ms |
| bench-validator-02 bench-validator-02:8545 | 1.33M | 1.31M | 98.6 % | known 9.86K | 5 ms | 25 ms |
| bench-validator-03 bench-validator-03:8545 | 3.46M | 3.46M | 100.0 % | none | 6 ms | 18 ms |
| bench-validator-04 bench-validator-04:8545 | 2.41M | 2.41M | 100.0 % | known 9 | 6 ms | 19 ms |
| bench-validator-05 bench-validator-05:8545 | 1.39M | 440K | 31.7 % | other 948K, known 83 | 1 ms | 24 ms |
| bench-validator-06 bench-validator-06:8545 | 502K | 457K | 91.1 % | known 22.4K | 6 ms | 39 ms |
| bench-validator-07 bench-validator-07:8545 | 2.74M | 2.74M | 100.0 % | none | 5 ms | 17 ms |
| bench-validator-08 bench-validator-08:8545 | 3.06M | 3.06M | 99.9 % | known 1.35K | 6 ms | 20 ms |
| bench-validator-09 bench-validator-09:8545 | 2.68M | 2.65M | 98.8 % | known 16.7K | 4 ms | 15 ms |
| bench-validator-10 bench-validator-10:8545 | 1.55M | 1.52M | 98.0 % | known 11.8K | 6 ms | 21 ms |
Shaded: phases outside the mixed window. Hairlines: epoch closes. Dashed orange: chaos events.
own senders vs chain total, per block-timestamp second
own vs chain gas used, per block-timestamp second
compute.googleapis.com/instance/cpu/utilization, 60-s mean, % of all vCPUs; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 22.3 % | 23.9 % | 26.4 % | 26.4 % |
| bench-validator-01 | validator | 37.1 % | 40.1 % | 48.3 % | 48.3 % |
| bench-validator-02 | validator | 37.2 % | 40.0 % | 45.3 % | 45.3 % |
| bench-validator-03 | validator | 40.5 % | 43.4 % | 48.4 % | 48.4 % |
| bench-validator-04 | validator | 38.6 % | 41.2 % | 48.3 % | 48.3 % |
| bench-validator-05 | validator | 31.2 % | 38.2 % | 45.7 % | 45.7 % |
| bench-validator-06 | validator | 36.8 % | 39.3 % | 46.0 % | 46.0 % |
| bench-validator-07 | validator | 38.9 % | 42.8 % | 47.2 % | 47.2 % |
| bench-validator-08 | validator | 39.6 % | 43.0 % | 47.2 % | 47.2 % |
| bench-validator-09 | validator | 39.0 % | 43.6 % | 49.1 % | 49.1 % |
| bench-validator-10 | validator | 38.1 % | 40.2 % | 48.7 % | 48.7 % |
One row per node, over the mixed phase.
agent processes/rss_usage summed over telcoin pids; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 3.43 GiB | 4.76 GiB | 4.87 GiB | 4.87 GiB |
| bench-validator-01 | validator | 5.55 GiB | 6.37 GiB | 6.41 GiB | 6.41 GiB |
| bench-validator-02 | validator | 5.68 GiB | 6.51 GiB | 6.61 GiB | 6.61 GiB |
| bench-validator-03 | validator | 5.01 GiB | 5.95 GiB | 6.00 GiB | 6.00 GiB |
| bench-validator-04 | validator | 5.03 GiB | 5.90 GiB | 5.99 GiB | 5.99 GiB |
| bench-validator-05 | validator | 5.20 GiB | 6.36 GiB | 6.40 GiB | 6.40 GiB |
| bench-validator-06 | validator | 5.30 GiB | 6.25 GiB | 6.29 GiB | 6.29 GiB |
| bench-validator-07 | validator | 4.93 GiB | 5.64 GiB | 5.71 GiB | 5.71 GiB |
| bench-validator-08 | validator | 5.15 GiB | 5.98 GiB | 6.11 GiB | 6.11 GiB |
| bench-validator-09 | validator | 5.00 GiB | 5.70 GiB | 5.80 GiB | 5.80 GiB |
| bench-validator-10 | validator | 5.67 GiB | 6.55 GiB | 6.70 GiB | 6.70 GiB |
One row per node, over the mixed phase.
agent memory/percent_used, state=used; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 9.3 % | 10.4 % | 10.4 % | 10.4 % |
| bench-validator-01 | validator | 23.5 % | 24.5 % | 24.6 % | 24.6 % |
| bench-validator-02 | validator | 23.5 % | 24.5 % | 24.6 % | 24.6 % |
| bench-validator-03 | validator | 19.1 % | 19.7 % | 19.7 % | 19.7 % |
| bench-validator-04 | validator | 19.6 % | 20.3 % | 20.3 % | 20.3 % |
| bench-validator-05 | validator | 19.8 % | 27.3 % | 27.5 % | 27.5 % |
| bench-validator-06 | validator | 20.6 % | 21.1 % | 21.2 % | 21.2 % |
| bench-validator-07 | validator | 19.6 % | 19.8 % | 19.8 % | 19.8 % |
| bench-validator-08 | validator | 20.6 % | 21.2 % | 21.3 % | 21.3 % |
| bench-validator-09 | validator | 19.9 % | 20.5 % | 20.6 % | 20.6 % |
| bench-validator-10 | validator | 23.1 % | 24.1 % | 24.1 % | 24.1 % |
One row per node, over the mixed phase.
instance/disk/{read,write}_bytes_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 111.4 MiB/s | 118.9 MiB/s | 125.8 MiB/s | 125.8 MiB/s | 800.0 MiB/s |
| bench-validator-01 | validator | 120.6 MiB/s | 124.7 MiB/s | 126.0 MiB/s | 126.0 MiB/s | 800.0 MiB/s |
| bench-validator-02 | validator | 122.3 MiB/s | 129.2 MiB/s | 130.1 MiB/s | 130.1 MiB/s | 800.0 MiB/s |
| bench-validator-03 | validator | 122.9 MiB/s | 129.0 MiB/s | 131.5 MiB/s | 131.5 MiB/s | 800.0 MiB/s |
| bench-validator-04 | validator | 122.5 MiB/s | 128.0 MiB/s | 129.3 MiB/s | 129.3 MiB/s | 800.0 MiB/s |
| bench-validator-05 | validator | 101.2 MiB/s | 123.2 MiB/s | 125.9 MiB/s | 125.9 MiB/s | 800.0 MiB/s |
| bench-validator-06 | validator | 122.5 MiB/s | 129.4 MiB/s | 129.7 MiB/s | 129.7 MiB/s | 800.0 MiB/s |
| bench-validator-07 | validator | 122.6 MiB/s | 128.1 MiB/s | 128.9 MiB/s | 128.9 MiB/s | 800.0 MiB/s |
| bench-validator-08 | validator | 122.8 MiB/s | 128.2 MiB/s | 129.4 MiB/s | 129.4 MiB/s | 800.0 MiB/s |
| bench-validator-09 | validator | 122.8 MiB/s | 130.1 MiB/s | 131.7 MiB/s | 131.7 MiB/s | 800.0 MiB/s |
| bench-validator-10 | validator | 122.6 MiB/s | 129.4 MiB/s | 131.6 MiB/s | 131.6 MiB/s | 800.0 MiB/s |
instance/disk/{read,write}_ops_count rate, all devices; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max | Limit |
|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 24.6K IOPS | 27.1K IOPS | 28.1K IOPS | 28.1K IOPS | 50K IOPS |
| bench-validator-01 | validator | 24.5K IOPS | 26.2K IOPS | 26.4K IOPS | 26.4K IOPS | 50K IOPS |
| bench-validator-02 | validator | 24.7K IOPS | 26.9K IOPS | 27.2K IOPS | 27.2K IOPS | 50K IOPS |
| bench-validator-03 | validator | 24.7K IOPS | 26.7K IOPS | 27.6K IOPS | 27.6K IOPS | 50K IOPS |
| bench-validator-04 | validator | 24.7K IOPS | 26.3K IOPS | 26.9K IOPS | 26.9K IOPS | 50K IOPS |
| bench-validator-05 | validator | 20.4K IOPS | 26.2K IOPS | 26.6K IOPS | 26.6K IOPS | 50K IOPS |
| bench-validator-06 | validator | 24.7K IOPS | 26.8K IOPS | 27.2K IOPS | 27.2K IOPS | 50K IOPS |
| bench-validator-07 | validator | 24.7K IOPS | 26.4K IOPS | 27.3K IOPS | 27.3K IOPS | 50K IOPS |
| bench-validator-08 | validator | 24.7K IOPS | 26.4K IOPS | 27K IOPS | 27K IOPS | 50K IOPS |
| bench-validator-09 | validator | 24.7K IOPS | 27.1K IOPS | 27.4K IOPS | 27.4K IOPS | 50K IOPS |
| bench-validator-10 | validator | 24.7K IOPS | 26.8K IOPS | 27.6K IOPS | 27.6K IOPS | 50K IOPS |
| Node | Machine | vCPU | Disk | IOPS limit | MiB/s limit | Basis |
|---|---|---|---|---|---|---|
| bench-observer-11 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-01 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-02 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-03 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-04 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-05 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-06 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-07 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-08 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-09 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
| bench-validator-10 | c3-highcpu-8 | 8 | hyperdisk-balanced 100 GB | 50K IOPS | 800.0 MiB/s | hyperdisk-balanced 100 GB: provisioned 50,000 IOPS, provisioned 800 MiB/s; c3 8 vCPU cap 50,000 IOPS / 800 MiB/s; reads and writes share this budget |
instance/network/sent_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 1.9 Mbps | 2.2 Mbps | 3.5 Mbps | 3.5 Mbps |
| bench-validator-01 | validator | 125.1 Mbps | 133.4 Mbps | 136.0 Mbps | 136.0 Mbps |
| bench-validator-02 | validator | 126.1 Mbps | 135.6 Mbps | 136.2 Mbps | 136.2 Mbps |
| bench-validator-03 | validator | 134.9 Mbps | 143.9 Mbps | 144.6 Mbps | 144.6 Mbps |
| bench-validator-04 | validator | 129.8 Mbps | 136.2 Mbps | 138.1 Mbps | 138.1 Mbps |
| bench-validator-05 | validator | 104.2 Mbps | 125.8 Mbps | 128.5 Mbps | 128.5 Mbps |
| bench-validator-06 | validator | 124.1 Mbps | 132.9 Mbps | 150.0 Mbps | 150.0 Mbps |
| bench-validator-07 | validator | 134.4 Mbps | 152.6 Mbps | 191.9 Mbps | 191.9 Mbps |
| bench-validator-08 | validator | 132.8 Mbps | 142.7 Mbps | 145.1 Mbps | 145.1 Mbps |
| bench-validator-09 | validator | 131.1 Mbps | 141.3 Mbps | 150.8 Mbps | 150.8 Mbps |
| bench-validator-10 | validator | 126.7 Mbps | 135.1 Mbps | 142.7 Mbps | 142.7 Mbps |
instance/network/received_bytes_count rate; summaries over the mixed phase
| Node | Class | Mean | p95 | Peak 60-s mean | Max |
|---|---|---|---|---|---|
| bench-observer-11 | observer | 8.3 Mbps | 10.5 Mbps | 30.9 Mbps | 30.9 Mbps |
| bench-validator-01 | validator | 11.2 Mbps | 13.4 Mbps | 17.9 Mbps | 17.9 Mbps |
| bench-validator-02 | validator | 11.1 Mbps | 13.8 Mbps | 14.4 Mbps | 14.4 Mbps |
| bench-validator-03 | validator | 15.1 Mbps | 16.9 Mbps | 18.0 Mbps | 18.0 Mbps |
| bench-validator-04 | validator | 13.0 Mbps | 14.9 Mbps | 17.5 Mbps | 17.5 Mbps |
| bench-validator-05 | validator | 15.5 Mbps | 45.7 Mbps | 90.4 Mbps | 90.4 Mbps |
| bench-validator-06 | validator | 9.8 Mbps | 11.6 Mbps | 14.5 Mbps | 14.5 Mbps |
| bench-validator-07 | validator | 13.6 Mbps | 16.2 Mbps | 17.8 Mbps | 17.8 Mbps |
| bench-validator-08 | validator | 14.3 Mbps | 17.5 Mbps | 17.7 Mbps | 17.7 Mbps |
| bench-validator-09 | validator | 13.5 Mbps | 16.7 Mbps | 19.3 Mbps | 19.3 Mbps |
| bench-validator-10 | validator | 11.4 Mbps | 14.9 Mbps | 19.0 Mbps | 19.0 Mbps |
| Metric | Query | Class | Nodes | Median of node means | Worst peak 60-s mean | Max |
|---|---|---|---|---|---|---|
| tn_primary_commit_latency_seconds | p95 | validator | 10 | 1,000 ms | 17.3 s | 23.8 s |
| tn_engine_execution_duration_seconds | p95 | validator | 10 | 1,540 ms | 2,687 ms | 3,250 ms |
| tn_engine_execution_duration_seconds | p95 | observer | 1 | 1,274 ms | 2,616 ms | 3,188 ms |
| tn_worker_batch_size_bytes | p95 | validator | 10 | 235.6 KiB | 246.4 KiB | 246.4 KiB |
| tn_worker_batch_transactions | p95 | validator | 9 | 491 | 1.48K | 1.48K |
| tn_executor_output_transactions | p95 | validator | 10 | 10.7K | 23.8K | 23.8K |
| tn_primary_consensus_output_bytes | max | validator | 10 | 513.9 KiB | 1.5 MiB | 1.6 MiB |
| tn_primary_consensus_output_bytes | max | observer | 1 | 516.1 KiB | 1.3 MiB | 1.9 MiB |
| tn_engine_queued_outputs | max | validator | 10 | 6.56 | 8 | 8 |
| tn_engine_queued_outputs | max | observer | 1 | 1.49 | 7.75 | 8 |
| tn_primary_round | max | validator | 10 | 1.6K | 3.39K | 3.41K |
| tn_primary_round | max | observer | 1 | 1.59K | 3.32K | 3.38K |
| tn_primary_committed_round | max | validator | 10 | 1.6K | 3.39K | 3.41K |
| tn_primary_committed_round | max | observer | 1 | 1.59K | 3.32K | 3.38K |
| tn_epoch_current | max | validator | 10 | 4.17 | 5 | 5 |
| tn_epoch_current | max | observer | 1 | 4.17 | 5 | 5 |
| tn_batch_builder_pending_pool_transactions | max | validator | 10 | 505 | 6.69K | 9.81K |
| tn_batch_builder_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_process_resident_memory_bytes | max | validator | 10 | 5.28 GiB | 6.63 GiB | 6.75 GiB |
| reth_process_resident_memory_bytes | max | observer | 1 | 3.42 GiB | 4.86 GiB | 4.90 GiB |
| tn_primary_subdags_committed_total | rate | validator | 10 | 1.26 | 1.44 | 1.44 |
| tn_primary_subdags_committed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_round_lag | lag | validator | 10 | 2.23 | 3.25 | 6 |
| tn_primary_round_lag | lag | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_seal_failures_total | rate | validator | 8 | 0.000409 | 0.0167 | 0.0222 |
| tn_node_mode | max | validator | 10 | 0 | 1 | 1 |
| tn_node_mode | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_total | rate | validator | 10 | 22.1 | 31.1 | 31.4 |
| tn_primary_vote_requests_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_created_total | rate | validator | 10 | 21 | 25.9 | 25.9 |
| tn_primary_votes_created_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_recast_total | rate | validator | 10 | 0 | 0.167 | 0.222 |
| tn_primary_votes_recast_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_requests_too_new_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="exec_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="lead_sleep"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="parents"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="persist"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="prelock_wait"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="sync_batches"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="validate"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_vote_timeouts_total{stage="vote_store"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_stage_seconds{stage="exec_wait"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lead_sleep"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="lock"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="parents"} | p95 | validator | 10 | 1 ms | 2 ms | 2 ms |
| tn_primary_vote_stage_seconds{stage="persist"} | p95 | validator | 10 | 3 ms | 5 ms | 5 ms |
| tn_primary_vote_stage_seconds{stage="prelock_wait"} | p95 | validator | 10 | 610 ms | 2,425 ms | 2,425 ms |
| tn_primary_vote_stage_seconds{stage="sync_batches"} | p95 | validator | 10 | 0 ms | 0 ms | 0 ms |
| tn_primary_vote_stage_seconds{stage="validate"} | p95 | validator | 10 | 3 ms | 19 ms | 19 ms |
| tn_primary_vote_stage_seconds{stage="vote_store"} | p95 | validator | 10 | 1 ms | 2 ms | 2 ms |
| tn_primary_leader_rounds_total{leader="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff",swapped="false",found="true"} | rate | validator | 10 | 0.128 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff",swapped="true",found="true"} | rate | validator | 10 | 0.0489 | 0.106 | 0.156 |
| tn_primary_leader_rounds_total{leader="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e",swapped="false",found="false"} | rate | validator | 10 | 0.00119 | 0.0215 | 0.0222 |
| tn_primary_leader_rounds_total{leader="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e",swapped="false",found="true"} | rate | validator | 10 | 0.126 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e",swapped="true",found="false"} | rate | validator | 10 | 0.000783 | 0.0167 | 0.0222 |
| tn_primary_leader_rounds_total{leader="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e",swapped="true",found="true"} | rate | validator | 10 | 0.0393 | 0.1 | 0.133 |
| tn_primary_leader_rounds_total{leader="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz",swapped="false",found="true"} | rate | validator | 10 | 0.0638 | 0.133 | 0.156 |
| tn_primary_leader_rounds_total{leader="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz",swapped="true",found="true"} | rate | validator | 10 | 0.00116 | 0.0442 | 0.066 |
| tn_primary_leader_rounds_total{leader="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r",swapped="false",found="true"} | rate | validator | 10 | 0.0045 | 0.106 | 0.133 |
| tn_primary_leader_rounds_total{leader="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r",swapped="true",found="true"} | rate | validator | 10 | 0.000358 | 0.0165 | 0.022 |
| tn_primary_leader_rounds_total{leader="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw",swapped="false",found="true"} | rate | validator | 10 | 0.127 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw",swapped="true",found="true"} | rate | validator | 10 | 0.0401 | 0.111 | 0.133 |
| tn_primary_leader_rounds_total{leader="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6",swapped="false",found="false"} | rate | validator | 10 | 0.00409 | 0.0778 | 0.111 |
| tn_primary_leader_rounds_total{leader="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6",swapped="false",found="true"} | rate | validator | 10 | 0.0638 | 0.15 | 0.156 |
| tn_primary_leader_rounds_total{leader="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6",swapped="true",found="false"} | rate | validator | 10 | 0.000734 | 0.033 | 0.044 |
| tn_primary_leader_rounds_total{leader="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6",swapped="true",found="true"} | rate | validator | 10 | 0.0151 | 0.122 | 0.133 |
| tn_primary_leader_rounds_total{leader="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1",swapped="false",found="false"} | rate | validator | 10 | 0.000358 | 0.0165 | 0.022 |
| tn_primary_leader_rounds_total{leader="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1",swapped="false",found="true"} | rate | validator | 10 | 0.124 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1",swapped="true",found="true"} | rate | validator | 10 | 0.0465 | 0.139 | 0.178 |
| tn_primary_leader_rounds_total{leader="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq",swapped="false",found="true"} | rate | validator | 10 | 0.127 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq",swapped="true",found="true"} | rate | validator | 10 | 0.039 | 0.117 | 0.156 |
| tn_primary_leader_rounds_total{leader="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ",swapped="false",found="true"} | rate | validator | 10 | 0.127 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ",swapped="true",found="true"} | rate | validator | 10 | 0.0481 | 0.133 | 0.178 |
| tn_primary_leader_rounds_total{leader="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3",swapped="false",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3",swapped="false",found="true"} | rate | validator | 10 | 0.0728 | 0.156 | 0.156 |
| tn_primary_leader_rounds_total{leader="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3",swapped="true",found="false"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_leader_rounds_total{leader="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3",swapped="true",found="true"} | rate | validator | 10 | 0.021 | 0.0889 | 0.111 |
| tn_primary_leader_commits_total{leader="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff"} | rate | validator | 10 | 0.177 | 0.256 | 0.311 |
| tn_primary_leader_commits_total{leader="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e"} | rate | validator | 10 | 0.165 | 0.244 | 0.289 |
| tn_primary_leader_commits_total{leader="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz"} | rate | validator | 10 | 0.065 | 0.167 | 0.2 |
| tn_primary_leader_commits_total{leader="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r"} | rate | validator | 10 | 0.00491 | 0.122 | 0.156 |
| tn_primary_leader_commits_total{leader="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw"} | rate | validator | 10 | 0.167 | 0.267 | 0.267 |
| tn_primary_leader_commits_total{leader="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6"} | rate | validator | 10 | 0.0789 | 0.25 | 0.267 |
| tn_primary_leader_commits_total{leader="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1"} | rate | validator | 10 | 0.17 | 0.272 | 0.311 |
| tn_primary_leader_commits_total{leader="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq"} | rate | validator | 10 | 0.166 | 0.25 | 0.267 |
| tn_primary_leader_commits_total{leader="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ"} | rate | validator | 10 | 0.175 | 0.259 | 0.289 |
| tn_primary_leader_commits_total{leader="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3"} | rate | validator | 10 | 0.0938 | 0.222 | 0.244 |
| tn_engine_queue_wait_seconds | p95 | validator | 10 | 8,008 ms | 21.5 s | 25.2 s |
| tn_engine_queue_wait_seconds | p95 | observer | 1 | 2,128 ms | 9,697 ms | 9,750 ms |
| tn_engine_persist_duration_seconds | p95 | validator | 10 | 474 ms | 914 ms | 927 ms |
| tn_engine_persist_duration_seconds | p95 | observer | 1 | 465 ms | 677 ms | 750 ms |
| tn_engine_busy_milliseconds_total | rate | validator | 10 | 986 | 1.01K | 1.05K |
| tn_engine_busy_milliseconds_total | rate | observer | 1 | 904 | 1K | 1.01K |
| tn_reth_persist_save_blocks_seconds | p95 | validator | 10 | 250 ms | 476 ms | 650 ms |
| tn_reth_persist_save_blocks_seconds | p95 | observer | 1 | 236 ms | 346 ms | 400 ms |
| tn_reth_persist_commit_seconds | p95 | validator | 10 | 250 ms | 467 ms | 472 ms |
| tn_reth_persist_commit_seconds | p95 | observer | 1 | 251 ms | 370 ms | 425 ms |
| tn_reth_persisted_accounts_total | rate | validator | 10 | 2.99K | 7.47K | 7.47K |
| tn_reth_persisted_accounts_total | rate | observer | 1 | 2.96K | 7.35K | 7.35K |
| tn_reth_persisted_storage_slots_total | rate | validator | 10 | 3.97K | 4.6K | 4.68K |
| tn_reth_persisted_storage_slots_total | rate | observer | 1 | 3.96K | 4.57K | 4.68K |
| tn_node_mode_transitions_total{to="cvv_active",reason="rejoin"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_node_mode_transitions_total{to="cvv_inactive",reason="epoch_behind"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="attempts_exhausted"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="epoch_closed"} | rate | validator | 1 | 0.0679 | 1.11 | 1.22 |
| tn_executor_rejoin_blocked_total{reason="epoch_mismatch"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="no_known_header"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_executor_rejoin_blocked_total{reason="round_gap"} | rate | validator | 1 | 0.267 | 1.23 | 1.29 |
| tn_executor_rejoin_blocked_total{reason="stale_head"} | rate | validator | 1 | 0.00669 | 0.187 | 0.25 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="committee"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_worker_sync_serves_denied_total{worker="0",tier="public"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="n2PTTgCQ5BpVPV1scTWG7eH97rTajbHamJHqwT3sEVD78UHvnZWQh6CdJVdxS4XYWNCPNh3WX3k35bZ8Z4auPmiXSw8jcJJCC6zmzFCyD8qD7AnsDvCsayweBC1zHvusPmr"} | rate | validator | 3 | 0.00952 | 1.5 | 2 |
| tn_worker_batch_requests_denied_total{worker="0",peer="ojnjgb2wFyUjzTxydgjVuYz9NZudiyZ42HAYhHfzh7xRwtafTJu6LGAPypt9rS7KzL3r3eApuCdrwd5HL9z3xxV2EFojLfXH4rFBy47GLnE3z4wP39whwX1D7bvqMw12skN"} | rate | validator | 3 | 0.0111 | 1.63 | 2.18 |
| tn_worker_batch_requests_denied_total{worker="0",peer="other"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_denied_total{worker="0",peer="pxKG6xH2cxvStH15MW9fhJYABPBvMw4Mzv5WdARz8DrhLL5fFc9E5yZWx2sAJv9cYY4uw5jLYSScFRbeqUd1R6BTXEnuRAqNHaLQM61WYKnviDtsYgb3UvRYh1JJuFvzJ2W"} | rate | validator | 3 | 0 | 0.15 | 0.2 |
| tn_worker_batch_requests_denied_total{worker="0",peer="s7bPgoys5Lf5DKaLhZiybr6SiogH78hCRgsR5fDdrYxunDjimFR7zJDcVcNcg2LiwQGGRxt8sNzLksasm5pZY1bhEQsCCagXrvm7wC4faTiy26fkNR1h7tM2Fd7tgqHMmCX"} | rate | validator | 3 | 0.0146 | 1.12 | 1.49 |
| tn_worker_batch_requests_denied_total{worker="0",peer="sPUdopQ9x1BRbEWwNBTFKiXt585Spgmm14qGd1SSsydpo6KvhoqQJ5ekYsbcu7oH6sXNkwanteVtSWHRtnvzayU46qdeTXPZEaCCkbpiNYvjrFHNrKchLYNtZ1ruGoy4CbG"} | rate | validator | 4 | 0.00698 | 1.7 | 2.27 |
| tn_worker_batch_requests_denied_total{worker="0",peer="sbQYUVCPr4Hv1MMVSNqzBjRPnpVkeu4TgdJubvXCKA5wND5Gia4SDQafJfyqU2tiWotf4wLKZEkvtnPasZbcyc47SNfYz5Pv4y2HieAtHRfNrsLmQMjXKn75cX99jg6t9us"} | rate | validator | 4 | 0.0096 | 1.53 | 2.04 |
| tn_worker_batch_requests_denied_total{worker="0",peer="tMRJcY2rYHeGbizi7Mx7VfksKVpzpPojBcxjX46rbZChTHiDHmk9URo6uCcvVid1gnGmnDhHo1CffUxeBcpmDWsNGnVz6u77CtToKv3SFhKNS2vdj2L7wDE6uJcgtQoDWSg"} | rate | validator | 3 | 0.0114 | 1.57 | 2.09 |
| tn_worker_batch_requests_denied_total{worker="0",peer="uV5rFtrqpQUa9JxdCDXjDgterkWBakX6JARNqcwKZ5H4anwJ2TbhVTs3JPaCvKqckRecBvx3c3xFWd98SbzHTpBXUk1whCVDFPpZYB7CFCukGD15EtnVQW3XmN5Jvo4kd6X"} | rate | validator | 3 | 0.00825 | 1.15 | 1.53 |
| tn_worker_batch_requests_denied_total{worker="0",peer="uWmSKMfEyTpK6gTwGDiPEFqpSUA77vbPFzed26SjRnVTiLvvZhMHD8XdviFYpmKAeaSYk2sbG3SGyZxLmA9bEyWTS5zVFhjhrkPfQzzKDbTY8oyUc8LfSZ7bujYACmuty5T"} | rate | validator | 4 | 0.0184 | 0.667 | 0.889 |
| tn_worker_batch_requests_denied_total{worker="0",peer="ye6GqWBvRQ7dJr6rdJRTo3WrcBbkW696U7dx3AngZLgz8EZmqkbWbQv5fUxKKy7B42pveckCDzbFTxuw6RqhckyGRP87R7L1yH49BV1Krd3mfU7z6p6to6wkBUqmRCxdeiT"} | rate | validator | 3 | 0 | 0.283 | 0.378 |
| tn_worker_batch_requests_failed_total{worker="0",peer="n2PTTgCQ5BpVPV1scTWG7eH97rTajbHamJHqwT3sEVD78UHvnZWQh6CdJVdxS4XYWNCPNh3WX3k35bZ8Z4auPmiXSw8jcJJCC6zmzFCyD8qD7AnsDvCsayweBC1zHvusPmr"} | rate | validator | 3 | 0.000635 | 0.417 | 0.556 |
| tn_worker_batch_requests_failed_total{worker="0",peer="ojnjgb2wFyUjzTxydgjVuYz9NZudiyZ42HAYhHfzh7xRwtafTJu6LGAPypt9rS7KzL3r3eApuCdrwd5HL9z3xxV2EFojLfXH4rFBy47GLnE3z4wP39whwX1D7bvqMw12skN"} | rate | validator | 3 | 0.00404 | 0.533 | 0.711 |
| tn_worker_batch_requests_failed_total{worker="0",peer="other"} | rate | validator | 2 | 0 | 0 | 0 |
| tn_worker_batch_requests_failed_total{worker="0",peer="pxKG6xH2cxvStH15MW9fhJYABPBvMw4Mzv5WdARz8DrhLL5fFc9E5yZWx2sAJv9cYY4uw5jLYSScFRbeqUd1R6BTXEnuRAqNHaLQM61WYKnviDtsYgb3UvRYh1JJuFvzJ2W"} | rate | validator | 3 | 0 | 0.15 | 0.2 |
| tn_worker_batch_requests_failed_total{worker="0",peer="s7bPgoys5Lf5DKaLhZiybr6SiogH78hCRgsR5fDdrYxunDjimFR7zJDcVcNcg2LiwQGGRxt8sNzLksasm5pZY1bhEQsCCagXrvm7wC4faTiy26fkNR1h7tM2Fd7tgqHMmCX"} | rate | validator | 3 | 0.00127 | 0.75 | 1 |
| tn_worker_batch_requests_failed_total{worker="0",peer="sPUdopQ9x1BRbEWwNBTFKiXt585Spgmm14qGd1SSsydpo6KvhoqQJ5ekYsbcu7oH6sXNkwanteVtSWHRtnvzayU46qdeTXPZEaCCkbpiNYvjrFHNrKchLYNtZ1ruGoy4CbG"} | rate | validator | 4 | 0.00254 | 0.483 | 0.644 |
| tn_worker_batch_requests_failed_total{worker="0",peer="sbQYUVCPr4Hv1MMVSNqzBjRPnpVkeu4TgdJubvXCKA5wND5Gia4SDQafJfyqU2tiWotf4wLKZEkvtnPasZbcyc47SNfYz5Pv4y2HieAtHRfNrsLmQMjXKn75cX99jg6t9us"} | rate | validator | 4 | 0.00556 | 0.35 | 0.467 |
| tn_worker_batch_requests_failed_total{worker="0",peer="tMRJcY2rYHeGbizi7Mx7VfksKVpzpPojBcxjX46rbZChTHiDHmk9URo6uCcvVid1gnGmnDhHo1CffUxeBcpmDWsNGnVz6u77CtToKv3SFhKNS2vdj2L7wDE6uJcgtQoDWSg"} | rate | validator | 3 | 0.00444 | 0.333 | 0.444 |
| tn_worker_batch_requests_failed_total{worker="0",peer="uV5rFtrqpQUa9JxdCDXjDgterkWBakX6JARNqcwKZ5H4anwJ2TbhVTs3JPaCvKqckRecBvx3c3xFWd98SbzHTpBXUk1whCVDFPpZYB7CFCukGD15EtnVQW3XmN5Jvo4kd6X"} | rate | validator | 3 | 0.0019 | 0.767 | 1.02 |
| tn_worker_batch_requests_failed_total{worker="0",peer="uWmSKMfEyTpK6gTwGDiPEFqpSUA77vbPFzed26SjRnVTiLvvZhMHD8XdviFYpmKAeaSYk2sbG3SGyZxLmA9bEyWTS5zVFhjhrkPfQzzKDbTY8oyUc8LfSZ7bujYACmuty5T"} | rate | validator | 4 | 0.00317 | 1.09 | 1.44 |
| tn_worker_batch_requests_failed_total{worker="0",peer="ye6GqWBvRQ7dJr6rdJRTo3WrcBbkW696U7dx3AngZLgz8EZmqkbWbQv5fUxKKy7B42pveckCDzbFTxuw6RqhckyGRP87R7L1yH49BV1Krd3mfU7z6p6to6wkBUqmRCxdeiT"} | rate | validator | 3 | 0 | 0 | 0 |
| tn_node_epoch_record_signers | max | validator | 10 | 7.17 | 10 | 10 |
| tn_node_epoch_record_signers | max | observer | 1 | 7.2 | 8 | 8 |
| tn_node_epoch_record_missing_signers_total{signer="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="47oQDzXUa9Kv8fzSat8zoggtm4MsQLWEiS8FG8u1HPff"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="4W4ApMvFcM9ipf73ecsU25MbSrzHu5m9kqJE5A17jT5e"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz"} | rate | validator | 10 | 0.000356 | 0.0165 | 0.022 |
| tn_node_epoch_record_missing_signers_total{signer="58pEnTaq2dYqsp6y1gGHu4wf5S9Ux6sJQepAbKcqR4qz"} | rate | observer | 1 | 0.000377 | 0.0137 | 0.0182 |
| tn_node_epoch_record_missing_signers_total{signer="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="5RAEhJabcANXvwckXuhzdDJXWnvTQjwCZ7fESk2FfB1r"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="5d3ZtLyzWxL75opPW68CCqBm1sMEY1KdwV26FRuGfDUw"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6"} | rate | validator | 10 | 0.000818 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="5txV47P6jAHi1Uvm6sfy7osyiYRzGo3uJoX1SfaVq3n6"} | rate | observer | 1 | 0.00092 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1"} | rate | validator | 10 | 0.000818 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="A7xHY55JTpwRz2z2wPD4V6axQajsVkGJA1oG9AUjo5W1"} | rate | observer | 1 | 0.00092 | 0.0167 | 0.0222 |
| tn_node_epoch_record_missing_signers_total{signer="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq"} | rate | validator | 10 | 0 | 0.0133 | 0.0177 |
| tn_node_epoch_record_missing_signers_total{signer="Ab1dyJbmy75Q7dYxNn9RtQW2maYpA8FYz6i8MaX8RKfq"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_record_missing_signers_total{signer="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ"} | rate | validator | 10 | 0 | 0.0154 | 0.0206 |
| tn_node_epoch_record_missing_signers_total{signer="CaobP5Gbf4YAuKj3qEbhtJu4FQ6MfmMuGJKPUocjqwZZ"} | rate | observer | 1 | 0.000377 | 0.0137 | 0.0182 |
| tn_node_epoch_record_missing_signers_total{signer="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3"} | rate | validator | 10 | 0 | 0.0158 | 0.021 |
| tn_node_epoch_record_missing_signers_total{signer="CioqH7gHDTeNpghq5o2w8AZyRjNpheFWsRQqEQCMy6J3"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="batch"} | p95 | validator | 9 | 18.2 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"} | p95 | validator | 4 | 14.5 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="epoch_record"} | p95 | observer | 1 | 14.5 s | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="failure",class="vote"} | p95 | validator | 1 | 9,506 ms | 29.0 s | 29.0 s |
| tn_network_outbound_request_duration_seconds{outcome="response",class="batch"} | p95 | validator | 4 | 448 ms | 847 ms | 847 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="epoch_record"} | p95 | validator | 3 | 437 ms | 479 ms | 481 ms |
| tn_network_outbound_request_duration_seconds{outcome="response",class="vote"} | p95 | validator | 6 | 1,993 ms | 2,381 ms | 2,384 ms |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="in"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_closed_admission_denied_total{network="bench",direction="out"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="in",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="address_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="duplicate_connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="invalid_limits"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="peer_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="poisoned"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="prefix_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="process_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="sources_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_source_admission_denied_total{network="bench",direction="out",reason="unsupported_address"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_admission_mode | max | validator | 10 | 2 | 2 | 2 |
| tn_network_admission_mode | max | observer | 1 | 2 | 2 | 2 |
| tn_network_admission_fallback | max | validator | 10 | 0 | 1 | 1 |
| tn_network_admission_fallback | max | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | validator | 10 | 0 | 2.82 | 3.99 |
| tn_network_outbound_request_failures_total{kind="connection"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | validator | 10 | 12 | 95.5 | 138 |
| tn_network_outbound_request_failures_total{kind="dial"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | validator | 10 | 1.95 | 15.6 | 16.8 |
| tn_network_outbound_request_failures_total{kind="io"} | rate | observer | 1 | 0.488 | 1.09 | 1.38 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | validator | 10 | 0.418 | 5.2 | 5.96 |
| tn_network_outbound_request_failures_total{kind="timeout"} | rate | observer | 1 | 0.00283 | 0.0202 | 0.0222 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_outbound_request_failures_total{kind="unsupported"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | validator | 10 | 0.014 | 56.7 | 82.6 |
| tn_network_inbound_requests_shed_total{class="batch",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="certificate_sync",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | validator | 10 | 0.395 | 1.08 | 1.4 |
| tn_network_inbound_requests_shed_total{class="epoch_record",reason="unsubscribed"} | rate | observer | 1 | 0.183 | 0.499 | 0.62 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | validator | 10 | 0.013 | 0.133 | 0.176 |
| tn_network_inbound_requests_shed_total{class="gossip",reason="unsubscribed"} | rate | observer | 1 | 0.0129 | 0.0293 | 0.0364 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | validator | 10 | 0.0199 | 0.224 | 0.288 |
| tn_network_inbound_requests_shed_total{class="other",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="admission"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="queue_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="serve_budget"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | validator | 10 | 0.259 | 21.8 | 31.7 |
| tn_network_inbound_requests_shed_total{class="vote",reason="unsubscribed"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_network_inbound_request_service_seconds{class="batch"} | p95 | validator | 5 | 60 ms | 209 ms | 209 ms |
| tn_network_inbound_request_service_seconds{class="epoch_record"} | p95 | validator | 5 | 5 ms | 5 ms | 5 ms |
| tn_network_inbound_request_service_seconds{class="epoch_record"} | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| tn_network_inbound_request_service_seconds{class="vote"} | p95 | validator | 7 | 616 ms | 2,425 ms | 2,425 ms |
| tn_network_established_connections | max | validator | 10 | 19.3 | 20 | 20 |
| tn_network_established_connections | max | observer | 1 | 19 | 20 | 20 |
| tn_worker_sync_streams_shed_total{reason="budget_exhausted"} | rate | validator | 1 | 0 | 0 | 0 |
| tn_worker_sync_streams_shed_total{reason="denied"} | rate | validator | 10 | 0.0654 | 2.18 | 2.91 |
| tn_worker_sync_streams_shed_total{reason="denied"} | rate | observer | 1 | 0.0179 | 0.65 | 0.867 |
| tn_worker_batch_fetch_duration_seconds | p95 | validator | 10 | 5 ms | 1,618 ms | 2,180 ms |
| tn_executor_batch_fetch_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_executor_batch_fetch_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_received_total | rate | validator | 10 | 15.1 | 17.3 | 17.3 |
| tn_primary_votes_received_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_vote_request_failures_total | rate | validator | 10 | 0.0164 | 1.96 | 2.64 |
| tn_primary_vote_request_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificates_formed_total | rate | validator | 10 | 2.52 | 2.88 | 2.89 |
| tn_primary_certificates_formed_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_certificate_form_duration_seconds | p95 | validator | 10 | 442 ms | 838 ms | 942 ms |
| tn_engine_outputs_executed_total | rate | validator | 10 | 1.26 | 1.49 | 1.51 |
| tn_engine_outputs_executed_total | rate | observer | 1 | 1.26 | 1.48 | 1.53 |
| tn_engine_persist_failures_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_engine_persist_failures_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_node_consensus_sync_distance | max | validator | 10 | 8.39 | 506 | 507 |
| tn_node_consensus_sync_distance | max | observer | 1 | 2.23 | 10.2 | 12 |
| reth_db_table_size{table="AccountChangeSets"} | max | validator | 10 | 457 MiB | 728 MiB | 729 MiB |
| reth_db_table_size{table="AccountChangeSets"} | max | observer | 1 | 458 MiB | 727 MiB | 729 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | validator | 10 | 51 MiB | 80 MiB | 80 MiB |
| reth_db_table_size{table="AccountsHistory"} | max | observer | 1 | 51 MiB | 80 MiB | 80 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | validator | 10 | 33 MiB | 58 MiB | 58 MiB |
| reth_db_table_size{table="HashedAccounts"} | max | observer | 1 | 33 MiB | 58 MiB | 58 MiB |
| reth_db_table_size{table="HashedStorages"} | max | validator | 10 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="HashedStorages"} | max | observer | 1 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | validator | 10 | 28 MiB | 48 MiB | 48 MiB |
| reth_db_table_size{table="PlainAccountState"} | max | observer | 1 | 28 MiB | 48 MiB | 48 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | validator | 10 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="PlainStorageState"} | max | observer | 1 | 41 MiB | 44 MiB | 44 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | validator | 10 | 537 MiB | 857 MiB | 859 MiB |
| reth_db_table_size{table="StorageChangeSets"} | max | observer | 1 | 539 MiB | 857 MiB | 859 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | validator | 10 | 103 MiB | 127 MiB | 127 MiB |
| reth_db_table_size{table="StoragesHistory"} | max | observer | 1 | 103 MiB | 127 MiB | 127 MiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | validator | 10 | 998 MiB | 1.58 GiB | 1.58 GiB |
| reth_db_table_size{table="TransactionHashNumbers"} | max | observer | 1 | 1000 MiB | 1.58 GiB | 1.58 GiB |
| reth_db_table_size{table="TransactionSenders"} | max | validator | 10 | 514 MiB | 832 MiB | 834 MiB |
| reth_db_table_size{table="TransactionSenders"} | max | observer | 1 | 516 MiB | 832 MiB | 834 MiB |
| reth_db_freelist | max | validator | 10 | 27.8K | 51.9K | 60K |
| reth_db_freelist | max | observer | 1 | 27.6K | 48.2K | 57.1K |
| reth_database_transaction_commit_write_duration_seconds | p95 | validator | 9 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_write_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | validator | 10 | 248 ms | 465 ms | 470 ms |
| reth_database_transaction_commit_sync_duration_seconds | p95 | observer | 1 | 250 ms | 370 ms | 425 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | validator | 10 | 248 ms | 465 ms | 470 ms |
| reth_database_transaction_commit_whole_duration_seconds | p95 | observer | 1 | 250 ms | 370 ms | 425 ms |
| tn_engine_recover_seconds | p95 | validator | 10 | 220 ms | 944 ms | 944 ms |
| tn_engine_recover_seconds | p95 | observer | 1 | 194 ms | 686 ms | 889 ms |
| tn_engine_evm_seconds | p95 | validator | 10 | 265 ms | 562 ms | 662 ms |
| tn_engine_evm_seconds | p95 | observer | 1 | 258 ms | 514 ms | 638 ms |
| tn_engine_root_seconds | p95 | validator | 10 | 496 ms | 1,017 ms | 1,450 ms |
| tn_engine_root_seconds | p95 | observer | 1 | 479 ms | 823 ms | 888 ms |
| tn_engine_finish_seconds | p95 | validator | 10 | 245 ms | 492 ms | 492 ms |
| tn_engine_finish_seconds | p95 | observer | 1 | 162 ms | 272 ms | 319 ms |
| tn_engine_forward_wait_seconds | p95 | validator | 10 | 2,386 ms | 24.2 s | 26.1 s |
| tn_engine_forward_wait_seconds | p95 | observer | 1 | 99 ms | 2,294 ms | 2,369 ms |
| tn_engine_channel_depth | max | validator | 10 | 1.13 | 12.2 | 14 |
| tn_engine_channel_depth | max | observer | 1 | 0.0185 | 0.5 | 2 |
| tn_engine_executed_transactions_total | rate | validator | 10 | 7.44K | 14K | 14K |
| tn_engine_executed_transactions_total | rate | observer | 1 | 7.38K | 13.7K | 13.7K |
| tn_executor_execution_wait_seconds | p95 | validator | 10 | 190 ms | 30.0 s | 30.0 s |
| tn_primary_commit_run_ahead_blocked_total | rate | validator | 10 | 0 | 0 | 0 |
| tn_primary_commit_run_ahead_blocked_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_consensus_exec_lag_rounds | max | validator | 10 | 17.1 | 170 | 172 |
| tn_primary_consensus_exec_lag_rounds | max | observer | 1 | 3.41 | 18.5 | 22 |
| tn_primary_commit_lag_rounds | max | validator | 10 | 2.23 | 3.25 | 6 |
| tn_primary_commit_lag_rounds | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_rejoin_grace_holds_total | rate | validator | 10 | 0 | 0.0134 | 0.0178 |
| tn_primary_rejoin_grace_holds_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_inactive_total | rate | validator | 10 | 0 | 31.1 | 31.4 |
| tn_primary_votes_refused_inactive_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_primary_votes_refused_execution_behind_total | rate | validator | 10 | 0.00819 | 0.728 | 0.911 |
| tn_primary_votes_refused_execution_behind_total | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_pool_age_seconds | p95 | validator | 10 | 6,447 ms | 30.0 s | 30.0 s |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="bytes_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="drained"} | rate | validator | 10 | 1.7 | 2.96 | 3.07 |
| tn_batch_builder_builds_total{reason="drained"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | validator | 10 | 1.57 | 5.8 | 5.84 |
| tn_batch_builder_builds_total{reason="gas_full"} | rate | observer | 1 | 0 | 0 | 0 |
| tn_worker_batch_report_seconds | p95 | validator | 10 | 0 ms | 2 ms | 3 ms |
| tn_primary_digest_wait_seconds | p95 | validator | 10 | 1,378 ms | 30.0 s | 30.0 s |
| tn_primary_headers_proposed_total{reason="max_delay"} | rate | validator | 10 | 0.00859 | 0.122 | 0.156 |
| tn_primary_headers_proposed_total{reason="min_delay"} | rate | validator | 10 | 2.53 | 2.88 | 2.89 |
| tn_primary_headers_proposed_total{reason="parents_and_threshold"} | rate | validator | 10 | 0.0138 | 2.55 | 2.87 |
| tn_primary_digests_requeued_total{cause="canceled"} | rate | validator | 10 | 0.00709 | 25 | 26 |
| tn_primary_digests_requeued_total{cause="committed"} | rate | validator | 10 | 0.0114 | 5.37 | 6.4 |
| tn_executor_consensus_pack_persist_seconds | p95 | validator | 10 | 7 ms | 10 ms | 12 ms |
| tn_batch_builder_seal_duration_seconds | p95 | validator | 10 | 259 ms | 881 ms | 881 ms |
| tn_worker_quorum_wait_seconds | p95 | validator | 10 | 258 ms | 842 ms | 842 ms |
| tn_primary_proposer_pending_digests | max | validator | 10 | 0.877 | 34.8 | 40 |
| tn_primary_proposer_pending_digests | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_transactions | max | validator | 10 | 487 | 6.27K | 9.81K |
| reth_transaction_pool_pending_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_pool_transactions | max | validator | 10 | 379 | 5.69K | 5.97K |
| reth_transaction_pool_queued_pool_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_all_transactions_by_hash | max | validator | 10 | 2.52K | 9.12K | 9.88K |
| reth_transaction_pool_all_transactions_by_hash | max | observer | 1 | 0 | 0 | 0 |
| reth_storage_providers_database_save_blocks_total | p95 | validator | 10 | 176 ms | 343 ms | 365 ms |
| reth_storage_providers_database_save_blocks_total | p95 | observer | 1 | 146 ms | 256 ms | 267 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | validator | 9 | 175 ms | 343 ms | 365 ms |
| reth_storage_providers_database_save_blocks_mdbx | p95 | observer | 1 | 146 ms | 256 ms | 267 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | validator | 10 | 33 ms | 83 ms | 91 ms |
| reth_storage_providers_database_save_blocks_sf | p95 | observer | 1 | 34 ms | 52 ms | 60 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | validator | 10 | 162 ms | 400 ms | 402 ms |
| reth_storage_providers_database_save_blocks_commit_mdbx | p95 | observer | 1 | 169 ms | 335 ms | 379 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | validator | 9 | 11 ms | 16 ms | 17 ms |
| reth_storage_providers_database_save_blocks_commit_sf | p95 | observer | 1 | 9 ms | 11 ms | 12 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | validator | 10 | 27 ms | 49 ms | 55 ms |
| reth_storage_providers_database_save_blocks_write_state | p95 | observer | 1 | 24 ms | 43 ms | 46 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | validator | 10 | 17 ms | 28 ms | 32 ms |
| reth_storage_providers_database_save_blocks_write_hashed_state | p95 | observer | 1 | 16 ms | 24 ms | 25 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | validator | 10 | 23 ms | 36 ms | 39 ms |
| reth_storage_providers_database_save_blocks_write_trie_updates | p95 | observer | 1 | 21 ms | 33 ms | 34 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_preparation_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_wallclock_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_gc_cputime_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | validator | 9 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_audit_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | validator | 10 | 5 ms | 5 ms | 5 ms |
| reth_database_transaction_commit_ending_duration_seconds | p95 | observer | 1 | 5 ms | 5 ms | 5 ms |
| tn_node_switch{name="digest_dedupe"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="digest_dedupe"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_close_task_drain"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="epoch_record_state_check"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="epoch_record_state_check"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="mdbx_random_advice"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="parent_grace"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="parent_grace"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="pipelined_persist"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="quorum_exec_block"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block"} | max | observer | 1 | 1 | 1 | 1 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="quorum_exec_block_starved_rounds"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="recovery_threads"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="recovery_threads"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="rpc_admission_backlog_outputs"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | validator | 10 | 0 | 0 | 0 |
| tn_node_switch{name="sealed_tx_keep"} | max | observer | 1 | 0 | 0 | 0 |
| tn_node_switch{name="vote_queue_fix"} | max | validator | 10 | 1 | 1 | 1 |
| tn_node_switch{name="vote_queue_fix"} | max | observer | 1 | 1 | 1 | 1 |
| tn_primary_epoch_entry_rounds_behind | max | validator | 10 | 3 | 35 | 35 |
| tn_primary_epoch_entry_rounds_behind | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_random_advice_bytes | max | validator | 10 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_random_advice_bytes | max | observer | 1 | 8192.00 GiB | 8192.00 GiB | 8192.00 GiB |
| tn_reth_mdbx_used_bytes | max | validator | 10 | 2.87 GiB | 4.46 GiB | 4.46 GiB |
| tn_reth_mdbx_used_bytes | max | observer | 1 | 2.88 GiB | 4.46 GiB | 4.46 GiB |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | validator | 10 | 43.2M | 74.5M | 74.8M |
| tn_reth_mdbx_page_ops_total{op="cow"} | max | observer | 1 | 35M | 66.8M | 67.4M |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | validator | 10 | 6.37K | 9.29K | 9.35K |
| tn_reth_mdbx_page_ops_total{op="msync"} | max | observer | 1 | 5.32K | 8.57K | 8.63K |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | validator | 10 | 1.08M | 2.49M | 2.5M |
| tn_reth_mdbx_page_ops_total{op="newly"} | max | observer | 1 | 821K | 2.06M | 2.09M |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | validator | 10 | 754K | 1.17M | 1.17M |
| tn_reth_mdbx_page_ops_total{op="prefault"} | max | observer | 1 | 622K | 1.07M | 1.08M |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="spill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="unspill"} | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | validator | 10 | 1 | 1 | 1 |
| tn_reth_mdbx_page_ops_total{op="wops"} | max | observer | 1 | 1 | 1 | 1 |
| tn_reth_sealed_txs_marked | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_marked | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_executed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_repooled_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_dropped_evicted_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_cleared_lag_total | max | observer | 1 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | validator | 10 | 0 | 0 | 0 |
| tn_reth_sealed_txs_late_marks_refused_total | max | observer | 1 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_batch_builder_sealed_txs_skipped_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_pending_pool_size_bytes | max | validator | 10 | 173.4 KiB | 2.2 MiB | 3.4 MiB |
| reth_transaction_pool_pending_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_queued_pool_size_bytes | max | validator | 10 | 136.3 KiB | 2.0 MiB | 2.1 MiB |
| reth_transaction_pool_queued_pool_size_bytes | max | observer | 1 | 0 B | 0 B | 0 B |
| reth_transaction_pool_pending_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_pending_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_queued_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_basefee_transactions_evicted | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_removed_transactions | max | validator | 10 | 1.78M | 3.89M | 3.9M |
| reth_transaction_pool_removed_transactions | max | observer | 1 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | validator | 10 | 0 | 0 | 0 |
| reth_transaction_pool_invalid_transactions | max | observer | 1 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | validator | 10 | 0 | 0 | 0 |
| tn_worker_duplicate_digest_suppressed_total{worker="0"} | max | observer | 1 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_duplicate_digest_suppressed_total | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_recovery_fallback_total | max | observer | 1 | 0 | 0 | 0 |
| tn_node_epoch_close_backlog_outputs | max | validator | 10 | 6.47 | 16 | 16 |
| tn_node_epoch_close_backlog_outputs | max | observer | 1 | 1.75 | 2 | 2 |
| tn_node_epoch_close_detect_lag_seconds | max | validator | 10 | 1,213 ms | 408.8 s | 408.8 s |
| tn_node_epoch_close_detect_lag_seconds | max | observer | 1 | 1,571 ms | 2,951 ms | 2,951 ms |
| tn_node_epoch_close_drain_seconds | max | validator | 10 | 4,476 ms | 13.0 s | 13.0 s |
| tn_node_epoch_close_drain_seconds | max | observer | 1 | 946 ms | 1,351 ms | 1,351 ms |
| tn_node_epoch_record_vote_skipped_total{reason="certified_before_close"} | max | validator | 3 | 1.56 | 3 | 3 |
| tn_node_epoch_record_vote_skipped_total{reason="not_in_committee"} | max | observer | 1 | 4.06 | 5 | 5 |
| tn_primary_parent_grace_total{outcome="complete"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="expired"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_parent_grace_total{outcome="preempted"} | max | validator | 10 | 0 | 0 | 0 |
| tn_primary_late_parents_total{origin="older_round"} | max | validator | 10 | 5.03K | 8.22K | 8.22K |
| tn_primary_late_parents_total{origin="own_round_behind"} | max | validator | 10 | 27.5K | 32.8K | 32.9K |
| tn_node_execution_backlog_outputs | max | validator | 10 | 9.04 | 22.5 | 24 |
| tn_node_execution_backlog_outputs | max | observer | 1 | 2.23 | 9.75 | 12 |
| tn_engine_unpersisted_outputs | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_unpersisted_outputs | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_built_height | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_built_height | max | observer | 1 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | validator | 10 | 0 | 0 | 0 |
| tn_engine_close_barrier_holds_total | max | observer | 1 | 0 | 0 | 0 |
tn_primary_commit_latency_seconds
tn_primary_round − tn_primary_committed_round
tn_engine_queued_outputs, max by node
The share of each validator's own headers that formed a certificate, per window (tools/cert-rate.py on the node logs). bench-validator-02, bench-validator-06 and bench-validator-10 certified fewer than 90 % of their own headers in a window where they proposed at least 50. Own leader % is the validator's share of the committed leaders and not committed % the share of its certificates no committed leader reached, both over the run window; the fair share is 100 / 10 validators = 10.0 %. Greyed: fewer than 50 headers proposed, not judged.
| Validator | Region | blast 0 preamble | blast 0 warmup | blast 0 mixed | blast 0 cooldown | blast 1 preamble | blast 1 warmup | blast 1 mixed | blast 1 cooldown | blast 2 preamble | blast 2 warmup | blast 2 mixed | blast 2 cooldown | run | Own leader % | Not committed % |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-validator-01 | us-west2 (Los Angeles) | 100.0 | 100.0 | 100.0 | 100.0 | 97.9 | 100.0 | 99.9 | 100.0 | 100.0 | 100.0 | 99.9 | 100.0 | 100.0 | 13.3 | 0.1 |
| bench-validator-02 below 90 % | europe-west2 (London) | 101.9 | 100.0 | 99.9 | 100.0 | 102.2 | 100.0 | 99.8 | 100.0 | 100.0 | 99.6 | 78.6 | 93.0 | 88.7 | 2.8 | 45.8 |
| bench-validator-03 | australia-southeast1 (Sydney) | 101.9 | 99.6 | 100.1 | 100.0 | 100.0 | 100.0 | 99.9 | 100.0 | 99.4 | 100.0 | 99.9 | 100.3 | 100.0 | 13.3 | 0.0 |
| bench-validator-04 | asia-northeast1 (Tokyo) | 101.9 | 99.6 | 100.1 | 101.4 | 97.9 | 99.5 | 99.8 | 100.8 | 99.4 | 100.0 | 100.0 | 100.0 | 100.0 | 13.7 | 0.1 |
| bench-validator-05 | us-east4 (Ashburn) | 101.9 | 99.6 | 100.1 | 100.0 | 97.8 | 99.0 | 99.7 | 100.0 | 100.0 | 100.0 | 99.9 | – | 99.9 | 7.5 | 17.5 |
| bench-validator-06 below 90 % | europe-west4 (Netherlands) | 98.1 | 99.5 | 99.9 | 100.0 | 100.0 | 100.0 | 99.6 | 99.2 | 99.4 | 99.6 | 44.5 | 92.3 | 70.7 | 0.3 | 61.0 |
| bench-validator-07 | asia-southeast1 (Singapore) | 100.0 | 100.0 | 100.1 | 100.0 | 97.9 | 100.0 | 99.9 | 100.0 | 100.0 | 100.2 | 100.0 | 100.3 | 100.0 | 13.7 | 0.0 |
| bench-validator-08 | asia-east2 (Hong Kong) | 101.9 | 99.6 | 100.1 | 101.4 | 97.9 | 100.0 | 100.0 | 100.0 | 99.4 | 100.2 | 99.9 | 100.3 | 100.0 | 13.7 | 0.0 |
| bench-validator-09 | asia-northeast3 (Seoul) | 100.0 | 100.0 | 100.1 | 98.6 | 100.0 | 100.0 | 99.9 | 100.8 | 100.0 | 100.2 | 95.8 | 100.3 | 97.8 | 13.4 | 0.1 |
| bench-validator-10 below 90 % | asia-south1 (Mumbai) | 98.1 | 99.1 | 99.8 | 101.4 | 100.0 | 99.5 | 99.8 | 100.0 | 100.0 | 100.0 | 68.2 | 100.0 | 83.3 | 9.8 | 7.5 |
| Node | Class | Peak RssAnon | Peak RssFile | Peak cache | PSI mem | PSI io | PSI cpu | db growth | static_files growth | Root used | OOM Δ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| bench-observer-11 | observer | 674 MiB | 4.33 GiB | – | 0.0 % | 2.5 % | 1.0 % | 4.00 GiB | 5.24 GiB | 19.0 % | 0 |
| bench-validator-01 | validator | 2.98 GiB | 3.51 GiB | – | 0.0 % | 3.4 % | 6.2 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-02 | validator | 3.00 GiB | 3.60 GiB | – | 0.0 % | 4.1 % | 6.9 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-03 | validator | 2.20 GiB | 3.81 GiB | – | 0.0 % | 2.7 % | 9.3 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-04 | validator | 2.29 GiB | 3.71 GiB | – | 0.0 % | 3.2 % | 8.3 % | 4.00 GiB | 5.24 GiB | 20.0 % | 0 |
| bench-validator-05 | validator | 3.47 GiB | 4.21 GiB | – | 0.0 % | 3.0 % | 7.0 % | 4.00 GiB | 5.24 GiB | 20.0 % | 0 |
| bench-validator-06 | validator | 2.43 GiB | 3.88 GiB | – | 0.0 % | 3.1 % | 7.7 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-07 | validator | 2.22 GiB | 3.50 GiB | – | 0.0 % | 3.3 % | 8.0 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-08 | validator | 2.46 GiB | 3.68 GiB | – | 0.0 % | 3.1 % | 8.0 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
| bench-validator-09 | validator | 2.33 GiB | 3.49 GiB | – | 0.0 % | 3.1 % | 7.3 % | 4.00 GiB | 5.24 GiB | 20.0 % | 0 |
| bench-validator-10 | validator | 2.93 GiB | 3.88 GiB | – | 0.0 % | 3.2 % | 6.5 % | 4.00 GiB | 5.24 GiB | 21.0 % | 0 |
/proc/<pid>/status of the telcoin process on the node with the largest RSS
Not collected: consensus-db/cache size.
/proc/pressure on the node with the highest memory PSI
Per role and per node, from sizing.json (p95 over the landed window; PSI and RssAnon max over the whole sampler run). The lines follow the hardware doc's tier rules: CPU p95 over 70 % of vCPUs; memory used p95 over 75 % of RAM, or telcoin RSS p95 over 85 % of RAM; disk write p95 over 60 % of the PD MiB/s limit, or write IOPS p95 over 60 % of the IOPS limit; IO pressure over 20 % or memory pressure over 5 % (PSI some avg10, max); network p95 over 50 % of 1 Gbps; root disk over 80 %. A line crossed on any node puts the role at the limit; a nonzero reading within 15 points of its line is tight; otherwise the role has headroom. OOM kills, disk throttling and degraded disk performance are flagged too.
| Signal | Class | Worst node | Value | Share of capacity | Line | Nodes over / tight | Status |
|---|---|---|---|---|---|---|---|
| CPU p95 | validator | bench-validator-09 | 43.6 % | 43.6 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | validator | bench-validator-05 | 27.3 % | 27.3 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | validator | bench-validator-10 | 7.04 GB | 41.0 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | validator | bench-validator-01 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | validator | bench-validator-05 | 3.73 GB | 21.7 % | info | – | info |
| Disk write p95 | validator | bench-validator-09 | 136 MB/s | 16.2 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | validator | bench-validator-09 | 27,058 | 54.1 % | > 60 % | 0 / 10 | tight |
| PSI io max | validator | bench-validator-02 | 4.1 % | 4.1 % | > 20 % | 0 / 0 | headroom |
| Root disk max | validator | bench-validator-03 | 20.1 % | 20.1 % | > 80 % | 0 / 0 | headroom |
| Network p95 | validator | bench-validator-07 | 153 Mbps | 15.3 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | validator | – | – | – | info | – | not collected |
| OOM kills | validator | – | – | any | any | – | none |
| Disk throttling | validator | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | validator | – | – | any sample | any | – | none |
| CPU p95 | observer | bench-observer-11 | 23.9 % | 23.9 % | > 70 % | 0 / 0 | headroom |
| Memory used p95 | observer | bench-observer-11 | 10.4 % | 10.4 % | > 75 % | 0 / 0 | headroom |
| telcoin RSS p95 | observer | bench-observer-11 | 5.11 GB | 29.7 % | > 85 % | 0 / 0 | headroom |
| PSI memory max | observer | bench-observer-11 | 0.0 % | 0.0 % | > 5 % | 0 / 0 | headroom |
| RssAnon peak | observer | bench-observer-11 | 0.71 GB | 4.1 % | info | – | info |
| Disk write p95 | observer | bench-observer-11 | 125 MB/s | 14.9 % | > 60 % | 0 / 0 | headroom |
| Write IOPS p95 | observer | bench-observer-11 | 27,047 | 54.1 % | > 60 % | 0 / 1 | tight |
| PSI io max | observer | bench-observer-11 | 2.5 % | 2.5 % | > 20 % | 0 / 0 | headroom |
| Root disk max | observer | bench-observer-11 | 18.6 % | 18.6 % | > 80 % | 0 / 0 | headroom |
| Network p95 | observer | bench-observer-11 | 11 Mbps | 1.1 % | > 50 % | 0 / 0 | headroom |
| Consensus cache peak | observer | – | – | – | info | – | not collected |
| OOM kills | observer | – | – | any | any | – | none |
| Disk throttling | observer | – | – | metric type unavailable in project | any | – | not collected |
| Disk performance degraded | observer | – | – | any sample | any | – | none |
| Node | Class | CPU p95 | Memory used p95 | telcoin RSS p95 | PSI memory max | RssAnon peak | Disk write p95 | Write IOPS p95 | PSI io max | Root disk max | Network p95 | Consensus cache peak |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Line | > 70 % | > 75 % | > 85 % | > 5 % | info | > 60 % | > 60 % | > 20 % | > 80 % | > 50 % | info | |
| bench-validator-01 | validator | 40.1 % | 24.5 % | 6.84 GB 40 % | 0.0 % | 3.20 GB 19 % | 131 MB/s 16 % | 26,119 52 % | 3.4 % | 19.9 % | 133 Mbps 13 % | – |
| bench-validator-02 | validator | 40.0 % | 24.5 % | 6.99 GB 41 % | 0.0 % | 3.22 GB 19 % | 135 MB/s 16 % | 26,774 54 % | 4.1 % | 20.0 % | 136 Mbps 14 % | – |
| bench-validator-03 | validator | 43.4 % | 19.7 % | 6.38 GB 37 % | 0.0 % | 2.36 GB 14 % | 135 MB/s 16 % | 26,633 53 % | 2.7 % | 20.1 % | 144 Mbps 14 % | – |
| bench-validator-04 | validator | 41.2 % | 20.3 % | 6.33 GB 37 % | 0.0 % | 2.46 GB 14 % | 134 MB/s 16 % | 26,312 53 % | 3.2 % | 19.8 % | 136 Mbps 14 % | – |
| bench-validator-05 | validator | 38.2 % | 27.3 % | 6.79 GB 40 % | 0.0 % | 3.73 GB 22 % | 129 MB/s 15 % | 26,143 52 % | 3.8 % | 19.2 % | 126 Mbps 13 % | – |
| bench-validator-06 | validator | 39.3 % | 21.1 % | 6.71 GB 39 % | 0.0 % | 2.61 GB 15 % | 136 MB/s 16 % | 26,729 53 % | 3.1 % | 20.0 % | 133 Mbps 13 % | – |
| bench-validator-07 | validator | 42.8 % | 19.8 % | 6.06 GB 35 % | 0.0 % | 2.38 GB 14 % | 134 MB/s 16 % | 26,356 53 % | 3.3 % | 20.1 % | 153 Mbps 15 % | – |
| bench-validator-08 | validator | 43.0 % | 21.2 % | 6.42 GB 37 % | 0.0 % | 2.64 GB 15 % | 134 MB/s 16 % | 26,340 53 % | 3.1 % | 20.0 % | 143 Mbps 14 % | – |
| bench-validator-09 | validator | 43.6 % | 20.5 % | 6.12 GB 36 % | 0.0 % | 2.51 GB 15 % | 136 MB/s 16 % | 27,058 54 % | 3.1 % | 19.8 % | 141 Mbps 14 % | – |
| bench-validator-10 | validator | 40.2 % | 24.1 % | 7.04 GB 41 % | 0.0 % | 3.15 GB 18 % | 136 MB/s 16 % | 26,750 53 % | 3.2 % | 20.0 % | 135 Mbps 14 % | – |
| bench-observer-11 | observer | 23.9 % | 10.4 % | 5.11 GB 30 % | 0.0 % | 0.71 GB 4 % | 125 MB/s 15 % | 27,047 54 % | 2.5 % | 18.6 % | 11 Mbps 1 % | – |
Small figures are the share of capacity: RAM for RSS, the PD limit for disk write (VM cap applied), 1 Gbps for network. Hover a cell for its JSON path.
A = Fleet c3-hdb · c3-highcpu-8 · r20261113-1630
B = Fleet c3-hdb · c3-highcpu-8 · r20261113-1351
tight c3-highcpu-8 validators at 8,539 sustained TPS: tight on disk. Disk tight: write IOPS p95 27,058, 54.1 % of the 50,000 IOPS limit (bench-validator-09), tight against the 60 % line. CPU has headroom: CPU p95 43.6 % of 8 vCPUs (bench-validator-09). Memory has headroom: memory used p95 27.3 % of RAM (bench-validator-05); telcoin RSS p95 7.04 GB, 41.0 % of 16 GiB RAM (bench-validator-10); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 153 Mbps, 15.3 % of 1 Gbps (bench-validator-07).
tight c3-highcpu-8 observers at 8,539 sustained TPS: tight on disk. Disk tight: write IOPS p95 27,047, 54.1 % of the 50,000 IOPS limit (bench-observer-11), tight against the 60 % line. CPU has headroom: CPU p95 23.9 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 10.4 % of RAM (bench-observer-11); telcoin RSS p95 5.11 GB, 29.7 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 11 Mbps, 1.1 % of 1 Gbps (bench-observer-11).
at the limit c3-highcpu-8 validators at 8,065 sustained TPS: at the limit on disk. Disk at the limit: write IOPS p95 20,050–20,146, up to 100.7 % of the 20,000 IOPS limit, over the 60 % line on all 10 validators (bench-validator-01, 02, 03, 04, 05, 06, 07, 08, 09, 10); IO pressure (PSI some avg10) max 14.9 % (bench-validator-08), tight against the 20 % line. CPU has headroom: CPU p95 36.0 % of 8 vCPUs (bench-validator-03). Memory has headroom: memory used p95 22.7 % of RAM (bench-validator-06); telcoin RSS p95 6.87 GB, 40.0 % of 16 GiB RAM (bench-validator-06); memory pressure (PSI some avg10) max 0.0 % (bench-validator-01). Network has headroom: network p95 130 Mbps, 13.0 % of 1 Gbps (bench-validator-07). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
at the limit c3-highcpu-8 observers at 8,065 sustained TPS: at the limit on disk. Disk at the limit: write IOPS p95 20,071, 100.4 % of the 20,000 IOPS limit on bench-observer-11, over the 60 % line; IO pressure (PSI some avg10) max 13.7 % (bench-observer-11), tight against the 20 % line. CPU has headroom: CPU p95 20.4 % of 8 vCPUs (bench-observer-11). Memory has headroom: memory used p95 12.4 % of RAM (bench-observer-11); telcoin RSS p95 4.54 GB, 26.4 % of 16 GiB RAM (bench-observer-11); memory pressure (PSI some avg10) max 0.0 % (bench-observer-11). Network has headroom: network p95 12 Mbps, 1.2 % of 1 Gbps (bench-observer-11). Next step: a faster volume (pd-ssd adds 30 IOPS and 0.48 MiB/s per GB up to the per-VM cap; past the cap, a larger VM).
Validators: Fleet A (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. Disk is at the limit on Fleet B (c3-highcpu-8) only. Fleet A (c3-highcpu-8) stays under the disk lines that Fleet B (c3-highcpu-8) crosses.
Observers: Fleet A (c3-highcpu-8) is tight on disk, with headroom on CPU, memory and network; Fleet B (c3-highcpu-8) is at the limit on disk, with headroom on CPU, memory and network. Disk is at the limit on Fleet B (c3-highcpu-8) only. Fleet A (c3-highcpu-8) stays under the disk lines that Fleet B (c3-highcpu-8) crosses.
| KPI | Fleet A | Fleet B | B ÷ A |
|---|---|---|---|
| Sustained mixed TPS (own) | 8,539.0 tx/s | 8,065.4 tx/s | 0.94× |
| Sustained mixed TPS (chain) | 8,539.0 tx/s | 8,065.4 tx/s | 0.94× |
| Landed / accepted | 99.9 % | 100.0 % | 1.00× |
| Reverted (own) | 0 | 0 | – |
| Latency p50 | 858 ms | 679 ms | 0.79× |
| Latency p95 | 3,528 ms | 14,662 ms | 4.16× |
| Gas / s (own) | 365M gas/s | 269M gas/s | 0.74× |
| Capacity at mix | 16,530.3 tx/s | 14,103.0 tx/s | 0.85× |
| Signal | Class | Fleet A | Fleet B | Line |
|---|---|---|---|---|
| CPU p95 | validator | 43.6 % headroom | 36.0 % headroom | > 70 % |
| Memory used p95 | validator | 27.3 % headroom | 22.7 % headroom | > 75 % |
| telcoin RSS p95 | validator | 7.04 GB 41.0 % headroom | 6.87 GB 40.0 % headroom | > 85 % |
| PSI memory max | validator | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | validator | 136 MB/s 16.2 % headroom | 110 MB/s 13.1 % headroom | > 60 % |
| Write IOPS p95 | validator | 27,058 54.1 % tight | 20,146 100.7 % at the limit | > 60 % |
| PSI io max | validator | 4.1 % headroom | 14.9 % tight | > 20 % |
| Root disk max | validator | 20.1 % headroom | 17.5 % headroom | > 80 % |
| Network p95 | validator | 153 Mbps 15.3 % headroom | 130 Mbps 13.0 % headroom | > 50 % |
| CPU p95 | observer | 23.9 % headroom | 20.4 % headroom | > 70 % |
| Memory used p95 | observer | 10.4 % headroom | 12.4 % headroom | > 75 % |
| telcoin RSS p95 | observer | 5.11 GB 29.7 % headroom | 4.54 GB 26.4 % headroom | > 85 % |
| PSI memory max | observer | 0.0 % headroom | 0.0 % headroom | > 5 % |
| Disk write p95 | observer | 125 MB/s 14.9 % headroom | 97 MB/s 11.5 % headroom | > 60 % |
| Write IOPS p95 | observer | 27,047 54.1 % tight | 20,071 100.4 % at the limit | > 60 % |
| PSI io max | observer | 2.5 % headroom | 13.7 % tight | > 20 % |
| Root disk max | observer | 18.6 % headroom | 15.9 % headroom | > 80 % |
| Network p95 | observer | 11 Mbps 1.1 % headroom | 12 Mbps 1.2 % headroom | > 50 % |
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
aligned on each fleet's mixed-phase start
Not collected: reference chains.