Repository navigation
feat(beacon): score gossipsub peers with lighthouse's parameters - #656
Conversation
…-636-638-gloas-live The scoring refresh names the dynamic topics for the digest current at each slot, so on this branch's runtime fork schedule they follow a switch within a slot; the docs say what does not follow (exit and slashing parameters, old-digest weights).
…36-638-gloas-live Brings gloas validator duties (produceBlockV4, envelope publication, PTC duties and payload attestations, gloas attestation data and aggregates, VC gloas support) onto the deployment branch, keeping every behavior of #626, #633, #636, #638, #646, #647-#652, #656, #658-#660 and the sync-committee and liveness endpoints. Conflict resolutions keep both sides: the attestation pool stays in Store (tmp) while the payload attestation pool is threaded through P2P and the RPC handles (feature); the aggregate endpoints keep tmp's liveness recording and attesting indices and add the feature's fork-header check and gloas pooling; the VC tests and fake execution client serve both fulu blobs and gloas V6. Semantic fixes: a. POST /eth/v2/beacon/blocks (gloas) calls publish_beacon_block(block, Vec::new()): gloas columns travel with the envelope. RecordingNetwork implements publish_beacon_block(block, sidecars) and both new methods. b. produceBlockV4 appends client versions to the graffiti exactly like produceBlockV3 (graffiti::execution_client_version run alongside the payload build, with_client_versions, Extension<OwnVersion>) and logs it. c. Attestation data, aggregate_attestation keep require_execution_client and require_validated for gloas slots; payload_attestation_data now applies the same two rules (503 without an execution client, or when the voted block's payload is unvalidated). d. Proposer duties v1 and v2 serve gloas epochs from a fulu or gloas state's proposer_lookahead; nothing refuses gloas any more; v2 keeps its dependent root. e. The VC's per-validator ProposerSettings apply to gloas proposals (graffiti in the BlockRequest, fee recipient compared with the bid's); a test pins the graffiti. VC tests updated to the ProposerSettings constructors. f. gloas production reads the attestation pool from Store and calls the stf with the ActiveBalanceCache the perf work added; fulu production and pack_operations are untouched (gloas blocks carry no pooled operations). g. Chain events are emitted by the chain actor only, so nothing on the RPC publish paths needed to move; gloas imports reach it unchanged. h. Cargo.lock unchanged; cargo check --locked passes.
4434d77 to
9c18979
Compare
## 🗒️ Description / Motivation
Merges `beacon-chain-integration` into `main`. This adds an **Ethereum
Beacon Chain** client (phase0 through fulu) to the ethlambda binary,
alongside the Lean consensus client:
- **`ethlambda beacon`**: a beacon node. It checkpoint-syncs, follows
the chain over gossip and req/resp, validates execution payloads through
an execution client's Engine API, and serves the Beacon API, including
the validator endpoints.
- **`ethlambda validator`**: a beacon validator client (attest,
aggregate, propose) that talks to any beacon node over the standard REST
API.
- **`ethlambda node`** (lean) keeps its behavior, apart from the items
under "Behavior changes for lean" below.
Both chains run on the same storage layer, P2P actor, RPC crate and
chain actor. `BeaconState` has a `Lean` variant holding lean's `State`,
and each shared handler dispatches on it once, at the top.
Each piece was reviewed as its own PR into the integration branch
(listed under "Related Issues / PRs"). This PR merges that branch.
## What Changed
| Area | Change |
|---|---|
| `ethlambda-types` (`beacon/`) | Containers for phase0 to fulu, presets
(compile-time `preset-minimal` feature), runtime fork config, and a
`Lean` variant on `ForkName` and `BeaconState` |
| `ethlambda-state-transition` (`beacon/`) | State transition, epoch
processing, fork upgrades, fork choice, BLS (`blst`), KZG (`c-kzg`),
gossip validation rules, DAS custody math |
| `ethlambda-ssz-tree` (**new**) | Persistent Merkle `List`/`Vector` for
the validator registry and balances, so consecutive states share
unchanged subtrees |
| `ethlambda-storage` | One `Store` for both chains. Beacon states are
stored as snapshots plus VCDIFF deltas and written on a background
thread. Adds a committee cache, and `db_version`/`chain`/`preset`
metadata that resume checks |
| `ethlambda-blockchain` | Beacon path in the chain actor
(`BlockChain::spawn_beacon`): import, holding blocks until their parent
or columns arrive, applying gossip aggregates to fork choice, block
production and aggregation |
| `ethlambda-engine` (**new**) | Engine API client with JWT auth:
`newPayload`, `forkchoiceUpdated`, `getPayload`, `getBlobs` |
| `ethlambda-p2p` | Beacon wire: blocks, aggregates, subnet attestations
and data columns are validated before relay;
`beacon_blocks_by_{range,root}/2` served and requested; range sync;
column custody and sampling; attestation subnet backbone; discv5 always
on for `beacon` |
| `ethlambda-rpc` | Beacon API under `/eth/v1` and `/eth/v2`: JSON by
default, SSZ on request. The router is picked by `Store::chain()`, so
`/lean/v0` and the Beacon API are never served together |
| `ethlambda-validator` (**new**) | Validator client: duties,
attestations, aggregation selection, proposals, keymanager API. It has
**no slashing protection** (recorded in `docs/spec_deviations.md`) |
| `bin/ethlambda` | `beacon` and `validator` sub-commands; `--network`
takes a built-in name (`mainnet`, `sepolia`, `hoodi`) or a directory of
published network files; beacon checkpoint sync; `benchmark import`
replays a real block corpus offline; `run_node` is one startup path for
both chains |
| CI / Makefile | One beacon spec-test job per preset, minimal-preset
check, `make test` split into `test-consensus` and `test-node` (one
runner's disk no longer fits both), benchmark-smoke and tooling jobs,
disk-cleanup action |
| Docs | New: `beacon_stf.md`, `beacon_wire.md`, `beacon_engine.md`,
`cli.md`. Updated: `data_storage.md`, `metrics.md`, `rpc.md`,
`checkpoint_sync.md`, `discovery.md`, `benchmarking.md`,
`spec_deviations.md`, README (beacon quickstart with ethrex) |
## Correctness / Behavior Guarantees
- **Separation:** beacon behavior lives in `beacon` modules that lean
code never reads, and those modules never read lean's. The shared crates
dispatch on the chain once per handler.
- **Lean state at a beacon accessor:** this panics, naming the function
it reached (`lean_state_unreachable` / `lean_fork_unreachable`), so the
caller never gets a wrong answer back. `ForkName::Lean` is left out of
`ForkName::ALL`, so neither `parse` nor a fork upgrade can reach it.
- **Spec conformance:** every consensus-specs `v1.6.1` fixture case
passes on both presets: 5705 mainnet and 40009 minimal, including 150
mainnet `fork_choice` cases. Gossip fixtures are pinned to
`v1.7.0-beta.1`.
- **Resume safety:** a data directory written by another chain, preset
or format version is refused at startup, before any state is read.
### ⚠️ Behavior changes for lean (`ethlambda node`)
- **Existing data directories are refused on resume.** A directory
written by `main` has no `db_version` key, so `Store::from_db_state`
returns `DbVersionMismatch { found: 0, expected: 4 }`. There is no
migration: an upgraded node needs a fresh data directory (or checkpoint
sync).
- `blst` and `c-kzg` are now dependencies of the lean binary too, since
the state-transition crate holds both chains' rules. They are not
feature-gated.
- libssz is a git dependency pinned by `rev`: the beacon containers need
`DerefMut`/`IndexMut` on `SszList` (lambdaclass/libssz#33), which no
crates.io release has yet.
- The `release-fast` profile (used for tests) now sets `debug-assertions
= true`.
- `P2P::spawn` now runs before the chain actor is spawned, so gossip
arriving in that short window is dropped.
- `/lean/v0/blocks/finalized` answers JSON when asked for it by name.
SSZ stays the default.
## Tests Added / Run
- **Beacon spec tests:** `make test-beacon` (or `test-beacon-mainnet` /
`test-beacon-minimal`). Each fixture case is a separately named test,
e.g. `cargo test -p ethlambda-state-transition --test beacon_spec_tests
--features beacon-spec-tests -- electra/attester_slashing`. CI runs one
job per preset.
- **`make test`** still needs no consensus-spec download: the beacon
spec target is behind the `beacon-spec-tests` feature, and the BLS/KZG
vector tests report as ignored without it.
- **Minimal preset:** the `Test minimal preset` CI job builds and runs
the `ethlambda-types` unit tests with `preset-minimal` on.
- **Benchmark smoke:** a CI job runs the `benchmark` sub-command end to
end.
- Unit and integration tests in every new crate (`ssz-tree`, `engine`,
`validator`) and in the beacon paths of storage, p2p, rpc and the chain
actor.
## Related Issues / PRs
<details>
<summary>PRs merged into the integration branch (reviewed on
<code>lambdaclass/ethlambda_private</code>)</summary>
**Types and state transition**
- lambdaclass/ethlambda_private#1 Beacon Chain containers under a
`beacon` namespace
- lambdaclass/ethlambda_private#2 Beacon Chain state transition, phase0
through fulu
- lambdaclass/ethlambda_private#37 keep the registry and balances in
persistent Merkle trees
**Follower, sync and storage**
- lambdaclass/ethlambda_private#3 follow mainnet gossip behind an
`ethlambda beacon` sub-command
- lambdaclass/ethlambda_private#6 unify the lean and beacon storage
layer
- lambdaclass/ethlambda_private#7 checkpoint sync for the beacon chain
- lambdaclass/ethlambda_private#8 serve beacon blocks by range and by
root
- lambdaclass/ethlambda_private#9 give the follower a chain, from the
anchor to the tip
- lambdaclass/ethlambda_private#11 fulu data availability sampling,
custody through the availability gate
- lambdaclass/ethlambda_private#13 validate execution payloads over the
Engine API
- lambdaclass/ethlambda_private#15 follow a network described by files
on disk
- lambdaclass/ethlambda_private#26 add sepolia and hoodi as built-in
networks
- lambdaclass/ethlambda_private#48, lambdaclass/ethlambda_private#52,
lambdaclass/ethlambda_private#53, lambdaclass/ethlambda_private#54: held
blocks, their columns and range batches
**P2P**
- lambdaclass/ethlambda_private#18 stop outbound req/resp requests
crossing protocols
- lambdaclass/ethlambda_private#23 rank dial candidates by custody
redundancy
- lambdaclass/ethlambda_private#28 count why peers drop us
- lambdaclass/ethlambda_private#38 validate beacon gossip before
propagating it
- lambdaclass/ethlambda_private#43 run every data column check in p2p
- lambdaclass/ethlambda_private#47 validate beacon aggregates and subnet
attestations before relaying
- lambdaclass/ethlambda_private#19 apply gossip aggregates to fork
choice, backbone two attestation subnets
**API and validator**
- lambdaclass/ethlambda_private#21 serve the Beacon API from `ethlambda
beacon`
- lambdaclass/ethlambda_private#44 report the preset and constants from
`/eth/v1/config/spec`
- lambdaclass/ethlambda_private#16 beacon-chain validator client
- lambdaclass/ethlambda_private#34 serve the validator API
- lambdaclass/ethlambda_private#40 aggregate and propose through
`ethlambda beacon`
- lambdaclass/ethlambda_private#56 pack other nodes' attestations into
produced blocks
**Performance and metrics**
- lambdaclass/ethlambda_private#10, lambdaclass/ethlambda_private#20,
lambdaclass/ethlambda_private#22, lambdaclass/ethlambda_private#24,
lambdaclass/ethlambda_private#33, lambdaclass/ethlambda_private#49:
registry scans, parallel pubkey validation, committee cache, background
state writes, BLS key validation
- lambdaclass/ethlambda_private#17, lambdaclass/ethlambda_private#31:
import timing by phase
- lambdaclass/ethlambda_private#25 benchmark block import by replaying a
real corpus
**CI**
- lambdaclass/ethlambda_private#12, lambdaclass/ethlambda_private#30,
lambdaclass/ethlambda_private#45: split lint/test jobs, benchmark smoke
job, runner selection
</details>
Open PRs based on `beacon-chain-integration`, **not** included here:
lambdaclass#626, lambdaclass#627, lambdaclass#629, lambdaclass#630, lambdaclass#632, lambdaclass#634, lambdaclass#636, lambdaclass#638, lambdaclass#642, lambdaclass#644 to lambdaclass#652,
lambdaclass#656, lambdaclass#658.
## ✅ Verification Checklist
- [ ] Ran `make fmt` (clean)
- [ ] Ran `make lint` (clippy with `-D warnings`, clean)
- [ ] Ran `make test` (`test-consensus` plus `test-node`, at
`release-fast`), all passing
- [ ] Ran `make test-beacon` (both presets), all passing
---------
Co-authored-by: Pablo Deymonnaz <pdeymon@fi.uba.ar>
Co-authored-by: Klaus Lungwitz <klaus.lungwitz@lambdaclass.com>
Co-authored-by: Klaus @ LambdaClass <18153834+klaus993@users.noreply.github.com>
Co-authored-by: Chase Wright <chase.wright@ethereum.org>
🤖 Codex Code ReviewLooks good overall. I did not spot a consensus-critical bug in fork choice / STF / attestation processing here; the PR is narrowly scoped to beacon-wire gossipsub scoring, and the math/tests are generally careful.
If you want, I can do a second pass focused only on the scoring formulas against Lighthouse semantics, or scan for any test gaps around epoch-boundary refresh behavior. Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Claude Code ReviewReview of PR 656: gossipsub peer scoring on the beacon wireI read the full diff but did not build it or run the tests. The design is sound and well documented. I found no consensus-affecting bugs. Scoring is beacon-only, lean is untouched, and the only change to state-transition code is making What looks right
Points to consider
Overall this is good to merge once Point 1 is addressed or consciously accepted. Points 2 and 5 are cheap polish. Automated review by Claude (Anthropic) · sonnet · custom prompt |
🤖 Kimi Code ReviewI'll review this PR focusing on correctness, security, performance, and Rust best practices. This is a substantial peer scoring implementation for gossipsub. Overall AssessmentThe code is well-structured, thoroughly documented, and includes comprehensive tests. The port from Lighthouse appears careful. However, I've identified several issues ranging from minor correctness concerns to potential security/availability issues. Critical Issues1. Integer Division Truncation in
|
| Priority | Item | File:Line | Issue |
|---|---|---|---|
| Compile error | 3 | swarm_adapter.rs:291 |
Missing HashMap import |
| High | 1 | scoring.rs:787-798 |
active = 0 gives incorrect aggregator count |
| Medium | 5 | swarm_adapter.rs:436-442 |
set_topic_params failure silently logged at debug |
| Medium | 7 | scoring.rs:410-413 |
Verify gossipsub accepts very small thresholds |
| Low | 8 | scoring.rs:251-305 |
Allocates 69 topics every slot |
| Low | 9 | swarm_adapter.rs:312-335 |
Peer iteration in hot loop |
| Low | 11 | scoring.rs:97 |
Visibility inconsistency |
| Style | — | — | Consider caching topic names |
Recommended Changes
-
Fix the compile error in
swarm_adapter.rs:use std::collections::HashMap; // Add this import
-
Add guard for
active = 0inexpected_aggregators_per_slot:fn expected_aggregators_per_slot(active: u64) -> (f64, u64) { if active == 0 { return (0.0, 0); } // ... rest }
-
Consider
warn!level forset_topic_paramsfailures, or at least document whydebug!is sufficient. -
Add a test for
disconnect_graylisted_peersbehavior, perhaps by mocking the gossipsub peer list.
The core scoring logic appears correct and the Lighthouse port is faithful. The MeshDeliveryGate is a thoughtful addition that addresses a real availability concern for startup/catch-up scenarios.
Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt
ilitteri
left a comment
There was a problem hiding this comment.
Two fork-rollover issues need to be addressed before merge.
-
High: old-fork topics retain live P3 scoring and can graylist honest peers.
refreshupdates dynamic parameters only forwire.fork_digest(crates/net/p2p/src/beacon/scoring.rs:561-564), leaving the previous digest's nonzero threshold and weight active. Old topics remain subscribed for two epochs while honest peers move traffic to the new digest; their old-topic counters then decay without deliveries untildisconnect_graylisted_peersacts on the resulting score (crates/net/p2p/src/swarm_adapter.rs:318-330). Using this PR's mainnet parameters with one million validators and a previously full aggregate counter, the old aggregate topic alone crosses the -16,000 graylist threshold after 53 silent slots, within the 64-slot retention window. Please retire/zero old-digest weights and P3 parameters, and apply gate changes to every held digest. -
Medium: new-fork topics have incomplete scoring. Full startup parameters include exit and slashing topics (
scoring.rs:230-234,:263-284), but refresh installs only dynamic topics, so those fixed topics receive no parameters under subsequent digests and invalid messages escape P4. Also, future-digest topics can be GRAFTed during the one-epoch pre-fork subscription window before parameters exist; pinned gossipsub does not initialize mesh state when parameters are later installed, leaving those peers without P1/P3 until re-GRAFT or reconnect. Please install complete parameters for current and upcoming held digests before subscriptions/GRAFTs.
Please add runtime rollover coverage for both cases, including graylist behavior and a mesh formed before activation.
ilitteri
left a comment
There was a problem hiding this comment.
Additional specification confirmation for the old-fork P3 finding: specs/altair/p2p-interface.md:442-445 says that, post-fork, peers propagating messages on pre-fork topics MUST NOT be scored negatively. Keeping the old digest's P3 threshold and weight active therefore is not only a graylist risk, it also violates the gossip-transition rule.
Lighthouse explicitly calls remove_topic_weight_except(new_fork_digest) at the transition to prevent peers on old topics from being penalized (beacon_node/network/src/service.rs:921-923). Prysm currently disables P3 mesh-delivery scoring, and installs fixed-topic parameters on every new subscription, which is consistent with the two recommended changes in the existing review.
|
Both fixed in dadac7a. Every held digest now gets the whole parameter set (exits and slashings included), by role, sent by
Tests:
The first two fail if the parameters are sent after the subscriptions, and the first fails if the switch skips the warmup restart. |
dadac7a to
8e90e57
Compare
Motivation
The beacon wire had no gossipsub peer scoring (
// TODO: set peer scoring params). Gossipsub kept exchanging with every peer, including ones that send invalid messages, break IWANT promises or cannot keep up, and the connection cap was the only bound on bad peers.Slow peers are the visible case on the followers. The mainnet follower logged 96k gossipsub
Send Queue fullwarnings in 22 h, from only 8 peers (one of them 60k). The Hoodi follower currently logs about 5000 a minute, almost all from 2 peers. Without scoring, nothing penalizes or drops those peers.Changes
p2p/src/beacon/scoring.rs(new)gossipsub_scoring_parameters.rs, ported formula for formula: thresholds, peer score parameters, and per-topic parameters forbeacon_block,beacon_aggregate_and_proof, all 64beacon_attestation_{n}subnets, exits and slashings.MeshDeliveryGate(the P3 sync gate below).DigestRoleandPeerScoring::hold, which score every held fork digest by its role (below). Unit tests for the derived parameters against lighthouse's values.p2p/src/lib.rsRefreshPeerScoring, first run at startup) from the head's current-epoch shuffling, under every held fork digest, and sends them to the swarm.p2p/src/beacon/transition.rsapplyhands the swarm every held digest's parameters before it subscribes to any of them.p2p/src/swarm_adapter.rsSwarmCommand::SetTopicScoreParams. On the existing 10 s tick, peers below the graylist are disconnected and peers are counted by score band.storage/src/committee_cache.rs,types/beacon/committees.rsCommitteeCache::head_current_committeesreads the pinned head's current-epoch shuffling (never builds one), andEpochCommittees::active_validator_counttakes the count from its length, so the refresh needs no registry scan.lean_gossipsub_peers_by_score{band},lean_gossipsub_score_disconnects_total,lean_gossipsub_mesh_delivery_scoring.beacon_wire.md"Peer scoring" section;metrics.md.Design decisions
Lighthouse's parameters. Teku, Lodestar and Grandine run the same ones, so a score here means what it means on most of mainnet. Thresholds: gossip -4000, publish -8000, graylist -16000. The slow-peer penalty (-10 per dropped message) is kept, since slow peers are what the followers see most.
mesh_nis ours (8), not lighthouse's 5. It only enters the first-message-delivery cap.P3 (mesh deliveries) is gated on our own sync. Lighthouse subscribes to these topics only once synced. This node subscribes at startup and, while catching up, ignores every aggregate voting for a block it has not imported. A mesh peer credited with no aggregates scores about -19.5k on aggregates alone (below the graylist), so a restart would graylist our whole aggregate mesh for our own lag. P3 is off while the head lags the wall clock by more than 4 slots, and back on only after one epoch within that lag, so the counters have refilled. While off, its threshold and weight are zero but its cap is not, so the counters keep counting to their real ceiling.
The gate reads head lag directly, not
SyncStatus. On the mainnet followerSyncStatusreportedsyncedthrough a 10-minute catch-up up to 98 slots behind: its network-stall rule reads a lagging freshest-imported block as a stalled network.Graylisted peers are disconnected; no ban list. Gossipsub alone only ignores a graylisted peer's RPCs, and the peer keeps a connection slot. A peer's negative score is retained for
retain_score(100 epochs) after it leaves, so one that reconnects comes back graylisted and is dropped again on the next pass.Columns, sync committee contributions and BLS changes stay unscored. This node
Ignores every contribution and change (no consumer), and a delivery is only credited once accepted, so P3 there would penalize the mesh for a gap that is ours. Only Prysm scores columns.All 64 attestation subnets get parameters, not just the backbone ones, because aggregator duties join others at runtime.
Every held fork digest is scored, by role. Around a boundary the node holds two digests' topics, and each gets the whole parameter set (exits and slashings included):
SUBSCRIBE_LEAD_EPOCHSaheadUNSUBSCRIBE_LAG_EPOCHSafterremove_topic_weight_except). The cost, as on lighthouse: invalid messages on the previous digest's topics stop costing P4 while it is held.Not in this PR
SyncStatusduring beacon catch-up.Testing
Run on the branch after merging
beacon-chain-integration(125dce1).cargo clippy --workspace --all-targets --profile release-fast -- -D warningscargo fmt --all --checkcargo test --profile release-fast -p ethlambda-p2p --libcargo test --profile release-fast -p ethlambda-storage --libAfter the fork-rollover fix (dadac7a), on the branch merged with
main:make lint(cargo clippy --locked --workspace --all-targets -- -D warnings)cargo fmt --all --checkcargo test --profile release-fast -p ethlambda-p2p --libThe rollover tests in
beacon::scoring:a_rollover_parametrizes_each_digest_before_joining_itruns the realtransition::applyacross a boundary with the swarm's commands captured in order;a_mesh_grafted_ahead_of_its_fork_is_scoredconnects two real swarms and checks the mesh grafted before the fork earns a score, against a control topic joined before its parameters that stays at 0;a_previous_digest_never_graylists_its_silent_meshis a tick-by-tick model of gossipsub's mesh-delivery arithmetic (a live run would need 53 slots of decay). The first two fail if the parameters are sent after the subscriptions, and the first fails if the switch skips the warmup restart.The beacon spec suites were not run: the only state-transition change makes
committee_count_per_slotpublic, and fork choice is untouched.