Skip to content

feat(rpc): emit gloas events and head_v2 on /eth/v1/events - #748

Draft
MegaRedHand wants to merge 4 commits into
mainfrom
feat/beacon-gloas-events
Draft

MegaRedHand wants to merge 4 commits into
mainfrom
feat/beacon-gloas-events

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

Summary

GET /eth/v1/events accepted every gloas topic but emitted none of them, so contributoor (ethPandaOps) had nothing to show for our nodes on Sepolia. This emits:

Topic Emitted when
head_v2 next to every head (pre-gloas with payload_status: "full"), and again for the same block on an empty -> full payload status change
block unchanged, plus builder_index and block_hash for gloas blocks (absent before)
execution_payload, execution_payload_available the chain actor imports an envelope
execution_payload_gossip an envelope passes gossip validation, or POST /eth/v1/beacon/execution_payload_envelopes accepts it
execution_payload_bid, proposer_preferences, payload_attestation_message passes gossip validation, or the matching POST accepts it

head is deprecated by the specification in favour of head_v2 and is still emitted. docs/rpc.md has the new rows, drops the stale "this build stops at fulu" sentence, and documents the execution_payload_available timing.

Design notes

  • Publisher. Bids, preferences and payload attestation messages are validated in ethlambda-p2p and never reach the chain actor, and p2p does not depend on ethlambda-blockchain (which owns EventBus). So ethlambda-network-api gets a GossipEventSink trait and a GossipEvent enum, EventBus implements the trait, and main.rs passes Some(Arc::new(events.clone())) into P2P::spawn (new last parameter). The p2p side is one function, publish_gossip_event, called from Validated::forward on Accept only, behind has_subscribers(). The API handlers already run next to the bus, so they emit directly through a small emit_accepted helper; the bus is an Option<Extension<EventBus>> there so routers built without one (the handlers' own tests) still serve.
  • Envelope gossip: sink, not BlockAnnouncement. Accepted gossip envelopes do reach the actor, but new_execution_payload_envelope carries no accepted-versus-queued flag, and gossip also forwards Queued envelopes through it. Following the block_gossip pattern would have meant a new parameter on that trait method and every implementor and test double. The sink already exists for the other three, takes Accept only, and touches one function.
  • execution_payload_available. Emitted right after execution_payload, from the same import. The actor imports an envelope only once the payload is verified and every sampled column is stored, which is what the event reports, so there is no later moment to wait for.
  • head_v2 dependent roots. current_epoch_dependent_root is the root at start_slot(epoch - 1) - 1 and next_epoch_dependent_root the root at start_slot(epoch) - 1, genesis root on underflow. These are head's previous_duty_dependent_root and current_duty_dependent_root renamed, so they come from the same NewChain::dependent_root.
  • payload_status. full for every pre-gloas head. From gloas on it follows Store::head_payload_status, with empty when none is recorded yet. The re-emission is driven by the EventView (the head and its recorded status as last reported), so it fires once per empty -> full transition and nothing else repeats (full -> empty is optional in the specification and left out). Both head_v2 emissions keep the 32-slot recency gate head has.
  • version on head_v2 is the head state's fork; the bid, message and preferences wrappers are always gloas.
  • block fields are Options with skip_serializing_if, so a pre-gloas block's JSON is byte-identical to before.
  • A head move now puts two beacon events on the shared ring instead of one. The existing head tests read through drain, which leaves head_v2 out; the new ones use drain_all.

Test plan

  • cargo fmt --all
  • cargo clippy -p ethlambda-network-api -p ethlambda-blockchain -p ethlambda-p2p -p ethlambda-rpc -p ethlambda --all-targets --profile release-fast -- -D warnings
  • cargo test -p ethlambda-blockchain --profile release-fast --lib -- events: JSON shape of every new payload against the specification examples (field names, quoting, version wrapper), the sink-to-topic mapping, the payload_status label.
  • cargo test -p ethlambda-blockchain --profile release-fast --lib -- head_v2 imported_envelope block_event_carries ... and the existing beacon event tests: head_v2 dependent roots across epochs 0 to 2 including underflow, the pre-gloas full, the empty -> full re-emission (and that full -> empty and a repeat do not re-emit), publish_envelope_imported, and the block event's bid fields.
  • cargo test -p ethlambda-rpc --profile release-fast --lib -- beacon::: an accepted bid and accepted preferences are announced once in their wrapper, a refused or already-pooled bid is not.
  • cargo test -p ethlambda-blockchain --profile release-fast --features beacon-spec-tests --lib -- beacon_envelope against the v1.7.0-beta.2 fixtures: 47 passed, including an_imported_envelope_emits_execution_payload_then_available, which drives the actor's real envelope import.
  • Not covered by a test: the p2p publish_gossip_event function, and the payload attestation and envelope POST handlers' emission (same helper as the bid and preferences handlers).

ethPandaOps' contributoor subscribes to head_v2 and the gloas topics, and
ethlambda accepted every one of them without ever sending any, so the lab's
gloas views stayed empty for our nodes on Sepolia.

head_v2 goes out next to every head (pre-gloas with payload_status "full")
and again for the same block when its payload status goes from empty to
full. The block event gains builder_index and block_hash for gloas blocks.
The chain actor publishes execution_payload and execution_payload_available
on envelope import. Bids, proposer preferences, payload attestation messages
and accepted envelopes are validated in ethlambda-p2p and never reach the
actor, and p2p does not depend on the crate owning the event bus, so p2p
reports them through a GossipEventSink trait in ethlambda-network-api that
EventBus implements. The API handlers that accept the same objects publish
on the bus they already hold.
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
ethPandaOps' contributoor subscribes to head_v2 and the gloas topics, and
ethlambda accepted every one of them without ever sending any, so the lab's
gloas views stayed empty for our nodes on Sepolia.

head_v2 goes out next to every head (pre-gloas with payload_status "full")
and again for the same block when its payload status goes from empty to
full. The block event gains builder_index and block_hash for gloas blocks.
The chain actor publishes execution_payload and execution_payload_available
on envelope import. Bids, proposer preferences, payload attestation messages
and accepted envelopes are validated in ethlambda-p2p and never reach the
actor, and p2p does not depend on the crate owning the event bus, so p2p
reports them through a GossipEventSink trait in ethlambda-network-api that
EventBus implements. The API handlers that accept the same objects publish
on the bus they already hold.

Cherry-picked from feat/beacon-gloas-events at 8274123.

Adapted onto #747: the p2p crate imports both `GossipEventSink` and the peer table types.
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
…ash (#749)

contributoor subscribes to the Beacon API `fast_confirmation` event, which
this node accepted but never emitted for lack of a Fast Confirmation Rule.
`forkchoiceUpdated` had the same gap: the spec's `get_safe_execution_block_hash`
is built on the rule, so the safe block was a stand-in (the justified
checkpoint's payload).

Implement the rule from consensus-specs `master` at f993cf2 (not the pinned
beta.2, which ships no FCR fixtures and lacks the gloas empty-slot discount,
precise committee weight, the restart-resilient root and the
`support_discount <= maximum_support` assertion), one Rust function per spec
function in `beacon::fast_confirmation`. The beacon actor runs it once per
slot at the tick, after the head recompute and before `forkchoiceUpdated`, and
skips it (resetting its variables) while the head is more than 32 slots behind
the clock. Each run publishes a `fast_confirmation` event and its confirmed
block becomes the engine's safe hash, which removes the documented
spec deviation.

The spec's helpers walk the registry or the committees per block, which is
tens of passes per slot at mainnet size, so a run flattens each balance source
once, folds every block's attestation score from the latest messages in one
pass and sums cached per-slot committee weights; only epoch-crossing and
empty-slot ranges deduplicate validators. Tests check those sums against the
spec's set-based definitions and `get_attestation_score`.

Cherry-picked from feat/beacon-fast-confirmation at 5247749.

Adapted onto #748: its gloas `ChainEvent` variants sit beside `BeaconFastConfirmation`, and `docs/rpc.md` lists both PRs' events, with `fast_confirmation` dropped from the never-emitted list.
MegaRedHand added a commit that referenced this pull request Nov 2, 2026
feat(rpc): emit single_attestation for gossip-accepted subnet attestations

ethPandaOps' contributoor subscribes to single_attestation and never to
attestation, and ethlambda accepted that topic without ever sending it, so
the lab showed no attestation data for our nodes.

Subnet attestations are validated in ethlambda-p2p and never reach the chain
actor, so p2p reports an accepted one through the GossipEventSink that #748
added, and the event bus maps it to a ChainEvent on Topic::SingleAttestation.
It is the highest-rate event the node has (every attestation on its backbone
subnets), so the attestation is cloned only when a client is subscribed, and
the payload is boxed to keep the broadcast ring sized for the smaller
variants. The payload is the existing electra SingleAttestation container, so
its JSON form is the specification's.

Second commit: feat(rpc): emit single_attestation for API-submitted attestations

This is the specification's "(from P2P or API)" half of single_attestation:
POST /eth/v2/beacon/pool/attestations now reports each attestation that
passed validation and was handed to the network actor, the way the other
API handlers report what a validator client or builder submits.

It is a separate commit because emitting our own validators' attestations,
with near-zero propagation time, can link validator indices to this node,
which is why contributoor makes the topic opt-in. Dropping this commit leaves
the gossip-accepted events alone; docs/spec_deviations.md would then need a
line saying API submissions are not reported.

Cherry-picked from feat/beacon-single-attestation-event at 6a62106.
Adapted onto #749: events.rs keeps both the BeaconFastConfirmation and SingleAttestation variants and topic arms.
MegaRedHand added a commit that referenced this pull request Nov 5, 2026
…rs (#773)

The chain actor waited for every engine_newPayload inside the import,
so the head moved only after EL + state transition and the actor sat
idle for the EL round trip, handling nothing else. Gloas envelopes did
the same.

A pre-gloas block that is an optimistic candidate now imports as
optimistic as soon as its consensus checks pass, and a gloas envelope
applies with its payload NOT_VALIDATED. The newPayload question goes out
first, from a tokio task that reports back with a NewPayloadAnswered
message, and the answer is applied then as optimistic-sync.md applies a
verdict. Optimism already gates attestation data, so validators still
attest only to validated payloads.

- Payloads are sent parent-before-child, at most 4 in flight, and
  forkchoiceUpdated names the newest payload on the head's chain whose
  request is not queued or in flight (walking the waiting payloads'
  parent hashes), sent again when the head's own answer arrives, so a
  catch-up still feeds the EL a head.
- At most 128 payloads wait for an answer; past that the import is held
  until answers free room, so catch-up cannot run unboundedly ahead.
- No answer after the retry ladder keeps the block optimistic and asks
  again each slot. optimistic-sync.md says such a block MUST NOT be
  applied; it already is, so this is documented as a deviation.
- An envelope answered INVALID with a null latestValidHash (how Engine
  API V2+ reports a block hash mismatch) is taken back out and only that
  copy refused, matching Lighthouse and Prysm. Blocks built on a FULL
  payload wait for its answer, so nothing is built on an envelope that
  gets taken back out.
- A VALID answer for the current head sends `head` again without the
  optimistic flag, for validator clients that react to head events.

Non-candidate blocks (merge transition) and servers without a mailbox
(offline replay, the import benchmark) still wait for the answer inline.

On nightly, that inline wait is #755's: the request goes out from a task
and the block's state transition runs meanwhile, so the inline path keeps
its overlap, speculation metrics and EngineAnswer telemetry rows, and
#773's own inline helper (ask_payload_inline) is dropped. The engine
section of an import that asks without waiting is a recorded zero. An
applied envelope still publishes #748's execution_payload and
execution_payload_available events, now before its answer. Answers that arrive later write no el_sent/el_verdict
telemetry rows (#765): those come only from an import that waited.

Cherry-picked from perf/async-new-payload at 33d1969 (f23010e,
33d1969).

Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
emit_accepted was inserted between decode_list's doc comment and its
signature, so its doc opened with decode_list's description and decode_list
had none. Two comments also still said the chain actor is the bus's only
publisher, which stopped being true once p2p and the submission endpoints
emit accepted objects.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant