diff --git a/CLAUDE.md b/CLAUDE.md index 9e917529..0293edaa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,7 +19,8 @@ crates/ ├─ src/lib.rs # BlockChain actor, tick events, validator duties ├─ src/store.rs # Fork choice store, block/attestation processing ├─ src/block_builder.rs # Block assembly (pre-built at previous slot's interval 4) - ├─ src/aggregation.rs # Always-on signature aggregation worker (own thread + Store clone) + ├─ src/aggregation.rs # Always-on aggregation worker (own thread + Store clone; also body-proof jobs) + ├─ src/body_proof.rs # Candidate block bodies: build, buffer, and the proposer's choice ├─ src/reaggregate.rs # Re-aggregation of block-borne votes on import ├─ src/sync_status.rs # Sync-gate tracker (suppresses duties while syncing) ├─ src/key_manager.rs # Validator key management and signing @@ -54,11 +55,11 @@ crates/ ### Tick-Based Validator Duties (5 intervals per slot; 4-second slots by default) ``` -Interval 0: Block published (at the slot boundary). The build+publish code path is merged into the previous slot's interval 4 (see below) and aligned to publish here; no attestation acceptance happens at interval 0. +Interval 0: Accept attestations (if proposing), then assemble+publish our block from the buffered BlockBodyProof candidates (or an empty body) Interval 1: Attestation production (all validators, including proposer) Interval 2: Aggregate publication (aggregators gossip the aggregates their worker produced). Proving itself is NOT confined to this interval: the worker runs continuously, and the actor buffers each finished aggregate until this tick. One that finishes DURING interval 2 or 3 is gossiped on arrival instead, since the window is already open and buffering would hold it a full slot. Interval 3: Safe target update (fork choice) -Interval 4: Accept accumulated attestations; build the NEXT slot's block and publish it aligned to that slot's interval 0 (build and publish merged into this tick) +Interval 4: Accept accumulated attestations; the worker packs the NEXT slot's candidate BlockBodyProof, gossiped as it finishes ``` ### Attestation Pipeline @@ -334,8 +335,8 @@ actual_slot = finalized_slot + 1 + relative_index ### Protocols - **Transport**: QUIC over UDP (TLS 1.3), plus TCP (noise + yamux) on the same port number as a fallback: a peer whose advertised `quic` doesn't answer can still be reached over TCP, and libp2p races both addresses within one dial (list order confers no preference; the default `dial_concurrency_factor` starts both handshakes) - Binding TCP puts `--gossipsub-port` in the HTTP servers' namespace, so it must now differ from `--api-port`/`--metrics-port` too. `NodeOptions::validate_ports` rejects every clash before anything binds -- **Gossipsub**: Blocks + Attestations (snappy raw compression) - - Topic: `/leanconsensus/{fork_digest}/{block|aggregation|attestation_N}/ssz_snappy` +- **Gossipsub**: Blocks + Attestations + candidate block bodies (snappy raw compression) + - Topic: `/leanconsensus/{fork_digest}/{block|aggregation|block_body_proof|attestation_N}/ssz_snappy` - `fork_digest` is a 4-byte hex string (no `0x` prefix); currently the dummy `12345678` agreed across clients - Mesh size: 8 (6-12 bounds), heartbeat: 700ms - **Req/Resp**: Status, BlocksByRoot, BlocksByRange (snappy frame compression + varint length) @@ -410,7 +411,7 @@ incremental, and line-tables-only debuginfo, so rebuilds are much faster than ### Aggregator Flag Required for Finalization - At least one node **must** be started with `--is-aggregator` to finalize blocks - Without this flag, attestations pass signature verification and are logged as "Attestation processed", but the signature is never stored for aggregation (the `is_aggregator` gate in `on_gossip_attestation`, `store.rs`), so blocks are always built with `attestation_count=0` -- The attestation pipeline: gossip → verify signature → store gossip signature (only if `is_aggregator`) → aggregation worker picks it up on its next selection round → publish at interval 2 → promote to known → pack into blocks +- The attestation pipeline: gossip → verify signature → store gossip signature (only if `is_aggregator`) → aggregation worker picks it up on its next selection round → publish at interval 2 → promote to known → packed into a candidate body proof at interval 4 - **Symptom**: `justified_slot=0` and `finalized_slot=0` indefinitely despite healthy block production and attestation gossip ### Runtime Aggregator Toggle (Hot-Standby Model) diff --git a/bin/ethlambda/src/benchmark/mod.rs b/bin/ethlambda/src/benchmark/mod.rs index 869ddcce..fe77b395 100644 --- a/bin/ethlambda/src/benchmark/mod.rs +++ b/bin/ethlambda/src/benchmark/mod.rs @@ -20,7 +20,7 @@ use ethlambda_blockchain::block_builder::ProposerConfig; use ethlambda_blockchain::metrics::BLOCK_PROPOSAL_ATTESTATION_BUILD_PHASES; use ethlambda_blockchain::store::{on_block_without_verification, produce_block_with_signatures}; use ethlambda_storage::{NEW_PAYLOAD_CAP, Store}; -use ethlambda_types::block::{MultiMessageAggregate, SignedBlock}; +use ethlambda_types::block::{BlockProof, SignedBlock}; use ethlambda_types::primitives::HashTreeRoot as _; use eyre::WrapErr as _; @@ -247,7 +247,7 @@ fn build_one_slot( // index. let signed_block = SignedBlock { message: block, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; on_block_without_verification(store, signed_block) .wrap_err_with(|| format!("importing the built block failed at slot {slot}"))?; diff --git a/crates/blockchain/src/aggregation.rs b/crates/blockchain/src/aggregation.rs index e2b42b0b..574cd95e 100644 --- a/crates/blockchain/src/aggregation.rs +++ b/crates/blockchain/src/aggregation.rs @@ -45,11 +45,17 @@ //! slot's committee aggregation. //! //! The actor parks the worker outright for as long as it needs the prover to -//! itself, or the node has no business aggregating: around its own block -//! build, and while the sync gate suppresses duties, so a node that is behind -//! spends the prover on the block import that closes the gap rather than on a -//! backlog the network has stopped waiting for. Both are [`PauseReason`]s (see +//! itself, or the node has no business aggregating: around its own proposal, +//! and while the sync gate suppresses duties, so a node that is behind spends +//! the prover on the block import that closes the gap rather than on a backlog +//! the network has stopped waiting for. Both are [`PauseReason`]s (see //! [`AggregationWorker::pause`] and [`AggregationWorker::set_paused`]). +//! +//! During the head-update interval the worker has one further duty: build the +//! candidate [`BlockBodyProof`] for the upcoming slot (see +//! [`body_proof::build_body_proof`]) and hand it to the actor, which gossips it +//! for that slot's proposer to adopt. It takes priority over aggregation there, +//! since it is the one job with a deadline. use std::collections::{HashMap, HashSet}; use std::sync::Arc; @@ -63,7 +69,7 @@ use ethlambda_types::{ ShortRoot, aggregator::AggregatorController, attestation::{AggregationBits, AttestationData, HashedAttestationData}, - block::{ByteList512KiB, SingleMessageAggregate}, + block::{BlockBodyProof, ByteList512KiB, SingleMessageAggregate}, chain_config::ChainConfig, constants::{INTERVALS_PER_SLOT, MIN_MILLISECONDS_PER_SLOT}, primitives::H256, @@ -74,7 +80,8 @@ use spawned_concurrency::tasks::ActorRef; use tokio_util::sync::CancellationToken; use tracing::{info, trace, warn}; -use crate::block_builder::{self, EntryScore}; +use crate::block_builder::{self, EntryScore, ProposerConfig}; +use crate::body_proof; use crate::{SlotInterval, metrics}; /// How long the worker waits before re-reading the pool when it found nothing @@ -147,6 +154,17 @@ const _: () = assert!( "EARLY_AGGREGATION_WINDOW must not reach past the slot boundary at the shortest cadence" ); +/// Offset within the slot at which the head-update interval — the slot's last +/// — begins. From here the worker's first duty is the next slot's candidate +/// body proof. +/// +/// Derived from the configured slot duration, like +/// [`vote_aggregation_offset_ms`]. +fn head_update_offset_ms(config: &ChainConfig) -> u64 { + // Slot 0 reduces `to_ms_since_genesis` to the offset within a slot. + SlotInterval::EndOfSlot.to_ms_since_genesis(0, config) +} + /// A single pre-prepared aggregation group. /// /// Built on the actor thread from a store snapshot; consumed by an off-thread @@ -306,15 +324,17 @@ impl Drop for PauseGuard { } } -/// Startup-fixed inputs the worker's vote-propagation gate needs. Both come -/// from the CLI and never change at runtime, so the worker owns a copy instead -/// of reaching back into the actor. +/// Startup-fixed inputs the worker needs. All come from the CLI and never +/// change at runtime, so the worker owns a copy instead of reaching back into +/// the actor. #[derive(Clone)] pub(crate) struct WorkerConfig { /// Number of attestation committees (= subnet count). pub(crate) attestation_committee_count: u64, /// Attestation subnets this node subscribes to. pub(crate) subscribed_subnets: HashSet, + /// Body-packing policy, shared with the proposer path. + pub(crate) proposer_config: ProposerConfig, } /// One successful aggregate announced to the actor, after the worker has @@ -336,6 +356,27 @@ impl Message for AggregateProduced { type Result = (); } +/// A candidate body proof the worker built for `slot`. +pub(crate) struct BodyProofProduced { + /// Slot the body was packed for: the one whose proposer may adopt it. + pub(crate) slot: u64, + pub(crate) body_proof: BlockBodyProof, + /// Wall time the merge took, observed on the worker thread. + pub(crate) elapsed: Duration, +} +impl Message for BodyProofProduced { + type Result = (); +} + +/// What the worker does with a turn of its loop. +enum WorkerJob { + /// Prove one aggregation group. Boxed: a job carries its whole aggregation + /// material, which dwarfs the other variant. + Aggregate(Box), + /// Build the candidate body proof for `slot`. + BodyProof { slot: u64 }, +} + /// What the worker is allowed to pick up, given where the slot is. /// /// The prover is single-threaded and the slot's committee aggregate is the one @@ -482,12 +523,27 @@ fn select_best_job(store: &Store, current_slot: u64, policy: JobPolicy) -> Optio head_state.historical_block_hashes.iter().copied().collect(); extended_historical_block_hashes.push(store.head().expect("head read works")); - let projected = block_builder::ProjectedState::from_head_state(&head_state); + // Seed the head votes too, not just the justification projection. Since a + // vote for an already-justified target is no longer filtered out, its only + // remaining value is the fork-choice weight it carries; without this the + // worker would score every such group at zero on both axes and prove none + // of them, leaving the pool empty exactly on the slots where every pooled + // vote names a settled target, which is the case this is meant to cover. + // + // The baseline is what the CHAIN carries, not what this node has seen. + // Every aggregate this worker produces is applied back into the pool and + // the fork-choice vote map together (`apply_aggregated_group` on the actor + // thread, then the next promote moves both new->known), so scoring against + // the seen-votes map would report zero for the very groups just proved. + let projected = block_builder::ProjectedState::from_head_state(&head_state) + .with_head_votes(store.extract_on_chain_votes()); // One round: the store is re-read before the next job, so a same-target // candidate re-tiers against the aggregate this one produced (once - // applied) rather than against an in-memory projection of it. - let (data_root, score) = pick_best_candidate( + // applied) rather than against an in-memory projection of it. That is also + // why the scored head voters are dropped here: there is no second in-memory + // round that could credit a validator twice. + let (data_root, score, _new_head_voters) = pick_best_candidate( &candidates, &projected, &known_block_roots, @@ -598,8 +654,8 @@ fn pick_best_candidate( extended_historical_block_hashes: &[H256], current_slot: u64, validator_count: usize, -) -> Option<(H256, EntryScore)> { - let mut best: Option<(H256, EntryScore)> = None; +) -> Option<(H256, EntryScore, HashSet)> { + let mut best: Option<(H256, EntryScore, HashSet)> = None; let mut best_key: Option<(u8, block_builder::OrderingKey)> = None; for (data_root, candidate) in candidates { @@ -613,7 +669,11 @@ fn pick_best_candidate( continue; } - let Some((score, _new_voters)) = + // Head votes ARE scored here: the projection above is seeded from + // `extract_on_chain_votes`. So this skip now means "adds nothing on + // EITHER axis" rather than "adds no justification voters", and a group + // whose target is already settled survives on its head votes alone. + let Some((score, _new_voters, new_head_voters)) = projected.score_entry(att_data, &candidate.coverage(), validator_count) else { trace_skipped_candidate("zero_new_voters", att_data, data_root); @@ -626,7 +686,7 @@ fn pick_best_candidate( let slot_bucket: u8 = if att_data.slot == current_slot { 0 } else { 1 }; let candidate_key = candidate_ordering_key(slot_bucket, &score, *data_root); if best_key.as_ref().is_none_or(|k| candidate_key < *k) { - best = Some((*data_root, score)); + best = Some((*data_root, score, new_head_voters)); best_key = Some(candidate_key); } } @@ -825,7 +885,7 @@ pub fn aggregate_job(job: AggregationJob) -> Option { /// /// - `insert_new_aggregated_payload` records the fork-choice votes before it /// pushes the payload, and it records them with a max-merge -/// (`should_replace_vote`) that gives the same map whatever order concurrent +/// (`AttestationData::supersedes`) that gives the same map whatever order concurrent /// writers arrive in. A promote landing in the middle moves votes from `new` /// to `known` rather than dropping them, so the worst interleaving leaves the /// vote or the payload to be promoted one tick later. Neither is lost. @@ -988,13 +1048,14 @@ pub(crate) fn spawn_aggregation_worker( /// Worker loop — runs on its own thread for the actor's lifetime. /// -/// Each round re-reads the pool through the store handle, picks the best job -/// ([`select_best_job`]), proves it, and hands the result to the actor as an -/// [`AggregateProduced`] message. With nothing to do — nothing eligible, +/// Each round re-reads the pool through the store handle, takes the job worth +/// doing right now ([`next_job`]), and hands the result to the actor: an +/// [`AggregateProduced`] for a proved group, a [`BodyProofProduced`] for the +/// upcoming slot's candidate body. With nothing to do — nothing eligible, /// parked for some [`PauseReason`], or no aggregation duty — it sleeps /// [`WORKER_IDLE_POLL`] and looks again. /// -/// `aggregate_mixed` cannot be interrupted, so both cancellation and the pause +/// leanVM proofs cannot be interrupted, so both cancellation and the pause /// reasons are only observed between jobs. fn run_aggregation_worker( mut store: Store, @@ -1009,63 +1070,131 @@ fn run_aggregation_worker( // The chain's time grid never changes at runtime, so one read covers the // worker's whole life. let time_config = *store.config(); + // Slot the last candidate body proof was packed for, so the head-update + // interval produces one candidate rather than a stream of them. + let mut body_proof_slot: Option = None; while !cancel.is_cancelled() { - let Some(job) = next_job(&store, &time_config, &aggregator, &paused, &config) else { + let Some(job) = next_job( + &store, + &time_config, + &aggregator, + &paused, + &config, + body_proof_slot, + ) else { std::thread::sleep(WORKER_IDLE_POLL); continue; }; - let slot = job.slot; - let raw_sigs = job.raw_ids.len(); - let children = job.children.len(); + let delivered = match job { + WorkerJob::Aggregate(job) => run_aggregate_job(*job, &mut store, &actor), + WorkerJob::BodyProof { slot } => { + // Marked before the build, not after: a failed or empty build + // would otherwise be retried for the rest of the interval. + body_proof_slot = Some(slot); + run_body_proof_job(slot, &store, &config, &actor) + } + }; - let job_start = Instant::now(); - let output = aggregate_job(job); - let elapsed = job_start.elapsed(); + if !delivered { + // Actor is gone; nothing would consume further work. + break; + } + } - let Some(output) = output else { - warn!( - slot, - raw_sigs, - children, - ?elapsed, - "Committee signature aggregation failed" - ); - metrics::inc_aggregator_skipped_other(1); - // A failure leaves the store exactly as it found it, so the next - // round re-reads the same pool and picks the same job. Sleep before - // looping: a proof that fails cheaply, before the prover runs, - // would otherwise spin this thread at full speed. - std::thread::sleep(WORKER_IDLE_POLL); - continue; - }; + info!("Aggregation worker stopped"); +} - info!( +/// Prove one aggregation group, store it, and announce it to the actor. +/// Returns false when the actor is gone. +fn run_aggregate_job( + job: AggregationJob, + store: &mut Store, + actor: &ActorRef, +) -> bool { + let slot = job.slot; + let raw_sigs = job.raw_ids.len(); + let children = job.children.len(); + + let job_start = Instant::now(); + let output = aggregate_job(job); + let elapsed = job_start.elapsed(); + + let Some(output) = output else { + warn!( slot, raw_sigs, children, - participants = output.participants.len(), ?elapsed, - "Committee signature aggregated" + "Committee signature aggregation failed" ); + metrics::inc_aggregator_skipped_other(1); + // A failure leaves the store exactly as it found it, so the next round + // re-reads the same pool and picks the same job. Sleep before looping: + // a proof that fails cheaply, before the prover runs, would otherwise + // spin this thread at full speed. + std::thread::sleep(WORKER_IDLE_POLL); + return true; + }; - // Store before announcing, so the pool the actor reads to publish, and - // the one the next selection round re-reads, both already account for - // this aggregate. - let produced = store_aggregate(&mut store, output, elapsed); - if actor.send(produced).is_err() { - // Actor is gone; nothing would consume further aggregates. - break; - } - } + info!( + slot, + raw_sigs, + children, + participants = output.participants.len(), + ?elapsed, + "Committee signature aggregated" + ); - info!("Aggregation worker stopped"); + // Store before announcing, so the pool the actor reads to publish, and the + // one the next selection round re-reads, both already account for this + // aggregate. + let produced = store_aggregate(store, output, elapsed); + actor.send(produced).is_ok() +} + +/// Build the candidate body proof for `slot` and send it to the actor. Returns +/// false when the actor is gone. +fn run_body_proof_job( + slot: u64, + store: &Store, + config: &WorkerConfig, + actor: &ActorRef, +) -> bool { + let job_start = Instant::now(); + let Some(body_proof) = body_proof::build_body_proof(store, slot, config.proposer_config) else { + return true; + }; + let elapsed = job_start.elapsed(); + + info!( + %slot, + attestation_count = body_proof.block_body.attestations.len(), + proof_bytes = body_proof.proof.proof.len(), + ?elapsed, + "Block body proof built" + ); + metrics::observe_body_proof_building(elapsed); + + actor + .send(BodyProofProduced { + slot, + body_proof, + elapsed, + }) + .is_ok() } /// One round of job selection: honor the role flag and the pause reasons, take -/// the slot from the store clock and the [`JobPolicy`] from where the wall -/// clock sits inside it, then ask [`select_best_job`] for the winner. `None` +/// the slot from the store clock and the position inside it from the wall +/// clock, then decide what is worth doing. +/// +/// In the head-update interval the next slot's candidate body proof comes +/// first, unless one was already built for that slot: it is the job with a +/// deadline (the proposer assembles before the slot boundary), while +/// aggregation work keeps just as well for the next round. Otherwise the best +/// aggregation job the [`JobPolicy`] admits wins ([`select_best_job`]). `None` /// means "nothing to do right now", which inside the early window is a /// deliberate answer rather than an idle one. fn next_job( @@ -1074,7 +1203,8 @@ fn next_job( aggregator: &AggregatorController, paused: &AtomicU8, config: &WorkerConfig, -) -> Option { + body_proof_slot: Option, +) -> Option { // The role flag is read here because the RPC thread writes it. Everything // the actor itself owns, the sync verdict included, reaches us as a // [`PauseReason`] instead of being re-derived from shared state. @@ -1098,15 +1228,17 @@ fn next_job( // proving it thin below `min_sigs`, or bucket a stale group as current and // hold it back to a boundary that has already passed. let slot = store.current_slot(); + let ms_into_slot = ms_into_slot(now_ms, slot, time_config); - let policy = job_policy( - ms_into_slot(now_ms, slot, time_config), - time_config, - store, - config, - ); + if ms_into_slot >= head_update_offset_ms(time_config) && body_proof_slot != Some(slot + 1) { + return Some(WorkerJob::BodyProof { slot: slot + 1 }); + } + + let policy = job_policy(ms_into_slot, time_config, store, config); select_best_job(store, slot, policy) + .map(Box::new) + .map(WorkerJob::Aggregate) } #[cfg(test)] @@ -1115,7 +1247,7 @@ mod tests { use ethlambda_storage::backend::InMemoryBackend; use ethlambda_types::constants::DEFAULT_MILLISECONDS_PER_SLOT; use ethlambda_types::{ - block::{Block, BlockBody, BlockHeader, MultiMessageAggregate, SignedBlock}, + block::{Block, BlockBody, BlockHeader, BlockProof, SignedBlock}, checkpoint::Checkpoint, state::{JustificationValidators, JustifiedSlots, State, StateConfig}, }; @@ -1212,7 +1344,7 @@ mod tests { state_root: H256::ZERO, body: BlockBody::default(), }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; store .insert_signed_block(root, signed_block) @@ -1396,9 +1528,10 @@ mod tests { justified_slots: JustifiedSlots::new(), finalized_slot: 0, current_votes: HashMap::new(), + head_votes: None, }; - let (picked_root, score) = pick_best_candidate( + let (picked_root, score, _head_voters) = pick_best_candidate( &candidates, &projected, &known_block_roots, @@ -1488,10 +1621,11 @@ mod tests { justified_slots: JustifiedSlots::new(), finalized_slot: 0, current_votes: HashMap::new(), + head_votes: None, }; // Round 1: A (6 new voters) outranks B (2 new voters); both Build tier. - let (picked_root, score) = pick_best_candidate( + let (picked_root, score, _head_voters) = pick_best_candidate( &candidates, &projected, &known_block_roots, @@ -1515,7 +1649,7 @@ mod tests { // Round 2: only B remains. Combined with A's now-recorded 6 voters, // B's 2 new voters cross 2/3 of 10 — B is re-tiered from what would // have been Build in isolation to Justify. - let (picked_root, score) = pick_best_candidate( + let (picked_root, score, _head_voters) = pick_best_candidate( &candidates, &projected, &known_block_roots, @@ -1572,12 +1706,20 @@ mod tests { assert!(select_best_job(&store, 0, JobPolicy::Open).is_none()); } - /// A group whose target is already justified (here: at or behind the - /// finalized boundary) can never justify or finalize anything further and - /// must never become a job, even with enough raw sigs to otherwise be - /// viable. + /// A group whose target sits at or behind the finalized boundary must + /// never become a job, even with enough raw sigs to otherwise be viable: + /// it can neither justify nor finalize anything, and a finalized target is + /// settled for good, so its votes carry no fork-choice signal worth proving + /// either. + /// + /// The rejection comes from `target_not_justifiable` + /// (`slot_is_justifiable_after` is false below the finalized slot), not + /// from the target being justified: a justified target *above* the + /// finalized boundary is deliberately still eligible, scored on its head + /// votes alone. See + /// `select_aggregates_a_justified_target_for_its_head_votes`. #[test] - fn select_skips_group_whose_target_is_already_justified() { + fn select_skips_group_whose_target_is_at_or_behind_finalized() { const NUM_VALIDATORS: usize = 10; const HEAD_SLOT: u64 = 20; const FINALIZED_SLOT: u64 = 10; @@ -1613,7 +1755,244 @@ mod tests { assert!( select_best_job(&store, 999, JobPolicy::Open).is_none(), - "a group targeting an already-justified slot must never become a job" + "a group targeting a finalized slot must never become a job" + ); + } + + /// Regression guard for the CALL SITE, not the accessor. + /// + /// `snapshot_aggregation_inputs` must score head votes against the votes + /// the CHAIN carries (`extract_on_chain_votes`), never against the votes + /// this node has merely seen (`extract_latest_known_attestations`). The two + /// look interchangeable and both compile, but the seen-votes map advances + /// in lockstep with the very pool these jobs are selected from + /// (`insert_new_aggregated_payload` writes `new_votes` + `new_payloads`, + /// then `promote_new_aggregated_payloads` drains both into their `known` + /// counterparts), so scoring against it reports zero for every group and + /// silently kills the whole head-vote axis. That shipped twice. + /// + /// So: promote a payload for this exact attestation, which populates + /// `known_votes` while leaving `on_chain_votes` empty. A job must still be + /// selected. Swapping the call site back to the seen-votes map makes this + /// assertion fail, which is the entire point of the test. + #[test] + fn snapshot_scores_head_votes_against_the_chain_not_against_seen_votes() { + const NUM_VALIDATORS: usize = 10; + const HEAD_SLOT: u64 = 20; + const FINALIZED_SLOT: u64 = 10; + const TARGET_SLOT: u64 = 12; + + let hashes: Vec = (0..HEAD_SLOT).map(|i| H256([(i + 1) as u8; 32])).collect(); + let mut head_state = make_head_state(HEAD_SLOT, NUM_VALIDATORS, &hashes); + head_state.latest_finalized = Checkpoint { + root: hashes[FINALIZED_SLOT as usize], + slot: FINALIZED_SLOT, + }; + ethlambda_state_transition::justified_slots_ops::extend_to_slot( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + ethlambda_state_transition::justified_slots_ops::set_justified( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + let mut store = new_test_store(head_state); + insert_test_block(&mut store, hashes[0], 0, H256::ZERO); + + let att_data = AttestationData { + slot: TARGET_SLOT, + head: Checkpoint { + root: hashes[0], + slot: 0, + }, + target: Checkpoint { + root: hashes[TARGET_SLOT as usize], + slot: TARGET_SLOT, + }, + source: Checkpoint { + root: hashes[0], + slot: 0, + }, + }; + let hashed = HashedAttestationData::new(att_data.clone()); + store.insert_gossip_signature(hashed.clone(), 0, dummy_sig()); + store.insert_gossip_signature(hashed.clone(), 1, dummy_sig()); + + // Put this very vote into the SEEN map, the way the worker's own output + // lands there, while leaving the on-chain map untouched. + let mut bits = AggregationBits::with_length(NUM_VALIDATORS).unwrap(); + bits.set(0, true).unwrap(); + bits.set(1, true).unwrap(); + store.insert_new_aggregated_payload(hashed, SingleMessageAggregate::empty(bits)); + store.promote_new_aggregated_payloads(); + assert!( + !store.extract_latest_known_attestations().is_empty(), + "fixture must actually populate the seen-votes map" + ); + assert!( + store.extract_on_chain_votes().is_empty(), + "fixture must leave the on-chain map empty: no block carried this" + ); + + assert!( + select_best_job(&store, 999, JobPolicy::Open).is_some(), + "a vote the chain does not carry is still worth proving, however \ + many times this node has already seen it" + ); + } + + /// The suppression direction, which the other tests never exercise: once a + /// block HAS carried the vote, the group is worth nothing on either axis + /// and must not become a job. + /// + /// Without this, "always selects" and "correctly selects" look identical: + /// an empty on-chain baseline makes every group score its full coverage, so + /// a test that only ever asserts `is_some()` passes even if the baseline is + /// ignored outright. + #[test] + fn select_skips_a_group_whose_vote_the_chain_already_carries() { + const NUM_VALIDATORS: usize = 10; + const HEAD_SLOT: u64 = 20; + const FINALIZED_SLOT: u64 = 10; + const TARGET_SLOT: u64 = 12; + + let hashes: Vec = (0..HEAD_SLOT).map(|i| H256([(i + 1) as u8; 32])).collect(); + let mut head_state = make_head_state(HEAD_SLOT, NUM_VALIDATORS, &hashes); + head_state.latest_finalized = Checkpoint { + root: hashes[FINALIZED_SLOT as usize], + slot: FINALIZED_SLOT, + }; + ethlambda_state_transition::justified_slots_ops::extend_to_slot( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + ethlambda_state_transition::justified_slots_ops::set_justified( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + let mut store = new_test_store(head_state); + insert_test_block(&mut store, hashes[0], 0, H256::ZERO); + + let att_data = AttestationData { + slot: TARGET_SLOT, + head: Checkpoint { + root: hashes[0], + slot: 0, + }, + target: Checkpoint { + root: hashes[TARGET_SLOT as usize], + slot: TARGET_SLOT, + }, + source: Checkpoint { + root: hashes[0], + slot: 0, + }, + }; + let hashed = HashedAttestationData::new(att_data.clone()); + store.insert_gossip_signature(hashed.clone(), 0, dummy_sig()); + store.insert_gossip_signature(hashed, 1, dummy_sig()); + + // Now put this exact vote ON CHAIN for both participants. + let mut bits = AggregationBits::with_length(NUM_VALIDATORS).unwrap(); + bits.set(0, true).unwrap(); + bits.set(1, true).unwrap(); + let block = SignedBlock { + message: Block { + slot: 1, + proposer_index: 0, + parent_root: hashes[0], + state_root: H256::ZERO, + body: BlockBody { + attestations: vec![ethlambda_types::attestation::AggregatedAttestation { + aggregation_bits: bits, + data: att_data, + }] + .try_into() + .unwrap(), + }, + }, + proof: BlockProof::default(), + }; + let block_root = { + use ethlambda_types::primitives::HashTreeRoot as _; + block.message.hash_tree_root() + }; + store + .insert_signed_block(block_root, block) + .expect("insert block carrying the vote"); + assert_eq!( + store.extract_on_chain_votes().len(), + 2, + "fixture must actually put the vote on chain" + ); + + assert!( + select_best_job(&store, 999, JobPolicy::Open).is_none(), + "the chain already carries this vote, so it adds nothing on either axis" + ); + } + + /// The counterpart: a target that is justified but still above the + /// finalized boundary DOES become a job, on the strength of its head votes + /// alone. + /// + /// This is the case that used to be filtered out wholesale. On a chain + /// whose justifiable rungs sit several slots apart, every pooled vote names + /// a settled target for slots at a time; dropping them all left the + /// aggregators with nothing to prove and the next proposer with no + /// candidate body to adopt. + #[test] + fn select_aggregates_a_justified_target_for_its_head_votes() { + const NUM_VALIDATORS: usize = 10; + const HEAD_SLOT: u64 = 20; + const FINALIZED_SLOT: u64 = 10; + const TARGET_SLOT: u64 = 12; // above finalized, and marked justified + + let hashes: Vec = (0..HEAD_SLOT).map(|i| H256([(i + 1) as u8; 32])).collect(); + let mut head_state = make_head_state(HEAD_SLOT, NUM_VALIDATORS, &hashes); + head_state.latest_finalized = Checkpoint { + root: hashes[FINALIZED_SLOT as usize], + slot: FINALIZED_SLOT, + }; + ethlambda_state_transition::justified_slots_ops::extend_to_slot( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + ethlambda_state_transition::justified_slots_ops::set_justified( + &mut head_state.justified_slots, + FINALIZED_SLOT, + TARGET_SLOT, + ); + let mut store = new_test_store(head_state); + insert_test_block(&mut store, hashes[0], 0, H256::ZERO); + + let att_data = AttestationData { + slot: TARGET_SLOT, + head: Checkpoint { + root: hashes[0], + slot: 0, + }, + target: Checkpoint { + root: hashes[TARGET_SLOT as usize], + slot: TARGET_SLOT, + }, + source: Checkpoint { + root: hashes[0], + slot: 0, + }, + }; + let hashed = HashedAttestationData::new(att_data); + store.insert_gossip_signature(hashed.clone(), 0, dummy_sig()); + store.insert_gossip_signature(hashed, 1, dummy_sig()); + + assert!( + select_best_job(&store, 999, JobPolicy::Open).is_some(), + "a justified target above the finalized boundary must still be proved for its head votes" ); } @@ -1833,6 +2212,10 @@ mod tests { let config = WorkerConfig { attestation_committee_count: 4, subscribed_subnets: HashSet::from([0, 1]), + proposer_config: ProposerConfig { + enable_proposer_aggregation: false, + max_attestations_per_block: 1, + }, }; // 10 validators over 4 committees: subnets 0 and 1 hold 3 each, so a // group gathering both needs 4 of those 6. diff --git a/crates/blockchain/src/block_builder.rs b/crates/blockchain/src/block_builder.rs index 4cb1128e..9c08451b 100644 --- a/crates/blockchain/src/block_builder.rs +++ b/crates/blockchain/src/block_builder.rs @@ -60,23 +60,9 @@ pub struct ProposerConfig { /// Build a valid block on top of this state. /// -/// Selects attestations via `select_attestations`, collapses entries sharing -/// the same `AttestationData` down to one (a block may carry at most one entry -/// per data; `on_block` rejects duplicates), and runs the STF once to seal the -/// state root. The proposer signature is NOT included; it is appended by the -/// caller. -/// -/// The collapse strategy is gated by `enable_proposer_aggregation`: -/// - **enabled**: same-data proofs are merged via recursive single-message -/// aggregation into a single union-coverage proof (leanSpec #510). Maximizes voter -/// coverage per entry at the cost of a leanVM aggregation per duplicated -/// data entry. -/// - **disabled** (default): the single best-coverage proof per data is kept -/// and the rest dropped. Skips the leanVM work; coverage is bounded by the -/// best individual proof. -/// -/// Either way the output has one entry per `AttestationData` and the -/// attestation-to-proof correspondence stays 1:1. +/// Picks the body's attestations via [`select_and_compact`], then runs the STF +/// once to seal the state root. The proposer signature is NOT included; it is +/// appended by the caller. /// /// `config.max_attestations_per_block` bounds how many distinct /// `AttestationData` entries are packed (a proposer-side self-limit). It is @@ -87,32 +73,106 @@ pub(crate) fn build_block( slot: u64, proposer_index: u64, parent_root: H256, - known_block_roots: &HashSet, - aggregated_payloads: &HashMap)>, + inputs: ProposalInputs<'_>, config: ProposerConfig, ) -> Result<(Block, Vec, PostBlockCheckpoints), StoreError> { info!(slot, proposer_index, "Building block"); + let (attestations, aggregated_signatures) = + select_and_compact(head_state, slot, parent_root, inputs, config)?; + + let (final_block, post_checkpoints) = seal_block( + head_state, + slot, + proposer_index, + parent_root, + BlockBody { attestations }, + )?; + + metrics::observe_block_proposal_attestation_data_selected(final_block.body.attestations.len()); + metrics::observe_block_proposal_aggregates_selected(aggregated_signatures.len()); + + Ok((final_block, aggregated_signatures, post_checkpoints)) +} + +/// Seal a block around `body`: run the state transition once to compute the +/// state root, and report the post-state checkpoints. +/// +/// The body need not be one this node packed. Running the STF is what makes +/// adopting a gossiped [`ethlambda_types::block::BlockBodyProof`] safe: an +/// `Err` here means those attestations are not valid on top of `head_state`, +/// and the state root is computed from the transition rather than trusted. +pub(crate) fn seal_block( + head_state: &State, + slot: u64, + proposer_index: u64, + parent_root: H256, + body: BlockBody, +) -> Result<(Block, PostBlockCheckpoints), StoreError> { + let mut block = Block { + slot, + proposer_index, + parent_root, + state_root: H256::ZERO, + body, + }; + let mut post_state = head_state.clone(); + // ethlambda runs the STF once after selection (it projects justification + // incrementally instead of re-running the STF per loop round), so this is + // a single `stf_simulate` observation per build. + let stf_start = Instant::now(); + process_slots(&mut post_state, slot)?; + process_block(&mut post_state, &block)?; + metrics::observe_block_proposal_phase("stf_simulate", stf_start.elapsed()); + block.state_root = post_state.hash_tree_root(); + + let post_checkpoints = PostBlockCheckpoints { + justified: post_state.latest_justified, + finalized: post_state.latest_finalized, + }; + + Ok((block, post_checkpoints)) +} + +/// Pick the attestations a block at `slot` should carry, and the proof that +/// goes with each. +/// +/// Selection (`select_attestations`) followed by the collapse every block needs +/// — one entry per `AttestationData`, since `on_block` rejects duplicates — +/// with no state transition and no block assembled, so it serves both the +/// proposer (`build_block`) and the aggregation worker building a candidate +/// body proof. +/// +/// The collapse strategy is gated by `enable_proposer_aggregation`: +/// - **enabled**: same-data proofs are merged via recursive single-message +/// aggregation into a single union-coverage proof (leanSpec #510). Maximizes +/// voter coverage per entry at the cost of a leanVM aggregation per +/// duplicated data entry. +/// - **disabled** (default): the single best-coverage proof per data is kept +/// and the rest dropped. Skips the leanVM work; coverage is bounded by the +/// best individual proof. +/// +/// Either way the output has one entry per `AttestationData` and the +/// attestation-to-proof correspondence stays 1:1. +pub(crate) fn select_and_compact( + head_state: &State, + slot: u64, + parent_root: H256, + inputs: ProposalInputs<'_>, + config: ProposerConfig, +) -> Result<(AggregatedAttestations, Vec), StoreError> { let select_start = Instant::now(); let selected = select_attestations( head_state, slot, parent_root, - known_block_roots, - aggregated_payloads, + inputs, config.max_attestations_per_block, ); metrics::observe_block_proposal_phase("select_payloads", select_start.elapsed()); let child_payloads_consumed = selected.len(); - // Each AttestationData may appear at most once per block (`on_block` - // rejects duplicates), so same-data entries must be collapsed to one. - // Gated by `enable_proposer_aggregation`: when enabled, proofs sharing an - // AttestationData are merged via recursive single-message aggregation into - // a union-coverage proof (leanSpec #510); when disabled, we skip that leanVM - // work and keep only the single best-coverage proof per data. Both paths - // log the entry / unique-entry counts they already compute. let compact_start = Instant::now(); let compacted = if config.enable_proposer_aggregation { compact_attestations(selected, head_state, slot)? @@ -121,40 +181,32 @@ pub(crate) fn build_block( keep_best_proof_per_data(selected, &running_votes, slot) }; metrics::observe_block_proposal_phase("compact", compact_start.elapsed()); + metrics::inc_block_proposal_child_payloads_consumed(child_payloads_consumed as u64); let (aggregated_attestations, aggregated_signatures): (Vec<_>, Vec<_>) = compacted.into_iter().unzip(); - let attestations: AggregatedAttestations = aggregated_attestations .try_into() .expect("attestation count exceeds limit"); - let mut final_block = Block { - slot, - proposer_index, - parent_root, - state_root: H256::ZERO, - body: BlockBody { attestations }, - }; - let mut post_state = head_state.clone(); - // ethlambda runs the STF once after selection (it projects justification - // incrementally instead of re-running the STF per loop round), so this is - // a single `stf_simulate` observation per build. - let stf_start = Instant::now(); - process_slots(&mut post_state, slot)?; - process_block(&mut post_state, &final_block)?; - metrics::observe_block_proposal_phase("stf_simulate", stf_start.elapsed()); - final_block.state_root = post_state.hash_tree_root(); - - metrics::inc_block_proposal_child_payloads_consumed(child_payloads_consumed as u64); - metrics::observe_block_proposal_attestation_data_selected(final_block.body.attestations.len()); - metrics::observe_block_proposal_aggregates_selected(aggregated_signatures.len()); - let post_checkpoints = PostBlockCheckpoints { - justified: post_state.latest_justified, - finalized: post_state.latest_finalized, - }; + Ok((attestations, aggregated_signatures)) +} - Ok((final_block, aggregated_signatures, post_checkpoints)) +/// The chain view `process_block_header` would produce on a candidate block at +/// `slot`: covering `[0, slot - 1]` with `parent_root` at the parent's slot and +/// `ZERO_HASH` for the empty slots in between. +/// +/// Lets a caller validate a vote's head/source/target roots against the chain +/// the block would extend, instead of waiting for the state transition — which +/// does not check them at all, and would happily carry a vote for a root this +/// node has never seen. +pub(crate) fn extended_chain_view(head_state: &State, slot: u64, parent_root: H256) -> Vec { + let parent_slot = head_state.latest_block_header.slot; + let num_empty_slots = slot.saturating_sub(parent_slot).saturating_sub(1) as usize; + let mut hashes: Vec = head_state.historical_block_hashes.iter().copied().collect(); + hashes.push(parent_root); + hashes.extend(std::iter::repeat_n(H256::ZERO, num_empty_slots)); + hashes } /// Tiered greedy attestation selection for block proposal. @@ -172,25 +224,21 @@ fn select_attestations( head_state: &State, slot: u64, parent_root: H256, - known_block_roots: &HashSet, - aggregated_payloads: &HashMap)>, + inputs: ProposalInputs<'_>, max_attestations_per_block: usize, ) -> Vec<(AggregatedAttestation, SingleMessageAggregate)> { + let ProposalInputs { + known_block_roots, + aggregated_payloads, + latest_head_votes, + } = inputs; + let mut selected: Vec<(AggregatedAttestation, SingleMessageAggregate)> = Vec::new(); if aggregated_payloads.is_empty() { return selected; } - // Chain view that `process_block_header` would produce on the candidate - // block: covering [0, slot - 1] with parent_root at parent.slot and - // ZERO_HASH for empty slots in between. Lets us validate source/target - // roots without waiting for the STF to drop mismatches. - let parent_slot = head_state.latest_block_header.slot; - let num_empty_slots = slot.saturating_sub(parent_slot).saturating_sub(1) as usize; - let mut extended_historical_block_hashes: Vec = - head_state.historical_block_hashes.iter().copied().collect(); - extended_historical_block_hashes.push(parent_root); - extended_historical_block_hashes.extend(std::iter::repeat_n(H256::ZERO, num_empty_slots)); + let extended_historical_block_hashes = extended_chain_view(head_state, slot, parent_root); let chain = ChainContext { aggregated_payloads, @@ -202,14 +250,15 @@ fn select_attestations( // Running per-target-root voter set, seeded from state and updated // incrementally as entries are selected. Mirrors the role of Eth2 // participation flags in Prysm/Lighthouse-style packing. - let mut projected = ProjectedState::from_head_state(head_state); + let mut projected = + ProjectedState::from_head_state(head_state).with_head_votes(latest_head_votes); let mut processed_data_roots: HashSet = HashSet::new(); // A block may carry at most `MAX_ATTESTATIONS_DATA` distinct entries // (`on_block` rejects more), so the proposer-side limit never exceeds it. let max_rounds = max_attestations_per_block.min(MAX_ATTESTATIONS_DATA); for _round in 0..max_rounds { - let Some((data_root, score, new_voters)) = + let Some((data_root, score, new_voters, new_head_voters)) = pick_best_candidate(&chain, &processed_data_roots, &projected) else { trace!( @@ -230,6 +279,7 @@ fn select_attestations( trace!( tier = ?score.tier, new_voters = score.new_voters, + new_head_voters = score.new_head_voters, target_slot = score.target_slot, target_root = %ShortRoot(&target_root.0), data_root = %ShortRoot(&data_root.0), @@ -238,6 +288,7 @@ fn select_attestations( ); projected.advance(score.tier, att_data, new_voters); + projected.advance_head_votes(att_data, new_head_voters); } selected @@ -246,16 +297,17 @@ fn select_attestations( /// Scan candidate attestation entries and pick the highest-scoring one. /// /// Skips entries already processed, those failing `entry_passes_filters` -/// (logging the reason), and those with zero new voters. Among remaining -/// entries, returns `(data_root, score, new_voters)` for the entry with the +/// (logging the reason), and those adding neither a justification voter nor a +/// head vote. Among remaining entries, returns +/// `(data_root, score, new_voters, new_head_voters)` for the entry with the /// best `EntryScore::ordering_key` (lower is better). Caller re-indexes /// `chain.aggregated_payloads[&data_root]` for `att_data` and `proofs`. fn pick_best_candidate( chain: &ChainContext<'_>, processed_data_roots: &HashSet, projected: &ProjectedState, -) -> Option<(H256, EntryScore, HashSet)> { - let mut best: Option<(H256, EntryScore, HashSet)> = None; +) -> Option<(H256, EntryScore, HashSet, HashSet)> { + let mut best: Option<(H256, EntryScore, HashSet, HashSet)> = None; let mut best_key: Option = None; for (data_root, (att_data, proofs)) in chain.aggregated_payloads { @@ -275,7 +327,7 @@ fn pick_best_candidate( .iter() .flat_map(|proof| proof.participant_indices()) .collect(); - let Some((score, new_voters)) = + let Some((score, new_voters, new_head_voters)) = projected.score_entry(att_data, &coverage, chain.validator_count) else { trace_skipped_attestation("zero_new_voters", att_data, data_root); @@ -284,7 +336,7 @@ fn pick_best_candidate( let candidate_key = score.ordering_key(*data_root); if best_key.as_ref().is_none_or(|k| candidate_key < *k) { - best = Some((*data_root, score, new_voters)); + best = Some((*data_root, score, new_voters, new_head_voters)); best_key = Some(candidate_key); } } @@ -292,6 +344,30 @@ fn pick_best_candidate( best } +/// What a proposer builds a block out of: the attestation pool plus the two +/// pieces of node-local state used to filter and score it. +/// +/// Grouped rather than passed as loose parameters because they travel together +/// through every entry point here and are all sourced from the same `Store` +/// read in `produce_block_with_signatures`. +pub(crate) struct ProposalInputs<'a> { + /// Roots this node holds a block for. A vote naming an unknown head is not + /// packable, since the state transition could not resolve it. + pub(crate) known_block_roots: &'a HashSet, + /// The attestation pool: `data_root -> (data, proofs)`. + pub(crate) aggregated_payloads: + &'a HashMap)>, + /// Per-validator latest head votes that the CHAIN already carries. + /// + /// Deliberately not the votes fork choice holds: that map advances in + /// lockstep with the pool these entries come from, so every entry would + /// score zero new head voters. See `ForkChoiceState::on_chain_votes`. + /// + /// Owned because `Store::extract_on_chain_votes` already returns + /// a clone, and the projection mutates it as entries are selected. + pub(crate) latest_head_votes: HashMap, +} + /// Static inputs to the attestation selection scan: the candidate pool and /// the chain-level facts used to filter and score entries. Built once before /// the round loop in `select_attestations`. @@ -316,6 +392,29 @@ pub(crate) struct ProjectedState { pub(crate) justified_slots: JustifiedSlots, pub(crate) finalized_slot: u64, pub(crate) current_votes: HashMap>, + /// Each validator's latest head vote that the CHAIN already carries, + /// advanced as entries are selected so a validator is not credited twice + /// across rounds. + /// + /// `None` turns head-vote scoring off entirely, which is not the same as + /// seeding an empty map: with no recorded vote every validator in an + /// entry's coverage reads as newly covered, so an empty map scores every + /// entry as maximally valuable. + /// + /// An empty map is a real state here, and it no longer means what it meant + /// when this was seeded from fork choice: a node that has just resumed + /// holds a full set of gossip-learned votes within a slot while it has + /// still seen no block, so `on_chain_votes` is empty and every entry scores + /// its whole coverage. That errs toward packing more rather than less, it + /// is capped by `max_attestations_per_block`, and it resolves on the first + /// import that carries attestations. What it must NOT be is a stand-in for + /// "not scoring head votes here", which is what `None` is for. + /// + /// Both production callers seed it. It stays optional because the scoring + /// tests construct projections directly, and because a caller that only + /// wants justification scoring should have to say so rather than pass an + /// empty map and get the opposite. + pub(crate) head_votes: Option>, } impl ProjectedState { @@ -327,9 +426,22 @@ impl ProjectedState { justified_slots: head_state.justified_slots.clone(), finalized_slot: head_state.latest_finalized.slot, current_votes: build_running_votes(head_state), + head_votes: None, } } + /// Seed the per-validator latest head votes, so scoring can value an + /// entry for the fork-choice weight it adds and not only for the + /// justification voters it brings. + /// + /// Takes the map by value: `Store::extract_on_chain_votes` + /// already hands out an owned clone, so there is nothing to gain by + /// borrowing it and the projection then owns what it mutates. + pub(crate) fn with_head_votes(mut self, head_votes: HashMap) -> Self { + self.head_votes = Some(head_votes); + self + } + /// Fold a selected entry into the projection: record its voters under the /// entry's `target.root`, then advance justification/finalization per /// `tier` (Finalize implies Justify). `new_voters` is the entry's marginal @@ -372,15 +484,132 @@ impl ProjectedState { } } + /// Fold a selected entry's head votes into the projection, so the next + /// round does not credit the same validator for the same head again. + /// + /// Separate from [`ProjectedState::advance`] because that one is handed the + /// entry's *marginal justification* voters, which is a strictly smaller set + /// than the coverage whose head votes this entry carries. Folding head + /// votes there would silently under-credit them. + pub(crate) fn advance_head_votes( + &mut self, + att_data: &AttestationData, + new_head_voters: impl IntoIterator, + ) { + let Some(head_votes) = self.head_votes.as_mut() else { + return; + }; + for validator_id in new_head_voters { + head_votes.insert(validator_id, att_data.clone()); + } + } + + /// The subset of `coverage` whose latest head vote this entry would + /// replace, per the LMD-GHOST latest-message rule + /// ([`AttestationData::supersedes`]). + /// + /// Measured against the votes the CHAIN already carries, so a validator + /// with no entry counts as new: no block has carried a vote for it, so this + /// entry is the first weight it would contribute on chain. + /// Whether applying this entry puts 2/3 of the validator set's latest head + /// votes on `att_data.head.root`. + /// + /// The head-side analogue of `crosses_2_3` for justification, and counted + /// the same way: over the projected POST-state, not the delta. A validator + /// counts when the entry moves it onto this head, or when it already names + /// this head and the entry does not move it elsewhere. + /// + /// `None` head votes means the axis is switched off, so no supermajority + /// can be claimed. + fn head_crosses_2_3( + &self, + att_data: &AttestationData, + new_head_voters: &HashSet, + validator_count: usize, + ) -> bool { + let Some(head_votes) = self.head_votes.as_ref() else { + return false; + }; + let head_root = att_data.head.root; + // Everyone this entry moves lands on `head_root` by construction. + let retained = head_votes + .iter() + .filter(|(vid, vote)| vote.head.root == head_root && !new_head_voters.contains(vid)) + .count(); + let total = retained + new_head_voters.len(); + 3 * total >= 2 * validator_count + } + + /// The subset of `coverage` whose latest head vote this entry would + /// replace, per the LMD-GHOST latest-message rule + /// ([`AttestationData::supersedes`]). + /// + /// Measured against the votes the CHAIN already carries, so a validator + /// with no entry counts as new: no block has carried a vote for it, so this + /// entry is the first weight it would contribute on chain. + pub(crate) fn new_head_voters( + &self, + att_data: &AttestationData, + coverage: &HashSet, + ) -> HashSet { + let Some(head_votes) = self.head_votes.as_ref() else { + return HashSet::new(); + }; + coverage + .iter() + .copied() + .filter(|vid| { + head_votes + .get(vid) + .is_none_or(|existing| att_data.supersedes(existing)) + }) + .collect() + } + + /// Whether `att_data`'s target is already justified in this projection. + /// + /// Shared by `score_entry` (selection) and `body_proof::count_body_voters` + /// (scoring an already-sealed candidate) so the two cannot drift: a target the + /// state transition has already justified is dropped from `justifications` on + /// justification (`state_transition::lib`), so `current_votes` holds no prior-voter + /// entry for it and scoring would otherwise count its entire coverage as new. + /// + /// Note this is NOT a filter. `entry_passes_filters` deliberately admits a + /// settled target, since its votes still carry fork-choice weight; what this + /// predicate decides is that they carry no justification value. + /// + /// An untracked target slot (commonly the head block's own slot, or any slot past + /// the head-seeded window's edge) is not yet justified as far as this projection + /// knows, so it stays eligible: `is_slot_justified` returning `None` reads as + /// "not justified", not "unknown". Exempt: the genesis self-vote (source == target + /// == slot 0), which fork-choice bootstrapping needs even though its target is + /// trivially "justified". + pub(crate) fn target_already_justified(&self, att_data: &AttestationData) -> bool { + !is_genesis_self_vote(att_data) + && justified_slots_ops::is_slot_justified( + &self.justified_slots, + self.finalized_slot, + att_data.target.slot, + ) + .unwrap_or(false) + } + /// Score a candidate entry from its realized validator `coverage` against /// this projection. /// - /// Returns `None` if `coverage` contributes zero validators relative to the - /// running voter set for `att_data.target.root` (no marginal value, drop). - /// On `Some`, the returned `HashSet` is the subset of `coverage` that is new - /// (caller uses it to `advance` the projection without re-scanning - /// `coverage`). A genesis self-vote cannot justify or finalize and is always - /// scored as tier 3. + /// Returns `None` only if the entry is worthless on *both* axes: it adds no + /// justification voter for `att_data.target.root` and no validator's head + /// vote either. An entry that adds head votes alone is kept, at + /// [`Tier::TargetAdvance`] if those votes carry the head past 2/3 and + /// [`Tier::Build`] otherwise, because its fork-choice weight is real even + /// when its target is already carried: dropping it is how a slot whose + /// votes all name a settled target ends up proposing nothing at all. + /// + /// On `Some`, the returned sets are the subsets of `coverage` that are new + /// on each axis, so the caller can `advance` and `advance_head_votes` the + /// projection without re-scanning `coverage`. A genesis self-vote cannot + /// justify or finalize, so it never reaches `Justify`/`Finalize`; it is + /// still eligible for `TargetAdvance` on its head weight. /// /// The caller resolves `coverage` and passes it in: block building unions a /// data's proof participants (see `pick_best_candidate`); committee-signature @@ -392,16 +621,33 @@ impl ProjectedState { att_data: &AttestationData, coverage: &HashSet, validator_count: usize, - ) -> Option<(EntryScore, HashSet)> { + ) -> Option<(EntryScore, HashSet, HashSet)> { + // A settled target contributes no justification voters, whatever its + // coverage. Scoring it normally would credit the whole bitfield as new: + // the state transition drops a justified target's `justifications` + // entry, so `current_votes` holds no prior voters for it and every + // participant would read as marginal. The entry survives on its head + // votes alone, at `Build` tier. + let target_settled = self.target_already_justified(att_data); + let prior_voters = self.current_votes.get(&att_data.target.root); - let prior_count = prior_voters.map_or(0, HashSet::len); + let prior_count = if target_settled { + 0 + } else { + prior_voters.map_or(0, HashSet::len) + }; - let new_voters: HashSet = coverage - .iter() - .copied() - .filter(|vid| prior_voters.is_none_or(|prior| !prior.contains(vid))) - .collect(); - if new_voters.is_empty() { + let new_voters: HashSet = if target_settled { + HashSet::new() + } else { + coverage + .iter() + .copied() + .filter(|vid| prior_voters.is_none_or(|prior| !prior.contains(vid))) + .collect() + }; + let new_head_voters = self.new_head_voters(att_data, coverage); + if new_voters.is_empty() && new_head_voters.is_empty() { return None; } @@ -419,35 +665,54 @@ impl ProjectedState { && (att_data.source.slot + 1..att_data.target.slot) .all(|s| !slot_is_justifiable_after(s, self.finalized_slot)); - let tier = if is_genesis_self_vote(att_data) || !crosses_2_3 { - Tier::Build - } else if finalizes { + // An entry that adds no justification voter cannot move the target past + // the threshold, whatever `prior_count` already sits at, so it cannot + // justify regardless of `crosses_2_3` — it is here for its head votes. + let justifies = !is_genesis_self_vote(att_data) && crosses_2_3 && !new_voters.is_empty(); + + // Same rule on the head axis, for the same reason: an entry that moves + // nobody's head vote did not bring the head anywhere, however much + // weight already sits there. Requiring a non-empty contribution is what + // keeps a settled entry from claiming a threshold it did not cross. + let advances_head = !new_head_voters.is_empty() + && self.head_crosses_2_3(att_data, &new_head_voters, validator_count); + + let tier = if justifies && finalizes { Tier::Finalize - } else { + } else if justifies { Tier::Justify + } else if advances_head { + Tier::TargetAdvance + } else { + Tier::Build }; let score = EntryScore { tier, new_voters: new_voters.len(), + new_head_voters: new_head_voters.len(), target_slot: att_data.target.slot, att_slot: att_data.slot, }; - Some((score, new_voters)) + Some((score, new_voters, new_head_voters)) } /// Validate a candidate entry against the projection and the given chain /// view. /// - /// Mirrors `state_transition::is_valid_vote`: the entry's head must be - /// known, its source must be justified, its (source, target) must match - /// the candidate-block chain view, `target.slot > source.slot`, target - /// must not already be justified, and target must be a justifiable slot - /// relative to the projected finalized slot. The genesis self-vote - /// (source == target == slot 0) is exempt from the `target.slot > - /// source.slot` and `target_already_justified` checks since fork-choice - /// bootstrapping needs it; STF will silently drop it, but it carries - /// fork-choice signal. + /// Narrower than `state_transition::is_valid_vote`: the entry's head must + /// be known, its source must be justified, its (source, target) must match + /// the candidate-block chain view, `target.slot > source.slot`, and target + /// must be a justifiable slot relative to the projected finalized slot. + /// + /// Deliberately does NOT reject an already-justified target, though + /// `is_valid_vote` skips one: that vote still carries fork-choice weight, + /// so it is scored rather than filtered (see the note at that check, and + /// [`ProjectedState::score_entry`]). + /// + /// The genesis self-vote (source == target == slot 0) is exempt from the + /// `target.slot > source.slot` check since fork-choice bootstrapping needs + /// it; STF will silently drop it, but it carries fork-choice signal. pub(crate) fn entry_passes_filters( &self, att_data: &AttestationData, @@ -479,18 +744,21 @@ impl ProjectedState { if !is_genesis_self_vote && att_data.target.slot <= att_data.source.slot { return Err("target_not_after_source"); } - // An untracked target slot (commonly the head block's own slot) is not yet - // justified, so it stays eligible. Same reasoning as the source check above. - if !is_genesis_self_vote - && justified_slots_ops::is_slot_justified( - &self.justified_slots, - self.finalized_slot, - att_data.target.slot, - ) - .unwrap_or(false) - { - return Err("target_already_justified"); - } + // An already-justified target is deliberately NOT rejected here. + // + // The state transition skips such a vote without rejecting the block + // (`is_valid_vote` returns `Ok(false)` and `process_attestations` does + // `continue`), while `insert_signed_block` records every attestation a + // block carries as a fork-choice vote regardless of that verdict. So + // the vote is worthless for justification and still valuable for + // LMD-GHOST. That is a question of value, not validity, and it is + // answered in `score_entry`, which zeroes the justification axis for a + // settled target and keeps its head-vote value. + // + // Rejecting it here is what left a slot whose votes all name a settled + // target with nothing to propose: on devnet-5 the justifiable rungs sit + // 3 slots apart, so two slots in every three had every pooled entry + // dropped at this line and built no candidate body at all. if !is_genesis_self_vote && !slot_is_justifiable_after(att_data.target.slot, self.finalized_slot) { @@ -514,14 +782,24 @@ pub(crate) enum Tier { Finalize = 1, /// Applying the entry crosses 2/3 on target but does not finalize. Justify = 2, - /// Adds marginal new voters toward target's 2/3 supermajority. - Build = 3, + /// Applying the entry brings 2/3 of validators' latest head votes onto the + /// entry's head root, without justifying anything. + /// + /// The LMD-GHOST analogue of `Justify`: it does not move the justification + /// checkpoint, but it settles the head, which is what a later target is + /// eventually chosen against. Ranks below `Justify` because finality beats + /// head weight, and above `Build` because crossing the threshold is worth + /// more than adding marginal weight below it. + TargetAdvance = 3, + /// Adds marginal new voters toward target's 2/3 supermajority, or head + /// weight below the head threshold. + Build = 4, } /// Tiered score for a candidate `AttestationData` entry during block building. /// -/// Lower `tier` wins. Entries with zero new voters relative to the running -/// per-target-root voter set are dropped (returned as `None`). +/// Lower `tier` wins. Entries that add neither a justification voter nor a +/// head vote are dropped (returned as `None`). /// /// The within-tier ordering is tier-dependent (leanSpec PR #1149): /// @@ -533,11 +811,17 @@ pub(crate) enum Tier { /// coverage leads: more `new_voters`, then larger `target_slot`, then larger /// `att_slot`. /// +/// `new_head_voters` sits immediately after `new_voters` in both tiers, so it +/// breaks a tie on justification value and never outranks it. Two entries that +/// bring the same justification voters are not equivalent: the one whose votes +/// also move more validators' latest head is worth more to fork choice. +/// /// In both tiers `data_root` (ascending) is the final deterministic tiebreak. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) struct EntryScore { pub(crate) tier: Tier, pub(crate) new_voters: usize, + pub(crate) new_head_voters: usize, /// Read only inside [`EntryScore::ordering_key`]; kept private. target_slot: u64, /// Read only inside [`EntryScore::ordering_key`]; kept private. @@ -545,31 +829,65 @@ pub(crate) struct EntryScore { } /// Total order over candidate entries; the smallest value is the best pick. -/// `tier` leads, then three tier-dependent `Reverse`-encoded priorities, then +/// `tier` leads, then four tier-dependent `Reverse`-encoded priorities, then /// `data_root` as the deterministic tiebreak. See [`EntryScore::ordering_key`]. -pub(crate) type OrderingKey = (Tier, Reverse, Reverse, Reverse, H256); +pub(crate) type OrderingKey = ( + Tier, + Reverse, + Reverse, + Reverse, + Reverse, + H256, +); impl EntryScore { /// Sort key where the smallest tuple is the best candidate. `tier` always - /// leads; the remaining three slots carry tier-dependent priorities (see + /// leads; the remaining four slots carry tier-dependent priorities (see /// the type-level docs), all encoded as `Reverse` so "larger is better". pub(crate) fn ordering_key(&self, data_root: H256) -> OrderingKey { + /// Filler for the `OrderingKey` slots a tier does not rank on. Any + /// constant works: identical across every entry in that tier, it can + /// never decide a comparison. + const ORDERING_UNUSED: Reverse = Reverse(0); + let more_new_voters = Reverse(self.new_voters as u64); + let more_new_head_voters = Reverse(self.new_head_voters as u64); let newer_target = Reverse(self.target_slot); let newer_att = Reverse(self.att_slot); match self.tier { - Tier::Build => ( + // Finality first: which checkpoint this moves, then how recent the + // vote is, and only then how much weight it carries. Head votes + // outrank justification voters here because by this point the + // target is already crossing 2/3, so the marginal justification + // voter is worth less than the head weight riding along with it. + Tier::Finalize | Tier::Justify => ( self.tier, - more_new_voters, newer_target, newer_att, + more_new_head_voters, + more_new_voters, data_root, ), - Tier::Finalize | Tier::Justify => ( + // The head is what this tier settles, and the target is not moving, + // so `newer_target` would be noise. Justification voters are not + // ranked at all: an entry here is chosen for head weight. + // `ORDERING_UNUSED` holds the two slots this tier does not rank on; + // being constant, it never discriminates. + Tier::TargetAdvance => ( self.tier, - newer_target, newer_att, + more_new_head_voters, + ORDERING_UNUSED, + ORDERING_UNUSED, + data_root, + ), + // Below every threshold, so raw progress toward one leads. + Tier::Build => ( + self.tier, more_new_voters, + more_new_head_voters, + newer_target, + newer_att, data_root, ), } @@ -891,8 +1209,12 @@ fn trace_skipped_attestation(reason: &'static str, att: &AttestationData, data_r mod tests { use super::*; use ethlambda_types::{ - attestation::{AggregatedAttestation, AggregationBits, AttestationData}, - block::{ByteList512KiB, MultiMessageAggregate, SignedBlock, SingleMessageAggregate}, + attestation::{ + AggregatedAttestation, AggregationBits, AttestationData, blank_xmss_signature, + }, + block::{ + BlockProof, ByteList512KiB, MultiMessageAggregate, SignedBlock, SingleMessageAggregate, + }, checkpoint::Checkpoint, state::State, }; @@ -948,9 +1270,10 @@ mod tests { justified_slots: JustifiedSlots::new(), finalized_slot: FINALIZED_SLOT, current_votes: HashMap::new(), + head_votes: None, }; - let (score, _) = projected + let (score, _, _) = projected .score_entry(&att_data, &coverage, NUM_VALIDATORS) .expect("entry contributes new voters"); @@ -961,6 +1284,488 @@ mod tests { ); } + /// An entry scored against a projection whose head votes were never seeded + /// must report zero new head voters. + /// + /// `None` must mean "do not score head votes", not "an empty map": with no + /// recorded vote every validator in coverage reads as newly covered, so an + /// empty map scores every entry as maximally valuable and `score_entry` + /// stops returning `None`. A caller wanting justification-only scoring has + /// to be able to say so without accidentally getting the opposite. + #[test] + fn head_vote_scoring_is_off_when_the_map_is_not_seeded() { + let projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::from([(H256::ZERO, HashSet::from([0, 1, 2]))]), + head_votes: None, + }; + let coverage: HashSet = HashSet::from([0, 1, 2]); + + assert!( + projected + .score_entry(&make_att_data(5), &coverage, 4) + .is_none(), + "an unseeded projection must fall back to justification-only scoring" + ); + } + + /// A vote whose target is fully covered still carries fork-choice weight, + /// so it is kept at `Build` rather than dropped. + #[test] + fn score_entry_keeps_an_entry_that_only_adds_head_votes() { + let att_data = AttestationData { + slot: 9, + head: Checkpoint { + slot: 8, + root: H256([8u8; 32]), + }, + target: Checkpoint { + slot: 6, + root: H256([6u8; 32]), + }, + source: Checkpoint { + slot: 3, + root: H256([3u8; 32]), + }, + }; + let coverage: HashSet = HashSet::from([0, 1, 2]); + + // Every voter already counted toward this target, so there is no + // justification value left; their recorded head vote is older. + let projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::from([(att_data.target.root, coverage.clone())]), + head_votes: Some(HashMap::from([ + (0, make_att_data(4)), + (1, make_att_data(4)), + (2, make_att_data(4)), + ])), + }; + + let (score, new_voters, new_head_voters) = projected + .score_entry(&att_data, &coverage, 4) + .expect("an entry that only moves heads is still worth carrying"); + + assert!( + new_voters.is_empty(), + "the target was already fully covered" + ); + assert_eq!(new_head_voters.len(), 3); + assert_eq!(score.new_head_voters, 3); + assert!( + score.tier > Tier::Justify, + "an entry adding no justification voter cannot justify, whatever \ + the prior count" + ); + assert_eq!( + score.tier, + Tier::TargetAdvance, + "3 of 4 validators moved onto this head crosses 2/3" + ); + } + + /// Worthless on both axes: already counted for the target, and every voter + /// already holds a newer head vote. + #[test] + fn score_entry_drops_an_entry_that_adds_neither_voters_nor_head_votes() { + let coverage: HashSet = HashSet::from([0, 1, 2]); + let projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::from([(H256::ZERO, coverage.clone())]), + head_votes: Some(HashMap::from([ + (0, make_att_data(9)), + (1, make_att_data(9)), + (2, make_att_data(9)), + ])), + }; + + assert!( + projected + .score_entry(&make_att_data(5), &coverage, 4) + .is_none(), + "a vote older than what fork choice already holds adds nothing" + ); + } + + /// Credited head votes do not count twice across selection rounds, and a + /// genuinely newer vote still does. + #[test] + fn advance_head_votes_prevents_double_counting_across_rounds() { + let mut projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::new(), + head_votes: Some(HashMap::new()), + }; + let coverage: HashSet = HashSet::from([0, 1]); + let first = make_att_data(5); + + let credited = projected.new_head_voters(&first, &coverage); + assert_eq!( + credited.len(), + 2, + "no recorded vote means every voter is new" + ); + + projected.advance_head_votes(&first, credited); + assert!( + projected.new_head_voters(&first, &coverage).is_empty(), + "the same entry must not be credited a second time" + ); + assert_eq!( + projected + .new_head_voters(&make_att_data(6), &coverage) + .len(), + 2, + "a later slot still supersedes what this block already credited" + ); + } + + /// A settled target is no longer filtered out, and is scored with its + /// justification axis zeroed: its coverage must NOT be credited as new + /// voters just because the state transition dropped its `justifications` + /// entry on justification. + #[test] + fn score_entry_zeroes_the_justification_axis_for_a_settled_target() { + const FINALIZED_SLOT: u64 = 0; + const TARGET_SLOT: u64 = 3; + + let mut justified_slots = JustifiedSlots::new(); + justified_slots_ops::extend_to_slot(&mut justified_slots, FINALIZED_SLOT, TARGET_SLOT); + justified_slots_ops::set_justified(&mut justified_slots, FINALIZED_SLOT, TARGET_SLOT); + + let att_data = AttestationData { + slot: 5, + head: Checkpoint { + slot: 4, + root: H256([4u8; 32]), + }, + target: Checkpoint { + slot: TARGET_SLOT, + root: H256([3u8; 32]), + }, + source: Checkpoint { + slot: 1, + root: H256([1u8; 32]), + }, + }; + let coverage: HashSet = HashSet::from([0, 1, 2, 3]); + + let projected = ProjectedState { + justified_slots, + finalized_slot: FINALIZED_SLOT, + // Empty, exactly as it is after the transition drops a justified + // target's tally. Without the settled-target guard the whole + // coverage would read as new. + current_votes: HashMap::new(), + head_votes: Some(HashMap::new()), + }; + + let (score, new_voters, new_head_voters) = projected + .score_entry(&att_data, &coverage, 4) + .expect("head votes keep the entry alive"); + + assert!( + new_voters.is_empty(), + "a settled target must credit no justification voters" + ); + assert_eq!(score.new_voters, 0); + assert_eq!(new_head_voters.len(), 4, "its head votes are still new"); + assert!( + score.tier > Tier::Justify, + "it cannot justify, so it must not be tiered as if it could" + ); + assert_eq!( + score.tier, + Tier::TargetAdvance, + "its head votes still carry the head past 2/3" + ); + } + + /// The filter must let a settled target through, since the state transition + /// skips such a vote without rejecting the block while still recording it + /// as a fork-choice vote. Rejecting it here is what left slots whose votes + /// all named a settled target with no candidate body at all. + #[test] + fn entry_passes_filters_admits_an_already_justified_target() { + const FINALIZED_SLOT: u64 = 0; + const TARGET_SLOT: u64 = 2; + + let mut justified_slots = JustifiedSlots::new(); + justified_slots_ops::extend_to_slot(&mut justified_slots, FINALIZED_SLOT, TARGET_SLOT); + justified_slots_ops::set_justified(&mut justified_slots, FINALIZED_SLOT, TARGET_SLOT); + // Source at slot 1 must read as justified for the filter to get past it. + justified_slots_ops::set_justified(&mut justified_slots, FINALIZED_SLOT, 1); + + let roots: Vec = (0..4u8).map(|i| H256([i + 1; 32])).collect(); + let att_data = AttestationData { + slot: 3, + head: Checkpoint { + slot: TARGET_SLOT, + root: roots[TARGET_SLOT as usize], + }, + target: Checkpoint { + slot: TARGET_SLOT, + root: roots[TARGET_SLOT as usize], + }, + source: Checkpoint { + slot: 1, + root: roots[1], + }, + }; + + let projected = ProjectedState { + justified_slots, + finalized_slot: FINALIZED_SLOT, + current_votes: HashMap::new(), + head_votes: None, + }; + let known: HashSet = roots.iter().copied().collect(); + + assert!( + projected.target_already_justified(&att_data), + "fixture must actually have a settled target" + ); + assert_eq!( + projected.entry_passes_filters(&att_data, &known, &roots), + Ok(()), + "a settled target is a scoring question, not a validity one" + ); + } + + /// Below the head threshold there is no `TargetAdvance`: the entry is + /// carrying weight, not settling anything. + #[test] + fn head_votes_below_two_thirds_stay_at_build() { + let att_data = make_att_data(5); + // 1 of 10 validators is nowhere near 2/3. + let coverage: HashSet = HashSet::from([0]); + + let projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::from([(att_data.target.root, coverage.clone())]), + head_votes: Some(HashMap::from([(0, make_att_data(4))])), + }; + + let (score, _, new_head_voters) = projected + .score_entry(&att_data, &coverage, 10) + .expect("it still moves a head vote"); + + assert_eq!(new_head_voters.len(), 1); + assert_eq!(score.tier, Tier::Build); + } + + /// An entry that moves nobody's head vote must not claim `TargetAdvance` + /// off weight that was already there. Same rule the justification axis + /// applies, and for the same reason: the threshold has to be crossed BY + /// this entry. + #[test] + fn an_entry_that_moves_no_head_vote_cannot_claim_target_advance() { + // A real target, not `make_att_data`'s genesis self-vote, so the entry + // can actually reach `Justify`. + let att_data = AttestationData { + slot: 5, + head: Checkpoint { + slot: 4, + root: H256([4u8; 32]), + }, + target: Checkpoint { + slot: 3, + root: H256([3u8; 32]), + }, + source: Checkpoint { + slot: 1, + root: H256([1u8; 32]), + }, + }; + let coverage: HashSet = HashSet::from([0, 1, 2]); + + // Everyone already voted for this exact data, so nothing moves, but the + // head is already at a supermajority. + let projected = ProjectedState { + justified_slots: JustifiedSlots::new(), + finalized_slot: 0, + current_votes: HashMap::new(), + head_votes: Some(HashMap::from([ + (0, att_data.clone()), + (1, att_data.clone()), + (2, att_data.clone()), + ])), + }; + + let (score, _, new_head_voters) = projected + .score_entry(&att_data, &coverage, 4) + .expect("it still adds justification voters"); + + assert!(new_head_voters.is_empty(), "no head vote moves"); + assert_eq!( + score.tier, + Tier::Justify, + "it justifies on its own axis, and must not be credited for a head \ + threshold it did not cross" + ); + } + + /// `TargetAdvance` ranks on recency first, then head weight. The target is + /// not moving at this tier, so `newer_target` is deliberately not consulted. + #[test] + fn target_advance_ranks_newer_attestation_over_more_head_votes() { + let newer_but_lighter = EntryScore { + tier: Tier::TargetAdvance, + new_voters: 0, + new_head_voters: 1, + target_slot: 2, + att_slot: 9, + }; + let older_but_heavier = EntryScore { + tier: Tier::TargetAdvance, + new_voters: 0, + new_head_voters: 500, + target_slot: 7, + att_slot: 8, + }; + + assert!( + newer_but_lighter.ordering_key(H256([1u8; 32])) + < older_but_heavier.ordering_key(H256([2u8; 32])), + "a fresher attestation wins even against far more head weight" + ); + } + + /// Within `TargetAdvance`, equal attestation slots fall through to head + /// weight, and justification voters never enter the comparison. + #[test] + fn target_advance_breaks_an_attestation_slot_tie_on_head_votes_only() { + let heavier = EntryScore { + tier: Tier::TargetAdvance, + new_voters: 0, + new_head_voters: 9, + target_slot: 1, + att_slot: 8, + }; + let lighter_but_more_justification_voters = EntryScore { + tier: Tier::TargetAdvance, + new_voters: 900, + new_head_voters: 8, + target_slot: 1, + att_slot: 8, + }; + + assert!( + heavier.ordering_key(H256([1u8; 32])) + < lighter_but_more_justification_voters.ordering_key(H256([2u8; 32])), + "head weight decides; justification voters are not ranked at this tier" + ); + } + + /// In the justify arm head votes now outrank justification voters, once + /// target and attestation slots tie. + #[test] + fn justify_ranks_head_votes_above_justification_voters() { + let more_head_votes = EntryScore { + tier: Tier::Justify, + new_voters: 1, + new_head_voters: 50, + target_slot: 4, + att_slot: 6, + }; + let more_justification_voters = EntryScore { + tier: Tier::Justify, + new_voters: 900, + new_head_voters: 49, + target_slot: 4, + att_slot: 6, + }; + + assert!( + more_head_votes.ordering_key(H256([1u8; 32])) + < more_justification_voters.ordering_key(H256([2u8; 32])), + "past the 2/3 target threshold the marginal justification voter is \ + worth less than head weight" + ); + } + + /// Tier still dominates every other term: a `Justify` entry with nothing + /// else going for it beats the best possible `TargetAdvance` entry, which + /// in turn beats the best possible `Build` entry. + #[test] + fn tier_dominates_every_other_ordering_term() { + let justify = EntryScore { + tier: Tier::Justify, + new_voters: 0, + new_head_voters: 0, + target_slot: 0, + att_slot: 0, + }; + let target_advance = EntryScore { + tier: Tier::TargetAdvance, + new_voters: u32::MAX as usize, + new_head_voters: u32::MAX as usize, + target_slot: u64::MAX, + att_slot: u64::MAX, + }; + let build = EntryScore { + tier: Tier::Build, + new_voters: u32::MAX as usize, + new_head_voters: u32::MAX as usize, + target_slot: u64::MAX, + att_slot: u64::MAX, + }; + let root = H256([1u8; 32]); + + assert!(justify.ordering_key(root) < target_advance.ordering_key(root)); + assert!(target_advance.ordering_key(root) < build.ordering_key(root)); + } + + /// Head votes break a tie on justification voters, and never outrank them. + #[test] + fn head_votes_break_a_tie_on_justification_voters() { + let root = H256::ZERO; + let base = EntryScore { + tier: Tier::Build, + new_voters: 3, + new_head_voters: 1, + target_slot: 5, + att_slot: 7, + }; + let more_head = EntryScore { + new_head_voters: 2, + ..base + }; + let more_voters = EntryScore { + new_voters: 4, + new_head_voters: 0, + ..base + }; + + assert!( + more_head.ordering_key(root) < base.ordering_key(root), + "with justification voters tied, more head votes must win" + ); + assert!( + more_voters.ordering_key(root) < more_head.ordering_key(root), + "head votes must not outrank justification voters" + ); + + // Same rule in the Justify arm, where new_voters sits later in the key. + let justify = EntryScore { + tier: Tier::Justify, + ..base + }; + let justify_more_head = EntryScore { + new_head_voters: 2, + ..justify + }; + assert!( + justify_more_head.ordering_key(root) < justify.ordering_key(root), + "the Justify arm must break its new_voters tie on head votes too" + ); + } + /// Regression test for https://github.com/lambdaclass/ethlambda/issues/259 /// /// Simulates a stall scenario by populating the payload pool with 50 @@ -1083,8 +1888,11 @@ mod tests { slot, proposer_index, parent_root, - &known_block_roots, - &aggregated_payloads, + ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes: HashMap::new(), + }, ProposerConfig { enable_proposer_aggregation: true, max_attestations_per_block: MAX_ATTESTATIONS_DATA, @@ -1101,11 +1909,16 @@ mod tests { ); // Substitute a worst-case-size proof to model what `propose_block` - // would attach. The actual SNARK can't be built without lean-multisig, - // but the size cap (`ByteList512KiB`) bounds the worst case. + // would attach: a 512 KiB attestation aggregate plus the fixed-size + // proposer signature. The actual SNARK can't be built without + // lean-multisig, but the size cap bounds the worst case. let _ = signatures; - let proof = MultiMessageAggregate::new( - ByteList512KiB::try_from(vec![0xAB; 512 * 1024]).expect("worst-case proof fits in cap"), + let proof = BlockProof::new( + blank_xmss_signature(), + MultiMessageAggregate::new( + ByteList512KiB::try_from(vec![0xAB; 512 * 1024]) + .expect("worst-case proof fits in cap"), + ), ); let signed_block = SignedBlock { message: block, @@ -1229,8 +2042,11 @@ mod tests { slot, proposer_index, parent_root, - &known_block_roots, - &aggregated_payloads, + ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes: HashMap::new(), + }, ProposerConfig { enable_proposer_aggregation: false, max_attestations_per_block: limit, @@ -1355,8 +2171,11 @@ mod tests { slot, proposer_index, parent_root, - &known_block_roots, - &aggregated_payloads, + ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes: HashMap::new(), + }, ProposerConfig { enable_proposer_aggregation: false, max_attestations_per_block: MAX_ATTESTATIONS_DATA, @@ -1661,8 +2480,11 @@ mod tests { slot, proposer_index, parent_root, - &known_block_roots, - &aggregated_payloads, + ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes: HashMap::new(), + }, ProposerConfig { enable_proposer_aggregation: true, max_attestations_per_block: MAX_ATTESTATIONS_DATA, @@ -1797,8 +2619,11 @@ mod tests { slot, proposer_index, parent_root, - &known_block_roots, - &aggregated_payloads, + ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes: HashMap::new(), + }, ProposerConfig { enable_proposer_aggregation: true, max_attestations_per_block: MAX_ATTESTATIONS_DATA, diff --git a/crates/blockchain/src/body_proof.rs b/crates/blockchain/src/body_proof.rs new file mode 100644 index 00000000..0acbe5f6 --- /dev/null +++ b/crates/blockchain/src/body_proof.rs @@ -0,0 +1,989 @@ +//! Block body proofs: candidate bodies a proposer can adopt instead of +//! building one itself. +//! +//! A [`BlockBodyProof`] pairs a candidate [`BlockBody`] with the Type-2 +//! aggregate binding its attestations. Since the proposer signature now sits +//! outside that aggregate (`BlockProof`), the aggregate no longer depends on +//! the block root, so any node can build one before the block exists — and the +//! merge, which is the expensive part of proposing, moves off the proposer's +//! critical path onto the aggregation worker and the gossip layer. +//! +//! This module owns both halves. [`build_body_proof`] is what the aggregation +//! worker runs during the head-update interval; [`BodyProofBuffer`] is the +//! bounded set of candidates a node collects each slot, from its own worker and +//! from gossip, and [`choose_body`] is how the slot's proposer picks one — or +//! decides an empty body is worth more. + +use std::collections::{HashMap, HashSet, VecDeque}; + +use ethlambda_crypto::signature::ValidatorPublicKey; +use ethlambda_state_transition::attestation_data_matches_chain; +use ethlambda_storage::Store; +use ethlambda_types::{ + attestation::{AttestationData, validator_indices}, + block::{ + Block, BlockBody, BlockBodyProof, ByteList512KiB, MultiMessageAggregate, + MultiMessageAggregateError, SingleMessageAggregate, + }, + primitives::{H256, HashTreeRoot as _}, + state::{State, Validator}, +}; +use spawned_concurrency::message::Message; +use tracing::{info, trace, warn}; + +use crate::block_builder::{self, PostBlockCheckpoints, ProposerConfig}; +use crate::metrics; +use crate::store::StoreError; + +/// Maximum candidates kept at once. One body proof per aggregator per slot +/// reaches a node, and only the freshest batch is worth anything to the next +/// proposer, so the buffer is a small ring rather than a growing pool. +pub(crate) const MAX_BODY_PROOF_CANDIDATES: usize = 8; + +/// Build a candidate body for `slot` off the store, and the Type-2 aggregate +/// binding its attestations. +/// +/// Runs on the aggregation worker, which is not the proposer and must not +/// mutate the store: unlike `produce_block_with_signatures` it takes the +/// current fork-choice head as the parent instead of advancing the store's +/// clock to `slot`, and it assembles no block — a body proof commits to no +/// parent, no state root and no proposer, which is exactly why it can be built +/// by a node that will not propose. Whoever adopts it re-validates the body +/// against its own state. +/// +/// Returns `None` when the pool yields no attestations: an empty body needs no +/// proof, and the proposer's empty-block fallback covers that case for free. +pub(crate) fn build_body_proof( + store: &Store, + slot: u64, + config: ProposerConfig, +) -> Option { + let parent_root = store.head().expect("head read works"); + let head_state = store + .get_state(&parent_root) + .expect("head state read works")?; + let aggregated_payloads = store.known_aggregated_payloads(); + let known_block_roots = store.get_block_roots().expect("block roots read works"); + + let inputs = block_builder::ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + // What the CHAIN carries, not what this node has seen: the pool this + // body is built from and the seen-votes map advance together, so + // scoring against the latter reports zero for every candidate. + latest_head_votes: store.extract_on_chain_votes(), + }; + + let (attestations, aggregates) = + block_builder::select_and_compact(&head_state, slot, parent_root, inputs, config) + .inspect_err(|err| warn!(%slot, %err, "Failed to select attestations for a body proof")) + .ok()?; + + if aggregates.is_empty() { + trace!(%slot, "No attestations to build a body proof from"); + return None; + } + + let proof = merge_attestation_aggregates(&head_state.validators, &aggregates) + .inspect_err(|err| warn!(%slot, %err, "Failed to build a body proof aggregate")) + .ok()?; + + Some(BlockBodyProof { + block_body: BlockBody { attestations }, + proof, + }) +} + +/// Merge per-attestation single-message aggregates into the one Type-2 a block +/// body carries. +/// +/// The components are the body's attestations and nothing else — with the +/// proposer signature outside the proof, no block root enters here, which is +/// what lets this run before the block exists. +fn merge_attestation_aggregates( + validators: &[Validator], + aggregates: &[SingleMessageAggregate], +) -> Result { + let mut merge_inputs: Vec<(Vec, ByteList512KiB)> = + Vec::with_capacity(aggregates.len()); + + for aggregate in aggregates { + let mut pubkeys = Vec::new(); + for vid in aggregate.participant_indices() { + let validator = validators + .get(vid as usize) + .ok_or(BodyProofError::ParticipantOutOfRange(vid))?; + let pubkey = ValidatorPublicKey::from_bytes(&validator.attestation_pubkey) + .map_err(|_| BodyProofError::PubkeyDecoding(vid))?; + pubkeys.push(pubkey); + } + merge_inputs.push((pubkeys, aggregate.proof.clone())); + } + + let merged = ethlambda_crypto::merge_type_1s_into_type_2(merge_inputs) + .map_err(|err| BodyProofError::Merge(err.to_string()))?; + + Ok(MultiMessageAggregate::from_bytes(merged.iter().as_slice())?) +} + +/// Why a body proof could not be built. Every variant means "propose without +/// this candidate": the proposer's own fallback still produces a valid block. +#[derive(Debug, thiserror::Error)] +pub(crate) enum BodyProofError { + #[error("attestation participant {0} is beyond the validator registry")] + ParticipantOutOfRange(u64), + #[error("could not decode the attestation pubkey of validator {0}")] + PubkeyDecoding(u64), + #[error("could not merge the attestation Type-1s into a Type-2: {0}")] + Merge(String), + #[error("merged attestation proof does not fit the block proof: {0}")] + ProofTooLarge(#[from] MultiMessageAggregateError), +} + +/// Self-message that assembles and publishes `slot`'s block. +/// +/// Scheduled by the block-publication tick when the candidate buffer is still +/// empty: the merge that produces a candidate spans the interval boundary, so +/// the batch for this slot often lands a couple of hundred milliseconds into +/// it. A message rather than an in-handler wait, because the actor has to keep +/// processing gossip in the meantime — that is what it is waiting for. +pub(crate) struct AssembleProposal { + pub(crate) slot: u64, + pub(crate) validator_id: u64, +} +impl Message for AssembleProposal { + type Result = (); +} + +/// A candidate body the proposer may adopt. +pub(crate) struct BodyProofCandidate { + pub(crate) body_proof: BlockBodyProof, + /// Whether the aggregate has already been established as valid: true for + /// one our own worker built, false for one that arrived on gossip. + /// + /// A proposer signs the block root exactly once per slot — the XMSS key is + /// one-time — so a candidate's proof has to be verified *before* signing, + /// not by importing the signed block and seeing whether it sticks. This + /// flag is what spares us that verification on our own proofs. + pub(crate) verified: bool, +} + +/// The body a proposer decided to build its block around, sealed and ready to +/// sign. +pub(crate) struct ChosenBody { + /// The block, `state_root` sealed by the state transition. + pub(crate) block: Block, + /// The aggregate binding the block's attestations: the adopted candidate's + /// proof, or an empty one for an empty body. + pub(crate) attestation_proof: MultiMessageAggregate, + /// Whether a candidate body proof was adopted (as opposed to falling back + /// to an empty body). + pub(crate) adopted: bool, +} + +/// How a sealed candidate compares to another. Higher is better: +/// finalization first, then justification, then how many voters the body adds +/// that the pre-state did not already have, then — all else equal — the +/// smaller body. +/// +/// The two voter terms are what keep a stale candidate out: its attestations +/// are already reflected in the state and its votes are no validator's latest, +/// so it adds nothing on either axis and loses to the empty body it ties on +/// checkpoints. +/// +/// `new_head_voters` sits directly after `new_voters`, matching +/// `EntryScore::ordering_key`: it breaks a tie on justification value and never +/// outranks it. Ordering the two this way is also what makes the body worth +/// carrying at all when every target has settled — a body that justifies +/// nothing but moves validators' latest head still beats an empty block. +#[derive(PartialEq, Eq, PartialOrd, Ord)] +struct BodyValue { + finalized_slot: u64, + justified_slot: u64, + new_voters: usize, + new_head_voters: usize, + /// Negated so that fewer attestations sorts higher. + fewer_attestations: isize, +} + +/// What a candidate body adds on top of the head state, on the two axes a +/// proposer cares about. Produced by [`count_body_voters`]. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub(crate) struct BodyVoters { + /// Validators the body adds toward justifying some target. + new_voters: usize, + /// Validators whose latest head vote the body would move. + new_head_voters: usize, +} + +impl BodyVoters { + /// Whether the body adds nothing on either axis, and so is not worth + /// carrying or keeping buffered. + fn is_scoreless(&self) -> bool { + self.new_voters == 0 && self.new_head_voters == 0 + } +} + +/// Choose the body for a block at `slot` from the buffered candidates, +/// falling back to an empty body. +/// +/// Each candidate is screened against the chain the block would extend, sealed +/// against `head_state` — the state transition both applies its attestations +/// and computes the state root — and scored by [`BodyValue`]. The best +/// candidate that beats the empty body is adopted, with three caveats: +/// +/// - a candidate carrying a vote that does not sit on that chain is dropped. +/// The state transition does not check those roots, so a body packed against +/// another node's view would otherwise be carried verbatim — valid, and +/// worthless; +/// - a candidate that arrived on gossip has its aggregate verified before it +/// is adopted, since the proposer signs the block root only once (XMSS keys +/// are one-time) and so cannot discover a bad proof by trying to import the +/// signed block; +/// - a candidate whose state transition or verification fails is dropped and +/// the next-best one considered. +/// +/// The empty-body fallback always succeeds and needs no prover work at all: +/// with the proposer signature outside the proof, an attestation-less block +/// carries no aggregate. +pub(crate) fn choose_body( + head_state: &State, + slot: u64, + proposer_index: u64, + parent_root: H256, + candidates: &BodyProofBuffer, + latest_head_votes: &HashMap, +) -> Result { + metrics::observe_body_proof_candidates(candidates.len()); + + let chain_view = block_builder::extended_chain_view(head_state, slot, parent_root); + + let (empty_block, empty_post) = block_builder::seal_block( + head_state, + slot, + proposer_index, + parent_root, + BlockBody::default(), + )?; + let empty_value = BodyValue { + finalized_slot: empty_post.finalized.slot, + justified_slot: empty_post.justified.slot, + new_voters: 0, + new_head_voters: 0, + fewer_attestations: 0, + }; + + let validator_count = head_state.validators.len(); + let mut ranked: Vec<(BodyValue, &BodyProofCandidate, Block, PostBlockCheckpoints)> = Vec::new(); + + for candidate in candidates.iter() { + let body = candidate.body_proof.block_body.clone(); + let attestation_count = body.attestations.len(); + if !body_votes_on_chain(&body, &chain_view) { + trace!( + %slot, + attestation_count, + "Rejected a candidate body proof: it votes off our chain" + ); + metrics::inc_body_proof_rejected("off_chain_vote"); + continue; + } + let voters = count_body_voters(head_state, &body, validator_count, latest_head_votes); + let sealed = block_builder::seal_block(head_state, slot, proposer_index, parent_root, body); + let (block, post) = match sealed { + Ok(sealed) => sealed, + Err(err) => { + // Expected, not exceptional: the candidate was packed against + // another node's view of the chain. + trace!(%slot, attestation_count, %err, "Rejected a candidate body proof"); + metrics::inc_body_proof_rejected("state_transition"); + continue; + } + }; + let value = BodyValue { + finalized_slot: post.finalized.slot, + justified_slot: post.justified.slot, + new_voters: voters.new_voters, + new_head_voters: voters.new_head_voters, + fewer_attestations: -(attestation_count as isize), + }; + if value <= empty_value { + trace!( + %slot, + attestation_count, + new_voters = voters.new_voters, + new_head_voters = voters.new_head_voters, + "Candidate body proof is worth no more than an empty body" + ); + continue; + } + ranked.push((value, candidate, block, post)); + } + + ranked.sort_by(|a, b| b.0.cmp(&a.0)); + + for (value, candidate, block, _post) in ranked { + if !candidate.verified + && let Err(err) = verify_body_proof(head_state, &candidate.body_proof) + { + warn!(%slot, %err, "Candidate body proof failed verification"); + metrics::inc_body_proof_rejected("verification"); + continue; + } + + info!( + %slot, + attestation_count = block.body.attestations.len(), + new_voters = value.new_voters, + new_head_voters = value.new_head_voters, + justified_slot = value.justified_slot, + finalized_slot = value.finalized_slot, + from_gossip = !candidate.verified, + "Adopted a candidate body proof" + ); + metrics::inc_block_body_from_proof(); + return Ok(ChosenBody { + block, + attestation_proof: candidate.body_proof.proof.clone(), + adopted: true, + }); + } + + info!( + %slot, + candidates = candidates.len(), + "No usable candidate body proof; proposing an empty block" + ); + metrics::inc_block_body_empty(); + Ok(ChosenBody { + block: empty_block, + attestation_proof: MultiMessageAggregate::default(), + adopted: false, + }) +} + +/// Whether every vote in a body sits on the chain the block would extend: +/// each one's source, target and head root found at its own slot in +/// `chain_view`. +/// +/// Deliberately narrower than the block builder's `entry_passes_filters`, +/// which also drops entries that are merely unhelpful — a target already +/// justified, a source not yet justified. Those are per-entry verdicts, and a +/// body is all-or-nothing: its proof binds exactly these attestations, so one +/// stale entry would cost the whole candidate and, often, leave the slot with +/// an empty block. A stale vote is already discounted by the new-voter score; +/// an off-chain vote is the one a proposer must not carry, and the state +/// transition would carry it happily. +fn body_votes_on_chain(body: &BlockBody, chain_view: &[H256]) -> bool { + body.attestations + .iter() + .all(|attestation| attestation_data_matches_chain(chain_view, &attestation.data)) +} + +/// Count the validators a body's attestations add on top of `head_state`. +/// +/// Uses the block builder's projection so the counts mean the same thing they +/// do during selection: per target root, voters the running set does not +/// already hold, and per validator, whether the vote would become their latest. +/// +/// Delegates entirely to `score_entry`, including its handling of a target the +/// state transition has already justified: that target contributes no +/// justification voters (its `justifications` entry was dropped on +/// justification, so scoring it naively would credit the entire coverage as +/// new) while its head votes still count. Not reimplemented here, so the +/// proposer's verdict on an already-sealed candidate and the selector's verdict +/// on a pool entry cannot drift. +/// +/// Two axes rather than one because a body whose targets have all settled can +/// still be the freshest fork-choice weight anyone has. +fn count_body_voters( + head_state: &State, + body: &BlockBody, + validator_count: usize, + latest_head_votes: &HashMap, +) -> BodyVoters { + let mut projected = block_builder::ProjectedState::from_head_state(head_state) + .with_head_votes(latest_head_votes.clone()); + let mut counts = BodyVoters::default(); + + for attestation in body.attestations.iter() { + let coverage: HashSet = validator_indices(&attestation.aggregation_bits).collect(); + + let Some((score, new_voters, new_head_voters)) = + projected.score_entry(&attestation.data, &coverage, validator_count) + else { + continue; + }; + counts.new_voters += new_voters.len(); + counts.new_head_voters += new_head_voters.len(); + projected.advance(score.tier, &attestation.data, new_voters); + projected.advance_head_votes(&attestation.data, new_head_voters); + } + + counts +} + +/// Verify a candidate's aggregate against the body it claims to bind: one +/// Type-2 component per attestation, each bound to that attestation's data +/// root and slot. +/// +/// The same check `verify_block_signatures` runs on import, minus the proposer +/// signature (which does not exist yet). +fn verify_body_proof(head_state: &State, body_proof: &BlockBodyProof) -> Result<(), StoreError> { + let attestations = &body_proof.block_body.attestations; + let validators = &head_state.validators; + let num_validators = validators.len() as u64; + + let mut pubkeys_per_component: Vec> = + Vec::with_capacity(attestations.len()); + let mut expected_bindings: Vec<(H256, u32)> = Vec::with_capacity(attestations.len()); + + for attestation in attestations.iter() { + let mut pubkeys = Vec::new(); + for vid in validator_indices(&attestation.aggregation_bits) { + let validator = + validators + .get(vid as usize) + .ok_or(StoreError::AttesterIndexOutOfRange { + validator_index: vid, + num_validators, + })?; + let pubkey = ValidatorPublicKey::from_bytes(&validator.attestation_pubkey) + .map_err(|_| StoreError::PubkeyDecodingFailed(vid))?; + pubkeys.push(pubkey); + } + pubkeys_per_component.push(pubkeys); + let slot = u32::try_from(attestation.data.slot) + .map_err(|_| StoreError::SlotOutOfRange(attestation.data.slot))?; + expected_bindings.push((attestation.data.hash_tree_root(), slot)); + } + + let _timing = metrics::time_pq_sig_aggregated_signatures_verification(); + ethlambda_crypto::verify_type_2_signature( + body_proof.proof.proof_bytes(), + pubkeys_per_component, + &expected_bindings, + ) + .map_err(StoreError::BlockProofVerificationFailed) +} + +/// Bounded, newest-first buffer of proposal candidates. +#[derive(Default)] +pub(crate) struct BodyProofBuffer { + candidates: VecDeque, +} + +impl BodyProofBuffer { + /// Record a candidate built by our own aggregation worker. + pub(crate) fn push_local(&mut self, body_proof: BlockBodyProof) { + self.push(BodyProofCandidate { + body_proof, + verified: true, + }); + } + + /// Record a candidate that arrived on gossip. Its aggregate is not + /// verified here: verification costs a full Type-2 check, and only the + /// slot's proposer ever needs the answer. + pub(crate) fn push_gossip(&mut self, body_proof: BlockBodyProof) { + self.push(BodyProofCandidate { + body_proof, + verified: false, + }); + } + + fn push(&mut self, candidate: BodyProofCandidate) { + self.candidates.push_front(candidate); + while self.candidates.len() > MAX_BODY_PROOF_CANDIDATES { + let dropped = self.candidates.pop_back(); + trace!( + dropped_attestations = dropped.map(|c| c.body_proof.block_body.attestations.len()), + "Evicted the oldest block body proof candidate" + ); + } + } + + /// Drop every candidate that adds nothing on top of `head_state`: no voter + /// toward justifying a target, and no validator's latest head vote either. + /// Returns how many were dropped. + /// + /// Run after a block is imported, which is exactly when a candidate can + /// become worthless: importing a block folds its attestations into the + /// state and records them as latest votes (`record_known_attestation_votes`), + /// so a candidate carrying that same body now scores zero on both axes, + /// while one carrying genuinely newer votes survives. + /// + /// Age is deliberately not the criterion. The merge that produces a + /// candidate takes seconds, so candidates routinely arrive a slot late and + /// a slot-old candidate may still hold the newest votes anyone has. What + /// matters is whether it still adds something, which is what this measures. + /// + /// Without this the ring never empties, and a permanently full ring has two + /// costs: the proposer re-scores dead entries every slot, and + /// `PROPOSAL_CANDIDATE_GRACE` never fires, since it waits only on an empty + /// buffer. + pub(crate) fn prune_scoreless( + &mut self, + head_state: &State, + latest_head_votes: &HashMap, + ) -> usize { + let validator_count = head_state.validators.len(); + let before = self.candidates.len(); + self.candidates.retain(|candidate| { + !count_body_voters( + head_state, + &candidate.body_proof.block_body, + validator_count, + latest_head_votes, + ) + .is_scoreless() + }); + before - self.candidates.len() + } + + /// Candidates newest first. + /// + /// Nothing is aged out by slot: see [`BodyProofBuffer::prune_scoreless`] + /// for why value, not age, is what decides. The ring bound keeps this + /// finite between prunes. + pub(crate) fn iter(&self) -> impl Iterator { + self.candidates.iter() + } + + pub(crate) fn len(&self) -> usize { + self.candidates.len() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use ethlambda_types::{ + attestation::{AggregatedAttestation, AggregationBits, AttestationData}, + block::{BlockHeader, MultiMessageAggregate}, + checkpoint::Checkpoint, + state::{JustificationValidators, JustifiedSlots, StateConfig, Validator}, + }; + use libssz_types::SszList; + + const NUM_VALIDATORS: usize = 4; + /// Slot of the head the candidates are packed on top of; the block under + /// construction is the next one. + const HEAD_SLOT: u64 = 1; + const BLOCK_SLOT: u64 = HEAD_SLOT + 1; + /// Round-robin proposer for [`BLOCK_SLOT`]; the state transition rejects + /// any other index. + const PROPOSER: u64 = BLOCK_SLOT % NUM_VALIDATORS as u64; + + fn genesis_root() -> H256 { + H256([1u8; 32]) + } + + /// The head block's root, as the state transition derives it: the hash of + /// `latest_block_header` once `process_slots` has filled in its state + /// root. Deriving it the same way the transition does is the only way a + /// fixture parent root matches. + fn head_root() -> H256 { + head_root_of(&head_state()) + } + + /// Like [`head_root`], but for a state that was modified from the plain + /// fixture (e.g. to mark a slot already justified). The header's + /// `state_root` embeds a hash of the whole state, so a different + /// `justified_slots` yields a different root; every test that mutates + /// `head_state()` must re-derive the parent root through here rather than + /// reusing [`head_root`]. + fn head_root_of(state: &State) -> H256 { + let mut state = state.clone(); + ethlambda_state_transition::process_slots(&mut state, BLOCK_SLOT) + .expect("advancing one slot works"); + state.latest_block_header.hash_tree_root() + } + + /// A chain of two blocks: genesis at slot 0 and the head at [`HEAD_SLOT`]. + /// `historical_block_hashes` covers `[0, HEAD_SLOT - 1]` — the header push + /// records the parent, never the block's own root — so the head root lands + /// there only once a block builds on it. + fn head_state() -> State { + State { + config: StateConfig { genesis_time: 1000 }, + slot: HEAD_SLOT, + latest_block_header: BlockHeader { + slot: HEAD_SLOT, + proposer_index: 0, + parent_root: genesis_root(), + state_root: H256::ZERO, + body_root: H256::ZERO, + }, + latest_justified: Checkpoint::default(), + latest_finalized: Checkpoint::default(), + historical_block_hashes: SszList::try_from(vec![genesis_root()]).unwrap(), + justified_slots: JustifiedSlots::new(), + validators: SszList::try_from( + (0..NUM_VALIDATORS) + .map(|i| Validator { + attestation_pubkey: [i as u8; 52], + proposal_pubkey: [i as u8; 52], + index: i as u64, + }) + .collect::>(), + ) + .unwrap(), + justifications_roots: Default::default(), + justifications_validators: JustificationValidators::new(), + } + } + + fn bits(indices: &[usize]) -> AggregationBits { + let max = indices.iter().copied().max().unwrap_or(0); + let mut bits = AggregationBits::with_length(max + 1).unwrap(); + for &i in indices { + bits.set(i, true).unwrap(); + } + bits + } + + /// A vote for the head, sourced at genesis: valid on top of + /// [`head_state`] and worth new voters. + fn head_vote(voters: &[usize]) -> AggregatedAttestation { + AggregatedAttestation { + aggregation_bits: bits(voters), + data: AttestationData { + slot: HEAD_SLOT, + head: Checkpoint { + root: head_root(), + slot: HEAD_SLOT, + }, + target: Checkpoint { + root: head_root(), + slot: HEAD_SLOT, + }, + source: Checkpoint { + root: genesis_root(), + slot: 0, + }, + }, + } + } + + /// A vote naming a head that is not on this chain, which the state + /// transition rejects. + fn off_chain_vote() -> AggregatedAttestation { + let mut attestation = head_vote(&[0]); + attestation.data.head.root = H256([9u8; 32]); + attestation.data.target.root = H256([9u8; 32]); + attestation + } + + fn candidate(attestations: Vec) -> BlockBodyProof { + BlockBodyProof { + block_body: BlockBody { + attestations: attestations.try_into().unwrap(), + }, + proof: MultiMessageAggregate::default(), + } + } + + fn choose(candidates: &BodyProofBuffer) -> ChosenBody { + choose_body( + &head_state(), + BLOCK_SLOT, + PROPOSER, + head_root(), + candidates, + &HashMap::new(), + ) + .expect("sealing an empty body always works") + } + + /// Fork choice's latest-vote map, holding `data` for each of `voters`. + fn latest_votes(voters: &[u64], data: &AttestationData) -> HashMap { + voters.iter().map(|v| (*v, data.clone())).collect() + } + + /// A state that already justified [`HEAD_SLOT`], its re-derived parent + /// root, and a vote targeting that settled slot. + fn already_justified_fixture() -> (State, H256, AggregatedAttestation) { + let mut state = head_state(); + let finalized_slot = state.latest_finalized.slot; + ethlambda_state_transition::justified_slots_ops::extend_to_slot( + &mut state.justified_slots, + finalized_slot, + HEAD_SLOT, + ); + ethlambda_state_transition::justified_slots_ops::set_justified( + &mut state.justified_slots, + finalized_slot, + HEAD_SLOT, + ); + let parent_root = head_root_of(&state); + + let vote = AggregatedAttestation { + aggregation_bits: bits(&[0, 1]), + data: AttestationData { + slot: HEAD_SLOT, + head: Checkpoint { + root: parent_root, + slot: HEAD_SLOT, + }, + target: Checkpoint { + root: parent_root, + slot: HEAD_SLOT, + }, + source: Checkpoint { + root: genesis_root(), + slot: 0, + }, + }, + }; + + (state, parent_root, vote) + } + + #[test] + fn choose_body_falls_back_to_an_empty_body() { + let chosen = choose(&BodyProofBuffer::default()); + + assert!(!chosen.adopted); + assert_eq!(chosen.block.body.attestations.len(), 0); + assert!( + chosen.attestation_proof.proof_bytes().is_empty(), + "an attestation-less block carries no aggregate" + ); + } + + #[test] + fn choose_body_adopts_a_candidate_that_adds_voters() { + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![head_vote(&[0, 1])])); + + let chosen = choose(&candidates); + + assert!(chosen.adopted); + assert_eq!(chosen.block.body.attestations.len(), 1); + } + + /// The state transition would carry a vote for an unknown root happily, so + /// this is the screen that keeps a body packed against another node's view + /// out of our block. + #[test] + fn choose_body_rejects_a_candidate_voting_off_chain() { + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![off_chain_vote()])); + + let chosen = choose(&candidates); + + assert!(!chosen.adopted); + } + + /// A body that adds no voters is worth no more than an empty one, so it + /// loses the tie rather than bloating the block. + #[test] + fn choose_body_ignores_a_candidate_with_no_new_voters() { + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![head_vote(&[])])); + + let chosen = choose(&candidates); + + assert!(!chosen.adopted); + } + + /// A body whose only vote targets a slot the state already justified must + /// score `new_voters == 0`: the state transition's `is_valid_vote` would + /// skip that same vote via `continue`, and the justified target's + /// `current_votes` entry is gone, so without that guard the whole + /// aggregation bitfield would be miscounted as new. + /// + /// Only the justification axis is pinned here. What those votes are still + /// worth as fork-choice weight is the next two tests' question. + #[test] + fn already_justified_target_scores_no_new_justification_voters() { + let (state, _parent_root, vote) = already_justified_fixture(); + let body = BlockBody { + attestations: vec![vote.clone()].try_into().unwrap(), + }; + + let voters = count_body_voters( + &state, + &body, + NUM_VALIDATORS, + &latest_votes(&[0, 1], &vote.data), + ); + + assert_eq!( + voters.new_voters, 0, + "a target the state already justified must not be credited as new" + ); + } + + /// The same body is still worth adopting while its votes are validators' + /// newest: the target has settled, but the head votes have not been seen. + /// This is the case that would otherwise leave the slot empty even though + /// the body carried the freshest fork-choice weight on the network. + #[test] + fn choose_body_adopts_an_already_justified_target_for_its_head_votes() { + let (state, parent_root, vote) = already_justified_fixture(); + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![vote])); + + let chosen = choose_body( + &state, + BLOCK_SLOT, + PROPOSER, + parent_root, + &candidates, + &HashMap::new(), + ) + .expect("sealing an empty body always works"); + + assert!( + chosen.adopted, + "a settled target does not make the head votes worthless" + ); + } + + /// Once fork choice already holds those very votes — which is what a block + /// import does — the body adds nothing on either axis and loses to empty. + #[test] + fn choose_body_ignores_a_candidate_stale_on_both_axes() { + let (state, parent_root, vote) = already_justified_fixture(); + let already_seen = latest_votes(&[0, 1], &vote.data); + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![vote])); + + let chosen = choose_body( + &state, + BLOCK_SLOT, + PROPOSER, + parent_root, + &candidates, + &already_seen, + ) + .expect("sealing an empty body always works"); + + assert!( + !chosen.adopted, + "a candidate adding neither voters nor head votes must not beat the empty body" + ); + } + + #[test] + fn prune_scoreless_drops_a_candidate_that_adds_nothing() { + let (state, _parent_root, vote) = already_justified_fixture(); + let already_seen = latest_votes(&[0, 1], &vote.data); + let mut buffer = BodyProofBuffer::default(); + buffer.push_local(candidate(vec![vote])); + + assert_eq!(buffer.prune_scoreless(&state, &already_seen), 1); + assert_eq!(buffer.len(), 0, "the ring must empty out, not stay full"); + } + + #[test] + fn prune_scoreless_keeps_a_candidate_whose_head_votes_are_new() { + let (state, _parent_root, vote) = already_justified_fixture(); + let mut buffer = BodyProofBuffer::default(); + buffer.push_local(candidate(vec![vote])); + + assert_eq!( + buffer.prune_scoreless(&state, &HashMap::new()), + 0, + "votes fork choice has not seen are still worth keeping" + ); + assert_eq!(buffer.len(), 1); + } + + /// The whole ring clears when every candidate has gone stale, which is what + /// lets `PROPOSAL_CANDIDATE_GRACE` fire again. + #[test] + fn prune_scoreless_empties_a_full_ring_of_stale_candidates() { + let (state, _parent_root, vote) = already_justified_fixture(); + let already_seen = latest_votes(&[0, 1], &vote.data); + let mut buffer = BodyProofBuffer::default(); + for _ in 0..MAX_BODY_PROOF_CANDIDATES { + buffer.push_local(candidate(vec![vote.clone()])); + } + assert_eq!(buffer.len(), MAX_BODY_PROOF_CANDIDATES); + + let dropped = buffer.prune_scoreless(&state, &already_seen); + + assert_eq!(dropped, MAX_BODY_PROOF_CANDIDATES); + assert_eq!(buffer.len(), 0); + } + + /// A target slot outside the projection's tracked `justified_slots` + /// window (the common case for anything between the head and the + /// candidate block) has no bit to read at all: `is_slot_justified` + /// returns `JustifiedSlotOutOfRange` there, which the deliberate + /// `.unwrap_or(false)` in `target_already_justified` must read as "not + /// justified" rather than propagate or treat as justified. Getting this + /// wrong would make every fresh target look stale. + #[test] + fn target_already_justified_treats_an_untracked_slot_as_not_justified() { + let state = head_state(); + let projected = block_builder::ProjectedState::from_head_state(&state); + + let att_data = AttestationData { + slot: HEAD_SLOT, + head: Checkpoint { + root: head_root(), + slot: HEAD_SLOT, + }, + target: Checkpoint { + root: head_root(), + slot: HEAD_SLOT, + }, + source: Checkpoint { + root: genesis_root(), + slot: 0, + }, + }; + + assert!( + !projected.target_already_justified(&att_data), + "an untracked target slot must stay eligible, not read as justified" + ); + } + + #[test] + fn choose_body_prefers_the_candidate_with_more_new_voters() { + let mut candidates = BodyProofBuffer::default(); + candidates.push_local(candidate(vec![head_vote(&[0])])); + candidates.push_local(candidate(vec![head_vote(&[0, 1, 2])])); + + let chosen = choose(&candidates); + + assert!(chosen.adopted); + let adopted = chosen + .block + .body + .attestations + .iter() + .next() + .expect("the adopted body carries its attestation"); + assert_eq!( + validator_indices(&adopted.aggregation_bits).count(), + 3, + "the wider candidate wins" + ); + } + + #[test] + fn buffer_keeps_the_newest_candidates() { + let mut buffer = BodyProofBuffer::default(); + for _ in 0..MAX_BODY_PROOF_CANDIDATES { + buffer.push_gossip(candidate(Vec::new())); + } + buffer.push_local(candidate(Vec::new())); + + assert_eq!(buffer.len(), MAX_BODY_PROOF_CANDIDATES); + assert!( + buffer.iter().next().expect("non-empty").verified, + "the newest candidate is at the front" + ); + assert_eq!( + buffer.iter().filter(|c| c.verified).count(), + 1, + "only the locally built candidate counts as verified" + ); + } +} diff --git a/crates/blockchain/src/events.rs b/crates/blockchain/src/events.rs index 5977bab6..e0fd2b02 100644 --- a/crates/blockchain/src/events.rs +++ b/crates/blockchain/src/events.rs @@ -324,7 +324,7 @@ mod tests { use ethlambda_storage::{ForkCheckpoints, backend::InMemoryBackend}; use ethlambda_types::constants::DEFAULT_MILLISECONDS_PER_SLOT; use ethlambda_types::{ - block::{Block, BlockBody, MultiMessageAggregate, SignedBlock}, + block::{Block, BlockBody, BlockProof, SignedBlock}, state::State, }; use std::sync::Arc; @@ -460,7 +460,7 @@ mod tests { state_root, body: BlockBody::default(), }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; store .insert_signed_block(root, signed_block) diff --git a/crates/blockchain/src/lib.rs b/crates/blockchain/src/lib.rs index e10ba854..1dcebd22 100644 --- a/crates/blockchain/src/lib.rs +++ b/crates/blockchain/src/lib.rs @@ -1,7 +1,6 @@ use std::collections::{HashMap, HashSet, VecDeque}; use std::time::{Duration, Instant, SystemTime}; -use ethlambda_crypto::signature::{ValidatorPublicKey, ValidatorSignature}; use ethlambda_network_api::{BlockChainToP2PRef, BlockSource, InitP2P}; use ethlambda_state_transition::is_proposer; use ethlambda_storage::{ALL_TABLES, Store}; @@ -12,12 +11,15 @@ use ethlambda_types::{ AggregationBits, AttestationData, HashedAttestationData, SignedAggregatedAttestation, SignedAttestation, bits_is_subset, validator_indices, }, - block::{ByteList512KiB, MultiMessageAggregate, SignedBlock}, + block::{BlockBodyProof, BlockProof, SignedBlock}, chain_config::ChainConfig, primitives::{H256, HashTreeRoot as _}, }; -use crate::aggregation::{AggregateProduced, AggregationWorker, PauseReason, WorkerConfig}; +use crate::aggregation::{ + AggregateProduced, AggregationWorker, BodyProofProduced, PauseReason, WorkerConfig, +}; +use crate::body_proof::{AssembleProposal, BodyProofBuffer}; use crate::key_manager::ValidatorKeyPair; use crate::sync_status::SyncStatusTracker; use spawned_concurrency::actor; @@ -34,6 +36,7 @@ pub use events::{ChainEvent, EventBus, Topic, UnknownTopic}; pub mod aggregation; pub mod block_builder; +pub(crate) mod body_proof; pub(crate) mod coverage; pub mod events; pub(crate) mod fork_choice_tree; @@ -73,6 +76,18 @@ pub struct BlockChainConfig { pub use ethlambda_types::block::MAX_ATTESTATIONS_DATA; pub use ethlambda_types::constants::{DEFAULT_MILLISECONDS_PER_SLOT, INTERVALS_PER_SLOT}; pub use sync_status::SyncStatusController; +/// How long the proposer waits at the slot boundary for a candidate body proof +/// when it has none yet. +/// +/// The merge that produces a candidate starts at the previous head-update +/// interval and routinely runs a couple of hundred milliseconds past the slot +/// boundary, so a proposer that assembled the instant its tick fired would +/// keep finding an empty buffer and publishing empty blocks. Waiting is much +/// cheaper than that: attestations are not due until interval 1, and a block +/// with the slot's votes in it is worth a few hundred milliseconds of +/// propagation. +const PROPOSAL_CANDIDATE_GRACE: Duration = Duration::from_millis(400); + /// Future-slot tolerance for gossip attestations, expressed in intervals. /// /// Bounds the clock skew the time check is willing to absorb when admitting a @@ -261,6 +276,8 @@ impl BlockChain { aggregator, pending_block_parents: HashMap::new(), aggregation_worker: None, + body_proof_candidates: BodyProofBuffer::default(), + pending_body_proofs: Vec::new(), pending_aggregates: HashMap::new(), last_tick_instant: None, attestation_committee_count, @@ -321,6 +338,17 @@ pub struct BlockChainServer { /// treat `None` as "not up yet". aggregation_worker: Option, + /// Candidate bodies the proposer may adopt for the upcoming slot: what our + /// own worker built plus what arrived on gossip. Pruned after each block + /// import of whatever that block made worthless, rather than cleared on a + /// tick: see [`body_proof::BodyProofBuffer::prune_scoreless`]. + body_proof_candidates: BodyProofBuffer, + + /// Candidate body proofs the worker produced and we have not gossiped yet. + /// Published during the head-update interval, the interval they are built + /// in, so the next slot's proposer sees them in time. + pending_body_proofs: Vec, + /// Aggregates the worker produced and not yet gossiped, keyed by /// attestation data root. The worker stores each one before announcing it, /// so the pool and its own next selection round already account for it; @@ -469,19 +497,46 @@ impl BlockChainServer { // at tick time), so it doubles as the wall-clock slot for the gate. pre_tick.diff_and_emit(&self.store, &self.events, slot); - // Per-interval duties for this tick. Intervals 0 (block publish) and 3 - // (safe-target update) are driven inside `store::on_tick` above, so they - // carry only a note below. + // Per-interval duties for this tick. Interval 3 (safe-target update) is + // driven inside `store::on_tick` above, so it carries only a note below. match interval { // ==== interval 0 ==== // - // No actor work at interval 0. The block is published here - // conceptually (at the slot boundary), but the build+publish code - // path runs at interval 4 of the previous slot — where it also - // advances the store to this slot's interval 0 before building (see - // `propose_block`). The real interval-0 tick is then skipped by the - // idempotency guard above, since the store clock is already here. - SlotInterval::BlockPublication => {} + // Assemble and publish our block, if we are this slot's proposer. + // + // Back at the slot boundary the protocol puts it, rather than + // prebuilt at the previous interval 4: the proposer no longer packs + // a body, it adopts one of the candidate body proofs gossiped + // during that interval, and those keep arriving until the boundary. + // Assembly is a state transition, a verification and a signature — + // no prover work — so it no longer needs an interval of headroom. + SlotInterval::BlockPublication => { + let proposer = (slot > 0) + .then(|| self.get_our_proposer(slot)) + .flatten() + .filter(|_| self.sync_status.duties_allowed()); + + if let Some(validator_id) = proposer { + if self.body_proof_candidates.len() > 0 { + self.assemble_proposal(slot, validator_id).await; + } else { + // Nothing to propose yet: the merge that produces a + // candidate spans the boundary, so this slot's batch is + // most likely still in flight. Come back for it rather + // than settling for an empty block. + info!( + %slot, + grace_ms = PROPOSAL_CANDIDATE_GRACE.as_millis() as u64, + "No candidate body proof yet; waiting before assembling" + ); + send_after( + PROPOSAL_CANDIDATE_GRACE, + _ctx.clone(), + AssembleProposal { slot, validator_id }, + ); + } + } + } // ==== interval 1 ==== // @@ -548,30 +603,20 @@ impl BlockChainServer { // ==== interval 4 ==== // - // Build and publish the NEXT slot's block here, one interval early, - // so the heavy leanVM work happens during this otherwise-idle - // interval. `propose_block` blocks the actor for the build and aligns - // publication to the slot boundary. Doing the whole proposal here — - // rather than stashing it for the interval-0 tick — keeps it robust: - // `on_tick` skips the interval-0 tick whenever this build overruns - // its interval. + // The candidate body proofs for the next slot are built here, on the + // worker, and published as they arrive (see the `BodyProofProduced` + // handler). This is the interval whose votes the next block carries: + // the store's promote ran just above, so the pool the worker packs + // from is the one the block will be judged against. SlotInterval::EndOfSlot => { - let next_slot = slot + 1; - let next_proposer = self - .get_our_proposer(next_slot) - .filter(|_| self.sync_status.duties_allowed()); - - if let Some(validator_id) = next_proposer { - // Park the aggregation worker for the build: both run - // leanVM proofs, and the block is the one with a deadline. - // The guard clears its own reason on the way out, including - // on `propose_block`'s early returns. - let _pause = self - .aggregation_worker - .as_ref() - .map(|worker| worker.pause(PauseReason::BlockBuild)); - self.propose_block(next_slot, validator_id).await; - } + // The buffer is not cleared here: our own worker's candidate can + // land either side of this tick, so a clear would race the batch + // it is making room for. Staleness is judged per read instead + // (`BodyProofBuffer::iter_fresh`). + // + // Anything the worker produced too late for its own interval + // goes out now, having missed the proposer it was built for. + self.publish_pending_body_proofs(slot); } } @@ -726,6 +771,55 @@ impl BlockChainServer { info!(%slot, count, "Published buffered aggregates"); } + /// Pause the aggregation worker and assemble this slot's block. + /// + /// Verifying a candidate's aggregate is leanVM work too, and the block is + /// the one with a deadline, so the worker sits out the assembly. The guard + /// clears its own reason on the way out, including on `propose_block`'s + /// early returns. + async fn assemble_proposal(&mut self, slot: u64, validator_id: u64) { + let _pause = self + .aggregation_worker + .as_ref() + .map(|worker| worker.pause(PauseReason::BlockBuild)); + self.propose_block(slot, validator_id).await; + } + + /// Gossip the candidate body proofs the worker produced, then clear the + /// buffer. + /// + /// Unlike an aggregate, a body proof has one slot in which it is worth + /// anything: the proposer it is meant for assembles before the next slot + /// opens. So an undeliverable one is dropped rather than held. + fn publish_pending_body_proofs(&mut self, slot: u64) { + let pending = std::mem::take(&mut self.pending_body_proofs); + if pending.is_empty() { + return; + } + let count = pending.len(); + + let Some(p2p) = self.p2p.as_ref() else { + debug!(%slot, count, "Dropping candidate body proofs: no P2P yet"); + return; + }; + + for body_proof in pending { + let _ = p2p + .publish_block_body_proof(body_proof) + .inspect_err(|err| error!(%err, "Failed to publish block body proof")); + } + info!(%slot, count, "Published candidate body proofs"); + } + + /// The slot the wall clock is in, which is what stamps a candidate body + /// proof's arrival: the store's clock only advances on ticks, and a + /// candidate can arrive between two of them. + fn wall_clock_slot(&self) -> u64 { + let time_config = *self.store.config(); + unix_now_ms().saturating_sub(time_config.genesis_time_ms()) + / time_config.milliseconds_per_slot + } + /// Returns the validator ID if any of our validators is the proposer for this slot. fn get_our_proposer(&self, slot: u64) -> Option { let head_state = self.store.head_state(); @@ -784,19 +878,21 @@ impl BlockChainServer { } } - /// Build the target slot's block and publish it, one interval early. + /// Assemble this slot's block from the candidate body proofs on hand and + /// publish it. /// - /// Runs at the previous slot's interval 4, blocking the actor for the build - /// (the expensive part is the leanVM single-message → multi-message - /// aggregate merge). It first - /// advances the store to the target slot's interval 0 (accepting - /// attestations) so the block is built on exactly the interval-0 state a - /// non-prebuilding proposer would see, then builds and publishes — aligned - /// to the slot boundary: if the build finishes before the slot opens we wait - /// out the remainder so the block is not published early; if it overran (the - /// common case under load) we publish at once. The whole proposal is - /// self-contained here, so it never depends on the interval-0 tick — which - /// `handle_tick` skips whenever this build overruns its interval. + /// Runs at the slot's own interval-0 tick. The proposer packs no body + /// itself: it adopts the most valuable candidate + /// (`body_proof::choose_body`), or signs an empty block when none is worth + /// more than one. What is left costs a state transition per candidate, one + /// aggregate verification and one signature — no prover work, and none at + /// all for an empty body — which is why this no longer needs to be + /// prebuilt an interval early. + /// + /// It re-runs the store's advance to this slot's interval 0 (accepting + /// attestations) so the candidates are judged against exactly the state the + /// block is built on, and so a tick that arrived late still proposes on the + /// right state. Both steps are idempotent. async fn propose_block(&mut self, slot: u64, validator_id: u64) { info!(%slot, %validator_id, "We are the proposer for this slot"); @@ -804,44 +900,36 @@ impl BlockChainServer { let slot_start_ms = time_config.genesis_time_ms() + SlotInterval::BlockPublication.to_ms_since_genesis(slot, &time_config); - // Build the block. `produce_block_with_signatures` advances the store to - // this slot's interval 0 (accepting attestations) before building — one - // interval ahead of the interval-4 tick we are running in — so the block - // is built on the interval-0 state rather than the previous slot's end - // state. Building early is safe because we publish below (nothing is - // stashed for a later tick), and the real interval-0 tick is then skipped - // by the idempotency guard in `on_tick`, since the store clock is already - // here. - // - // That interval-0 catch-up can move head/justified/finalized (it is the - // same attestation-acceptance step a non-proposing node runs at its - // interval-0 tick). Snapshot around the build so those moves surface as - // chain events here, matching an observer node; otherwise they would - // land outside every snapshot window and be silently absorbed into the - // later block-import diff's baseline. + // The interval-0 catch-up inside `produce_block_from_candidates` can + // move head/justified/finalized (it is the same attestation-acceptance + // step a non-proposing node runs at its interval-0 tick). Snapshot + // around it so those moves surface as chain events here, matching an + // observer node; otherwise they would land outside every snapshot + // window and be silently absorbed into the later block-import diff's + // baseline. let pre_build = ChainEventSnapshot::capture(&self.store); let timing = metrics::time_block_building(); - let build_result = store::produce_block_with_signatures( + let chosen = store::produce_block_from_candidates( &mut self.store, slot, validator_id, - self.proposer_config, + &self.body_proof_candidates, ) - .inspect_err(|err| error!(%slot, %validator_id, %err, "Failed to build block")); - - // `get_proposal_head` advances the store (interval-0 catch-up) inside - // `produce_block_with_signatures` *before* the build can fail, so emit - // the resulting head/checkpoint moves on both paths — a build failure - // must not strand a real finalization move outside every snapshot - // window. Ordered before the freshly built block's own import (which - // emits its `block` + head/checkpoint events). The catch-up advanced - // the store to `slot`'s interval 0, so the head-recency gate uses `slot`. + .inspect_err(|err| error!(%slot, %validator_id, %err, "Failed to assemble block")); + + // The catch-up runs before the assembly can fail, so emit the resulting + // head/checkpoint moves on both paths — a failure must not strand a + // real finalization move outside every snapshot window. Ordered before + // the freshly built block's own import (which emits its `block` + + // head/checkpoint events). The catch-up advanced the store to `slot`'s + // interval 0, so the head-recency gate uses `slot`. pre_build.diff_and_emit(&self.store, &self.events, slot); - let Ok((block, single_message_aggregates, _post_checkpoints)) = build_result else { + let Ok(chosen) = chosen else { metrics::inc_block_building_failures(); return; }; + let block = chosen.block; coverage::emit_proposal_coverage( &self.store, @@ -849,7 +937,9 @@ impl BlockChainServer { block.body.attestations.iter(), ); - // Sign the block root with the proposal key + // Sign the block root with the proposal key. Exactly once per slot: the + // XMSS key is one-time, which is why an adopted candidate's aggregate + // is verified before we get here rather than by trying the import. let block_root = block.hash_tree_root(); let Ok(proposer_signature) = self .key_manager @@ -860,118 +950,31 @@ impl BlockChainServer { return; }; - // Wrap the proposer's raw XMSS signature into a singleton - // single-message aggregate SNARK, then merge it with every attestation - // single-message aggregate into the single multi-message aggregate. - let head_state = self.store.head_state(); - let validators = &head_state.validators; - let Some(proposer_validator) = validators.get(validator_id as usize) else { - error!(%slot, %validator_id, "Proposer index out of range when assembling block"); - metrics::inc_block_building_failures(); - return; - }; - - // Decode the proposer's proposal pubkey once and reuse it both for the - // singleton single-message aggregate wrap and for the multi-message - // aggregate merge inputs. - let Ok(proposer_pubkey) = ValidatorPublicKey::from_bytes( - &proposer_validator.proposal_pubkey, - ) - .inspect_err( - |err| error!(%slot, %validator_id, %err, "Failed to decode proposer proposal pubkey"), - ) else { - metrics::inc_block_building_failures(); - return; - }; - - let Ok(proposer_validator_signature) = - ValidatorSignature::from_bytes(&proposer_signature).inspect_err(|err| { - error!(%slot, %validator_id, %err, "Failed to decode proposer signature bytes") - }) - else { - metrics::inc_block_building_failures(); - return; - }; - let Ok(proposer_proof_bytes) = ethlambda_crypto::aggregate_signatures( - vec![proposer_pubkey.clone()], - vec![proposer_validator_signature], - &block_root, - slot as u32, - ) - .inspect_err( - |err| error!(%slot, %validator_id, %err, "Failed to wrap proposer signature as single-message aggregate"), - ) else { - metrics::inc_block_building_failures(); - return; - }; - - let mut merge_inputs: Vec<(Vec, ByteList512KiB)> = - Vec::with_capacity(single_message_aggregates.len() + 1); - let mut resolve_failed = false; - for sma in &single_message_aggregates { - let mut pubkeys = Vec::new(); - for vid in sma.participant_indices() { - let Some(validator) = validators.get(vid as usize) else { - error!(%slot, %validator_id, vid, "Participant out of range while resolving pubkeys"); - resolve_failed = true; - break; - }; - match ValidatorPublicKey::from_bytes(&validator.attestation_pubkey) { - Ok(pk) => pubkeys.push(pk), - Err(err) => { - error!(%slot, %validator_id, vid, %err, "Failed to decode attestation pubkey"); - resolve_failed = true; - break; - } - } - } - if resolve_failed { - break; - } - merge_inputs.push((pubkeys, sma.proof.clone())); - } - if resolve_failed { - metrics::inc_block_building_failures(); - return; - } - merge_inputs.push((vec![proposer_pubkey], proposer_proof_bytes)); - - // Merge yields raw lean-multisig type-2 bytes. Per-component - // participants are rederived at verify time from - // `block.body.attestations[i].aggregation_bits` plus - // `block.proposer_index`, so nothing else needs persisting. - let merged_bytes = match ethlambda_crypto::merge_type_1s_into_type_2(merge_inputs) { - Ok(bytes) => bytes, - Err(err) => { - error!(%slot, %validator_id, %err, "Failed to merge Type-1s into Type-2"); - metrics::inc_block_building_failures(); - return; - } - }; - let proof = match MultiMessageAggregate::from_bytes(merged_bytes.iter().as_slice()) { - Ok(p) => p, - Err(err) => { - error!(%slot, %validator_id, %err, "Failed to build multi-message aggregate"); - metrics::inc_block_building_failures(); - return; - } - }; + // The proposer signature is carried raw, outside the aggregate, so + // assembling the envelope needs no prover work — for an empty body, + // none at all. let signed_block = SignedBlock { message: block, - proof, + proof: BlockProof::new(proposer_signature, chosen.attestation_proof), }; - // Stop timing here: the build is done, and the alignment wait below must - // not count toward the block-building metric. + // Stop timing here: the assembly is done, and the alignment wait below + // must not count toward the block-building metric. drop(timing); - info!(%slot, %validator_id, "Finished building block"); + info!( + %slot, + %validator_id, + adopted_body_proof = chosen.adopted, + attestation_count = signed_block.message.body.attestations.len(), + "Finished assembling block" + ); let now_ms = unix_now_ms(); - // Align publication to the slot boundary. If the build finished before - // the slot opened, wait out the remainder so the block is not published - // early; if it overran, publish immediately. + // Never publish ahead of the slot boundary. Assembly runs at the + // interval-0 tick, so this only bites when the tick fired early against + // a wall clock that has since drifted back. if now_ms < slot_start_ms { let wait_ms = slot_start_ms.saturating_sub(now_ms); tokio::time::sleep(Duration::from_millis(wait_ms)).await; @@ -1029,10 +1032,7 @@ impl BlockChainServer { } // Block import has no ready-made "now" slot like `on_tick`'s, so // compute the wall-clock slot fresh for the head-recency gate. - let time_config = *self.store.config(); - let wall_clock_slot = unix_now_ms().saturating_sub(time_config.genesis_time_ms()) - / time_config.milliseconds_per_slot; - pre_import.diff_and_emit(&self.store, &self.events, wall_clock_slot); + pre_import.diff_and_emit(&self.store, &self.events, self.wall_clock_slot()); metrics::update_head_slot(self.store.head_slot()); let latest_justified_slot = self @@ -1049,12 +1049,51 @@ impl BlockChainServer { metrics::update_latest_finalized_slot(latest_finalized_slot); metrics::update_validators_count(self.key_manager.validator_ids().len() as u64); + self.prune_scoreless_body_proofs(); + for table in ALL_TABLES { metrics::update_table_bytes(table.name(), self.store.estimate_table_bytes(table)); } Ok(()) } + /// Drop buffered candidate bodies that the block just imported made + /// worthless. + /// + /// Sits here rather than in `on_block` because the proposer's own block + /// reaches the store through `process_and_publish_block`, which never + /// enters the `on_block` cascade; this is the one point every successful + /// import passes through. + /// + /// Skipped while syncing: a node that is behind proposes nothing, so its + /// candidate buffer is not worth maintaining, and this keeps the scan off + /// the per-block backfill path. + fn prune_scoreless_body_proofs(&mut self) { + if !self.sync_status.duties_allowed() || self.body_proof_candidates.len() == 0 { + return; + } + + let head_root = self.store.head().expect("head read works"); + let Ok(Some(head_state)) = self.store.get_state(&head_root) else { + return; + }; + // Must be the same baseline selection uses. Against the seen-votes + // map every candidate scores zero on both axes, so this would drop the + // whole ring on precisely the slots the head-vote axis exists to serve. + let latest_head_votes = self.store.extract_on_chain_votes(); + + let dropped = self + .body_proof_candidates + .prune_scoreless(&head_state, &latest_head_votes); + if dropped > 0 { + info!( + dropped, + remaining = self.body_proof_candidates.len(), + "Pruned candidate body proofs that add nothing to our state" + ); + } + } + /// Process a newly received block. fn on_block(&mut self, signed_block: SignedBlock) { let mut queue = VecDeque::new(); @@ -1452,6 +1491,7 @@ impl BlockChainServer { WorkerConfig { attestation_committee_count: self.attestation_committee_count, subscribed_subnets: self.subscribed_subnets.clone(), + proposer_config: self.proposer_config, }, )); } @@ -1471,7 +1511,7 @@ impl BlockChainServer { // --- Manual Handler impls for network-api messages --- use ethlambda_network_api::p2p_to_block_chain::{ - NewAggregatedAttestation, NewAttestation, NewBlock, + NewAggregatedAttestation, NewAttestation, NewBlock, NewBlockBodyProof, }; impl Handler for BlockChainServer { @@ -1519,6 +1559,51 @@ impl Handler for BlockChainServer { } } +impl Handler for BlockChainServer { + async fn handle(&mut self, msg: AssembleProposal, _ctx: &Context) { + self.assemble_proposal(msg.slot, msg.validator_id).await; + } +} + +impl Handler for BlockChainServer { + async fn handle(&mut self, msg: BodyProofProduced, _ctx: &Context) { + let attestation_count = msg.body_proof.block_body.attestations.len(); + info!( + slot = msg.slot, + attestation_count, + elapsed = ?msg.elapsed, + "Built a candidate body proof" + ); + + // Our own candidate counts as verified: we merged the aggregate, so the + // proposer path can skip the Type-2 check on it. Buffered separately + // from the publication copy, which the gossip call consumes. + self.body_proof_candidates + .push_local(msg.body_proof.clone()); + self.pending_body_proofs.push(msg.body_proof); + + // Publish as soon as it exists. The merge routinely runs seconds past + // the head-update interval it started in, so holding it for the next + // head-update tick would waste it entirely. + self.publish_pending_body_proofs(self.wall_clock_slot()); + } +} + +impl Handler for BlockChainServer { + async fn handle(&mut self, msg: NewBlockBodyProof, _ctx: &Context) { + let attestation_count = msg.body_proof.block_body.attestations.len(); + // Not verified here: a Type-2 check costs about as much as verifying a + // block, and only the slot's proposer ever needs the answer. It pays + // for the one candidate it decides to adopt. + self.body_proof_candidates.push_gossip(msg.body_proof); + trace!( + attestation_count, + candidates = self.body_proof_candidates.len(), + "Buffered a gossiped block body proof" + ); + } +} + impl Handler for BlockChainServer { async fn handle(&mut self, msg: NewAggregatedAttestation, _ctx: &Context) { let arrival_ms = unix_now_ms(); diff --git a/crates/blockchain/src/metrics.rs b/crates/blockchain/src/metrics.rs index 31e533f3..38a693f3 100644 --- a/crates/blockchain/src/metrics.rs +++ b/crates/blockchain/src/metrics.rs @@ -388,6 +388,46 @@ static LEAN_AGGREGATED_PROOF_SIZE_BYTES: std::sync::LazyLock = .unwrap() }); +static LEAN_BLOCK_BODY_PROOF_BUILDING_TIME_SECONDS: std::sync::LazyLock = + std::sync::LazyLock::new(|| { + register_histogram!( + "lean_block_body_proof_building_time_seconds", + "Time taken to build a candidate block body proof", + vec![0.1, 0.25, 0.5, 0.75, 1.0, 1.5, 2.0, 3.0, 4.0] + ) + .unwrap() + }); + +static LEAN_BLOCK_BODY_PROOF_CANDIDATES: std::sync::LazyLock = + std::sync::LazyLock::new(|| { + register_histogram!( + "lean_block_body_proof_candidates", + "Candidate block body proofs a proposer had to choose from", + vec![0.0, 1.0, 2.0, 3.0, 4.0, 6.0, 8.0] + ) + .unwrap() + }); + +static LEAN_BLOCK_BODY_SOURCE_TOTAL: std::sync::LazyLock = + std::sync::LazyLock::new(|| { + register_int_counter_vec!( + "lean_block_body_source_total", + "Where a proposed block's body came from, by source", + &["source"] + ) + .unwrap() + }); + +static LEAN_BLOCK_BODY_PROOF_REJECTED_TOTAL: std::sync::LazyLock = + std::sync::LazyLock::new(|| { + register_int_counter_vec!( + "lean_block_body_proof_rejected_total", + "Candidate block body proofs a proposer rejected, by reason", + &["reason"] + ) + .unwrap() + }); + static LEAN_COMMITTEE_SIGNATURES_AGGREGATION_TIME_SECONDS: std::sync::LazyLock = std::sync::LazyLock::new(|| { register_histogram!( @@ -758,6 +798,17 @@ const AGGREGATOR_SKIP_REASONS: &[&str] = &[ "other", ]; +/// Label values for `lean_block_body_source_total`: a proposer either adopted +/// a gossiped candidate body or fell back to an empty one. Both are seeded at +/// zero so a dashboard can read the ratio from the first block. +const BLOCK_BODY_SOURCES: &[&str] = &["body_proof", "empty"]; + +/// Label values for `lean_block_body_proof_rejected_total`: a candidate +/// carrying a vote this node cannot place on the chain the block would extend, +/// one whose attestations its own state transition rejected, and one whose +/// aggregate failed verification. +const BODY_PROOF_REJECT_REASONS: &[&str] = &["off_chain_vote", "state_transition", "verification"]; + static LEAN_AGGREGATOR_SKIPPED_TOTAL: std::sync::LazyLock = std::sync::LazyLock::new(|| { register_int_counter_vec!( @@ -868,6 +919,16 @@ pub fn init() { for &reason in AGGREGATOR_SKIP_REASONS { LEAN_AGGREGATOR_SKIPPED_TOTAL.with_label_values(&[reason]); } + // Block body proofs: both body sources and both reject reasons are seeded + // so a dashboard can read the adoption ratio from the first block. + std::sync::LazyLock::force(&LEAN_BLOCK_BODY_PROOF_BUILDING_TIME_SECONDS); + std::sync::LazyLock::force(&LEAN_BLOCK_BODY_PROOF_CANDIDATES); + for &source in BLOCK_BODY_SOURCES { + LEAN_BLOCK_BODY_SOURCE_TOTAL.with_label_values(&[source]); + } + for &reason in BODY_PROOF_REJECT_REASONS { + LEAN_BLOCK_BODY_PROOF_REJECTED_TOTAL.with_label_values(&[reason]); + } } // --- Public API --- @@ -1010,6 +1071,40 @@ pub fn observe_committee_signatures_aggregation(elapsed: std::time::Duration) { LEAN_COMMITTEE_SIGNATURES_AGGREGATION_TIME_SECONDS.observe(elapsed.as_secs_f64()); } +/// Observe how long the worker took to build a candidate body proof: the +/// merge of the body's attestation Type-1s into one Type-2. +pub fn observe_body_proof_building(elapsed: Duration) { + LEAN_BLOCK_BODY_PROOF_BUILDING_TIME_SECONDS.observe(elapsed.as_secs_f64()); +} + +/// Observe how many candidate body proofs a proposer had to choose from. +/// A zero reading means the block could only be empty. +pub fn observe_body_proof_candidates(count: usize) { + LEAN_BLOCK_BODY_PROOF_CANDIDATES.observe(count as f64); +} + +/// A proposed block's body came from a candidate body proof. +pub fn inc_block_body_from_proof() { + LEAN_BLOCK_BODY_SOURCE_TOTAL + .with_label_values(&["body_proof"]) + .inc(); +} + +/// A proposed block carried an empty body: no candidate was usable. +pub fn inc_block_body_empty() { + LEAN_BLOCK_BODY_SOURCE_TOTAL + .with_label_values(&["empty"]) + .inc(); +} + +/// A candidate body proof a proposer rejected, by reason (see +/// [`BODY_PROOF_REJECT_REASONS`]). +pub fn inc_body_proof_rejected(reason: &str) { + LEAN_BLOCK_BODY_PROOF_REJECTED_TOTAL + .with_label_values(&[reason]) + .inc(); +} + /// One vote-aggregation interval passed with this node holding no aggregation /// duty. Bookkeeping label that lets dashboards separate "no duty" from /// genuine misses. diff --git a/crates/blockchain/src/reaggregate.rs b/crates/blockchain/src/reaggregate.rs index 13b433c5..fdcda4c0 100644 --- a/crates/blockchain/src/reaggregate.rs +++ b/crates/blockchain/src/reaggregate.rs @@ -15,7 +15,11 @@ //! 1. Only deconstructing when the chain is in sync — backfilling nodes //! must not flood gossip with rederived aggregates. //! 2. Skipping attestations whose target is at or behind the store's -//! justified checkpoint — they carry no fork-choice value. +//! justified checkpoint. Note this is NOT because such a vote is worthless: +//! selection deliberately packs one for the LMD-GHOST weight it carries. +//! It is because the vote is already ON CHAIN, in the very block being +//! imported, so splitting it back into the pool would let it be repacked +//! indefinitely while paying a SNARK for each round. //! 3. Skipping attestations whose participants are already a subset of the //! local union for that data — nothing to recover. //! 4. Capping the number of splits per imported block at @@ -71,11 +75,12 @@ pub fn reaggregate_from_block( let validators = &parent_state.validators; let num_validators = validators.len() as u64; - // Per-component pubkeys: one entry per body attestation in order, then - // the proposer entry. Layout is invariant per block, so it's resolved - // once and reused for every split call below. + // Per-component pubkeys: one entry per body attestation in order. The + // attestation aggregate no longer carries a proposer component (the + // proposer signature lives outside it), so the layout is attestations + // only. Resolved once and reused for every split call below. let mut pubkeys_per_component: Vec> = - Vec::with_capacity(attestations.len() + 1); + Vec::with_capacity(attestations.len()); for att in &attestations { let mut pubkeys = Vec::new(); for vid in validator_indices(&att.aggregation_bits) { @@ -93,15 +98,6 @@ pub fn reaggregate_from_block( } pubkeys_per_component.push(pubkeys); } - if block.proposer_index >= num_validators { - return Vec::new(); - } - let Ok(proposer_pubkey) = - ValidatorPublicKey::from_bytes(&validators[block.proposer_index as usize].proposal_pubkey) - else { - return Vec::new(); - }; - pubkeys_per_component.push(vec![proposer_pubkey]); let candidates = select_candidates(store, &attestations); if candidates.is_empty() { @@ -123,8 +119,8 @@ pub fn reaggregate_from_block( }; // Step 1: SNARK-split this attestation's component out of the block's - // merged multi-message aggregate proof. - let merged_bytes = signed_block.proof.proof_bytes(); + // attestation multi-message aggregate proof. + let merged_bytes = signed_block.proof.attestation_proof.proof_bytes(); let split_bytes = match ethlambda_crypto::split_type_2_by_message( merged_bytes, pubkeys_per_component.clone(), diff --git a/crates/blockchain/src/spec_test_runner.rs b/crates/blockchain/src/spec_test_runner.rs index cc2bebc2..43282722 100644 --- a/crates/blockchain/src/spec_test_runner.rs +++ b/crates/blockchain/src/spec_test_runner.rs @@ -80,6 +80,14 @@ pub fn rejection_reason(err: &StoreError) -> Option { StoreError::AttestationTooFarInFuture { .. } => RejectionReason::AttestationTooFarInFuture, StoreError::AggregateVerificationFailed(_) => RejectionReason::InvalidSignature, StoreError::BlockProofVerificationFailed(_) => RejectionReason::InvalidBlockProof, + // The proposer signature and the attestation aggregate are both + // components of the block proof, so a failure in either is the same + // spec rejection even though we carry them as separate wire fields. An + // attestation-free block carries no aggregate at all, so stray proof + // bytes are a malformed block proof rather than a distinct reason. + StoreError::ProposerSignatureDecodingFailed + | StoreError::ProposerSignatureVerificationFailed + | StoreError::UnexpectedAttestationProof => RejectionReason::InvalidBlockProof, StoreError::EmptyAggregationBits => RejectionReason::EmptyAggregationBits, StoreError::NotProposer { .. } => RejectionReason::WrongProposer, StoreError::DuplicateAttestationData { .. } => RejectionReason::DuplicateAttestationData, diff --git a/crates/blockchain/src/store.rs b/crates/blockchain/src/store.rs index 2ee1bb46..260ed8d8 100644 --- a/crates/blockchain/src/store.rs +++ b/crates/blockchain/src/store.rs @@ -18,7 +18,8 @@ use tracing::{info, trace, warn}; use crate::{ GOSSIP_DISPARITY_INTERVALS, INTERVALS_PER_SLOT, MAX_ATTESTATIONS_DATA, SlotInterval, - block_builder::{PostBlockCheckpoints, ProposerConfig, build_block}, + block_builder::{PostBlockCheckpoints, ProposalInputs, ProposerConfig, build_block}, + body_proof::{self, BodyProofBuffer, ChosenBody}, metrics, }; @@ -901,8 +902,60 @@ fn get_proposal_head(store: &mut Store, slot: u64) -> H256 { store.head().expect("store head exists") } +/// Produce the block for `slot` from the buffered candidate body proofs. +/// +/// The proposer's path: it packs no body of its own any more. Advancing the +/// store to `slot`'s interval 0 (which promotes the pending attestations) +/// still happens here, so the candidates are judged against the same state the +/// block will be built on, then [`body_proof::choose_body`] picks the body and +/// seals the block. +pub(crate) fn produce_block_from_candidates( + store: &mut Store, + slot: u64, + validator_index: u64, + candidates: &BodyProofBuffer, +) -> Result { + let head_root = get_proposal_head(store, slot); + let head_state = store + .get_state(&head_root) + .expect("head state exists") + .ok_or(StoreError::MissingParentState { + parent_root: head_root, + slot, + })?; + + let num_validators = head_state.validators.len() as u64; + if !is_proposer(validator_index, slot, num_validators) { + return Err(StoreError::NotProposer { + validator_index, + slot, + }); + } + + // What the CHAIN carries. By the time this runs at interval 0, the + // slot's aggregates have already been promoted into the seen-votes map, so + // that map holds the very `AttestationData` these candidate bodies carry + // and would score every one of them at zero. + let latest_head_votes = store.extract_on_chain_votes(); + + body_proof::choose_body( + &head_state, + slot, + validator_index, + head_root, + candidates, + &latest_head_votes, + ) +} + /// Produce a block and per-aggregated-attestation signature payloads for the target slot. /// +/// Packs a body straight from this node's own pool, which is what the +/// aggregation worker does for a candidate body proof (via +/// `body_proof::build_body_proof`) and what the offline block-building +/// benchmark measures. A live proposer instead adopts a candidate body through +/// [`produce_block_from_candidates`]. +/// /// Returns the finalized block and attestation signature payloads aligned /// with `block.body.attestations`. pub fn produce_block_with_signatures( @@ -935,6 +988,21 @@ pub fn produce_block_with_signatures( let known_block_roots = store.get_block_roots().unwrap(); + // The latest vote per validator the CHAIN already carries, so selection can + // value an entry for the head weight it would ADD and not only for the + // justification voters it brings. + // + // Deliberately not `extract_latest_known_attestations`: that map is written + // in lockstep with the aggregated-payload pool this block is built from, so + // every entry would score zero new head voters and the axis would be dead. + let latest_head_votes = store.extract_on_chain_votes(); + + let inputs = ProposalInputs { + known_block_roots: &known_block_roots, + aggregated_payloads: &aggregated_payloads, + latest_head_votes, + }; + let (block, signatures, post_checkpoints) = { let _timing = metrics::time_block_building_payload_aggregation(); build_block( @@ -942,8 +1010,7 @@ pub fn produce_block_with_signatures( slot, validator_index, head_root, - &known_block_roots, - &aggregated_payloads, + inputs, config, )? }; @@ -1013,6 +1080,20 @@ pub enum StoreError { #[error("Validator signature verification failed")] SignatureVerificationFailed, + /// Kept apart from [`Self::SignatureDecodingFailed`] because the proposer + /// signature is a component of the block proof, so the spec reports its + /// failure as an invalid block proof rather than an invalid signature. + #[error("Block proposer signature could not be decoded")] + ProposerSignatureDecodingFailed, + + /// See [`Self::ProposerSignatureDecodingFailed`] for why this is distinct + /// from [`Self::SignatureVerificationFailed`]. + #[error("Block proposer signature verification failed")] + ProposerSignatureVerificationFailed, + + #[error("Block carries an attestation proof but has no attestations")] + UnexpectedAttestationProof, + #[error("Block slot {0} exceeds u32 range")] SlotOutOfRange(u64), @@ -1115,13 +1196,18 @@ pub enum StoreError { BlockTooFarInFuture { block_slot: u64, current_slot: u64 }, } -/// Full verification of a signed block's merged multi-message aggregate proof. +/// Full verification of a signed block's proof. +/// +/// The proof has two independent parts: /// -/// Structural pre-checks (fast fail) ensure the merged proof's `info` list lines -/// up with the block body (one entry per attestation plus a trailing proposer -/// entry; messages, slots, and participants match what the body declares). -/// On success, the lean-multisig devnet5 `verify_type_2` primitive runs the -/// SNARK verifier over the merged proof bytes against the resolved pubkey set. +/// 1. The proposer's raw XMSS signature over the block root, verified directly +/// against the proposer's `proposal_pubkey` with the hash-based verifier. +/// 2. The attestation aggregate: a lean-multisig Type-2 over the body +/// attestations only. Structural pre-checks (fast fail) ensure its `info` +/// list lines up with the block body (one entry per attestation; messages, +/// slots, and participants match what the body declares), then the +/// `verify_type_2` SNARK verifier runs over the proof bytes. A block with no +/// attestations carries no aggregate. /// /// Exposed publicly so RPC handlers (notably the Hive test-driver /// `verify_signatures/run` endpoint) can run the exact same verification path @@ -1161,35 +1247,20 @@ pub fn verify_block_signatures( } let block_root = block.hash_tree_root(); - let structural_elapsed = total_start.elapsed(); - - // Resolve pubkeys per multi-message aggregate component for verify_type_2 and rederive the - // expected (message, slot) bindings from the block body. Attestation - // components use each participant's attestation_pubkey; the trailing - // proposer component uses the proposal_pubkey of `block.proposer_index`. - let expected_components = attestations.len() + 1; - let mut pubkeys_per_component: Vec> = - Vec::with_capacity(expected_components); - let mut expected_bindings: Vec<(H256, u32)> = Vec::with_capacity(expected_components); - - for attestation in attestations.iter() { - let mut pubkeys = Vec::new(); - for vid in validator_indices(&attestation.aggregation_bits) { - let out_of_range = StoreError::AttesterIndexOutOfRange { - validator_index: vid, - num_validators, - }; - let validator = validators.get(vid as usize).ok_or(out_of_range)?; - let pk = ValidatorPublicKey::from_bytes(&validator.attestation_pubkey) - .map_err(|_| StoreError::PubkeyDecodingFailed(vid))?; - pubkeys.push(pk); - } - pubkeys_per_component.push(pubkeys); - let slot_u32 = u32::try_from(attestation.data.slot) - .map_err(|_| StoreError::SlotOutOfRange(attestation.data.slot))?; - expected_bindings.push((attestation.data.hash_tree_root(), slot_u32)); - } - + // Slot narrowing is a range check, so it closes out the structural segment + // rather than landing between two timers and escaping both. + let block_slot_u32 = + u32::try_from(block.slot).map_err(|_| StoreError::SlotOutOfRange(block.slot))?; + // One instant ends the structural segment and starts the crypto one, so the + // two reported components sum to `total_elapsed` with no gap between them. + let structural_end = std::time::Instant::now(); + let structural_elapsed = structural_end.duration_since(total_start); + + // 1. Verify the proposer's raw XMSS signature over the block root. It is + // carried outside the attestation aggregate, so it is checked directly + // against the proposer's proposal pubkey with the hash-based verifier. + // Counted in `crypto_elapsed` below along with the aggregate, so this + // cost is reported rather than falling between the timers. let proposer_out_of_range = StoreError::ProposerIndexOutOfRange { proposer_index: block.proposer_index, num_validators, @@ -1199,30 +1270,69 @@ pub fn verify_block_signatures( .ok_or(proposer_out_of_range)?; let proposer_pubkey = ValidatorPublicKey::from_bytes(&proposer_validator.proposal_pubkey) .map_err(|_| StoreError::PubkeyDecodingFailed(block.proposer_index))?; - pubkeys_per_component.push(vec![proposer_pubkey]); - let block_slot_u32 = - u32::try_from(block.slot).map_err(|_| StoreError::SlotOutOfRange(block.slot))?; - expected_bindings.push((block_root, block_slot_u32)); + let proposer_signature = ValidatorSignature::from_bytes(&signed_block.proof.proposer_signature) + .map_err(|_| StoreError::ProposerSignatureDecodingFailed)?; + if !proposer_signature.is_valid(&proposer_pubkey, block_slot_u32, &block_root) { + return Err(StoreError::ProposerSignatureVerificationFailed); + } - let merged_bytes = signed_block.proof.proof_bytes(); + // 2. Verify the attestation aggregate (Type-2 over the body attestations + // only). A block with no attestations carries no aggregate; reject a + // stray proof rather than silently ignoring it. + if attestations.is_empty() { + if !signed_block + .proof + .attestation_proof + .proof_bytes() + .is_empty() + { + return Err(StoreError::UnexpectedAttestationProof); + } + } else { + // Resolve pubkeys per Type-2 component and rederive the expected + // (message, slot) bindings from the block body. Each component uses its + // participants' attestation_pubkeys. + let mut pubkeys_per_component: Vec> = + Vec::with_capacity(attestations.len()); + let mut expected_bindings: Vec<(H256, u32)> = Vec::with_capacity(attestations.len()); + + for attestation in attestations.iter() { + let mut pubkeys = Vec::new(); + for vid in validator_indices(&attestation.aggregation_bits) { + let out_of_range = StoreError::AttesterIndexOutOfRange { + validator_index: vid, + num_validators, + }; + let validator = validators.get(vid as usize).ok_or(out_of_range)?; + let pk = ValidatorPublicKey::from_bytes(&validator.attestation_pubkey) + .map_err(|_| StoreError::PubkeyDecodingFailed(vid))?; + pubkeys.push(pk); + } + pubkeys_per_component.push(pubkeys); + let slot_u32 = u32::try_from(attestation.data.slot) + .map_err(|_| StoreError::SlotOutOfRange(attestation.data.slot))?; + expected_bindings.push((attestation.data.hash_tree_root(), slot_u32)); + } - let crypto_start = std::time::Instant::now(); - ethlambda_crypto::verify_type_2_signature( - merged_bytes, - pubkeys_per_component, - &expected_bindings, - ) - .map_err(StoreError::BlockProofVerificationFailed)?; - let crypto_elapsed = crypto_start.elapsed(); + let merged_bytes = signed_block.proof.attestation_proof.proof_bytes(); + ethlambda_crypto::verify_type_2_signature( + merged_bytes, + pubkeys_per_component, + &expected_bindings, + ) + .map_err(StoreError::BlockProofVerificationFailed)?; + } + let total_end = std::time::Instant::now(); + let crypto_elapsed = total_end.duration_since(structural_end); + let total_elapsed = total_end.duration_since(total_start); - let total_elapsed = total_start.elapsed(); info!( slot = block.slot, attestation_count = attestations.len(), ?structural_elapsed, ?crypto_elapsed, ?total_elapsed, - "Block multi-message aggregate proof verified" + "Block proof verified" ); Ok(()) @@ -1285,24 +1395,24 @@ mod tests { use ethlambda_types::{ attestation::{AggregatedAttestation, AggregationBits, AttestationData}, block::{ - AggregatedAttestations, BlockBody, MultiMessageAggregate, SignedBlock, - SingleMessageAggregate, + AggregatedAttestations, BlockBody, BlockProof, SignedBlock, SingleMessageAggregate, }, checkpoint::Checkpoint, state::State, }; - /// Test helper: placeholder block proof bytes. + /// Test helper: placeholder block proof. /// - /// In production the merged proof is the raw `compress_without_pubkeys()` - /// output of `merge_many_type_1`, which can only be built by the - /// lean-multisig prover. Tests that don't go through - /// `verify_block_signatures` use an empty blob. + /// In production the attestation aggregate is the raw + /// `compress_without_pubkeys()` output of `merge_many_type_1`, which can + /// only be built by the lean-multisig prover, and the proposer signature is + /// a real XMSS signature. Tests that don't go through + /// `verify_block_signatures` use an empty proof. fn make_signed_block_proof( _proposer_index: u64, _attestation_proofs: Vec, - ) -> MultiMessageAggregate { - MultiMessageAggregate::default() + ) -> BlockProof { + BlockProof::default() } fn make_bits(indices: &[usize]) -> AggregationBits { @@ -1809,7 +1919,7 @@ mod tests { }; let signed_block = SignedBlock { message: block, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let result = on_block_without_verification(&mut store, signed_block); @@ -1851,7 +1961,7 @@ mod tests { }; let signed_block = SignedBlock { message: block, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let result = on_block_without_verification(&mut store, signed_block); @@ -1909,7 +2019,7 @@ mod tests { state_root: H256::ZERO, body: BlockBody { attestations }, }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let result = verify_block_signatures(&state, &out_of_range_attester); assert!( @@ -1931,7 +2041,7 @@ mod tests { state_root: H256::ZERO, body: BlockBody::default(), }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let result = verify_block_signatures(&state, &out_of_range_proposer); assert!( diff --git a/crates/blockchain/state_transition/src/lib.rs b/crates/blockchain/state_transition/src/lib.rs index d03f74b2..4302eb32 100644 --- a/crates/blockchain/state_transition/src/lib.rs +++ b/crates/blockchain/state_transition/src/lib.rs @@ -970,6 +970,78 @@ mod tests { ); } + /// A vote whose target is already justified must be SKIPPED, not rejected: + /// `is_valid_vote` returns `Ok(false)` and the loop does `continue`, so the + /// block carrying it still applies cleanly. + /// + /// The block builder relies on this. It deliberately packs such votes, + /// because they carry no justification value but still move LMD-GHOST + /// (`insert_signed_block` records every attestation a block carries as a + /// fork-choice vote, whatever this function decides). If the transition + /// ever started erroring here instead, every proposer packing a settled + /// target would produce blocks the network rejects. + #[test] + fn process_attestations_skips_an_already_justified_target_without_rejecting_the_block() { + const NUM_VALIDATORS: usize = 4; + let r1 = H256([1u8; 32]); + let r2 = H256([2u8; 32]); + + let mut justified_slots = JustifiedSlots::new(); + justified_slots_ops::extend_to_slot(&mut justified_slots, 0, 1); + justified_slots_ops::set_justified(&mut justified_slots, 0, 1); + + let mut state = State { + config: StateConfig { genesis_time: 0 }, + slot: 3, + latest_block_header: BlockHeader { + slot: 2, + proposer_index: 0, + parent_root: r1, + state_root: H256::ZERO, + body_root: BlockBody::default().hash_tree_root(), + }, + latest_justified: Checkpoint { slot: 1, root: r1 }, + latest_finalized: Checkpoint { + slot: 0, + root: H256::ZERO, + }, + historical_block_hashes: SszList::try_from(vec![H256::ZERO, r1, r2]).unwrap(), + justified_slots, + validators: SszList::try_from(make_validators(NUM_VALIDATORS)).unwrap(), + justifications_roots: SszList::try_from(vec![]).unwrap(), + justifications_validators: JustificationValidators::new(), + }; + + // Target slot 1 is already justified above; source is genesis. + let vote = AggregatedAttestation { + aggregation_bits: make_bits(&[0, 1, 2], NUM_VALIDATORS), + data: AttestationData { + slot: 2, + head: Checkpoint { slot: 1, root: r1 }, + target: Checkpoint { slot: 1, root: r1 }, + source: Checkpoint { + slot: 0, + root: H256::ZERO, + }, + }, + }; + + let before = state.latest_justified; + let atts: AggregatedAttestations = vec![vote].try_into().unwrap(); + + process_attestations(&mut state, &atts) + .expect("an already-justified target is skipped, not an error"); + + assert_eq!( + state.latest_justified, before, + "the skipped vote must not move justification" + ); + assert!( + state.justifications_roots.is_empty(), + "the skipped vote must not open a tally for a settled target" + ); + } + /// leanSpec #1178: `process_attestations` on a state with no validators is /// rejected with a typed error. Belt-and-suspenders: the header stage already /// rejects an empty registry first in the normal flow, but the flat-vote diff --git a/crates/common/test-fixtures/src/fork_choice.rs b/crates/common/test-fixtures/src/fork_choice.rs index c6c9e58c..d11b76dc 100644 --- a/crates/common/test-fixtures/src/fork_choice.rs +++ b/crates/common/test-fixtures/src/fork_choice.rs @@ -8,7 +8,7 @@ use crate::{ TestState, deser_xmss_hex, }; use ethlambda_types::attestation::XmssSignature; -use ethlambda_types::block::{MultiMessageAggregate, SignedBlock}; +use ethlambda_types::block::{BlockProof, SignedBlock}; use ethlambda_types::primitives::H256; use serde::{Deserialize, Deserializer}; use std::collections::HashMap; @@ -206,7 +206,7 @@ impl BlockStepData { pub fn to_blank_signed_block(&self) -> SignedBlock { SignedBlock { message: self.to_block(), - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), } } } diff --git a/crates/common/test-fixtures/src/verify_signatures.rs b/crates/common/test-fixtures/src/verify_signatures.rs index a425ac81..edac8b9f 100644 --- a/crates/common/test-fixtures/src/verify_signatures.rs +++ b/crates/common/test-fixtures/src/verify_signatures.rs @@ -11,7 +11,8 @@ //! proof: { proof: { data: "0x" } } use crate::{Block, RejectionReason, TestInfo, TestState}; -use ethlambda_types::block::{MultiMessageAggregate, SignedBlock}; +use ethlambda_types::attestation::blank_xmss_signature; +use ethlambda_types::block::{BlockProof, MultiMessageAggregate, SignedBlock}; use serde::Deserialize; use std::collections::HashMap; use std::fmt; @@ -137,17 +138,23 @@ impl TestSignedBlock { /// /// The container carries the raw lean-multisig wire in the /// `MultiMessageAggregate` stored by `SignedBlock.proof`. + /// + /// NOTE: these fixtures use the leanSpec #799 layout (proposer folded into + /// one merged Type-2). This client now carries the proposer signature + /// outside the attestation aggregate, so the merged bytes land in + /// `attestation_proof` with an empty proposer signature. The verify spec + /// tests therefore fail against these fixtures until they are regenerated. pub fn try_into_signed_block_with_proofs(self) -> Result { let bytes = self .proof .decode() .map_err(|err| SignedBlockConvertError::InvalidProofHex(err.to_string()))?; let len = bytes.len(); - let proof = MultiMessageAggregate::from_bytes(&bytes) + let attestation_proof = MultiMessageAggregate::from_bytes(&bytes) .map_err(|_| SignedBlockConvertError::ProofTooLarge(len))?; Ok(SignedBlock { message: self.block.into(), - proof, + proof: BlockProof::new(blank_xmss_signature(), attestation_proof), }) } } diff --git a/crates/common/types/src/attestation.rs b/crates/common/types/src/attestation.rs index c0b6a2f1..45f8a634 100644 --- a/crates/common/types/src/attestation.rs +++ b/crates/common/types/src/attestation.rs @@ -35,6 +35,28 @@ pub struct AttestationData { pub source: Checkpoint, } +impl AttestationData { + /// Whether this vote supersedes `other` as a validator's latest message. + /// + /// The LMD-GHOST latest-message rule: the later slot wins, and a tie is + /// broken by data root. Breaking the tie on a total order rather than on + /// arrival matters because the seen-votes map is written from more than + /// one place (block import, gossip payload insertion, the aggregation + /// worker), so an order-dependent rule would let two nodes that saw the + /// same votes in different orders disagree about the head. The on-chain + /// vote map is the exception, written only on block import, and relies on + /// the same total order to stay independent of import interleaving. + /// + /// Lives here rather than beside fork choice because the vote map is + /// maintained in the storage layer, which does not depend on the fork + /// choice crate; `ethlambda-types` is what storage, blockchain and fork + /// choice all share. + pub fn supersedes(&self, other: &AttestationData) -> bool { + self.slot > other.slot + || (self.slot == other.slot && self.hash_tree_root() > other.hash_tree_root()) + } +} + /// Validator attestation bundled with its signature. /// ///
@@ -214,6 +236,52 @@ impl From for HashedAttestationData { mod tests { use super::*; + fn att_data(slot: u64, head_root: u8) -> AttestationData { + AttestationData { + slot, + head: Checkpoint { + slot, + root: H256([head_root; 32]), + }, + target: Checkpoint::default(), + source: Checkpoint::default(), + } + } + + #[test] + fn supersedes_prefers_the_later_slot() { + let earlier = att_data(4, 1); + let later = att_data(5, 1); + + assert!(later.supersedes(&earlier)); + assert!(!earlier.supersedes(&later)); + } + + #[test] + fn supersedes_is_irreflexive() { + let vote = att_data(4, 1); + + assert!( + !vote.supersedes(&vote), + "a vote does not replace an identical one, or record_vote would clone every duplicate" + ); + } + + /// Same slot: the data root decides, so two nodes that saw the same votes + /// in different orders still agree on which one is a validator's latest. + #[test] + fn supersedes_breaks_a_slot_tie_on_data_root_and_is_antisymmetric() { + let a = att_data(4, 1); + let b = att_data(4, 2); + + assert_ne!(a.hash_tree_root(), b.hash_tree_root()); + assert_eq!( + a.supersedes(&b), + !b.supersedes(&a), + "exactly one of the two must win the tie" + ); + } + /// Build an `AggregationBits` of `len` bits with the indices in `set` flipped on. fn bits(len: usize, set: &[usize]) -> AggregationBits { let mut b = AggregationBits::with_length(len).unwrap(); diff --git a/crates/common/types/src/block.rs b/crates/common/types/src/block.rs index 5c5508a2..2025848c 100644 --- a/crates/common/types/src/block.rs +++ b/crates/common/types/src/block.rs @@ -4,22 +4,27 @@ use libssz_derive::{HashTreeRoot, SszDecode, SszEncode}; use libssz_types::SszList; use crate::{ - attestation::{AggregatedAttestation, AggregationBits, validator_indices}, + attestation::{ + AggregatedAttestation, AggregationBits, XmssSignature, blank_xmss_signature, + validator_indices, + }, primitives::{self, ByteList, H256}, }; // Convenience trait for calling hash_tree_root() without a hasher argument use primitives::HashTreeRoot as _; -/// Envelope carrying a block and the single merged proof binding every -/// signature it depends on. +/// Envelope carrying a block and its [`BlockProof`]. +/// +/// The proof keeps the proposer's raw signature separate from the attestation +/// aggregate (see [`BlockProof`]). /// ///
/// /// `HashTreeRoot` is intentionally not derived: consumers never hash a /// `SignedBlock` directly — they always hash the inner `Block`. Keeping the /// envelope structurally minimal also means the on-chain root is independent -/// of how the merged proof is serialised. +/// of how the proof is serialised. /// ///
#[derive(Clone, SszEncode, SszDecode)] @@ -27,16 +32,23 @@ pub struct SignedBlock { /// The block being signed. pub message: Block, - /// Single full-block proof covering attestations and the proposer signature. - pub proof: MultiMessageAggregate, + /// Full-block proof: proposer signature + attestation aggregate. + pub proof: BlockProof, } -// Manual Debug impl because the merged proof bytes are large and opaque. +// Manual Debug impl because the proof bytes are large and opaque. impl core::fmt::Debug for SignedBlock { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { f.debug_struct("SignedBlock") .field("message", &self.message) - .field("proof", &format_args!("<{} bytes>", self.proof.proof.len())) + .field( + "proposer_signature", + &format_args!("<{} bytes>", self.proof.proposer_signature.len()), + ) + .field( + "attestation_proof", + &format_args!("<{} bytes>", self.proof.attestation_proof.proof.len()), + ) .finish() } } @@ -88,6 +100,105 @@ pub enum MultiMessageAggregateError { ProofTooLarge(usize), } +// ============================================================================ +// Block proof (proposer signature outside the attestation aggregate) +// ============================================================================ + +/// A full-block proof: the proposer's raw signature plus the attestation +/// aggregate, carried as two independent fields. +/// +/// ```text +/// attestations = [att0, att1] -> (proposer_signature, aggregate([att0, att1])) +/// attestations = [] -> (proposer_signature, empty-proof) +/// ``` +/// +/// `proposer_signature` is the proposer's raw XMSS signature over the block +/// root — the same fixed-size [`XmssSignature`] wire type carried by +/// `SignedAttestation`. It is verified directly against the proposer's +/// `proposal_pubkey` with the hash-based XMSS verifier, so it never enters the +/// lean-multisig prover/verifier. +/// +/// `attestation_proof` is the lean-multisig Type-2 over the block body's +/// attestations *only* — the proposer is no longer one of its components, so +/// it is empty when the block carries no attestations. +/// +///
+/// +/// `HashTreeRoot` is intentionally not derived (as on `SignedAttestation`): +/// `XmssSignature` is a fixed-size byte vector here, but the spec Merkleizes +/// the signature as a container, so a derived root would diverge. Nothing +/// hashes a `BlockProof`. +/// +///
+#[derive(Debug, Clone, PartialEq, Eq, SszEncode, SszDecode)] +pub struct BlockProof { + /// The proposer's raw XMSS signature over the block root. + pub proposer_signature: XmssSignature, + /// Type-2 aggregate over the body attestations (empty if there are none). + pub attestation_proof: MultiMessageAggregate, +} + +impl BlockProof { + /// Build a proof from a proposer signature and an attestation aggregate. + pub fn new( + proposer_signature: XmssSignature, + attestation_proof: MultiMessageAggregate, + ) -> Self { + Self { + proposer_signature, + attestation_proof, + } + } +} + +impl Default for BlockProof { + /// A blank proof: the structurally-valid all-zero XMSS placeholder used by + /// genesis-style anchor blocks (see [`blank_xmss_signature`]) plus an empty + /// attestation aggregate. `XmssSignature` is fixed-size and has no empty + /// form, so the blank doubles as the genesis placeholder. + fn default() -> Self { + Self { + proposer_signature: blank_xmss_signature(), + attestation_proof: MultiMessageAggregate::default(), + } + } +} + +// ============================================================================ +// Block body proof +// ============================================================================ + +/// A candidate block body together with the aggregate that binds its +/// attestations: everything a proposer needs for a block except the header and +/// its own signature. +/// +/// Only possible because the proposer signature sits outside the aggregate +/// (see [`BlockProof`]): the Type-2 over a body's attestations does not depend +/// on the block root, so it can be built by a node that is not the proposer, +/// before the block exists. Aggregators gossip these so the slot's proposer +/// can adopt one instead of running the merge itself. +/// +/// A proposer that adopts one carries `proof` verbatim as +/// [`BlockProof::attestation_proof`], so the pair is only meaningful together: +/// the proof binds exactly these attestations, in this order. +#[derive(Clone, SszEncode, SszDecode)] +pub struct BlockBodyProof { + /// The candidate body, carrying the attestations the proof binds. + pub block_body: BlockBody, + /// Type-2 aggregate over `block_body`'s attestations. + pub proof: MultiMessageAggregate, +} + +// Manual Debug impl because the proof bytes are large and opaque. +impl core::fmt::Debug for BlockBodyProof { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + f.debug_struct("BlockBodyProof") + .field("attestations", &self.block_body.attestations.len()) + .field("proof", &format_args!("<{} bytes>", self.proof.proof.len())) + .finish() + } +} + // ============================================================================ // Single-message aggregate // ============================================================================ @@ -95,13 +206,12 @@ pub enum MultiMessageAggregateError { // Wire format mirrors leanSpec PR #717: `SingleMessageAggregate` is a flat // `{ participants, proof }` pair. The signed `message` and `slot` are NOT // carried on the envelope — verifiers rederive each component's binding -// from the surrounding block body (attestation `data` + slot for body -// components, block root + slot for the proposer component). +// from the surrounding block body (attestation `data` + slot). // -// `MultiMessageAggregate` carries the raw lean-multisig type-2 bytes. -// Component participant bitfields come from -// `block.body.attestations[i].aggregation_bits` (and `block.proposer_index` for -// the trailing proposer entry). +// `MultiMessageAggregate` carries the raw lean-multisig type-2 bytes for the +// body attestations only; the proposer signature is carried separately in +// `BlockProof::proposer_signature`. Component participant bitfields come from +// `block.body.attestations[i].aggregation_bits`. /// Maximum number of distinct `AttestationData` entries permitted in a single /// block. Canonical home for the cap shared across `ethlambda-blockchain`, @@ -307,11 +417,13 @@ mod tests { }; let signed = SignedBlock { message: block, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let bytes = signed.to_ssz(); let decoded = SignedBlock::from_ssz_bytes(&bytes).expect("decode"); - assert_eq!(decoded.proof.proof.len(), 0); + // Default proof: empty attestation aggregate + the blank XMSS placeholder. + assert_eq!(decoded.proof.attestation_proof.proof.len(), 0); + assert_eq!(decoded.proof.proposer_signature, blank_xmss_signature()); assert_eq!(decoded.message.slot, signed.message.slot); assert_eq!( decoded.message.proposer_index, @@ -330,4 +442,42 @@ mod tests { assert_eq!(&encoded[4..], proof_bytes); assert_eq!(aggregate.proof_bytes(), proof_bytes); } + + #[test] + fn signed_block_ssz_round_trip_with_proposer_signature() { + let block = Block { + slot: 9, + proposer_index: 2, + parent_root: H256::ZERO, + state_root: H256::ZERO, + body: BlockBody::default(), + }; + // A distinctive, full-size XMSS signature blob (fixed `SIGNATURE_SIZE`). + let proposer_bytes: Vec = (0..crate::attestation::SIGNATURE_SIZE) + .map(|i| (i % 251) as u8) + .collect(); + let proposer_signature = XmssSignature::try_from(proposer_bytes.clone()).unwrap(); + let attestation_bytes: Vec = (0..64).collect(); + let signed = SignedBlock { + message: block, + proof: BlockProof::new( + proposer_signature, + MultiMessageAggregate::from_bytes(&attestation_bytes).unwrap(), + ), + }; + + let bytes = signed.to_ssz(); + let decoded = SignedBlock::from_ssz_bytes(&bytes).expect("decode"); + + assert_eq!( + &*decoded.proof.proposer_signature, + proposer_bytes.as_slice() + ); + assert_eq!( + decoded.proof.attestation_proof.proof_bytes(), + attestation_bytes + ); + assert_eq!(decoded.message.slot, 9); + assert_eq!(decoded.message.proposer_index, 2); + } } diff --git a/crates/net/api/src/lib.rs b/crates/net/api/src/lib.rs index d6ec647d..cd2a3dfb 100644 --- a/crates/net/api/src/lib.rs +++ b/crates/net/api/src/lib.rs @@ -1,6 +1,6 @@ use ethlambda_types::{ attestation::{SignedAggregatedAttestation, SignedAttestation}, - block::SignedBlock, + block::{BlockBodyProof, SignedBlock}, primitives::H256, }; use spawned_concurrency::error::ActorError; @@ -17,6 +17,9 @@ pub trait BlockChainToP2P: Send + Sync { &self, attestation: SignedAggregatedAttestation, ) -> Result<(), ActorError>; + /// Gossip a candidate block body and the aggregate binding its + /// attestations, for the next slot's proposer to adopt. + fn publish_block_body_proof(&self, body_proof: BlockBodyProof) -> Result<(), ActorError>; fn fetch_block(&self, root: H256) -> Result<(), ActorError>; } @@ -43,6 +46,9 @@ pub trait P2PToBlockChain: Send + Sync { &self, attestation: SignedAggregatedAttestation, ) -> Result<(), ActorError>; + /// A candidate block body plus attestation aggregate seen on gossip. Kept + /// as a proposal candidate; see `BlockChainServer`'s body-proof buffer. + fn new_block_body_proof(&self, body_proof: BlockBodyProof) -> Result<(), ActorError>; } // --- Init messages --- diff --git a/crates/net/p2p/src/gossipsub/handler.rs b/crates/net/p2p/src/gossipsub/handler.rs index 1ba5cea3..9b28ff70 100644 --- a/crates/net/p2p/src/gossipsub/handler.rs +++ b/crates/net/p2p/src/gossipsub/handler.rs @@ -2,7 +2,7 @@ use ethlambda_network_api::BlockSource; use ethlambda_types::{ ShortRoot, attestation::{SignedAggregatedAttestation, SignedAttestation}, - block::SignedBlock, + block::{BlockBodyProof, SignedBlock}, primitives::HashTreeRoot as _, }; use libp2p::gossipsub::Event; @@ -12,8 +12,8 @@ use tracing::{error, info, trace}; use super::{ encoding::{compress_message, decompress_message}, messages::{ - AGGREGATION_TOPIC_KIND, ATTESTATION_SUBNET_TOPIC_PREFIX, BLOCK_TOPIC_KIND, - attestation_subnet_topic, + AGGREGATION_TOPIC_KIND, ATTESTATION_SUBNET_TOPIC_PREFIX, BLOCK_BODY_PROOF_TOPIC_KIND, + BLOCK_TOPIC_KIND, attestation_subnet_topic, }, }; use crate::{P2PServer, metrics}; @@ -96,6 +96,35 @@ pub async fn handle_gossipsub_message(server: &mut P2PServer, event: Event) { ); } } + Some(BLOCK_BODY_PROOF_TOPIC_KIND) => { + trace!( + kind = "block_body_proof", + peer_count, "P2P message received" + ); + let compressed_len = message.data.len(); + let Ok(uncompressed_data) = decompress_message(&message.data) + .inspect_err(|err| error!(%err, "Failed to decompress gossipped block body proof")) + else { + return; + }; + metrics::observe_gossip_block_body_proof_size(uncompressed_data.len(), compressed_len); + + let Ok(body_proof) = BlockBodyProof::from_ssz_bytes(&uncompressed_data) + .inspect_err(|err| error!(?err, "Failed to decode gossipped block body proof")) + else { + return; + }; + info!( + attestation_count = body_proof.block_body.attestations.len(), + proof_bytes = body_proof.proof.proof.len(), + "Received block body proof from gossip" + ); + if let Some(ref blockchain) = server.blockchain { + let _ = blockchain.new_block_body_proof(body_proof).inspect_err( + |err| error!(%err, "Failed to forward block body proof to blockchain"), + ); + } + } Some(kind) if kind.starts_with(ATTESTATION_SUBNET_TOPIC_PREFIX) => { trace!(kind = "attestation", peer_count, "P2P message received"); let compressed_len = message.data.len(); @@ -197,6 +226,28 @@ pub async fn publish_block(server: &mut P2PServer, signed_block: SignedBlock) { ); } +pub async fn publish_block_body_proof(server: &mut P2PServer, body_proof: BlockBodyProof) { + let attestation_count = body_proof.block_body.attestations.len(); + + // Encode to SSZ + let ssz_bytes = body_proof.to_ssz(); + + // Compress with raw snappy + let compressed = compress_message(&ssz_bytes); + + metrics::observe_gossip_block_body_proof_size(ssz_bytes.len(), compressed.len()); + + // Publish to the block-body-proof topic + server + .swarm_handle + .publish(server.block_body_proof_topic.clone(), compressed); + info!( + attestation_count, + proof_bytes = body_proof.proof.proof.len(), + "Published block body proof to gossipsub" + ); +} + pub async fn publish_aggregated_attestation( server: &mut P2PServer, attestation: SignedAggregatedAttestation, diff --git a/crates/net/p2p/src/gossipsub/messages.rs b/crates/net/p2p/src/gossipsub/messages.rs index 11664750..31dd8c7e 100644 --- a/crates/net/p2p/src/gossipsub/messages.rs +++ b/crates/net/p2p/src/gossipsub/messages.rs @@ -10,6 +10,10 @@ pub const ATTESTATION_SUBNET_TOPIC_PREFIX: &str = "attestation"; /// /// Full topic format: `/leanconsensus/{FORK_DIGEST}/aggregation/ssz_snappy` pub const AGGREGATION_TOPIC_KIND: &str = "aggregation"; +/// Topic kind for candidate block body + attestation aggregate gossip. +/// +/// Full topic format: `/leanconsensus/{FORK_DIGEST}/block_body_proof/ssz_snappy` +pub const BLOCK_BODY_PROOF_TOPIC_KIND: &str = "block_body_proof"; /// Build the block gossipsub topic. pub fn block_topic() -> libp2p::gossipsub::IdentTopic { @@ -25,6 +29,13 @@ pub fn aggregation_topic() -> libp2p::gossipsub::IdentTopic { )) } +/// Build the block-body-proof gossipsub topic. +pub fn block_body_proof_topic() -> libp2p::gossipsub::IdentTopic { + libp2p::gossipsub::IdentTopic::new(format!( + "/leanconsensus/{FORK_DIGEST}/{BLOCK_BODY_PROOF_TOPIC_KIND}/ssz_snappy" + )) +} + /// Build an attestation subnet gossipsub topic for the given subnet. pub fn attestation_subnet_topic(subnet_id: u64) -> libp2p::gossipsub::IdentTopic { libp2p::gossipsub::IdentTopic::new(format!( diff --git a/crates/net/p2p/src/gossipsub/mod.rs b/crates/net/p2p/src/gossipsub/mod.rs index b50ea4fd..a252ebc8 100644 --- a/crates/net/p2p/src/gossipsub/mod.rs +++ b/crates/net/p2p/src/gossipsub/mod.rs @@ -5,5 +5,8 @@ mod messages; pub use encoding::decompress_message; pub use handler::{ handle_gossipsub_message, publish_aggregated_attestation, publish_attestation, publish_block, + publish_block_body_proof, +}; +pub use messages::{ + aggregation_topic, attestation_subnet_topic, block_body_proof_topic, block_topic, }; -pub use messages::{aggregation_topic, attestation_subnet_topic, block_topic}; diff --git a/crates/net/p2p/src/lib.rs b/crates/net/p2p/src/lib.rs index c2b04359..f6ff89ed 100644 --- a/crates/net/p2p/src/lib.rs +++ b/crates/net/p2p/src/lib.rs @@ -9,6 +9,7 @@ use ethlambda_network_api::{ InitBlockChain, P2PToBlockChainRef, block_chain_to_p2p::{ FetchBlock, PublishAggregatedAttestation, PublishAttestation, PublishBlock, + PublishBlockBodyProof, }, }; use ethlambda_storage::Store; @@ -42,8 +43,9 @@ use crate::{ spawn_discovery, }, gossipsub::{ - aggregation_topic, attestation_subnet_topic, block_topic, publish_aggregated_attestation, - publish_attestation, publish_block, + aggregation_topic, attestation_subnet_topic, block_body_proof_topic, block_topic, + publish_aggregated_attestation, publish_attestation, publish_block, + publish_block_body_proof, }, req_resp::{ BLOCKS_BY_RANGE_PROTOCOL_V1, BLOCKS_BY_ROOT_PROTOCOL_V1, Codec, @@ -226,6 +228,7 @@ pub struct BuiltSwarm { pub(crate) attestation_committee_count: u64, pub(crate) block_topic: libp2p::gossipsub::IdentTopic, pub(crate) aggregation_topic: libp2p::gossipsub::IdentTopic, + pub(crate) block_body_proof_topic: libp2p::gossipsub::IdentTopic, /// Every dial target per bootnode; see [`dial_addrs`]. Empty entries are never /// inserted; see [`bootnode_dial_addrs`]. pub(crate) bootnode_addrs: HashMap>, @@ -399,6 +402,15 @@ pub fn build_swarm(config: SwarmConfig) -> Result { .subscribe(&aggregation_topic) .unwrap(); + // Subscribe to the block-body-proof topic (all nodes: any node may be the + // next slot's proposer, and a proposer adopts a body proof it received). + let block_body_proof_topic = block_body_proof_topic(); + swarm + .behaviour_mut() + .gossipsub + .subscribe(&block_body_proof_topic) + .unwrap(); + // The committee metric should reflect validator membership only, not // aggregator-only subscriptions. let metric_subnet = config @@ -426,6 +438,7 @@ pub fn build_swarm(config: SwarmConfig) -> Result { attestation_committee_count: config.attestation_committee_count, block_topic, aggregation_topic, + block_body_proof_topic, bootnode_addrs, }) } @@ -475,6 +488,7 @@ impl P2P { attestation_committee_count: built.attestation_committee_count, block_topic: built.block_topic, aggregation_topic: built.aggregation_topic, + block_body_proof_topic: built.block_body_proof_topic, connected_peers: HashSet::new(), pending_root_requests: HashMap::new(), outbound_requests: HashMap::new(), @@ -519,6 +533,7 @@ pub struct P2PServer { pub(crate) attestation_committee_count: u64, pub(crate) block_topic: libp2p::gossipsub::IdentTopic, pub(crate) aggregation_topic: libp2p::gossipsub::IdentTopic, + pub(crate) block_body_proof_topic: libp2p::gossipsub::IdentTopic, pub(crate) connected_peers: HashSet, pub(crate) pending_root_requests: HashMap, @@ -639,6 +654,12 @@ impl Handler for P2PServer { } } +impl Handler for P2PServer { + async fn handle(&mut self, msg: PublishBlockBodyProof, _ctx: &Context) { + publish_block_body_proof(self, msg.body_proof).await; + } +} + impl Handler for P2PServer { async fn handle(&mut self, msg: FetchBlock, _ctx: &Context) { let root = msg.root; diff --git a/crates/net/p2p/src/metrics.rs b/crates/net/p2p/src/metrics.rs index 01c4685e..df7b0fea 100644 --- a/crates/net/p2p/src/metrics.rs +++ b/crates/net/p2p/src/metrics.rs @@ -96,6 +96,25 @@ static LEAN_GOSSIP_AGGREGATION_SIZE_BYTES: LazyLock = LazyLock::ne .unwrap() }); +static LEAN_GOSSIP_BLOCK_BODY_PROOF_SIZE_BYTES: LazyLock = LazyLock::new(|| { + register_histogram_vec!( + "lean_gossip_block_body_proof_size_bytes", + "Bytes size of a gossip block body proof message", + &["compression"], + vec![ + 10_000.0, + 50_000.0, + 100_000.0, + 250_000.0, + 500_000.0, + 1_000_000.0, + 2_000_000.0, + 5_000_000.0 + ] + ) + .unwrap() +}); + /// Observe the size of a gossip block message, recording both the raw SSZ /// size and the snappy-compressed on-wire size. pub fn observe_gossip_block_size(raw: usize, snappy: usize) { @@ -118,6 +137,17 @@ pub fn observe_gossip_attestation_size(raw: usize, snappy: usize) { .observe(snappy as f64); } +/// Observe the size of a gossip block body proof message, recording both the +/// raw SSZ size and the snappy-compressed on-wire size. +pub fn observe_gossip_block_body_proof_size(raw: usize, snappy: usize) { + LEAN_GOSSIP_BLOCK_BODY_PROOF_SIZE_BYTES + .with_label_values(&["raw"]) + .observe(raw as f64); + LEAN_GOSSIP_BLOCK_BODY_PROOF_SIZE_BYTES + .with_label_values(&["snappy"]) + .observe(snappy as f64); +} + /// Observe the size of a gossip aggregated attestation message, recording both /// the raw SSZ size and the snappy-compressed on-wire size. pub fn observe_gossip_aggregation_size(raw: usize, snappy: usize) { diff --git a/crates/net/p2p/src/req_resp/handlers.rs b/crates/net/p2p/src/req_resp/handlers.rs index 56913ad0..fe13136c 100644 --- a/crates/net/p2p/src/req_resp/handlers.rs +++ b/crates/net/p2p/src/req_resp/handlers.rs @@ -591,7 +591,7 @@ mod tests { use ethlambda_storage::{ForkCheckpoints, backend::InMemoryBackend}; use ethlambda_types::constants::DEFAULT_MILLISECONDS_PER_SLOT; use ethlambda_types::{ - block::{Block, BlockBody, MultiMessageAggregate}, + block::{Block, BlockBody, BlockProof}, state::State, }; use std::sync::Arc; @@ -605,7 +605,7 @@ mod tests { state_root: H256::ZERO, body: BlockBody::default(), }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), } } diff --git a/crates/net/rpc/src/lib.rs b/crates/net/rpc/src/lib.rs index 32858471..09a71bfc 100644 --- a/crates/net/rpc/src/lib.rs +++ b/crates/net/rpc/src/lib.rs @@ -462,7 +462,7 @@ mod tests { #[tokio::test] async fn test_get_latest_finalized_block() { use ethlambda_types::{ - block::{Block, BlockBody, MultiMessageAggregate, SignedBlock}, + block::{Block, BlockBody, BlockProof, SignedBlock}, checkpoint::Checkpoint, primitives::{H256, HashTreeRoot as _}, }; @@ -486,7 +486,7 @@ mod tests { let block_root = block.header().hash_tree_root(); let signed_block = SignedBlock { message: block, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; // Persist the signed block and mark it as the latest finalized checkpoint. @@ -530,7 +530,7 @@ mod tests { #[tokio::test] async fn test_get_latest_finalized_block_serves_genesis_with_placeholder_proof() { - use ethlambda_types::block::{MultiMessageAggregate, SignedBlock}; + use ethlambda_types::block::{BlockProof, SignedBlock}; use libssz::SszEncode; // Genesis-anchored store: `init_store` writes the header + state but no @@ -555,7 +555,7 @@ mod tests { .unwrap(); let expected = SignedBlock { message: genesis_block.message.clone(), - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), }; let expected_ssz = expected.to_ssz(); diff --git a/crates/storage/src/store.rs b/crates/storage/src/store.rs index 1d4ba5e2..3be3f5c2 100644 --- a/crates/storage/src/store.rs +++ b/crates/storage/src/store.rs @@ -13,9 +13,7 @@ use ethlambda_types::{ AggregatedAttestation, AggregationBits, AttestationData, HashedAttestationData, bits_is_subset, bits_same_set, validator_indices, }, - block::{ - Block, BlockBody, BlockHeader, MultiMessageAggregate, SignedBlock, SingleMessageAggregate, - }, + block::{Block, BlockBody, BlockHeader, BlockProof, SignedBlock, SingleMessageAggregate}, chain_config::ChainConfig, checkpoint::Checkpoint, constants::INTERVALS_PER_SLOT, @@ -356,6 +354,32 @@ type BlockRootIndexChanges = (Vec, Vec); struct ForkChoiceState { known_votes: HashMap, new_votes: HashMap, + /// Latest vote per validator that a BLOCK has carried, as opposed to one + /// this node merely learned about. + /// + /// Maintained only from `insert_signed_block`, never from gossip or the + /// aggregation pool, which is the whole point: the vote maps and the + /// payload buffers advance in lockstep from the same data, at both stages. + /// [`Store::insert_new_aggregated_payload`] writes `new_votes` and + /// `new_payloads` in one call, and [`Store::promote_new_aggregated_payloads`] + /// then drains `new_votes` into `known_votes` and `new_payloads` into + /// `known_payloads`, also in one call. So `known_votes` and the pool that + /// `known_aggregated_payloads` serves are two views of the same set of + /// votes, and a candidate body built out of that pool can never carry a + /// vote newer than `known_votes`. Scoring a body's head-vote value against + /// `known_votes` therefore always yields zero and is the wrong question. + /// The one that matters when deciding what to PACK is whether the chain + /// already carries the vote, which is exactly what this map answers. + /// + /// Not fork-aware: a vote carried by a block on an abandoned branch still + /// counts as on-chain here. That only ever makes a vote look less novel + /// than it is, so the failure mode is packing slightly less rather than + /// double-counting, and keeping it fork-aware would cost an ancestor walk + /// per scoring call to save nothing at the scale we run at. + /// + /// Bounded by the validator-set size: one entry per validator, replaced in + /// place, so it needs no pruning as the chain advances. + on_chain_votes: HashMap, } /// Bounded buffer for gossip signatures with FIFO eviction. @@ -1316,12 +1340,12 @@ impl Store { let sig_key = encode_slot_root_key(header.slot, root); let proof = match view.get(Table::BlockProof, &sig_key).expect("get") { Some(proof_bytes) => { - MultiMessageAggregate::from_ssz_bytes(&proof_bytes).expect("valid block proof") + BlockProof::from_ssz_bytes(&proof_bytes).expect("valid block proof") } // Synthesis only covers the genesis-style anchor (slot 0). For any // other slot a missing proof (pruned finalized block, or genuine // corruption) surfaces as `None` rather than a fabricated block. - None if header.slot == 0 => MultiMessageAggregate::default(), + None if header.slot == 0 => BlockProof::default(), None => return None, }; @@ -1501,12 +1525,6 @@ impl Store { // ============ Attestation Extraction ============ - fn should_replace_vote(existing: &AttestationData, candidate: &AttestationData) -> bool { - candidate.slot > existing.slot - || (candidate.slot == existing.slot - && candidate.hash_tree_root() > existing.hash_tree_root()) - } - fn record_vote( votes: &mut HashMap, validator_id: u64, @@ -1514,12 +1532,20 @@ impl Store { ) { let should_replace = votes .get(&validator_id) - .is_none_or(|existing| Self::should_replace_vote(existing, data)); + .is_none_or(|existing| data.supersedes(existing)); if should_replace { votes.insert(validator_id, data.clone()); } } + /// Record a block's attestations as fork-choice votes. + /// + /// Called from `insert_signed_block` only, so it doubles as the one place + /// that learns a vote is now ON CHAIN. Both maps are updated: `known_votes` + /// is what fork choice weighs, `on_chain_votes` is the baseline block + /// production scores a candidate body's head-vote value against. See + /// [`ForkChoiceState::on_chain_votes`] for why the two cannot be the same + /// map. fn record_known_attestation_votes(&self, attestations: &[AggregatedAttestation]) { let mut fork_choice = self.fork_choice.lock().unwrap(); for attestation in attestations { @@ -1529,6 +1555,11 @@ impl Store { validator_id, &attestation.data, ); + Self::record_vote( + &mut fork_choice.on_chain_votes, + validator_id, + &attestation.data, + ); } } } @@ -1538,6 +1569,18 @@ impl Store { self.fork_choice.lock().unwrap().known_votes.clone() } + /// Extract the latest vote per validator that a block has already carried. + /// + /// The baseline for scoring how much fork-choice weight a candidate body + /// would ADD to the chain. Deliberately not + /// [`Self::extract_latest_known_attestations`]: that map is written in + /// lockstep with the aggregated-payload pool bodies are built from, so + /// scoring against it reports zero for every candidate. See + /// [`ForkChoiceState::on_chain_votes`]. + pub fn extract_on_chain_votes(&self) -> HashMap { + self.fork_choice.lock().unwrap().on_chain_votes.clone() + } + /// Extract per-validator latest attestations from new (pending) payloads. pub fn extract_latest_new_attestations(&self) -> HashMap { self.fork_choice.lock().unwrap().new_votes.clone() @@ -1970,7 +2013,7 @@ mod tests { state_root: H256::ZERO, body: BlockBody::default(), }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), } } @@ -1989,7 +2032,7 @@ mod tests { attestations: attestations.try_into().unwrap(), }, }, - proof: MultiMessageAggregate::default(), + proof: BlockProof::default(), } } @@ -2144,6 +2187,108 @@ mod tests { assert_eq!(votes[&3], data); } + /// The pool and `known_votes` are written in lockstep, so a candidate body + /// built from the pool can never carry a vote newer than `known_votes`. + /// `on_chain_votes` must NOT move with them, or head-vote scoring reports + /// zero for every candidate and the whole axis is dead. + /// + /// This is the regression guard for exactly that: scoring a body's + /// head-vote value against `known_votes` looks reasonable and silently + /// always returns nothing. + #[test] + fn aggregated_payloads_move_known_votes_but_never_on_chain_votes() { + let mut store = Store::test_store(); + let data = make_att_data_for_target(8, root(8)); + + store.insert_new_aggregated_payload( + HashedAttestationData::new(data.clone()), + make_proof_for_validator(0), + ); + store.promote_new_aggregated_payloads(); + + assert_eq!( + store.extract_latest_known_attestations()[&0], + data, + "the pool write must reach the fork-choice map" + ); + assert!( + store.extract_on_chain_votes().is_empty(), + "no block has carried this vote, so it is not on chain" + ); + } + + /// The other half: a block import is what makes a vote on-chain, and it + /// must move BOTH maps. + #[test] + fn insert_signed_block_records_on_chain_votes() { + let mut store = Store::test_store(); + let data = make_att_data_for_target(8, root(8)); + let block = signed_block_with_attestations( + 1, + H256::ZERO, + vec![AggregatedAttestation { + aggregation_bits: make_proof_for_validators(&[1, 3]).participants, + data: data.clone(), + }], + ); + let block_root = block.message.hash_tree_root(); + + store + .insert_signed_block(block_root, block) + .expect("insert signed block"); + + let on_chain = store.extract_on_chain_votes(); + assert_eq!(on_chain[&1], data); + assert_eq!(on_chain[&3], data); + assert_eq!( + store.extract_latest_known_attestations()[&1], + data, + "a block import still feeds fork choice as before" + ); + } + + /// A pooled vote strictly newer than what the chain carries must read as + /// new against the on-chain baseline. This is the property the whole fix + /// turns on: if it fails, candidates score zero again. + #[test] + fn a_pooled_vote_newer_than_the_chain_supersedes_the_on_chain_baseline() { + let mut store = Store::test_store(); + let on_chain_data = make_att_data_for_target(8, root(8)); + let block = signed_block_with_attestations( + 1, + H256::ZERO, + vec![AggregatedAttestation { + aggregation_bits: make_proof_for_validator(0).participants, + data: on_chain_data.clone(), + }], + ); + store + .insert_signed_block(block.message.hash_tree_root(), block) + .expect("insert signed block"); + + // A later attestation from the same validator, still only in the pool. + let fresher = make_att_data_for_target(9, root(9)); + store.insert_new_aggregated_payload( + HashedAttestationData::new(fresher.clone()), + make_proof_for_validator(0), + ); + store.promote_new_aggregated_payloads(); + + let on_chain = store.extract_on_chain_votes(); + assert_eq!( + on_chain[&0], on_chain_data, + "the pool must not advance the on-chain baseline" + ); + assert!( + fresher.supersedes(&on_chain[&0]), + "the pooled vote must read as new against what the chain carries" + ); + assert!( + !fresher.supersedes(&store.extract_latest_known_attestations()[&0]), + "and must read as NOT new against known_votes, which is the bug this fixes" + ); + } + #[test] fn prune_old_block_proofs_within_retention() { let backend = Arc::new(InMemoryBackend::new()); @@ -3200,7 +3345,7 @@ mod tests { .expect("genesis block must be retrievable with synthetic proof"); assert_eq!(signed.message.slot, 0); - assert_eq!(signed.proof, MultiMessageAggregate::default()); + assert_eq!(signed.proof, BlockProof::default()); } /// The synthesis branch must be confined to the slot-0 anchor: a diff --git a/docs/architecture.md b/docs/architecture.md index ed59e183..9b0eaa57 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -67,11 +67,11 @@ order. That walk and the actor split the duties: | Interval | In `store::on_tick` | In the actor | | --- | --- | --- | -| 0 | accept new attestations, if we propose this slot | nothing: the build ran at the previous interval 4 | +| 0 | accept new attestations, if we propose this slot | assemble and publish our block from the candidate body proofs on hand | | 1 | nothing | produce attestations | | 2 | nothing | publish the aggregates the worker produced | | 3 | update the safe target | nothing | -| 4 | accept accumulated attestations | build and publish the next slot's block | +| 4 | accept accumulated attestations | build and gossip the next slot's candidate body proof (on the worker) | See [Slots and Intervals](./slots_and_intervals.md) for what each duty means at the protocol level, and why the proposer builds one interval early. @@ -118,6 +118,27 @@ delay the aggregate the slot is waiting on. From interval 2 on, whatever is in h aggregated. Separately, the actor raises a pause flag around its own block build, since that competes for the same prover. +### Proposing from a gossiped body + +The heavy part of proposing was never picking attestations, it was merging their proofs +into the one aggregate a block body carries. That merge does not involve the block root — +the proposer's signature rides beside the aggregate in `BlockProof`, not inside it — so +whoever holds the proofs can do it, before the block exists. + +ethlambda splits proposing along that line. During the head-update interval each aggregator +packs a candidate body for the next slot, merges its proofs on the aggregation worker, and +gossips the pair as a `BlockBodyProof` (`crates/blockchain/src/body_proof.rs`). The next +slot's proposer keeps a bounded buffer of what arrives — its own worker's candidate +included — and at the slot boundary adopts the most valuable one that survives its checks: +votes it can place on the chain it is extending, attestations its own state transition +accepts, an aggregate that verifies. Nothing qualifying means an empty block, which is a +real option rather than a failure: an attestation-less block carries no aggregate and needs +no prover call. + +The proposer verifies before it signs, not after. XMSS signing keys are one-time, so a +proposer gets exactly one signature per slot and cannot discover a bad candidate by +importing the block and seeing whether it sticks. + Block import runs a second, smaller aggregation path. `reaggregate.rs` splits an imported block's merged proof back into per-attestation aggregates and folds them into the local pool, which is how a node that only saw a vote inside a block gets its fork-choice weight. diff --git a/docs/benchmarking.md b/docs/benchmarking.md index 924437b7..62e9e84d 100644 --- a/docs/benchmarking.md +++ b/docs/benchmarking.md @@ -1,7 +1,14 @@ # Benchmarking block building -`ethlambda benchmark` measures block building the way the node performs it when -it proposes, against a reproducible synthetic workload, with no devnet running. +`ethlambda benchmark` measures block building — packing a body out of the +attestation pool and sealing it — against a reproducible synthetic workload, +with no devnet running. + +That path is now the aggregation worker's, not the proposer's: since +[block body proofs](./slots_and_intervals.md#block-body-proofs), a proposer +adopts a candidate body someone else packed. What the benchmark measures is +unchanged, but read its numbers as the cost of producing a candidate body proof +rather than the cost of a proposal. Block building is otherwise only observable through the Prometheus histograms a live node exports. Those are noisy, depend on whatever the network happened to diff --git a/docs/data_storage.md b/docs/data_storage.md index 506fdb5c..25b3d71e 100644 --- a/docs/data_storage.md +++ b/docs/data_storage.md @@ -120,7 +120,7 @@ The eight variants of the `Table` enum (`crates/storage/src/api/tables.rs`): | ----------------- | ----------- | ----------------------------------------- | -------------------------------- | | `BlockHeaders` | root | `BlockHeader` | never | | `BlockBodies` | root | `BlockBody` | never | -| `BlockProof` | slot ‖ root | aggregate proof (`MultiMessageAggregate`) | yes: finalized older than ~1 day | +| `BlockProof` | slot ‖ root | `BlockProof` (proposer signature + attestation aggregate) | yes: finalized older than ~1 day | | `BlockRoots` | slot | block root (`H256`) | never | | `States` | root | full `State` snapshot | never | | `StateDiffs` | root | `StateDiff` | never | @@ -160,8 +160,9 @@ anchors, whose bodies are either empty or unavailable. Never pruned. ### BlockProof -`slot ‖ root → MultiMessageAggregate`. This table stores the block's **merged -aggregate proof blob**. It is keyed by `slot ‖ root` so that pruning can scan in +`slot ‖ root → BlockProof`. This table stores the block's **proof pair**: the +proposer's raw XMSS signature over the block root, plus the aggregate over the +body's attestations. It is keyed by `slot ‖ root` so that pruning can scan in slot order and stop early. Stored separately from headers/bodies because the genesis block has no proof. diff --git a/docs/metrics.md b/docs/metrics.md index d8d9dee2..a0a978ee 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -118,10 +118,29 @@ The metrics below are not part of the [leanMetrics specification](https://github |------|------|-------|-------------------------|--------|---------| | `lean_aggregated_proof_size_bytes` | Histogram | Bytes size of an aggregated signature proof's `proof_data` field | On aggregated signature production | | 1024, 4096, 16384, 65536, 131072, 262144, 524288, 1048576 | +### Block Body Proofs + +Candidate block bodies and the aggregate binding their attestations, gossiped by +aggregators during the head-update interval for the next slot's proposer to adopt. See +[Slots and Intervals](./slots_and_intervals.md#interval-4-head-update). + +| Name | Type | Usage | Sample collection event | Labels | Buckets | +|------|------|-------|-------------------------|--------|---------| +| `lean_block_body_proof_building_time_seconds` | Histogram | Time taken to build a candidate block body proof (the attestation Type-1 → Type-2 merge) | On body proof production | | 0.1, 0.25, 0.5, 0.75, 1, 1.5, 2, 3, 4 | +| `lean_block_body_proof_candidates` | Histogram | Candidate body proofs a proposer had to choose from; `0` means the block could only be empty | On block production | | 0, 1, 2, 3, 4, 6, 8 | +| `lean_block_body_source_total` | Counter | Where a proposed block's body came from | On block production | source=body_proof,empty | | +| `lean_block_body_proof_rejected_total` | Counter | Candidate body proofs a proposer rejected | On block production | reason=off_chain_vote,state_transition,verification | | + +Since the proposer no longer packs a body, `lean_block_building_time_seconds` now measures +assembly only — choosing a candidate, verifying it, signing — and the merge it used to +include shows up in `lean_block_body_proof_building_time_seconds` on whichever node built +the candidate. + ### Network Sizes | Name | Type | Usage | Sample collection event | Labels | Buckets | |------|------|-------|-------------------------|--------|---------| +| `lean_gossip_block_body_proof_size_bytes` | Histogram | Bytes size of a gossip block body proof message (raw SSZ or snappy on-wire) | On gossip body proof send/receive | compression=raw,snappy | 10000, 50000, 100000, 250000, 500000, 1000000, 2000000, 5000000 | | `lean_gossip_block_size_bytes` | Histogram | Bytes size of a gossip block message (raw SSZ or snappy on-wire) | On gossip block send/receive | compression=raw,snappy | 10000, 50000, 100000, 250000, 500000, 1000000, 2000000, 5000000 | | `lean_gossip_attestation_size_bytes` | Histogram | Bytes size of a gossip attestation message (raw SSZ or snappy on-wire) | On gossip attestation send/receive | compression=raw,snappy | 512, 1024, 2048, 4096, 8192, 16384 | | `lean_gossip_aggregation_size_bytes` | Histogram | Bytes size of a gossip aggregated attestation message (raw SSZ or snappy on-wire) | On gossip aggregation send/receive | compression=raw,snappy | 1024, 4096, 16384, 65536, 131072, 262144, 524288, 1048576 | diff --git a/docs/slots_and_intervals.md b/docs/slots_and_intervals.md index 3b01e45f..48d84262 100644 --- a/docs/slots_and_intervals.md +++ b/docs/slots_and_intervals.md @@ -13,7 +13,7 @@ milliseconds rather than expressed as a fraction of the slot. | 1 | t+800 ms | [Vote propagation](#interval-1-vote-propagation) | every validator | a signed attestation, on its subnet topic | | 2 | t+1600 ms | [Vote aggregation](#interval-2-vote-aggregation) | aggregators | an aggregated attestation, on the `aggregation` topic | | 3 | t+2400 ms | [Safe target computation](#interval-3-safe-target-computation) | every validator | nothing: local bookkeeping | -| 4 | t+3200 ms | [Head update](#interval-4-head-update) | every validator | nothing: local bookkeeping | +| 4 | t+3200 ms | [Head update](#interval-4-head-update) | every validator | in ethlambda: a candidate block body proof, on the `block_body_proof` topic | ```text ONE SLOT (4000 ms) @@ -24,7 +24,7 @@ milliseconds rather than expressed as a fraction of the slot. │ block │ vote │ vote │safe target │ head │ │ proposal │propagation │aggregation │computation │ update │ └────────────┴────────────┴────────────┴────────────┴────────────┘ - ◄───────────── gossiped ─────────────▶ ◄───── local only ───────▶ + ◄───────────── gossiped ─────────────▶ ◄─ local ─▶ ◄─ gossiped ─▶ ``` The grid comes from a genesis timestamp every node shares, so the schedule needs no @@ -59,11 +59,13 @@ Genesis occupies slot 0, so proposals start at slot 1, and nothing forces a slot filled: a proposer that is offline or too slow leaves an empty slot, and the next block simply points its parent root at an older block. -> **In ethlambda:** block proposal is merged into the previous slot's head-update -> interval: the proposer advances its store to the next slot, builds the block there, -> and holds publication until the slot boundary. That buys the build one extra interval -> of headroom and leaves no actor work at the block-proposal tick itself. The aggregation -> worker is paused for the duration, so the build does not share the prover with it. +> **In ethlambda:** the proposer packs no body of its own. It adopts the most valuable +> of the candidate [block body proofs](#block-body-proofs) gossiped during the previous +> interval — validated against its own state, and only if it is worth more than an empty +> body — or else signs an empty block, which costs no prover work at all. What is left of +> proposing is a state transition, one aggregate verification and one signature, so it +> runs at this tick rather than being prebuilt an interval early. The aggregation worker +> is paused for the duration, so the assembly does not share the prover with it. ## Interval 1: Vote propagation @@ -126,3 +128,27 @@ what keeps a validator's fork-choice view from shifting under it mid-slot. The s is the exception: it reads the unpromoted buffer directly, which is how it stays a view of this slot alone. See [why staged promotion](./lmd_ghost.md#why-staged-promotion) for the reasoning. + +> **In ethlambda:** the promote is followed by a second duty for aggregators, described +> below: pack a candidate body for the next slot and gossip it as a block body proof. + +## Block body proofs + +An ethlambda addition, not in leanSpec. The costly part of proposing is not picking +attestations, it is merging their proofs into the single aggregate a block body carries. +That merge does not depend on the block root — the proposer's signature is carried beside +the aggregate, not inside it — so it need not be done by the proposer, and need not wait +for the slot to open. + +So the promote at interval 4 is followed by a second duty for aggregators: pack a candidate +body for the next slot out of the pool as it now stands, merge its proofs, and gossip the +pair as a `BlockBodyProof` on its own topic. The next slot's proposer collects whatever +arrives and, at the slot boundary, adopts the most valuable candidate that survives its own +checks. + +The proposer keeps the last word. A candidate is dropped if any of its votes does not sit +on the chain the block extends, if its attestations do not survive the state transition, or +if its aggregate fails verification; and it is adopted only if it justifies more, finalizes +more, or adds voters the state does not already have. Otherwise the block +goes out empty, which is cheap enough to be a real option: an attestation-less block +carries no aggregate and needs no prover call. diff --git a/docs/spec_deviations.md b/docs/spec_deviations.md index 48907f61..369e7611 100644 --- a/docs/spec_deviations.md +++ b/docs/spec_deviations.md @@ -18,6 +18,27 @@ grid. - **leanSpec:** `aggregate()` is called inline and synchronously from `tick_interval`, at interval 2 only. It walks every attestation data with fresh evidence, with no worker, no gate, and no separation between producing an aggregate and publishing it. - **Equivalence:** the worker produces the same aggregates over a slot, at different times; what a block may carry is unchanged. Where the two can differ is count: a slot whose proving overruns publishes fewer aggregates than the synchronous path would, which affects how many votes are included rather than signature validity. +## Proposer signature outside the block proof + +The block proof is a pair — the proposer's raw signature and the attestation aggregate — +rather than one merged proof over both. + +- **ethlambda:** `SignedBlock.proof` is a `BlockProof { proposer_signature, attestation_proof }` (`crates/common/types/src/block.rs`). `proposer_signature` is the raw XMSS signature over the block root, verified directly against the proposer's `proposal_pubkey` with the hash-based verifier; `attestation_proof` is the lean-multisig Type-2 over the body's attestations only, and is empty when the block carries none (`verify_block_signatures`, `crates/blockchain/src/store.rs`). +- **leanSpec:** the proposer signature is wrapped as a singleton Type-1 and merged into a single block Type-2 alongside every attestation. +- **Why:** the merged form makes the proposer signature the reason a block needs a prover at all — even an attestation-less one — and it ties the merge to the block root, so nothing can be merged before the block exists. Splitting removes prover work from the empty case entirely and is what makes a gossiped block body proof possible. +- **Consequence:** this is a wire-format divergence. The signature and SSZ fixtures no longer apply, and a node running this cannot interop with one that does not. + +## Block body proofs, and a proposer that packs no body + +Candidate bodies are built by aggregators and gossiped; the proposer adopts one instead of +packing its own. + +- **ethlambda:** during the head-update interval the aggregation worker packs a candidate body for the next slot and merges its attestation proofs into one Type-2, and the actor gossips the pair as a `BlockBodyProof` on `/leanconsensus/{fork_digest}/block_body_proof/ssz_snappy` (`crates/blockchain/src/body_proof.rs`). At the slot boundary the proposer scores the candidates it has collected and adopts the most valuable one — or signs an empty block if none beats one (`choose_body`). +- **leanSpec:** the proposer selects attestations from its own pool, merges their proofs itself, and does both inside its proposal slot. +- **Why:** the merge is the one part of proposing that costs seconds, and it does not need the proposer, the block root, or the slot. Moving it to the aggregators that already hold the proofs takes it off the critical path; the proposer is left with a state transition, a verification and a signature. +- **Safety:** the proposer keeps the last word. A candidate is dropped if any of its votes does not sit on the chain the block extends (`attestation_data_matches_chain` — the state transition does not check those roots), if its attestations do not survive that transition, or if its aggregate fails verification; the state root is computed from the transition rather than trusted. Verification happens before signing: XMSS keys are one-time, so a proposer cannot try a candidate, fail the import, and try another. The screen deliberately stops there: a body is all-or-nothing, so dropping one for a merely stale entry would cost the whole block, and staleness is already discounted by the adoption score. +- **Consequence:** a proposer whose candidates all fail, or that received none, proposes an empty block instead of packing what its own pool holds. On a chain with no aggregator gossiping body proofs, every block is empty. + ## Attestation scoring on block building Attestations are scored and selected when packing a block, rather than taken in