Repository navigation
perf(crypto): run the Silver-style verification on silver_crypto_simd - #772
Draft
MegaRedHand wants to merge 8 commits into
Draft
MegaRedHand wants to merge 8 commits into
MegaRedHand wants to merge 8 commits into
Conversation
silver_crypto_simd (gattaca-com/silver's crypto crate) runs BLS12-381 point decoding, subgroup checks, pairing products and the PeerDAS cell operations eight lanes at a time on AVX-512 IFMA, which every Zen 4/5 host we run has, and falls back to blst and portable Rust elsewhere through the same API. Its decoders return blst's own verdict on every input, and its cell verifier the spec's, so it can sit behind the existing spec-named functions unchanged. BLS (`ethlambda_crypto::bls`): key and signature decoding, `verify`, `fast_aggregate_verify`, `aggregate_verify` and the block batch now go through it; cache misses are decoded in chunks of `KEY_DECODE_CHUNK` across rayon. Point sums, committee sums, signing and key generation stay on blst, since the crate has no faster route for them. Its pairing product drops a pair holding the identity where blst refused an identity key, so `core_verify` and the batch refuse that key themselves before any pairing (the `P, -P` cancellation tests cover it). KZG (`beacon::kzg`): `compute_cells` and `verify_cell_kzg_proof_batch` run on it; commitments, blob proofs, cell proofs and recovery stay on c-kzg. Its verifier answers false for malformed input, so a rejection is followed by the spec's own input assertions to keep `Err` (fixture `output: null`) apart from `Ok(false)`. The module now has its own `Cell` byte array, converted to c-kzg's type only at calls into c-kzg. The verifier and extender take `&mut self` and are costly to build, so they live in small pools. Differential tests compare every swapped function with blst or c-kzg on valid, corrupt, off-subgroup and malformed inputs; on an IFMA host they exercise the vector kernels.
blst's batch spread each set's hashing, key weighting and Miller loop over its thread pool; the first silver version did all of it on the calling thread and measured 2.2x slower than blst at 12 sets and 4.1x at 64 on ethlambda-5. A batch is now cut into chunks of `LANES - 1` sets, each its own random linear combination (fresh weights, own signature sum, own final exponentiation), checked in parallel with each chunk's hashes in parallel too. A chunk plus its signature-sum pair fills one pass of the Miller-loop lanes, and soundness holds per chunk, since a forged set must still pass its own chunk's check.
…ypto_simd Merge perf/silver-crypto-simd into perf/silver-style-verification and re-point the latter's blst usage patterns at the new backend, so the two speedups compound instead of excluding each other: the H(m) cache, message- grouped batching and registry warm-up keep their shape, and the pairings, hashes, weighted sums and key decoding behind them run on 8-lane IFMA code. - primitives.rs now holds the silver point and pairing calls; the blst hash-to-G2 `unsafe` and the blst Pairing code are gone. Every pairing check still refuses an identity key, since the silver product drops pairs holding one. - hashed_messages caches G2Point, same two generations, cap and metrics. - verify, fast_aggregate_verify and SignatureSet::verify use core_verify with H(m) from the cache; aggregate_verify hashes through it in parallel. - verify_signature_sets sorts by message, groups runs of equal messages and checks chunks of at most LANES - 1 runs in parallel, each its own random linear combination; a chunk whose weighted signature sum is the point at infinity is refused. - warm_pubkeys decodes the cache misses with the batched decoder, one rayon task per decode chunk.
…rayon A chunk holds up to a lane-width of distinct messages, and a cache miss costs a full hash to G2 per message, so computing them one after another left a block's batch waiting on serial hashing that the blst-backed batch already ran in parallel.
|
Hi, thank you for looking to adopt Following up on "Signature decode is not faster" in the description: decoding is now the dominant cost (22.3 ms of the 512-vote batch, against 0.84 ms of pairing). |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #771. Runs the parent's Silver usage patterns on
silver_crypto_simd = "0.1.0"(AVX-512 IFMA, 8 lanes), and supersedes #764, whose backend swap this merges in.What changes on top of the parent
G2Point; key decode, pairings andcompute_cells/verify_cell_kzg_proof_batchrun on silver (CellBatchVerifier/CellExtenderpools). blst stays for sums, signing, keygen and the cachedPublicKeys; c-kzg stays for commitments, blob proofs, cell proofs and recovery.bls/primitives.rsnow holds silver calls and nounsafe. Hash-to-G2 is silver's, and the hash cache storesG2Point.PairingProduct: runs of equal messages are split into chunks of at mostLANES - 1runs. Each chunk is its own random-weight equation, with its own weights and its own refusal of an infinite signature sum. Chunks run in parallel, and so do the runs inside a chunk. An identity key is refused before the product, since silver'sPairingProductdrops identity pairs.CellBatchVerifierthrough the mergedverify_cell_kzg_proof_batch. No p2p or gossip code changes here.Benchmark (ethrex-office-4, Ryzen 9 9950X3D, idle; nightly
2b51a3336as control)compute_cellsµsverifyµs (cached message)eth_aggregate_pubkeys, 256 cached keys, µsThe parent PR has the full tables and caveats.
Known issues
cooldown-checkfails until 2026-11-17: the crate was published 2026-11-03.execution_payloadis slower with the crate present, though that phase has no BLS or KZG: by p50, +0.8 ms cold and +0.2 ms warm. perf(crypto): decode, pair and check cells through silver_crypto_simd #764 showed the same on ethlambda-5. Unexplained.SignatureChecker::check_all); that is not ported.Tests
make lint;ethlambda-cryptolib 67;ethlambda-state-transitionlib 734; beacon spec tests 7493 passed, 977 ignored;ethlambda-p2plib 384;ethlambda-blockchainlib 317;ethlambdabin 287. #764's differential tests against blst and c-kzg are kept.