devnet: EVM port scheme + upgrade-flow hardening + evmigration gas auto-sizing#175
Merged
Conversation
Routine version bumps in devnet/go.mod: golang.org/x/crypto 0.48.0->0.49.0, x/net, x/sys, x/term, x/text, klauspost/compress, and genproto pins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Publish EVM JSON-RPC host ports only for EVM-enabled chain versions (chain.version >= evm_from_version, default cutover v1.20.0). A pre-EVM devnet now publishes no EVM ports, which removes the host 8545 bind collision with stray/standalone lumerad processes. Renumber the per-validator EVM host ports to a contiguous block, stride 100, starting at 8645 (8545 left free): http=8645+i*100, ws=8646+i*100, metrics=8647+i*100, geth=8648+i*100 - generators: add DefaultJSONRPCMetricsPort/DefaultGethMetricsPort, a semver-compare helper, and gate the four EVM port mappings on the resolved chain version (with tests for both branches) - config: add metrics_port/geth_metrics_port to validators.json and 0.0.0.0 metrics/geth bind addresses + enable_metrics to config.json - scripts: validator-setup writes metrics-address/geth-metrics-address and enables telemetry on EVM versions; start.sh passes --metrics - docs: document the devnet docker host-port scheme and refresh the Remix and OpenRPC playground per-validator port tables Container-side ports are unchanged (8545/8546/6065/8100); only the host-published mappings differ, so integration tests that read the in-container app.toml need no changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add devnet-remote-upgrade-version: in-place upgrade of a devnet running on REMOTE_DEVNET_HOST to a pre-downloaded version, preserving chain data. When the upgrade crosses the pre-EVM -> EVM boundary (old < cutover <= new), it regenerates docker-compose.yml for the new version (now with the EVM host ports), rsyncs it, and runs apply-evm-ports.sh on the host to reapply the metrics/geth bind config and recreate the containers. Data lives in host bind mounts, so it survives the recreate. Also stage gen-activity-config.toml into the shared release dir during devnet-new-remote-version using rsync --ignore-existing, so the config is present on the host but an edited copy is never clobbered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Design for the version-gated EVM port publishing and the contiguous x100 per-validator host-port renumber. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When -config was not passed, tests-gen-activity loaded the default "gen-activity-config.toml" relative to the current working directory and silently ignored a missing file. Running the staged binary from any other cwd (e.g. /shared/release/tests-gen-activity invoked with cwd=/root) thus skipped the entire config — including chain-id from [common] — leaving the tool to fall back to flag defaults. Resolve the default config next to the executable first (falling back to the cwd-relative default), and log which config was loaded or that none was found so a miss is no longer silent. An explicit -config is untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register the v1.20.0 release (lumera v1.20.0-rc5, supernode v2.6.0-rc1) for devnet version workflows (download/build/upgrade). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an optional "asset" override to the lumera entry of binaries.json; when set, download-binaries.sh uses it verbatim instead of the versioned lumera_<tag>_linux_amd64.tar.gz convention. Configures v1.20.0 to pull the version-less pre-release asset lumera_linux_amd64.tar.gz. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vote-all.sh: include the primary validator (was excluded; proposer is not auto-voted), use deterministic fixed gas (auto+1.3x sat right on a vote's real usage -> random out-of-gas), and top up vesting voters that have 0 spendable. upgrade.sh: derive the upgrade plan height from the live voting period (parsed from the Go duration, e.g. 10m0s) + measured block time so the height lands after voting; resolve the proposal id to active/passed proposals only (never reuse FAILED ones); refuse the binary swap unless the chain actually halted for the upgrade; wait for the halt rather than an unreachable height. submit-upgrade-proposal.sh: set explicit --gas on submit/deposit (default 200k silently failed for proposal submission). submit-params-proposal.sh (new): reusable MsgUpdateParams proposal helper (reads current params, patches one key, dynamic authority/deposit, --gas set). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Spec + bite-sized plan for auto-sizing migration tx gas (--gas auto with a record-count formula fallback, clamped to block max_gas) and documenting validator migration. Migration fees are waived, so gas is an execution limit only; the max_validator_delegations cap is sized to keep a within-cap migration inside the 25M block-gas ceiling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… fallback) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…validator-migration guide
…ements Live devnet migrations (2026-06-23) showed the analytical 200000 + 7000/record fallback was ~100-200x too low: real cost is ~6M base + ~688k/record (validator) / ~1.33M/record (account). Bump MIGRATION_GAS_BASE 200000 -> 6000000 and MIGRATION_GAS_PER_RECORD 7000 -> 1500000 (conservative: uses the higher account marginal with margin). Migration fees are waived and block.max_gas=-1 on devnet and mainnet, so over-estimating the fallback is free; --gas auto remains the accurate path. Update the unit-test expectations accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e from live findings Live devnet measurements (2026-06-23) showed the original analytical gas formula (200000 + 7000/record) was ~100-200x too low; recalibrated to 6M base + 1.5M/record. Block max_gas is -1 (unlimited) on devnet and mainnet — not 25M — so gas is not a blocker. Documents the real constraint: --gas auto simulate takes ~107s for a 5149-record validator, exceeding CometBFT's default 10s timeout_broadcast_tx_commit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5df6623b96
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens devnet ops and evmigration tooling by (1) version-gating and renumbering devnet EVM host ports, (2) improving governance upgrade/voting flow reliability, and (3) making migration broadcasts more reliable via automatic gas sizing and better tx wait/indexing handling.
Changes:
- Version-gated devnet EVM port publishing + new contiguous per-validator ×100 host port scheme (including metrics/geth metrics).
- evmigration scripts:
--gas autowith fallback formula, record-count plumbing, and improvedwait_for_txbehavior when indexing lags. - Multisig migration UX improvements (signer-order visibility, signer index guidance) plus expanded/updated operator documentation and tests.
Reviewed changes
Copilot reviewed 39 out of 40 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/scripts/migrate-multisig.bats | Extends multisig bats assertions to match new signer-order/index guidance output. |
| tests/scripts/fixtures/lumerad-shim.sh | Adds shim behavior to simulate “tx not indexed yet” for query tx to test wait_for_tx fallback polling. |
| tests/scripts/devnet-start.bats | New bats test asserting start.sh run starts lumerad before waiting for blocks. |
| tests/scripts/common.bats | Adds bats coverage for wait_for_tx polling query tx when wait-tx returns early. |
| scripts/migrate-validator.sh | Exposes migration record count for gas fallback sizing. |
| scripts/migrate-multisig.sh | Adds signer-order logging/instructions; combine now simulates gas with --gas auto; improves signer index guidance. |
| scripts/migrate-account.sh | Sets migration record count from estimate for gas fallback sizing. |
| scripts/evmigration-common.sh | Implements migration gas auto-sizing + fallback and improves wait_for_tx reliability with query polling. |
| scripts/evmigration-common_test.sh | Adds pure helper unit checks for gas sizing helpers (standalone bash test). |
| Makefile.devnet | Adds remote in-place upgrade target and related remote staging behaviors. |
| docs/lumera-ports.md | Documents the devnet Docker EVM host-port scheme and version gate; spacing/format fixes. |
| docs/evm-integration/user-guides/validator-migration.md | Updates validator cap value, documents required RPC timeout changes for --gas auto, and post-migration revert guidance. |
| docs/evm-integration/user-guides/migration.md | Adds a validator migration section explaining gas scaling, --gas auto, timeout considerations, and updated caps. |
| docs/evm-integration/user-guides/migration-scripts.md | Updates script guide (formatting + new gas-sizing behavior + multisig signer-order guidance). |
| docs/evm-integration/guides/remix-guide.md | Updates local devnet EVM RPC port references to the new host-port scheme. |
| docs/evm-integration/guides/openrpc-playground.md | Updates port mapping examples and references to the new host-port scheme + version gate note. |
| docs/design/evmigration-scripts-design.md | Updates design doc to reflect gas auto-sizing behavior and fallback/clamp logic. |
| docs/design/2026-06-22-validator-migration-gas-plan.md | Adds implementation plan doc for validator migration gas sizing workstream. |
| docs/design/2026-06-22-validator-migration-gas-design.md | Adds design doc capturing rationale and post-implementation calibration notes. |
| docs/design/2026-06-22-devnet-evm-port-scheme-design.md | Adds design doc for the version-gated EVM port publishing + renumbering scheme. |
| devnet/tests/gen-activity/main.go | Improves config path resolution to prefer config next to the executable; adds logging on config load/miss. |
| devnet/tests/gen-activity/config_path_test.go | Adds unit tests for config path resolution behavior. |
| devnet/scripts/vote-all.sh | Hardens voting behavior: fixed gas default, include primary validator, and adds vesting-voter fee top-up support. |
| devnet/scripts/validator-setup.sh | Wires JSON-RPC metrics/geth metrics addresses and telemetry enablement for EVM builds. |
| devnet/scripts/upgrade.sh | Hardens upgrade flow: duration parsing, voting-period-derived upgrade height, halt-aware waiting/guarding, active-proposal filtering. |
| devnet/scripts/submit-upgrade-proposal.sh | Adds explicit gas to proposal submission and deposit broadcasts. |
| devnet/scripts/submit-params-proposal.sh | New script to submit single-parameter MsgUpdateParams proposals by patching on-chain params. |
| devnet/scripts/start.sh | Starts lumerad before waiting for blocks in run mode; gates --metrics flag on EVM builds. |
| devnet/scripts/download-binaries.sh | Supports version-less release asset filenames via lumera.asset override in binaries config. |
| devnet/scripts/apply-evm-ports.sh | New script to apply EVM port/metrics binding changes to an already-running devnet without wiping data. |
| devnet/go.sum | Updates devnet module dependency checksums for upgraded dependencies. |
| devnet/go.mod | Updates devnet module dependency versions (security/maintenance bumps). |
| devnet/generators/docker-compose.go | Adds semver gating for EVM port publishing and emits metrics/geth metrics host mappings when enabled. |
| devnet/generators/docker-compose_test.go | Adds generator tests asserting EVM-port gating and the new host-port mapping scheme. |
| devnet/generators/config.go | Adds default constants for JSON-RPC metrics and geth metrics container ports. |
| devnet/config/validators.json | Updates per-validator JSON-RPC host ports to ×100 blocks and adds metrics/geth metrics ports. |
| devnet/config/gen-activity-config.toml | New default config staged alongside tests-gen-activity for consistent runs from any cwd. |
| devnet/config/config.json | Adds chain-level JSON-RPC metrics and geth metrics address configuration and metrics enable flag. |
| devnet/config/config.go | Extends validator JSON-RPC config schema to include metrics/geth metrics host ports. |
| devnet/config/binaries.json | Adds a version entry supporting explicit asset filename override. |
Comments suppressed due to low confidence (1)
scripts/migrate-multisig.sh:469
- combine() relies on resolve_chain_id, but (unlike generate/sign/submit) it never checks whether CHAIN_ID was actually resolved before passing
--chain-id "$CHAIN_ID"tocombine-proof. If the RPC endpoint is unreachable or status doesn't return.node_info.network, this ends up invoking the CLI with an empty chain-id value and fails with a less actionable error. Add an explicit post-resolve check with a targeted message.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
a-ok123
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch combines three related devnet and evmigration workstreams.
1. Devnet infrastructure
8545/8645/...).download-binaries, remote EVM-cutover upgrade staging, gen-activity config path fixes, and related docs.lumeradbefore waiting for run-mode block progression.2. evmigration gas auto-sizing
The migration scripts re-key every delegation/unbonding/redelegation, so fixed CLI gas was too low for validators. This branch makes migration broadcasts size gas automatically:
lumerad_txbroadcasts with--gas auto --gas-adjustment 1.5, with a conservative record-count fallback.migrate-multisig.sh combinealso uses--gas auto.3. Multisig migration reliability and operator guidance
Follow-up devnet multisig drills exposed two UX/reliability gaps: tx indexing could make submit look hung even after broadcast, and signer-order mistakes could create an unintended destination address.
wait_for_txnow falls back to visiblequery txpolling, logs progress, and proceeds to on-chain verification when tx indexing times out.migrate-multisig.sh generateprints the on-chain legacy signer order, destination signer order, and explicit same-index signing instructions.migrate-multisig.sh signprints matched legacy/new signer indices and warns before the underlying CLI rejects mismatched signer positions.--nosortsigner ordering, clarify pubkey seeding/fee failure behavior, and expand troubleshooting for wrong destination order, mismatched--new, missing quorum, and tx-index timeout.Live validation
Test plan
bats tests/scripts— 149/149 passing.markdownlint docs/evm-integration/user-guides/migration-scripts.md docs/evm-integration/user-guides/migration.md— passing.git diff --check— passing.