Skip to content

Add beacon chain update plugin and cranker executable#280

Open
brianjohnson5972 wants to merge 65 commits into
masterfrom
feature/crank_queues_apy
Open

Add beacon chain update plugin and cranker executable#280
brianjohnson5972 wants to merge 65 commits into
masterfrom
feature/crank_queues_apy

Conversation

@brianjohnson5972
Copy link
Copy Markdown
Contributor

Summary

  • Introduces beacon_chain_update_plugin, a new nodeop plugin that periodically fetches Ethereum beacon chain state from beaconcha.in and pushes updates into on-chain smart
    contracts.
  • Adds cranker, a lightweight standalone executable that runs only the plugins needed for beacon chain updates (no full node required).
  • Adds a cron expression parser to cron_plugin for human-readable schedule configuration.
  • Fixes big-integer type routing in libfc's JSON parser.
  • Adds a bounded-retry identify_block_for_transaction method to ethereum_client.

What the plugin does

beacon_chain_update_plugin runs on a configurable cron schedule and:

  1. Fetches queue lengths from beaconcha.in/api/v2/ethereum/queues — both entry_queue and exit_queue — and calls DepositManager.setEntryQueue and WithdrawalQueue.setWithdrawDelay
    with the estimated wait time in seconds.
  2. Fetches current APY from beaconcha.in/api/v1/ethstore/latest and calls DepositManager.updateApyBPS with the value expressed in basis points.
  3. Finalizes epochs on a separate configurable interval by calling OPP.finalizeEpoch.

All on-chain calls are submitted via the existing outpost_ethereum_client_plugin. Block confirmation is awaited using a new identify_block_for_transaction future.

Configuration options (all via nodeop config):

  • beacon-chain-queue-url — queue endpoint (default: beaconcha.in)
  • beacon-chain-apy-url — APY endpoint (default: beaconcha.in)
  • beacon-chain-api-key — Bearer token for beaconcha.in API
  • beacon-chain-contracts-addrs — JSON map of contract name → address
  • beacon-chain-update-interval — cron schedule for queue/APY updates
  • beacon-chain-finalize-epoch-interval — cron schedule for epoch finalization

Changes by area

plugins/beacon_chain_update_plugin/ (new)

  • Plugin implementation with pimpl pattern; pure utility functions extracted to beacon_chain_update_detail.hpp for testability.
  • Contract client structs (OPP, deposit_manager, withdrawal_queue) wrapping ethereum_contract_client.
  • HTTP fetchers using Boost.Beast over TLS for both the queue and APY endpoints.
  • Loop capture bug fixed: cron job lambdas capture shared_ptr + string by value rather than dangling loop-local references.
  • Unsigned wrap guard in get_queue_length: returns nullopt if the API returns a past timestamp rather than wrapping to a huge value.

plugins/beacon_chain_update_plugin/test/ (new)

  • 15 Boost.Test cases covering get_field_from_object, get_queue_length (including missing fields, wrong types, past timestamps), and apy_fraction_to_bps (including floating-point
    epsilon robustness).

programs/cranker/ (new)

  • Minimal main.cpp that boots signature_provider_manager_plugin, outpost_ethereum_client_plugin, cron_plugin, and beacon_chain_update_plugin — no chain, no p2p.

plugins/cron_plugin/ (new files)

  • cron_parser.hpp/cpp: parses standard 5-field and extended 6-field (milliseconds) cron expressions into cron_service::job_schedule objects, with full range/step/list syntax.

libraries/libfc/src/network/ethereum/ethereum_client.cpp

  • identify_block_for_transaction(tx_hash): polls eth_getTransactionReceipt on a detached thread, returning a std::future<uint64_t> for the block number. Bounded to 600 attempts
    (10 min); captures weak_from_this() to prevent use-after-free if the client is destroyed before the transaction mines.
  • Fixed ABI call encoding: 0x prefix added to encoded call data; to_hex called with with_prefix=true.

libraries/libfc/src/io/json.cpp

  • Fixed big-integer type routing in number_from_stream. The min_len boundary values were all off by one (min_str.size() - 1 instead of min_str.size()), causing 18-digit negative
    numbers ≥ 922... to fall through to int128, 20-digit unsigned values to be routed to uint256 instead of uint128, and the int256/uint256 overflow checks to use inverted direction
    comparisons. All four boundaries and their comparison operators are corrected.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds new infrastructure for periodically fetching Ethereum beacon-chain metrics and pushing them on-chain, plus a lightweight runner executable to operate those updates without a full node.

Changes:

  • Introduces beacon_chain_update_plugin to fetch queue/APY data from beaconcha.in and submit contract updates via the outpost Ethereum client.
  • Adds cranker as a standalone executable that initializes only the plugin subset required for beacon-chain updates.
  • Adds a cron expression parser to cron_plugin and updates libfc Ethereum/JSON utilities (tx block identification + big-integer JSON parsing fixes).

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
programs/cranker/src/main.cpp New minimal entrypoint that boots only the plugins required for beacon-chain updates.
programs/cranker/README.md Documents cranker usage and CLI configuration.
programs/cranker/CMakeLists.txt Adds build target for the cranker executable.
programs/CMakeLists.txt Adds cranker subdirectory to the programs build.
plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp Updates client parsing and makes getters const.
plugins/outpost_ethereum_client_plugin/include/sysio/outpost_ethereum_client_plugin.hpp Makes client/ABI getters const.
plugins/cron_plugin/test/test_cron_parser.cpp Adds unit tests for cron expression parsing behavior.
plugins/cron_plugin/src/services/cron_parser.cpp Implements cron string parsing into cron_service::job_schedule.
plugins/cron_plugin/include/sysio/services/cron_parser.hpp Exposes cron parser API.
plugins/cron_plugin/CRON_PARSER_USAGE.md Usage documentation for the new cron parser.
plugins/CMakeLists.txt Adds beacon chain update plugin to the plugins build.
plugins/beacon_chain_update_plugin/test/test_beacon_chain_update_plugin.cpp Adds unit tests for beacon-chain update parsing utilities.
plugins/beacon_chain_update_plugin/test/main.cpp Boost.Test entrypoint for beacon-chain update plugin tests.
plugins/beacon_chain_update_plugin/test/CMakeLists.txt Builds and registers the beacon-chain update plugin test binary.
plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Implements the plugin: scheduling, HTTP fetch, contract tx submission, and parsing helpers.
plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_plugin.hpp Declares the beacon chain update plugin.
plugins/beacon_chain_update_plugin/include/sysio/beacon_chain_update_detail.hpp Declares testable parsing/utility helpers (queue length, APY conversion).
plugins/beacon_chain_update_plugin/CMakeLists.txt Builds the new plugin and links required dependencies (cron/outpost/curl).
libraries/libfc/src/network/ethereum/ethereum_client.cpp Fixes call/tx hex prefixing and adds identify_block_for_transaction.
libraries/libfc/include/fc/network/ethereum/ethereum_client.hpp Declares the new identify_block_for_transaction API.
libraries/libfc/src/network/ethereum/ethereum_abi.cpp Adjusts ABI decoding to tolerate missing name (e.g., receive/constructor).
libraries/libfc/include/fc/network/ethereum/ethereum_abi.hpp Adds receive to ABI invoke target type enum/reflection.
libraries/libfc/src/io/json.cpp Fixes numeric routing boundaries for big integers when parsing JSON.
CRON_PARSER_SUMMARY.md High-level documentation summary of the cron parser addition.
cmake/chain-tools.cmake Adds beacon_chain_update_plugin to the default chain_target() link set.
Comments suppressed due to low confidence (2)

plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp:94

  • chain_id is now always passed as a value (default-constructed to 0) even when the spec omits the optional chain id. This changes behavior vs the previous std::optional approach and can cause transactions to be signed with chainId=0. Consider keeping std::optional<fc::uint256> chain_id and only setting it when parts.size() == 4 (otherwise pass std::nullopt into ethereum_client).
      fc::uint256 chain_id;
      fc::ostring chain_id_str;
      if (parts.size() == 4) {
         chain_id_str = parts[3];
         if (chain_id_str.has_value())
            chain_id = fc::to_uint256(chain_id_str.value());
      } else {
         ilog("chainId: none");
      }

      auto  sig_provider = plug_sig->get_provider(sig_id);
      my->add_client(id,
                     std::make_shared<ethereum_client_entry_t>(
                        id,
                        url,
                        sig_provider,
                        std::make_shared<ethereum_client>(sig_provider, url,
                           chain_id)));

cmake/chain-tools.cmake:33

  • Adding beacon_chain_update_plugin to the global chain_target() link list will pull this plugin (and its dependencies like libcurl/OpenSSL) into every executable that uses chain_target (including unrelated programs/tests). If only cranker needs this plugin, prefer linking it only to that target instead of the global list.
            prometheus_plugin
            resource_monitor_plugin
            state_history_plugin
            signature_provider_manager_plugin
            outpost_client_plugin
            outpost_ethereum_client_plugin
            outpost_solana_client_plugin
            beacon_chain_update_plugin
            test_control_api_plugin
            test_control_plugin
            trace_api_plugin
            chain_plugin
            appbase
            ${PLUGIN_DEFAULT_DEPENDENCIES}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread libraries/libfc/src/io/json.cpp
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread libraries/libfc/src/network/ethereum/ethereum_client.cpp Outdated
Comment thread libraries/libfc/src/network/ethereum/ethereum_abi.cpp Outdated
Comment on lines +4 to +6
file(GLOB_RECURSE SRC_FILES src/*.cpp src/*.hpp)

chain_target(${TARGET_NAME} SOURCE_FILES ${SRC_FILES})
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cranker is intended to be lightweight, but using chain_target() will link in the full default plugin set from cmake/chain-tools.cmake (net, producer, state_history, etc.), increasing binary size and dependencies. Consider creating a minimal executable target (add_executable + target_link_libraries) that links only the required libraries/plugins for cranker.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@brianjohnson5972 brianjohnson5972 Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed beacon_chain_update_plugin from chain-tools.cmake, but leaving chain_target

Comment thread programs/cranker/README.md Outdated
Comment thread libraries/libfc/src/io/json.cpp Outdated
parse_components(vo.inputs, "inputs");
parse_components(vo.outputs, "outputs");
bool missed = true;
if(deferred_name) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had initially put this in for reporting what we don't expect to see, and determine if something else needs to be reported. Not sure if we should drop or at least keep it in here while we are still working on ethereum client scripts

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to throwing exception to identify what we may be missing.

Comment thread libraries/libfc/src/network/ethereum/ethereum_client.cpp Outdated
Comment thread libraries/libfc/src/network/ethereum/ethereum_client.cpp Outdated
@brianjohnson5972 brianjohnson5972 requested a review from qhool April 3, 2026 13:39
Comment thread cmake/chain-tools.cmake Outdated
Comment thread libraries/libfc/src/io/json.cpp Outdated
Comment thread libraries/libfc/src/network/ethereum/ethereum_client.cpp Outdated
Copy link
Copy Markdown
Contributor

@heifner heifner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review of cranker/beacon_chain_update_plugin changes. 16 findings across 5 files.

Comment thread plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/wire_eth_maintenance_plugin/src/wire_eth_maintenance_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread libraries/libfc/src/network/ethereum/ethereum_client.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Copy link
Copy Markdown
Collaborator

@jglanz jglanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plus all the other stuff we just discussed

* thread that polls eth_getTransactionReceipt until the receipt is available.
* @throws fc::network::json_rpc::json_rpc_exception if the initial RPC call fails.
*/
std::future<uint64_t> identify_block_for_transaction(const std::string& tx_hash);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

template <typename Fn, typename ... Args>
std::expected<decltype(Fn(Args...)), fc::exception> retry(retry_config_t config, Fn fn, Args&& ...) { ... }

Comment thread libraries/libfc/src/io/json.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Comment thread plugins/beacon_chain_update_plugin/src/beacon_chain_update_plugin.cpp Outdated
Copy link
Copy Markdown
Collaborator

@jglanz jglanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took 2 secs to look and saw some glaring issues. Remove the curl dep - rebase on to feature/opp-part2 - this should have near 0 impact - remove changes from sig prov plugin, etc. the 2 modifications to existing code that we discussed was adding receive and enhancing the cron parsing, but i even question the need for that

  • curl should not be a dep for fc,i know its used for clio, but thats legacy

brianjohnson5972 and others added 21 commits April 28, 2026 11:28
…nto `wire-sysio` removing a potential circular dep with `wire-libraries-ts`

## Overview

Moved `protoc-gen-<solana|solidity>` plugins and protobuf-bundler into `wire-sysio` removing a potential circular dep with `wire-libraries-ts`

Additionally, it adds a build options `BUILD_OPP_BUNDLES` (default is `ON`), which allows gating of the OPP bundler

The output by default is `<wire-sysio>/build/opp/<solidity|typescript|solana>` and can be link with either `pnpm` or `npm` package managers from there in the case you'd like to use them for local development.
@brianjohnson5972
Copy link
Copy Markdown
Contributor Author

@jglanz here is the description of all the changes to existing files:

cmake/dependencies.boost.cmake

Adding url to boost dependencies for http api support

libraries/custom_appbase/include/sysio/chain/app.hpp

Preventing the app plumbing from revealing the api key.

libraries/libfc/network/ethereum/ethereum_abi.cpp

Added receive for "payable" action. And added support to fc::from_variant for receive contract methods with no name. Added default false to_hex parameter to contract_encode_data to pass to the to_hex function.

libraries/libfc/network/ethereum/ethereum_client.cpp

Added method get_block_for_transaction as a helper function to retrieve the transaction receipt, if it exists, and return the block number that the transaction is part of.

libraries/libfc/src/io/json.cpp

The json processing is (incorrectly) dropping the fact that integer values that are wrapped by quotes, were originally quoted, and is processed just as a bare integer. Fixing this caused a ripple effect in many unit tests and determined that providing support for integers greater than int64 as in256 would be the solution at this time and writing up a Jira task to deal with the complete fix when we are past the critical timeline. Also added returning a variant variable on all paths in that method to ensure ellision is allowed.

plugins/cron_plugin/include/sysio/services/cron_service.hpp

Added blocking_retry

plugins/cron_plugin/src/services/cron_parser.cpp

Added support for cron schedule format: "* * * * " (and claude through in a millisecond version: " * * * * *") to allow configuring a cron schedule for a job taking advantage of the expansive interface for the cron_service.

plugins/outpost_ethereum_client_plugin/src/outpost_ethereum_client_plugin.cpp

Change three existing implicitly const methods to have the explicit "const" so they could be called using a const&.

brianjohnson5972 and others added 4 commits May 5, 2026 09:47
The compiled .wasm/.abi artifacts under contracts/ were modified on this
branch as a build byproduct. Restore them to the origin/master versions so
this branch carries no contract diff into the master merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
*/
void wire_eth_maintenance_plugin::interrupt() {
ilog("interrupt");
app().executor().stop();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done for you by custom_appbase. No need to call it here.

wire_eth_maintenance_plugin& wire_plug = app().get_plugin<wire_eth_maintenance_plugin>();
exe.set_stop_executor_cb([&wire_plug]() {
ilog("Exiting cranker");
wire_plug.interrupt();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed as wire_plug.interrupt() only does what the default does anyway.

for(const auto& client_spec : client_specs) {
ilog("found beacon chain outpost addresses: {}", client_spec);
fc::variant addrs = fc::json::from_file<fc::variant>(client_spec);
const auto addrs_obj = addrs.get_object();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be const auto&

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants