Sangbida/bwatch wallet#9298
Draft
sangbida wants to merge 17 commits into
Draft
Conversation
The next commits move wallet UTXO and tx tracking off chaintopology and onto bwatch. bwatch doesn't maintain a blocks table, but the legacy utxoset, transactions and channeltxs tables all have FOREIGN KEY references into blocks(height) (CASCADE / SET NULL), so we can't just retarget the existing tables. Instead, introduce parallel tables (our_outputs, our_txs) without the blocks(height) FK. The new bwatch-driven code writes only to these, the legacy tables stay populated by the existing code path during this release so downgrade still works, and a future release can drop them once we're past the downgrade window. Schema only here — wallet handlers that write into these tables and the backfill from outputs/transactions land in subsequent commits. Co-authored-by: Cursor <cursoragent@cursor.com>
Helpers for the bwatch watch_found handlers that land in the next commits. They arrive unwired so each handler commit stays small and the tree compiles at every step. The writers deliberately parallel got_utxo() / wallet_transaction_add() instead of replacing them: the legacy outputs/transactions tables must stay populated this release so downgrade works without a rescan. Co-authored-by: Cursor <cursoragent@cursor.com>
Use a single wallet/spk/<keyidx>/<form> owner for wallet address scriptpubkey watches. The form suffix keeps watchman pending-op ids unique when the same HD key is watched as p2wpkh, p2tr and p2sh-p2wpkh. The handler parses the keyindex from the owner suffix and recovers the address form from the matched output script, so all wallet address forms share the same recording path. Co-authored-by: Cursor <cursoragent@cursor.com>
Populate our_outputs/our_txs from the legacy outputs/transactions tables so the bwatch wallet path starts with the same state as the existing wallet. Use outputs rather than utxoset because it already contains only wallet-owned rows and carries wallet metadata like reservations and channel-close info. Downgrade only drops the new tables: later commits keep mirroring writes into the legacy tables, so no copy-back migration is needed. Co-authored-by: Cursor <cursoragent@cursor.com>
Add the wallet/utxo/<txid>:<outnum> dispatch entry: on watch_found, mark the UTXO spent in our_outputs, refresh the spending tx in our_txs, and emit a withdrawal coin movement; on watch_revert, clear spendheight so the UTXO becomes unspent again.
When bwatch sees an unconfirmed wallet change output, keep watching the same scriptpubkey so we get the later confirmation notification. Use a perennial watch because the output is already known; bwatch only needs to tell us if it later appears in a block. Co-authored-by: Cursor <cursoragent@cursor.com>
The legacy tables rely on their blocks(height) ON DELETE SET NULL foreign keys to mark rows unconfirmed/unspent when a block is reorged out. our_outputs/our_txs deliberately carry no blocks FK (bwatch does not maintain a blocks table), so block disconnect and rollback must clear their blockheight/spendheight fields explicitly. Co-authored-by: Cursor <cursoragent@cursor.com>
Make our_outputs the wallet's live UTXO table while mirroring each write into legacy outputs for one-release downgrade support. That keeps old binaries up to date without a downgrade copy-back or rescan. The mirroring stops in the release that removes chaintopology, freezing all the legacy tables at the same height. Co-authored-by: Cursor <cursoragent@cursor.com>
The wallet's live UTXO state now lives in our_outputs; point the raw-SQL assertions at it (spent means spendheight IS NOT NULL, reserved means reserved_til > 0). Co-authored-by: Cursor <cursoragent@cursor.com>
Same raw-SQL switch as test_connection.py: read the wallet's UTXO state from our_outputs. Co-authored-by: Cursor <cursoragent@cursor.com>
Upgrade a pre-bwatch snapshot db and check the migration mirrored outputs/transactions into our_outputs/our_txs, listfunds still reports the old UTXOs, and the wallet can receive and withdraw new funds. Co-authored-by: Cursor <cursoragent@cursor.com>
Now that the bwatch wallet path records every relevant transaction in our_txs, point the wallet's transaction readers at that table. wallet_transaction_add keeps dual-writing the legacy transactions table: the close path still inserts into channeltxs, whose transaction_id foreign key points at transactions(id), and wallet_get_funding_spend joins it. That legacy write can only go away with channeltxs itself. Co-authored-by: Cursor <cursoragent@cursor.com>
Change addresses are no longer bech32-only (p2tr is the default form), so the old name was misleading. Pure rename, no functional change; also drop the unused txfilter.h include. Co-authored-by: Cursor <cursoragent@cursor.com>
The four wallet_datastore_{get,create,update,remove} helpers used to
require the caller to be inside a wallet transaction; otherwise the
underlying db_prepare_v2 fatals at db/utils.c:103 with "Attempting to
prepare a db_stmt outside of a transaction".
watchman persists its pending bwatch ops through these helpers from
plugin callbacks that run outside any transaction, so wrap one on
demand.
Co-authored-by: Cursor <cursoragent@cursor.com>
The flag is registered by the bwatch plugin, not lightningd, so peek at the parsed configvars. Without it ld->watchman stays NULL and the watchman_* entry points are no-ops, leaving chain_topology as the only chain watcher: bwatch and the legacy path must not race each other. The bwatch pytests opt in explicitly (with rescan=0) instead of enabling bwatch globally. Co-authored-by: Cursor <cursoragent@cursor.com>
init_wallet_scriptpubkey_watches walks every HD key (BIP32 + BIP86) up
to {bip32,bip86}_max_index + keyscan_gap and arms a watch for each
form, so bwatch can report deposits from the very first block it
scans. wallet_get_newindex does the same for fresh keys, keeping
coverage as the wallet grows.
The per-UTXO watch on unconfirmed change is now redundant (the
perennial per-key watch already covers that scriptpubkey), so drop it.
Co-authored-by: Cursor <cursoragent@cursor.com>
The wallet no longer writes UTXO state to the legacy outputs table's status/spend_height columns, so tests that peek at the database directly must read our_outputs instead. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
26.09 FREEZE August 5th: Non-bugfix PRs not ready by this date will wait for 26.12.
RC1 is scheduled on August 17th
The final release is scheduled for September 7th.
Checklist
Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:
tools/lightning-downgrade