The docker-compose we use is here:

https://github.com/btcpayserver/BTCPayServer.Lightning/blob/master/tests/docker-compose.yml

btcpayserver/lightning:v0.9.3-1-dev is with my workaround.

You can easily test what happen by:

1. Make your change locally
2. Create a dummy commit (you need to commit because the docker build is cloning the repo)
3. docker build -t btcpayserver/lightning:v0.9.3-1-dev .
4. docker-compose down -v
5. docker-compose up -d
6. docker logs <lightning_container>

the docker-compose down -v make sure that the bug will be reproduced, as it makes sure bitcoind will call bitcoin-wallet create when starting, which exposed the bug.

Note that sometimes it works, so you need another round of down/up.

@cdecker I am not sure. I tried to use rpcwait, locally and as you pointed out, it was hanging. However, before trying my current solution, I tried inserting rpcwait directly in the gather_args function, and it did not worked.

Maybe it is because the bitcoin-cli we are using is rather old (0.18). Though I checked it had support for rpcwait.

Filtering for the last x forwards, settled vs failed would be nice as well. With a large node, listforwards takes several seconds and filtering/sorting with jq several more.

If anyone was wondering about resolving the AssumeUTXO changes in #19806 with the deglobalizing-chainman changes, here's an update: https://github.com/bitcoin/bitcoin/pull/19806#issuecomment-768673523.

The conclusion was that the PRs are mergeable in any order without much pain if the last commit of the AssumeUTXO changes in #19806 are slightly modified.

<!--cf906140f33d8803c4a75a2196329ecb--> 🐙 This pull request conflicts with the target branch and needs rebase.

_{Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".}

<!--cf906140f33d8803c4a75a2196329ecb--> 🐙 This pull request conflicts with the target branch and needs rebase.

_{Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".}

<!-- Note: This website is for bug reports, not general questions. Do not post issues about non-bitcoin versions of Electrum. -->

My Electrum finally updated to latest version today, and now, when I try to send funds, I get this error: " The server returned an error when broadcasting the transaction. Consider trying to connect to a different server, or updating Electrum. " Unknown error

My username is Wraydawn.

Looking for concept ACKs

Rationale: allow users to inspect the contents of their newly created descriptor wallets.

Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:

• add an option to export xprv
• with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes

The output is compatible with importdescriptors command so it could be easily used for backup/recover purposes.

Output example:

[
  {
    "desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
    "timestamp": 1296688602,
    "active": false,
    "range": [
      0,
      999
    ],
    "next": 0
  }
]

re-utACK

Hi all, I've had a chat offline with @jamesob about potential conflicts between this PR (#19806) and all of chainman-deglobalizing (#20158).

Thankfully, there is only one substantial conflict of note: the commit 385cb331bbf48cfba7b1e77e180d856100e1fdf1 pushed to #19806 yesterday which added lock annotations to the ActiveChain{,state}() suite of methods makes both rebase orders significantly harder, as it imposes additional locking requirements on its callers which are not easily scripted-diffable.

What I propose is that for now in #19806, we just lock ::cs_main inside ActiveChain{,state}() (mirroring how ::Chain{,state}Active() works) so that the two PRs don't block each other. I have a modified version of #19806 here which demonstrates this change: https://github.com/dongcarl/bitcoin/tree/2021-01-au.activate-rebased-with-alt-locking. Of course, the lock annotation is preferable in the longer term, and I'd be happy to publish a separate PR for that.

With my proposed modification, there won't be many conflicts, and I've tested that by playing out both merge orders:

1. Modified #19806 gets merged first, then #20158 (branch: https://github.com/dongcarl/bitcoin/tree/2021-01-au.activate-rebased-with-alt-locking)
2. #20158 gets merged first, then modified #19806 (branch: https://github.com/dongcarl/bitcoin/tree/2021-01-autxo-rebase-me-first)

Both seem to be fairly straightforward, and arrive at the exact same tree (checked with git diff).

TL;DR #19806 and #20158 can be merged in any order without much pain if we redo #19806's 385cb331bbf48cfba7b1e77e180d856100e1fdf1 like https://github.com/dongcarl/bitcoin/commit/08c6ece20ed398d7ed07b095353cb2a4bec7af2b and push the annotations in 385cb331bbf48cfba7b1e77e180d856100e1fdf1 to a followup PR.

Tried rerunning, and it breaks in different ways every time. I can't see how to run a single job, it only lets me do them all :(

Modified the variable name as requested.

Also modified the code to use historical fiat values when viewing an existing transaction's details. Otherwise, "current rate" is used for "Create Transaction" dialog.

Additionally, fiat values are only displayed if the user has actually selected a currency in the fiat settings.

Took me a while (stressing under valgrind) to reproduce this, then longer to figure out how it happened.

Turns out io_new_conn() can fail if the init function fails. In our case, this can happen if connect() immediately returns an error (inside io_connect). But we've already set the finish function, which (if this was the last address), will free connect, making the assignment connect->conn = ... write to a freed address.

Either way, if it fails, try_connect_one_addr() has taken care to update connect->conn, or free connect, and the caller should not do it.

Here's the valgrind trace:

==384981== Invalid write of size 8
==384981==    at 0x11127C: try_connect_one_addr (connectd.c:880)
==384981==    by 0x112BA1: destroy_io_conn (connectd.c:708)
==384981==    by 0x141459: destroy_conn (poll.c:244)
==384981==    by 0x14147F: destroy_conn_close_fd (poll.c:250)
==384981==    by 0x149EB9: notify (tal.c:240)
==384981==    by 0x149F8B: del_tree (tal.c:402)
==384981==    by 0x14A51A: tal_free (tal.c:486)
==384981==    by 0x140036: io_close (io.c:450)
==384981==    by 0x1400B3: do_plan (io.c:401)
==384981==    by 0x140134: io_ready (io.c:423)
==384981==    by 0x141A57: io_loop (poll.c:445)
==384981==    by 0x112CB0: main (connectd.c:1703)
==384981==  Address 0x4d67020 is 64 bytes inside a block of size 160 free'd
==384981==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==384981==    by 0x14A020: del_tree (tal.c:421)
==384981==    by 0x14A51A: tal_free (tal.c:486)
==384981==    by 0x1110C5: try_connect_one_addr (connectd.c:806)
==384981==    by 0x112BA1: destroy_io_conn (connectd.c:708)
==384981==    by 0x141459: destroy_conn (poll.c:244)
==384981==    by 0x14147F: destroy_conn_close_fd (poll.c:250)
==384981==    by 0x149EB9: notify (tal.c:240)
==384981==    by 0x149F8B: del_tree (tal.c:402)
==384981==    by 0x14A51A: tal_free (tal.c:486)
==384981==    by 0x140036: io_close (io.c:450)
==384981==    by 0x1405DC: io_connect_ (io.c:345)
==384981==  Block was alloc'd at
==384981==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==384981==    by 0x149CF1: allocate (tal.c:250)
==384981==    by 0x14A3C6: tal_alloc_ (tal.c:428)
==384981==    by 0x1114F2: try_connect_peer (connectd.c:1526)
==384981==    by 0x111717: connect_to_peer (connectd.c:1558)
==384981==    by 0x1124F5: recv_req (connectd.c:1627)
==384981==    by 0x1188B2: handle_read (daemon_conn.c:31)
==384981==    by 0x13FBCB: next_plan (io.c:59)
==384981==    by 0x140076: do_plan (io.c:407)
==384981==    by 0x140113: io_ready (io.c:417)
==384981==    by 0x141A57: io_loop (poll.c:445)
==384981==    by 0x112CB0: main (connectd.c:1703)

Signed-off-by: Rusty Russell rusty@rustcorp.com.au Fixes: #4343

Needs a release note (can be done in a follow-up.)

IMO m_prefercmcpt better, otherwise name collusion is too close.

AFAIU this comment tries to hint the double opt-in for CB to be bidirectional, maybe it can be clearer. "However, we request new block announcements using CMPCTBLOCK only after receiving a SENDCMPCT from peer."

I think this variable should be better named m_bip152_bandwidth_mode, right now when it sets to false it means low-bandwidth, which is counter-intuitive given variable name and comment (src/net.h, L533).

Maybe we can rename SHORT_IDS_BLOCK_VERSION to SENDCMPCT_VERSION ? A block of short-txn-ids is better known as a compact block or do we have ambiguity here ?

Current usage of fProvidersHeaderAndIDs is confusing to me.

We latched it to true at SENCMPCT reception, following this comment, it does mean this peer has signaled its availability to serve CMPCTBLOCKS.

But per-BIP152 , "Upon receipt of a "sendcmpct" message with the first and second integers set to 1, the node SHOULD announce new blocks by sending a cmpctblock message.", so a SENDCMPCT reception should only modify our behavior and doesn't constitute a commitment of the peer to send us CMPCTBLOCKS ?

I think we should keep to verify NODE_WITNESS existence among peer services, that's a different network namespace than SENDCMPCT, they might diverge.

I believe that's an oversight compared to "Specification for version 2" but at least should avoid to promote a buggy peer as a CB one ?

Edit: reestablished at 06ab570, you should modify 55c9231 message to warn reviewers

Constify test value same as net_processing, CMPCTBLOCKS_VERSION ?

Hello,

i dont know what it is but i wanted to install Bitcoin Core V.0.21.0 but ervery time at a random Point this comes:Corrupted block database detected. and then the program closed and wont work anymore.

My specs:

AMD Ryzen 3600XT Nvidia 2060Ti Gaming X 4TB western digital Windows 10 Home

debug.log

It looks like this

2021-01-26T13:32:25Z Bitcoin Core version v0.21.0 (release build) 2021-01-26T13:32:25Z Qt 5.9.8 (static), plugin=windows (static) 2021-01-26T13:32:25Z System: Windows 10 (10.0), x86_64-little_endian-llp64 2021-01-26T13:32:25Z Screen: \.\DISPLAY1 1920x1080, pixel ratio=2.0 2021-01-26T13:32:25Z Screen: \.\DISPLAY2 1920x1080, pixel ratio=1.0 2021-01-26T13:32:25Z Assuming ancestors of block 0000000000000000000b9d2ec5a352ecba0592946514a92f14319dc2b367fc72 have valid signatures. 2021-01-26T13:32:25Z Setting nMinimumChainWork=00000000000000000000000000000000000000001533efd8d716a517fe2c5008 2021-01-26T13:32:25Z Using the 'shani(1way,2way)' SHA256 implementation 2021-01-26T13:32:25Z Using RdSeed as additional entropy source 2021-01-26T13:32:25Z Using RdRand as an additional entropy source 2021-01-26T13:32:26Z Default data directory C:\Users\Loggorus\AppData\Roaming\Bitcoin 2021-01-26T13:32:26Z Using data directory D:\Bitcoin\chain 2021-01-26T13:32:26Z Config file: D:\Bitcoin\chain\bitcoin.conf (not found, skipping) 2021-01-26T13:32:26Z Using at most 125 automatic connections (2048 file descriptors available) 2021-01-26T13:32:26Z GUI: "registerShutdownBlockReason: Successfully registered: Bitcoin Core wurde noch nicht sicher beendet..." 2021-01-26T13:32:26Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements 2021-01-26T13:32:26Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements 2021-01-26T13:32:26Z Script verification uses 11 additional threads 2021-01-26T13:32:26Z scheduler thread start 2021-01-26T13:32:26Z Using wallet directory D:\Bitcoin\chain 2021-01-26T13:32:26Z init message: Verifiziere Wallet(s)... 2021-01-26T13:32:26Z init message: Lade Sperrliste... 2021-01-26T13:32:26Z SetNetworkActive: true 2021-01-26T13:32:26Z Using /16 prefix for IP bucketing 2021-01-26T13:32:26Z Cache configuration: 2021-01-26T13:32:26Z * Using 2.0 MiB for block index database 2021-01-26T13:32:26Z * Using 8.0 MiB for chain state database 2021-01-26T13:32:26Z * Using 1990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space) 2021-01-26T13:32:26Z init message: Lade Blockindex... 2021-01-26T13:32:26Z Switching active chainstate to Chainstate [ibd] @ height -1 (null) 2021-01-26T13:32:26Z Opening LevelDB in D:\Bitcoin\chain\blocks\index 2021-01-26T13:32:27Z Opened LevelDB successfully 2021-01-26T13:32:27Z Using obfuscation key for D:\Bitcoin\chain\blocks\index: 0000000000000000 2021-01-26T13:32:29Z LoadBlockIndexDB: last block file = 128 2021-01-26T13:32:29Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=621, size=125788462, heights=293039...294068, time=2014-03-29...2014-04-04) 2021-01-26T13:32:29Z Checking all blk files are present... 2021-01-26T13:32:29Z Opening LevelDB in D:\Bitcoin\chain\chainstate 2021-01-26T13:32:30Z Opened LevelDB successfully 2021-01-26T13:32:30Z Using obfuscation key for D:\Bitcoin\chain\chainstate: b7ab9a346d3beacb 2021-01-26T13:32:30Z Loaded best chain: hashBestChain=000000000000000066c41c5c2265a9f821212622215eba24ba7ebf7a3f6ebfef height=293630 date=2014-04-01T20:41:44Z progress=0.059174 2021-01-26T13:32:30Z init message: Verifiziere Blöcke... 2021-01-26T13:32:32Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started 2021-01-26T13:32:32Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s) 2021-01-26T13:32:32Z init message: Verifiziere Blöcke... 2021-01-26T13:32:32Z Verifying last 6 blocks at level 3 2021-01-26T13:32:32Z [0%]...ERROR: ReadBlockFromDisk: Deserialize or I/O error - CAutoFile::read: end of file: iostream error at FlatFilePos(nFile=128, nPos=37424931) 2021-01-26T13:32:32Z ERROR: VerifyDB(): *** ReadBlockFromDisk failed at 293630, hash=000000000000000066c41c5c2265a9f821212622215eba24ba7ebf7a3f6ebfef 2021-01-26T13:32:32Z : Corrupted block database detected. Please restart with -reindex or -reindex-chainstate to recover. 2021-01-26T13:32:46Z init message: Lade Blockindex... 2021-01-26T13:32:46Z should not be overwriting a chainstate 2021-01-26T13:32:46Z Error: Error opening block database 2021-01-26T13:32:48Z Shutdown: In progress... 2021-01-26T13:32:48Z scheduler thread exit 2021-01-26T13:32:48Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started 2021-01-26T13:32:48Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s) 2021-01-26T13:32:49Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) started 2021-01-26T13:32:49Z FlushStateToDisk: write coins cache to disk (0 coins, 0kB) completed (0.00s) 2021-01-26T13:32:50Z Shutdown: done

Try acehack1 on Instagram he can help you

re-ACK 647b81b

Changes since last just address the review comments.

You need to call with CDiskBlockPos, not with CBlockIndex*.

Pushed to fix doxygen comment.

That was unintentional. Fixed.

I have found the problem. Depending on the Linux install you chose, you may have QT5 and QT4 already installed

Doing

sudo apt autoremove '.*qt5.*-dev'
sudo apt autoremove '.*qt4.*-dev'

then install the depends fixes the problem

@NicolasChoukroun Please I am having the same problem. And I tried the commands above , my system could not locate them.

Thanks for the review @ccdle12. Comments addressed. Ready for further review.

That makes sense. Thanks!