Bitcoin Core addrindex + counterparty server on Pi 2

General Watercooler
1

This chart shows times taken for CP to parse recent blockchain blocks.

Pasted image735×556 8.4 KB

There’s no other workload (such as bitcoin CLI and counterparty client).

For some reason every now and then there are huge outliers that I haven’t been to figure out.
For example block https://blockchain.info/block-height/389326 took more than 4,800 seconds and had 1,692 bitcoin transactions and only 2 Counterparty transactions, so it would seem that the slowness comes from Bitcoin Core.

If we consider whether the previous block may have created a backlog of Counterparty transactions, the answer is no - http://blockscan.com/tx?block=389325 had only 7 Counterparty transactions. The same block had 774 Bitcoin transactions so while we may expect it took about 1,500 seconds to digest it (approximately 1/3 of 4,800 seconds), in fact it took only 68 seconds.

Looking at the longest parse times evidently the number of bitcoin transactions is invariably over 1,000 and the number of Counterparty transactions is close to zero.

The green line is smoothed to see if there’s a delay between peaks of BTC transactions that could potentially be attributed to addrindex, but this data series is too short (I looked only at the current server.log).

With this I have only two suspicions:

  • addrindex behavior should be investigated with performance tracing tools
  • it seems that worst delays appear when there’s over 1,000 bitcoin transactions per block

It is important to add that because of this outliers, Counterparty Server is behind the blockchain most of time, so although the chart doesn’t look too bad, it is bad. SSD seems a must-have to run the both services on one Pi.

Maybe I’ll take another look at this when I have nothing else to do :slight_smile:

1 Like
2

have you tried this with “minrelaytxfee=0.0001” ? does this help?

3

I am using the recommended settings (including the one you mentioned) and I do see a lot of rejected garbage transactions in logs, so that’s not entering the blockchain DB.

IO wait times are really where the problem is and they come from Bitcoin Core. But it’s hard to tell:

  • How much of that is addrindex (this depends on the number of transactions, especially with new addresses which adds to write IO)
  • How much of total Bitcoin Core activity is due to Counterparty RPC calls (this also depends on the number of transactions and block size)

Workload characterization is a time consuming process and someone who understands cpp and Python better would be able to do this faster, but even a developer would take some time (otherwise someone would have done it already).

One thing that I’m not using, by the way, is the high number of RPC threads in bitcoin.conf because I discovered that there’s no difference in my case (Pi 2 can barely move, let alone drive 1,000 RPC threads).

To tinker with these counterparty-lib (maybe lower or higher value?) settings and related Bitcoin Core settings may be interesting, but when it takes several hours (!) for counterparty-server to start, it’s easy to blow a day on just this:

DEFAULT_RPC_BATCH_SIZE = 20     # A 1 MB block can hold about 4200 transactions.
BACKEND_RPC_BATCH_NUM_WORKERS = 6

Source: https://github.com/CounterpartyXCP/counterparty-lib/blob/74a21a60f6ba8b8f5f74532c05a6c6595f486e20/counterpartylib/lib/config.py#L93

For the first parameter I’d like to test a value that’s the same as the RPC threads value in bitcoin.conf (4, by default), but I’m still waiting for counterparty-server to start processing blocks (it crashed due to the “transaction not in tx cache” error and I started it before I went to sleep).