# Catastrophic Failure

**URL:** <https://forums.counterparty.io/t/catastrophic-failure/5424>\
**Category:** Support\
**Created:** [April 20, 2019, 3:21am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424 "2019-04-20T03:21:00Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 20, 2019, 3:21am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/1 "2019-04-20T03:21:01Z")

</div>

## Where can I find the log files as the node suffered a failure. fednode ps: core@nodeserver:~/federatednode$ fednode ps Name Command State Ports

federatednode\_armory-utxsvr- start.sh Exit 255 6590/tcp, 6591/tcp  
testnet\_1  
federatednode\_armory-utxsvr\_1 start.sh Exit 255 6590/tcp, 6591/tcp  
federatednode\_bitcoin- start.sh Exit 255  
testnet\_1  
federatednode\_bitcoin\_1 start.sh Exit 255  
federatednode\_counterblock- start.sh Exit 0  
testnet\_1  
federatednode\_counterblock\_1 start.sh Exit 0  
federatednode\_counterparty- start.sh Exit 0  
testnet\_1  
federatednode\_counterparty\_1 start.sh Exit 0  
federatednode\_counterwallet\_1 start.sh Exit 255 0.0.0.0:443-\>443/tcp,  
0.0.0.0:80-\>80/tcp  
federatednode\_indexd- npm start Exit 255  
testnet\_1  
federatednode\_indexd\_1 npm start Exit 255  
federatednode\_mongodb\_1 docker-entrypoint.sh mongod Exit 255  
federatednode\_redis\_1 docker-entrypoint.sh redis Exit 255

core@nodeserver:~/federatednode$ fednode start  
This script requires root access (via sudo) to run. Please enter your sudo password below.  
[sudo] password for core:  
Starting bitcoin-testnet … error  
Starting mongodb … error  
Starting redis … error  
Starting bitcoin … error  
Starting counterparty … error  
Starting counterparty-testnet … error  
Starting indexd-testnet … error  
Starting armory-utxsvr … error  
Starting counterblock … error  
Starting armory-utxsvr-testnet … error  
Starting counterblock-testnet … error  
Starting counterwallet … error  
Starting indexd … error

ERROR: for bitcoin-testnet Cannot start service bitcoin-testnet: OCI runtime create failed: container with id exists: ba4ecabed42c49675a78b9d45f073317590447579e69cd52c4ae25c4e3b8e2b4: unknown

ERROR: for mongodb Cannot start service mongodb: OCI runtime create failed: container with id exists: a2b6f7e43d5a798641862e22a4f0a5c435e5a9cce50d78f99bb61ac22aa591c7: unknown

ERROR: for redis Cannot start service redis: OCI runtime create failed: container with id exists: 077f40c179f62a655c89015380e16f4fd92f39ecd745552b00909a8ff6c576d8: unknown

ERROR: for bitcoin Cannot start service bitcoin: OCI runtime create failed: container with id exists: 3714445f6a52e508be36acc5680c9cd55cba3217bc9ef93ea90c1925f043de47: unknown

ERROR: for indexd Cannot start service indexd: OCI runtime create failed: container with id exists: 21ee8361f4fa1d3ba1809e21d7efc0a0043ffc741452df330f4427d8bbfa1145: unknown

ERROR: for indexd-testnet Cannot start service indexd-testnet: OCI runtime create failed: container with id exists: c5406101c363dd408f53b5f0be60f40dd643e4e8d782e0d0b680e7572782262e: unknown  
ERROR: No containers to start

I left the node running with bitcoind and indexd on mainnet, as testnet was giving the following error when starting the indexd-testnet:  
ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos

How to recover from this?

TY

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 21, 2019, 4:33am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/2 "2019-04-21T04:33:21Z")

</div>

Okay, no one?

I will just do a reinstall then…😀

TY

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 22, 2019, 7:52am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/3 "2019-04-22T07:52:49Z")

</div>

Hi, anybody any idea why this keeps hapening when starting indexd:  
bitcoin\_1 | 2019-04-22T07:15:01Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:15:01Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:15:29Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:16:30Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:17:30Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:18:19Z New outbound peer connected: version: 70015, blocks=572732, peer=14  
bitcoin\_1 | 2019-04-22T07:18:31Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:19:30Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:20:30Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:21:31Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:22:32Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:23:31Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)  
bitcoin\_1 | 2019-04-22T07:24:31Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: iostream error at CBlockDiskPos(nFile=0, nPos=124450586)

TY

---

<div class="post-metadata">

**Author:** ![jdogresorg](https://yyz2.discourse-cdn.com/flex004/user_avatar/forums.counterparty.io/jdogresorg/32/1349_2.png) [@jdogresorg](https://forums.counterparty.io/u/jdogresorg)\
**Post date:** [April 25, 2019, 3:22am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/4 "2019-04-25T03:22:04Z")

</div>

This seems to be a disk IO error.

Are you running fednode on hardware or in a VM? Is your hard drive an SSD? This issue seems to pop up on EBS-backed AWS instances.

Also, have you let bitcoin and bitcoin-testnet sync fully before starting up indexd and indexd-testnet?

> <https://github.com/bitcoin/bitcoin/issues/5668>

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 26, 2019, 2:28pm UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/5 "2019-04-26T14:28:56Z")

</div>

Thanks for the reply.

Yes the Fednode is running on metal not VM, while both blockchains are fully synced. I figured it would be the HDD as it is ofc much slower and gets overwhelmed., I will move the Os etc…to a SSD and only use the HDD for the data\_Volumes. I will try if I can get the fednode to run fully only on testnet first. I have Bitcoin-testnet, indexd-testnet, armory-utxsvr-testnet and counterparty-testnet all synced running, Counterblock-testnet gives the following error when starting it up after all are synced: Counterblock is unable to find a file called : GeoIP.dat:  
core@nodeserver:~/federatednode$ fednode tail counterblock-testnet  
Attaching to federatednode\_counterblock-testnet\_1  
counterblock-testnet\_1 | ERROR:server: Unhandled Exception  
counterblock-testnet\_1 | Traceback (most recent call last):  
counterblock-testnet\_1 | File “/usr/local/bin/counterblock”, line 9, in   
counterblock-testnet\_1 | load\_entry\_point(‘counterblock’, ‘console\_scripts’, ‘counterblock’)()  
counterblock-testnet\_1 | File “/counterblock/counterblock/ **init**.py”, line 7, in server\_main  
counterblock-testnet\_1 | server.main()  
counterblock-testnet\_1 | File “/counterblock/counterblock/server.py”, line 146, in main  
counterblock-testnet\_1 | StartUpProcessor.run\_active\_functions()  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/processor/ **init**.py”, line 171, in run\_active\_functions  
counterblock-testnet\_1 | func[‘function’](\*args, \*\*kwargs)  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/modules/counterwallet.py”, line 613, in init  
counterblock-testnet\_1 | module\_config[‘GEOIP’] = pygeoip.GeoIP(geoip\_data\_path)  
counterblock-testnet\_1 | File “/usr/local/lib/python3.5/dist-packages/pygeoip/ **init**.py”, line 118, in **init**  
counterblock-testnet\_1 | self.\_fp = codecs.open(filename, ‘rb’, ENCODING)  
counterblock-testnet\_1 | File “/usr/lib/python3.5/codecs.py”, line 895, in open  
counterblock-testnet\_1 | file = builtins.open(filename, mode, buffering)  
counterblock-testnet\_1 | FileNotFoundError: [Errno 2] No such file or directory: ‘/root/.local/share/counterblock/GeoIP.dat’  
counterblock-testnet\_1 | INFO:log: Logging txes to ‘/root/.cache/counterblock/log/server.testnet.tx.log’  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/assets  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/counterwallet  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/dex  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/transaction\_stats  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/betting  
counterblock-testnet\_1 | INFO:server: counterblock command specified: server  
counterblock-testnet\_1 | INFO:server: counterblock Version 1.4.0 starting …  
counterblock-testnet\_1 | INFO:database: Connecting to mongoDB backend …  
counterblock-testnet\_1 | INFO:cache: Connecting to redis @ redis  
counterblock-testnet\_1 | INFO:counterwallet: Loading config at: /root/.config/counterblock/counterwallet.testnet.conf  
counterblock-testnet\_1 | INFO:counterwallet: Armory-utxsvr hostname: armory-utxsvr-testnet  
counterblock-testnet\_1 | INFO:counterwallet: Checking/updating GeoIP.dat …  
counterblock-testnet\_1 | INFO:counterwallet: Downloading GeoIP.dat  
counterblock-testnet\_1 | gzip: GeoIP.dat.gz: No such file or directory  
counterblock-testnet\_1 | b’’  
counterblock-testnet\_1 | ERROR:server: Unhandled Exception  
counterblock-testnet\_1 | Traceback (most recent call last):  
counterblock-testnet\_1 | File “/usr/local/bin/counterblock”, line 9, in   
counterblock-testnet\_1 | load\_entry\_point(‘counterblock’, ‘console\_scripts’, ‘counterblock’)()  
counterblock-testnet\_1 | File “/counterblock/counterblock/ **init**.py”, line 7, in server\_main  
counterblock-testnet\_1 | server.main()  
counterblock-testnet\_1 | File “/counterblock/counterblock/server.py”, line 146, in main  
counterblock-testnet\_1 | StartUpProcessor.run\_active\_functions()  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/processor/ **init**.py”, line 171, in run\_active\_functions  
counterblock-testnet\_1 | func[‘function’](\*args, \*\*kwargs)  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/modules/counterwallet.py”, line 613, in init  
counterblock-testnet\_1 | module\_config[‘GEOIP’] = pygeoip.GeoIP(geoip\_data\_path)  
counterblock-testnet\_1 | File “/usr/local/lib/python3.5/dist-packages/pygeoip/ **init**.py”, line 118, in **init**  
counterblock-testnet\_1 | self.\_fp = codecs.open(filename, ‘rb’, ENCODING)  
counterblock-testnet\_1 | File “/usr/lib/python3.5/codecs.py”, line 895, in open  
counterblock-testnet\_1 | file = builtins.open(filename, mode, buffering)  
counterblock-testnet\_1 | FileNotFoundError: [Errno 2] No such file or directory: ‘/root/.local/share/counterblock/GeoIP.dat’  
counterblock-testnet\_1 | INFO:log: Logging txes to ‘/root/.cache/counterblock/log/server.testnet.tx.log’  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/assets  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/counterwallet  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/dex  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/transaction\_stats  
counterblock-testnet\_1 | INFO:module: Loading Plugin Module lib/modules/betting  
counterblock-testnet\_1 | INFO:server: counterblock command specified: server  
counterblock-testnet\_1 | INFO:server: counterblock Version 1.4.0 starting …  
counterblock-testnet\_1 | INFO:database: Connecting to mongoDB backend …  
counterblock-testnet\_1 | INFO:cache: Connecting to redis @ redis  
counterblock-testnet\_1 | INFO:counterwallet: Loading config at: /root/.config/counterblock/counterwallet.testnet.conf  
counterblock-testnet\_1 | INFO:counterwallet: Armory-utxsvr hostname: armory-utxsvr-testnet  
counterblock-testnet\_1 | INFO:counterwallet: Checking/updating GeoIP.dat …  
counterblock-testnet\_1 | INFO:counterwallet: Downloading GeoIP.dat  
counterblock-testnet\_1 | gzip: GeoIP.dat.gz: No such file or directory  
counterblock-testnet\_1 | b’’  
counterblock-testnet\_1 | ERROR:server: Unhandled Exception  
counterblock-testnet\_1 | Traceback (most recent call last):  
counterblock-testnet\_1 | File “/usr/local/bin/counterblock”, line 9, in   
counterblock-testnet\_1 | load\_entry\_point(‘counterblock’, ‘console\_scripts’, ‘counterblock’)()  
counterblock-testnet\_1 | File “/counterblock/counterblock/ **init**.py”, line 7, in server\_main  
counterblock-testnet\_1 | server.main()  
counterblock-testnet\_1 | File “/counterblock/counterblock/server.py”, line 146, in main  
counterblock-testnet\_1 | StartUpProcessor.run\_active\_functions()  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/processor/ **init**.py”, line 171, in run\_active\_functions  
counterblock-testnet\_1 | func[‘function’](\*args, \*\*kwargs)  
counterblock-testnet\_1 | File “/counterblock/counterblock/lib/modules/counterwallet.py”, line 613, in init  
counterblock-testnet\_1 | module\_config[‘GEOIP’] = pygeoip.GeoIP(geoip\_data\_path)  
counterblock-testnet\_1 | File “/usr/local/lib/python3.5/dist-packages/pygeoip/ **init**.py”, line 118, in **init**  
counterblock-testnet\_1 | self.\_fp = codecs.open(filename, ‘rb’, ENCODING)  
counterblock-testnet\_1 | File “/usr/lib/python3.5/codecs.py”, line 895, in open  
counterblock-testnet\_1 | file = builtins.open(filename, mode, buffering)  
counterblock-testnet\_1 | FileNotFoundError: [Errno 2] No such file or directory: ‘/root/.local/share/counterblock/GeoIP.dat’  
federatednode\_counterblock-testnet\_1 exited with code 0

So counterblock cannot find GeoIP.dat after Counterwallet cannot download GeoIP.dat?:

counterblock-testnet\_1 | INFO:counterwallet: Checking/updating GeoIP.dat …  
counterblock-testnet\_1 | INFO:counterwallet: Downloading GeoIP.dat  
counterblock-testnet\_1 | gzip: GeoIP.dat.gz: No such file or directory  
counterblock-testnet\_1 | b’’

Why cant Counterwallet download GeoIP.dat?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![jdogresorg](https://yyz2.discourse-cdn.com/flex004/user_avatar/forums.counterparty.io/jdogresorg/32/1349_2.png) [@jdogresorg](https://forums.counterparty.io/u/jdogresorg)\
**Post date:** [April 26, 2019, 4:45pm UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/6 "2019-04-26T16:45:37Z")

</div>

GeoIP was discontinued and we should be using GeoLite2 shortly after this PR is merged.

> <https://github.com/CounterpartyXCP/counterblock/pull/176>

You can get the existing GeoIP.dat file from [http://j-dog.net/misc/GeoIP.dat](http://j-dog.net/misc/GeoIP.dat)

to get things working again place the file in /root/.local/share/counterblock/ in your counterblock docker container.

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 27, 2019, 3:52am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/7 "2019-04-27T03:52:15Z")

</div>

Great TY its running full stack on testnet, only this message came up after starting counterblock-testnet:  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:blockfeed: Cannot contact counterparty-server (via get\_running\_info): ‘NoneType’ object is not subscriptable  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…  
counterblock-testnet\_1 | WARNING:util: Bad status code returned: ‘503’. result body: ‘{“message”: “Server error”, “data”: “Counterparty database is behind backend.”, “code”: -32000}’. – Waiting 3 seconds before trying again…

As far as I can see it looks that it is fully insync seeing the time stamps. I will wait to see if it catches up.

On mainnet I am still getting the following error after starting counterparty:  
counterparty\_1 | [2019-04-27 02:43:17][INFO] Running v1.1.4 of counterparty-server.  
counterparty\_1 | [2019-04-27 02:43:17][INFO] Running v9.56.1 of counterparty-lib.  
counterparty\_1 | [2019-04-27 02:43:17][INFO] Acquiring lock.  
counterparty\_1 | [2019-04-27 02:43:17][INFO] Connecting to database (SQLite 3.24.0-r1).  
counterparty\_1 | [2019-04-27 02:43:18][INFO] Connecting to backend.  
counterparty\_1 | [2019-04-27 02:44:18][ERROR] Unhandled Exception  
counterparty\_1 | Traceback (most recent call last):  
counterparty\_1 | File “/usr/local/bin/counterparty-server”, line 9, in   
counterparty\_1 | load\_entry\_point(‘counterparty-cli’, ‘console\_scripts’, ‘counterparty-server’)()  
counterparty\_1 | File “/counterparty-cli/counterpartycli/ **init**.py”, line 16, in server\_main  
counterparty\_1 | server.main()  
counterparty\_1 | File “/counterparty-cli/counterpartycli/server.py”, line 164, in main  
counterparty\_1 | server.start\_all(db)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/server.py”, line 453, in start\_all  
counterparty\_1 | connect\_to\_backend()  
counterparty\_1 | File “/counterparty-lib/counterpartylib/server.py”, line 447, in connect\_to\_backend  
counterparty\_1 | backend.getblockcount()  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/ **init**.py”, line 42, in getblockcount  
counterparty\_1 | return BACKEND().getblockcount()  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/indexd.py”, line 148, in getblockcount  
counterparty\_1 | return rpc(‘getblockcount’, [])  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/indexd.py”, line 87, in rpc  
counterparty\_1 | return rpc\_call(payload)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/indexd.py”, line 51, in rpc\_call  
counterparty\_1 | raise BackendRPCError(‘Cannot communicate with backend at `{}`. (server is set to run on {}, is backend?)’.format(util.clean\_url\_for\_log(url), network))  
counterparty\_1 | counterpartylib.lib.backend.indexd.BackendRPCError: Cannot communicate with backend at `http://XXXXXXXX@0.0.0.0:8332`. (server is set to run on mainnet, is backend?)  
federatednode\_counterparty\_1 exited with code 0

All the config files are kept in the default, no changes have been made. except added my own other nodes on my private network.

Thanks again for the help, J-dog

---

<div class="post-metadata">

**Author:** ![jdogresorg](https://yyz2.discourse-cdn.com/flex004/user_avatar/forums.counterparty.io/jdogresorg/32/1349_2.png) [@jdogresorg](https://forums.counterparty.io/u/jdogresorg)\
**Post date:** [April 27, 2019, 4:06am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/8 "2019-04-27T04:06:25Z")

</div>

This looks like an indexd communication issue. Indexd needs to be up and fully synced before starting up Counterparty.

You can check the status of indexd via http://IP\_ADDRESS:8332/status

The response will look something like this

```auto
{
  "chainBlock": 573417,
  "indexBlock": 573417,
  "network": "mainnet",
  "blocksBehind": 0,
  "ready": true
}

```

Once your indexd shows “ready”:true, then try restarting counterparty and see if you still have communication issues between counterparty and indexd.

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 27, 2019, 5:03am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/9 "2019-04-27T05:03:01Z")

</div>

I get a 404 on that, thats the docker-ufw issue I still need to resolve as I cannot keep using the dockerized bitcoin nodes, but this came up…so I will let it sync:

 ![image](https://canada1.discourse-cdn.com/flex004/uploads/counterpartytalk/original/2X/b/bc48fcbfdefc72c0f8279997bee522af42415ab7.png)

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 27, 2019, 5:57am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/10 "2019-04-27T05:57:11Z")

</div>

FYI: I solved the ERROR: ReadBlockFromDisk: Deserialize or I/O error by replacing the faulty damaged blocks instead of doing a full 10-day -reindex…

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [April 27, 2019, 2:05pm UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/11 "2019-04-27T14:05:03Z")

</div>

http://IP\_ADDRESS:8432/status works, gives:  
{“chainBlock”:573482,“indexBlock”:264126,“network”:“mainnet”,“blocksBehind”:309356,“ready”:false}

TY

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [May 3, 2019, 10:13pm UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/12 "2019-05-03T22:13:56Z")

</div>

Finally inexd has caught up… next error:  
counterparty\_1 | [2019-05-03 22:10:34][ERROR] Unhandled Exception  
counterparty\_1 | Traceback (most recent call last):  
counterparty\_1 | File “/usr/local/bin/counterparty-server”, line 9, in   
counterparty\_1 | load\_entry\_point(‘counterparty-cli’, ‘console\_scripts’, ‘counterparty-server’)()  
counterparty\_1 | File “/counterparty-cli/counterpartycli/ **init**.py”, line 16, in server\_main  
counterparty\_1 | server.main()  
counterparty\_1 | File “/counterparty-cli/counterpartycli/server.py”, line 164, in main  
counterparty\_1 | server.start\_all(db)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/server.py”, line 466, in start\_all  
counterparty\_1 | blocks.follow(db)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/blocks.py”, line 1240, in follow  
counterparty\_1 | tx\_index = list\_tx(db, block\_hash, block\_index, block\_time, tx\_hash, tx\_index, tx\_hex)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/blocks.py”, line 934, in list\_tx  
counterparty\_1 | source, destination, btc\_amount, fee, data = get\_tx\_info(tx\_hex)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/blocks.py”, line 453, in get\_tx\_info  
counterparty\_1 | return \_get\_tx\_info(tx\_hex, block\_parser, block\_index)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/blocks.py”, line 467, in \_get\_tx\_info  
counterparty\_1 | return get\_tx\_info1(tx\_hex, block\_index, block\_parser=block\_parser)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/blocks.py”, line 566, in get\_tx\_info1  
counterparty\_1 | vin\_tx = backend.getrawtransaction(ib2h(vin.prevout.hash))  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/ **init**.py”, line 52, in getrawtransaction  
counterparty\_1 | return BACKEND().getrawtransaction(tx\_hash, verbose=verbose, skip\_missing=skip\_missing)  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/indexd.py”, line 157, in getrawtransaction  
counterparty\_1 | return getrawtransaction\_batch([tx\_hash], verbose=verbose, skip\_missing=skip\_missing)[tx\_hash]  
counterparty\_1 | File “/counterparty-lib/counterpartylib/lib/backend/indexd.py”, line 234, in getrawtransaction\_batch  
counterparty\_1 | raise BackendRPCError(’{} (txhash:: {})’.format(response[‘error’], tx\_hash\_call\_id.get(response.get(‘id’, ‘??’), ‘??’)))  
counterparty\_1 | counterpartylib.lib.backend.indexd.BackendRPCError: {‘message’: ‘No such mempool or blockchain transaction. Use gettransaction for wallet transactions.’, ‘code’: -5} (txhash:: 3ecfd580d5ef4f42bde272c86468d46f71458bc7dff383fd5baa04ffb86a3250)

federatednode\_counterparty\_1 exited with code 0

thanks in advance…

---

<div class="post-metadata">

**Author:** ![jdogresorg](https://yyz2.discourse-cdn.com/flex004/user_avatar/forums.counterparty.io/jdogresorg/32/1349_2.png) [@jdogresorg](https://forums.counterparty.io/u/jdogresorg)\
**Post date:** [May 5, 2019, 11:18pm UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/13 "2019-05-05T23:18:54Z")

</div>

> [@jeroensbel](#):
>
> 3ecfd580d5ef4f42bde272c86468d46f71458bc7dff383fd5baa04ffb86a3250

A quick search for this TX shows a previous issue open at [https://github.com/CounterpartyXCP/counterparty-lib/issues/1022](https://github.com/CounterpartyXCP/counterparty-lib/issues/1022)

You might want to try using the counterparty bootstrap image rather than re-parsing all the transactions from 5+ years ago to build the CP database tx by tx.

You should be able to download the bootstrap images via

```auto
rm ~/fednode/data/counterparty/counterparty.db
rm ~/fednode/data/counterparty/counterparty.testnet.db
fednode restart counterparty counterparty-testnet

```

That should remove he CP database, and restart counterparty, which will then download the latest bootstrap images which have all transactions parsed through 2018 and part of 2019.

---

<div class="post-metadata">

**Author:** ![jeroensbel](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@jeroensbel](https://forums.counterparty.io/u/jeroensbel)\
**Post date:** [May 6, 2019, 12:14am UTC](https://forums.counterparty.io/t/catastrophic-failure/5424/15 "2019-05-06T00:14:43Z")

</div>

Thanks J-dog

Downloading bootstrap db´s as I write.  
I still have not been able to let indexd and my core node talk, I need to have it set up this way as the 1Tb drive has filled itself up with the full counterparty stack and the 2 full blockchains, indexes, etc. I had to delete the counterparty bitcoin node.

I have send a private message to chiguireitor but no response yet.
