NOTE: this is a proposal for a CIP, this hasn’t been accepted / assigned a CIP number yet!
Title: P2SH data encoding; at least it's not P2PKH Authors: Ruben de Vries Type: Standards
Motivation
Counterparty currently has the following data encoding schemes:
-
opreturnthe best solution, doesn’t pollute the UTXO set, is the cheapest, but limited to a maximum 80 bytes -
multisigthe 2nd best solution, only temporarly pollutes the UTXO set, has no upper limit -
pubkeyhashthe worst solution, permanently pollutes the UTXO set, but also has no upper limit
For (literally) 99% of Counterparty transactions opreturn is enough (though some people forcefully use multisig encoding).
Previously for the rest we’d fall back to using multisig encoding since it has no real drawbacks.
Unfortunately since Bitcoin Core v0.12.1 there’s a DoS protection (called bytespersigop) that has essentially made bare multisig non-standard.
So we need an alternative for larger amounts of data and we want to avoid pubkeyhash encoding because it pollutes the UTXO set too much.
With the EVM coming there will also be more transactions with larger amounts of data, so we need this more than ever!
Rationale
It’s possible to embed data in P2SH redeemScripts by doing 2 transactions
where the first one sets up P2SH outputs with redeemScripts that allow us to put the data in the scriptSig of the spending transaction.
With this method we can easily put in a lot more data than the other methods, does not polute the UTXO set
and signature data can be pruned by Bitcoin nodes that wish to do so.
The original proof of concept, by Peter Todd, of this can be found here: https://github.com/petertodd/python-bitcoinlib/blob/master/examples/publish-text.py
Specification
The Basics
This process requires 2 transactions, the first to setup 1 or more P2SH outputs and then the second spending the P2SH outputs and placing the data we wish to embed in the scriptSig.
The Redeem Script
The data is split chunks of max X size, then for each chunk we create a redeemScript as follows:
redeemScript = {{ data }} OP_HASH160 {{ hash160(data) }} OP_EQUALVERIFY {{ pubkey }} OP_CHECKSIGVERIFY {{ n }} OP_DROP OP_DEPTH 0 OP_EQUAL
dissecting this into pieces we have:
{{ data }} OP_HASH160 {{ hash160(data) }} OP_EQUALVERIFY
this is where we place the data and the OP_HASH160 {{ hash160(data) }} OP_EQUALVERIFY ensure the data can’t be changed.
{{ pubkey }} OP_CHECKSIGVERIFY
this ensures the output needs to be signed by the owner, this can be replaced by other scripts such as a multisig or CLTV or similar things that were previously already put into P2SH outputs.
{{ n }} OP_DROP
n is an incrementing number to ensure that each output is unique even when the data chunks aren’t.
OP_DEPTH 0 OP_EQUAL
this prevents scriptSig malleability.
The Output Script
The output script placed in the first transaction is then:
ouputScript = OP_HASH160 {{ hash160([redeemScript]) }} OP_EQUALVERIFY
The real deal
Below we’ll describe how a Counterparty send would look.
Transaction 1
Inputs
- 1 + n UTXOs from the
sourceaddress to have enough BTC to pay the fees for both the first and the second transaction.
Outputs
- 1 output to
sourcewith enough value to pay the fee for the second transaction. - 1 + n P2SH outputs following the above method with
DUSTvalue. - 1 change output to send any excess BTC back to
source(optional; but in practice always there)
Transaction 2
Inputs
- 1 input spending the
sourceoutput from Transaction 1 - 1 + n inputs spending the P2SH outputs, including the data in the
scriptSig
Outputs
- 1 output to specify the destination of the Counterparty transaction (in some types of transactions this is omitted) with
DUSTvalue. - 1
opreturnoutput encoding the data'CNTRPTY' + 'P2SH'to signal that the data is found in the P2SH inputs, with 0 value.
Fees & Coin Selection
The first transaction has send enough BTC to the second transaction so the second transaction can pay for it’s own fee without having to add extra inputs.
So we calculate the value of the source output in the first transaction to be:
estimated_size_of_tx2 = 10 # base size of TX
estimated_size_of_tx2 += 181 # for source input
estimated_size_of_tx2 += 29 * count(destination_outputs) # for destination outputs
estimated_size_of_tx2 += sizeof(data) # for the data
estimated_size_of_tx2 += count(data_p2sh_outputs) * (181 + 9) # for the overhead of each data output being spend
estimated_fee_for_tx2 = (estimated_size_of_tx2 / 1000) * fee_per_kb
source_output_value = count(data_p2sh_outputs) * DUST + count(destination_outputs) * DUST + estimated_fee_for_tx2
This means the amount of BTC to require when doing coinselection is:
estimated_size_of_tx1_without_inputs = 10 # base size of a TX
estimated_size_of_tx1_without_inputs += 29 # for source output
estimated_size_of_tx1_without_inputs += count(data_p2sh_outputs) * 29 # for P2SH data outputs
inputs = []
for UTXO in coinselection:
inputs.append(UTXO)
estimated_size_of_tx1 = estimated_size_of_tx1_without_inputs + count(inputs) * 181
estimated_fee_for_tx1 = (estimated_size_of_tx1 / 1000) * fee_per_kb
if sum(inputs) >= estimated_fee_for_tx1 + estimated_fee_for_tx2 + source_output_value + count(data_p2sh_outputs) * DUST:
break
The Counterparty API
In practice this means the create_* API calls will have to start returning a list of 1 or more transactions which the client signs and broadcasts,
so clients need to adapt to this new style and always assume they need to sign and broadcast N transactions.
Pre-segwit we won’t be able to have the second transaction spend from the first transaction until the first transaction has been signed,
this means the client actually needs to do 2 API calls, where the second has the txId of the signed first transaction as param.
Once we can use segwit this problem is gone! However not all clients will straight away be able to sign segwit transactions (requires upgrades of the libraries they use).
Because of this being quite a hassle we propose to have 2 node configs / API params to control all of this: segwit and old_style_api.
old_style_api
When old_style_api = True the API will continue to function as normal, returning a single transaction, as string.
Once old_style_api = False the API will (across all create_* transactions) return a list of 1+ transactions (even when it’s just 1).
While old_style_api = True P2SH encoding will not be used unless explcitily set (so - the default - encoding=auto will not use P2SH encoding).
segwit
When segwit = False the first API call will return [tx_hex, None] signalling that it needs a second call with p2sh_pretx_txid added as param,
which should be the txId of the signed first transaction.
The second API call will then return [None, tx_hex].
When segwit = True the API call will return [tx_hex, tx_hex] and will no longer require a second API call.
EVM Transactions
Because the nature of EVM transactions being large® amounts of data,
the EVM API calls (create_publish and create_broadcast) will require old_style_api == False.
Child pays for Parent
The current scheme ensures both transactions pay fee for their own size, in the (near) future when the ‘Child pays for Parent’ functionality
that has been added to Bitcoin Core is widely adopted we can change this so that the second transaction pays a larger portion of the total fee (still needs to pay enough to be relayed).
This will ensure that the first transaction is never mined without the second transaction.
Backwards Compatibility
This is a new encoding that will be completely unrecognised by older clients, any clients who don’t upgrade would loose consensus with the nodes that did upgrade.
It also affects how the Counterparty API works (see above).
Copyright
This document is placed in the public domain.
