Problem:
Trolls can place ridiculously big orders on the DEX very cheaply, matching everything on the order book, and then never sending BTCpay.
Why:
* XCP sellers don’t set fee_required high enough (many people use 0)
* With a single order & fee trolls can match a LOT of orders
Fix:
* fee_required should mean fee required per XCP matched
* matching proceeds deterministically, as usual, but halts once fee_provided has been exhausted
* fee is payable in BTC (to miners, as usual) or in equiv. amount of XCP (intended amount flagged in the order, nominally converted to BTC at the same rate as each order matched)
* fees payed in XCP would be auto-refunded on successful BTCpay. Failure to BTCpay would result in destruction of XCP fee (not given to seller to avoid creating perverse incentives to game the system)
Advantages of this protocol:
* Simple and robust. No unnecessary complexity or magic numbers (“orders over X amount get treated differently”, requiring you to change the protocol when the value that X represents drifts too far)
* the cost of small orders remains very low
* no barrier to entry for someone with only BTC
* the nominal cost of large orders goes up, but people making large orders probably have at least a small amount of XCP (or can buy some first), so the real cost is closer to free if they pay a large fee in XCP and it is refunded
* the cost to lock up large amounts of XCP on the order book is prohibitively high (fee losses proportional to the amount of XCP you lock in fake matches)
Of course, fee mechanisms only work when people use them. While not part of the actual protocol, counterpartyd could default fee_required to something non-zero; 99% of people will use the default after all.
For example, imagine a default fee_required (per XCP matched) of 0.00002* BTC. The net effect would be that:
* orders for under 5 XCP would require no more than the standard miner fee (0.0001) they’re forced to pay anyway
* larger orders for, e.g… 100 XCP, would need to pay 0.002 (ie. 2 mBTC; not so cheap)
* people placing large orders would typically pay the extra fee (above the standard miners fee) in XCP, which is refunded upon BTCpay
* trolls trying to lock up orders in bulk - e.g… 1000 XCP worth, would have to waste 20mBTC each time
The effectiveness of the trolls could be further reduced by having a standard timeout for all matched orders. e.g… regardless of individual order timeouts, once matched timeout = 10* blocks.
It isn’t unreasonable, IMO, to expect people to remain online while settling an order. counterpartyd could automatically issue BTCpay upon seeing the match. For safety counterpartyd could also refuse to BTCpay within e.g… 5* blocks of match expiration. This mechanism would allow you to squeeze the “standard match expiration” down as low as safe, cutting back the number of blocks trolls can lock that 1000 XCP worth of orders for for their 20 mBTC fee.
* numbers in this post are illustrative only. someone wiser and more experienced can tweak them to something optimal, I just want to get the idea across. The protocol should be as simple and robust and unchanging as possible, but counterpartyd defaults could be tweaked whenever needed to encourage optimal market behavior.
One more refinement: the value for the “standard match expiration” I talked about shouldn’t be baked into the protocol. In future it might need to change (eg. if the bitcoin network becomes busy and it takes longer on average to get BTCpay into a block.) This is a protocol that you can’t easily change, without requiring every user to upgrade at the same time (otherwise you break consensus on which orders were valid, and therefore on who owns how much XCP, and therefore people lose money.)
Instead, an order to buy BTC must specify two things - the main timeout (how long the order is valid for) and also the “match timeout” (how long a match with this order has to complete BTCpay.) Counterpartyd would default this to a standard value (eg. 10 blocks) which you could freely tweak at any time without breaking the protocol. I’m willing to be 95% of people would use the default value.
So conceptually, supposing I want to leave an order to sell XCP on the order book and go to sleep:
* I set expiration to 100 blocks, so the order will be valid for a long time
* I set fee_required to a small amount per XCP matched (typically the default, eg. 0.00002)
* BUT, I don’t want a troll to be able to pay 0.00002 per XCP to lock my order up for the entire night, SO:
* I set the match_expiration to 10 blocks (or, typically, the default.) Now any match has only 10 blocks to BTCpay, or it fails and my order remains open for others to match.
If I want to leave an order to buy XCP on the order book and go to sleep:
* I set expiration to 100 blocks
* As soon as somebody matches my order, while I’m asleep, I can trust counterpartyd to do the right thing in my absence - if match_expiration remaining is > than some safety margin configured in counterpartyd (eg. 5 blocks) then it will automatically BTCpay
I’m having trouble understanding the new rules you propose.
Will you please bold them the way I did for the optional parameters I proposed?
If I understand it correctly, I could place an order to sell 1 xcp for 0.000001 BTC and charge a 0.1 BTC fee? and I can set the time for the maximum allowable to let my order clog the book for the maximum time period?
Is that correct? not sure if I understand your new rules correctly.
Also, the automation you propose at the end should be separate from the core protocol imho.
The automation for btcpay when combined with your fees could be catastrophic if someone set high fees for low amounts.
i.e. they sell you 1 of your 100 requested xcp with a 100% miner fee and your client automatically btcpays that - ouch.
I’d advocate a more decentralized user customizable solution such as the one MPT and I worked out and described in my sister thread here.
Hi Jeremy,
I appreciate you taking the time to read my proposal and reply. I think you may have misunderstood some parts of it. My proposal is really quite simple, and I am afraid that I have expressed it rather more verbosely than necessary. It is certainly 100% decentralised. I’ll attempt to summarise.
But first:
[quote author=jeremy link=topic=69.msg326#msg326 date=1391906767]
The automation for btcpay when combined with your fees could be catastrophic if someone set high fees for low amounts.
i.e. they sell you 1 of your 100 requested xcp with a 100% miner fee and your client automatically btcpays that - ouch.
[/quote]
You’re confused about how the existing protocol works. The fee_required is paid when you post an order, as a way to show that you intend to follow through with BTCpay to settle the order. You never pay any fee with BTCpay (other than the standard bitcoin miner fee), and I am not proposing otherwise.
Summary Version
I believe that the existing protocol is pretty good, and propose minor tweaks to it, while attempting to avoid as much complexity as possible.
Key ideas:
* fees must be proportional to the magnitude of order(s) matched
* fees should be payable in BTC or equivalent XCP (with XCP being either fully refunded or destroyed)
* match timeouts should be separate from order timeouts (typically you want to leave an order open for a long time, but you expect BTCpay promptly after a match is obtained)
* magic numbers in the protocol should be avoided. It must be simple to understand, simple to implement, and flexible.
Protocol changes:
* fee_required should be interpreted as fee per XCP matched (and matching should stop once it is exhaused)
* orders require a new field to specify xcp_fee_provided, if any
* orders require a new field to specify match_timeout (the protocol should enforce a minimum value for this)
Client changes:
* automatic sending of BTCpay upon match, while outside an expiration safety margin (eg. 5 blocks)
* fee_required should default to something sensible, eg. 0.00002 per XCP matched
* match_timeout should default to something sensible, eg. 10 blocks
* if XCP are available in an account, default to pay fee fee_required with XCP instead of BTC (or at least prompt the user) as this is refundable
This will enable:
* short match_timeouts to become typical (because everyone honest instantly auto settles with BTCpay)
* higher fees to become typical (because people can pay them with XCP, which is refunded, so nothing is lost)
* low value orders to pay low fees, so it doesn’t matter if they pay in BTC (may not even exceed the bitcoin miner fee, which you have to pay anyway)
I hope this is clearer 
-Gareth
[quote author=gacrux link=topic=69.msg340#msg340 date=1391950788]
Summary Version
I believe that the existing protocol is pretty good, and propose minor tweaks to it, while attempting to avoid as much complexity as possible.
Key ideas:
* fees must be proportional to the magnitude of order(s) matched
* fees should be payable in BTC or equivalent XCP (with XCP being either fully refunded or destroyed)
* match timeouts should be separate from order timeouts (typically you want to leave an order open for a long time, but you expect BTCpay promptly after a match is obtained)
* magic numbers in the protocol should be avoided. It must be simple to understand, simple to implement, and flexible.
Protocol changes:
* fee_required should be interpreted as fee per XCP matched (and matching should stop once it is exhaused)
* orders require a new field to specify xcp_fee_provided, if any
* orders require a new field to specify match_timeout (the protocol should enforce a minimum value for this)
Client changes:
* automatic sending of BTCpay upon match, while outside an expiration safety margin (eg. 5 blocks)
* fee_required should default to something sensible, eg. 0.00002 per XCP matched
* match_timeout should default to something sensible, eg. 10 blocks
* if XCP are available in an account, default to pay fee fee_required with XCP instead of BTC (or at least prompt the user) as this is refundable
This will enable:
* short match_timeouts to become typical (because everyone honest instantly auto settles with BTCpay)
* higher fees to become typical (because people can pay them with XCP, which is refunded, so nothing is lost)
* low value orders to pay low fees, so it doesn’t matter if they pay in BTC (may not even exceed the bitcoin miner fee, which you have to pay anyway)
I hope this is clearer
-Gareth
[/quote]
This is a very good proposal. I’ll get to work on it right away. (Feel free to submit pull requests, too!)
[quote author=PhantomPhreak link=topic=69.msg346#msg346 date=1391962015]
[quote author=gacrux link=topic=69.msg340#msg340 date=1391950788]
Summary Version
I believe that the existing protocol is pretty good, and propose minor tweaks to it, while attempting to avoid as much complexity as possible.
Key ideas:
* fees must be proportional to the magnitude of order(s) matched
* fees should be payable in BTC or equivalent XCP (with XCP being either fully refunded or destroyed)
* match timeouts should be separate from order timeouts (typically you want to leave an order open for a long time, but you expect BTCpay promptly after a match is obtained)
* magic numbers in the protocol should be avoided. It must be simple to understand, simple to implement, and flexible.
Protocol changes:
* fee_required should be interpreted as fee per XCP matched (and matching should stop once it is exhaused)
* orders require a new field to specify xcp_fee_provided, if any
* orders require a new field to specify match_timeout (the protocol should enforce a minimum value for this)
Client changes:
* automatic sending of BTCpay upon match, while outside an expiration safety margin (eg. 5 blocks)
* fee_required should default to something sensible, eg. 0.00002 per XCP matched
* match_timeout should default to something sensible, eg. 10 blocks
* if XCP are available in an account, default to pay fee fee_required with XCP instead of BTC (or at least prompt the user) as this is refundable
This will enable:
* short match_timeouts to become typical (because everyone honest instantly auto settles with BTCpay)
* higher fees to become typical (because people can pay them with XCP, which is refunded, so nothing is lost)
* low value orders to pay low fees, so it doesn’t matter if they pay in BTC (may not even exceed the bitcoin miner fee, which you have to pay anyway)
I hope this is clearer
-Gareth
[/quote]
This is a very good proposal. I’ll get to work on it right away. (Feel free to submit pull requests, too!)
[/quote]
Why do you think that it’s important for fees to be able to be specified in XCP, as well as BTC? I see your reasoning. This adds a lot of complexity, however.
EDIT: I also think that for simplicity of implementation, fees should be specified not per unit traded, but per match. User-friendly interfaces, then, can just set the default fees to be proportional to the order size.
EDIT #2: Check out this commit, which adds support for a ‘fee remaining’ variable which tracks how much of the fee has been allotted to each order match. Make sure that I didn’t make any mistakes.
EDIT #3: This commit lets all order matches awaiting BTC payment expire after 10 blocks.
another way to fight spam is escrowable insurance for order, for example seller may send an order that requires buyer to have N xcp on his address, if buyer fails to send BtcPay within time, that N xcp will be transferred to seller, of course this won’t work for buyers who don’t have xcp.
btw what prevent someone from making up order book on the bid side, but never send btcpay
Forcing all BTCpay to be conducted within a specific time and/or including a Auto BTCPay option are both bad ideas because:
BTC offers currently can set the time they intend to btcpay by and both of these suggestions remove that power of the btc order placer.
i.e the suggestions remove power from the client, the user of the protocol and would be going backwards instead of forwards.
Canceling BTC offers before their intended timeout is a bad idea because the BTC order placer knows best when they want to review the orders they collected and pay them. That’s the benefit of paying to put their order on the market. If Asset offerors don’t want to wait that long they should put shorter expiration times (or match timeout variable time if implemented).
Locking up a btc offer with a 10 round match time when the btc offer already stated “I will pay by 100 rounds” is removing the power from a trader to make that decision they can currently make.
Further, burning more fees are not going to make the problems better or solve them, there is no clear explanation of how burning more miner fees per order help active traders more than create another hurdle for decentralized trading altogether.
I’ve been working out the glaring problems of the protocol over chat with MPT, but the points I made are fairly accurate for the more obvious issues.
A particularly useful easy fix to the order matching problems that alpha testers experienced is that would be suggestion 4 that I proposed which ADDS NO EXTRA FEES and solves the demonstrated order book problem with MINIMAL changes to the protocol.
Can you explain the goals/objectives/reasons for the “key ideas” you stated here?
What are we are trying to get at or implement?
Shouldn’t the traders be given freedom to use the protocol however they see fit? (high/low fees burned or paid to who they want to trade with in what quantities they want to trade per btcpay in the timeframe they select?)
Please, just implement suggestion (4) to allow btc traders to choose the timeframe they want to have and more fairly to minimize the effects of “gaming” the market from other random test orders but also not add MORE fees unless orders require it. The traders clearly want to not require (more) fees as you see from the order book the most popular choice is none.
Here’s a link to suggestion 4 and all the explanation for how it could fit into a more comprehensive solution: https://forums.counterparty.co/index.php/topic,71.msg321.html#msg321
But here’s the technical details of how the protocol would change:
NEW CODE:
Orders offering btc DO NOT/WILL NOT MATCH WITH asset offers if the asset expiration time left is less than the btc offer expiration time left.
Orders requesting btc DO NOT/WILL NOT MATCH btc offers if the asset expiration time left is less than the btc offer expiration time left.
Your proposal is not so different I just believe the one I detailed is more explicit with how it fixes and adds functionality to the protocol.
[quote author=jeremy link=topic=69.msg354#msg354 date=1391984995]
Further, burning more fees are not going to make the problems better or solve them, there is no clear explanation of how burning more miner fees per order help active traders more than create another hurdle for decentralized trading altogether.
[/quote]
Burning fees is a disincentive towards trolling the network with fake BTC sell orders; the idea is that the disincentivization is in proportion to the size of the burn (the [tt]fee_required[/tt]). It by no means ‘creates a hurdle for decentralized trading’: the [tt]fee_required[/tt] is chosen by the BTC buyer in every case, and is, on the contrary, a feature of decentralized trading. Whether there are other ways to disincentivize trolls is another issue, but it’s important not to dismiss the optional implementation of fees as pointless.
“Gaming the network” as you call it is indeed itself a feature of decentralized trading and a “free market”, and while it can be made expensive to game the network (either by burning or escrowing funds, or whatever), one cannot, in a truly free market, disincentivize it altogether.
[quote author=PhantomPhreak link=topic=69.msg346#msg346 date=1391962015]
This is a very good proposal. I’ll get to work on it right away. (Feel free to submit pull requests, too!)
[/quote]
Thank you
Unfortunately, I don’t have a lot of spare time at the moment, or I would have. Code is often more efficient for communication than English. If I get enough time to myself, forking and tinkering with counterparty is high on my TODO list.
[quote author=PhantomPhreak link=topic=69.msg347#msg347 date=1391962577]
Why do you think that it’s important for fees to be able to be specified in XCP, as well as BTC? I see your reasoning. This adds a lot of complexity, however.
[/quote]
To allow for the proportionately large fees for large orders that I propose. Fee magnitude must be coupled to order magnitude. But losing high fees in BTC introduces considerable friction for large orders. An option to “pay” (escrow) fees with XCP seems to be the obvious solution.
I’m imagining a situation where prompt BTCpay upon order match is the norm for well behaved clients, and where fees are typically “paid” in XCP - so a person with a small amount of XCP in his account can trade without it costing him anything.
[quote author=PhantomPhreak link=topic=69.msg347#msg347 date=1391962577]
EDIT: I also think that for simplicity of implementation, fees should be specified not per unit traded, but per match. User-friendly interfaces, then, can just set the default fees to be proportional to the order size.
[/quote]
I agree - fee_required can retain its current meaning, as long as fee_provided is used up proportionally during matching.
If I match with 50% of your order, I should have to provide 50% of your fee_required (in BTC, XCP, or a mixture of the two.)
It is also very important that the client (counterpartyd) do the right thing automatically, and default required_fee = BASE_FEE * order magnitude. This should be user-over-rideable of course (counterpartyd must always work in the best interests of the person running it at all times.) But not having it auto-calculated in proportion to the order magnitude will result in everybody setting it to 0.00001 (or worse, zero!) because they don’t understand it, which makes the whole mechanism ineffective.
[quote author=PhantomPhreak link=topic=69.msg347#msg347 date=1391962577]
EDIT #2: Check out this commit, which adds support for a ‘fee remaining’ variable which tracks how much of the fee has been allotted to each order match. Make sure that I didn’t make any mistakes.
[/quote]
Awesome. I had a quick skim through and it looks like you’ve understood my idea That’s what I had imagined. Unfortunately I don’t have time to study the code in detail right now to spot any bugs (I’m at work; lunchtime’s over.) It’s good to see “if tx1[‘block_index’] >=” though - cutting over hardforking protocol changes at a specific future block is definitely the way to go. All you need now is a way to ensure everybody upgrades before then
(Have you considered publishing the latest counterpartyd version in a feed, and having counterpartyd warn the user if it spots a larger number in the feed that its own internal version number?)
[quote author=PhantomPhreak link=topic=69.msg347#msg347 date=1391962577]
EDIT #3: This commit lets all order matches awaiting BTC payment expire after 10 blocks.
[/quote]
That one is clear at a glance I like it.
In future you might want to migrate that magic number (“10 blocks”) out of the protocol and let orders specify it (with a default of 10 blocks, which everyone will typically use.) That would allow you to easily react to eg. future bitcoin network congestion pushing up the average time it takes to get BTCpay into a block (just tweak the default in counterpartyd; no protocol change, users can upgrade whenever they like and nothing breaks.)
But for now the simplicity of this is compelling (and its much better than the current situation.)
[quote author=romerun link=topic=69.msg350#msg350 date=1391965970]
another way to fight spam is escrowable insurance for order, for example seller may send an order that requires buyer to have N xcp on his address, if buyer fails to send BtcPay within time, that N xcp will be transferred to seller, of course this won’t work for buyers who don’t have xcp.
[/quote]
This is essentially what my idea is. I’ve just simplified it, by removing the concept of a separate seller-specified escrow amount. The extra complexity of running essentially two parallel order books (one using BTC “fees”, the other using XCP “escrows”, which can’t interact with each other) is unwarranted. A fee is an escrow, essentially. The point isn’t that you always lose it, you just have to lose it when you fail to settle with BTCpay (and in fact it works best this way.)
They’re the identical concept. Just:
* you must always pay a small BTC fee to the bitcoin mining network to place an order, so let’s count this toward the amount “escrowed”
* unfortunately, we can’t refund any BTC sent to miners, even if your order settles correctly (we would if we could!)
* you can certainly pay part or all of the fee/escrow with XCP, and that WILL be returned to you when you settle (because we can return it.)
* the seller should not receive any benefit if you fail to settle. ie. the “escrow” must be destroyed in this case. Otherwise, sellers will have a perverse incentive to try to prevent their orders settling (ping flooding the guy trying to send BTCpay? Setting insanely small expiry times in the hope your BTCpay doesn’t make it into a block on time? Use your imagination.)
[quote author=romerun link=topic=69.msg350#msg350 date=1391965970]
btw what prevent someone from making up order book on the bid side, but never send btcpay
[/quote]
Isn’t that the exact issue we’re trying to solve?
If the fee/escrow amount you stand to lose is significant, you are less likely to back out of sending BTCpay. You certainly won’t be able to afford to do so repeatedly.
Would like to add another proposal for allowing users to match specific buy/sell orders vs dex matching based on best price.
This ties back to the idea of reputation and if you look at the block data buyers who have consistently followed up with their BTCPay normally also do so for future orders. It would be quite easy to have an independent 3rd party optional reputation/score based system for specific address
The order command should be something like the following
i.e counterpartyd.py --order --offer_hash=XXXXXXXXXXXXXXXXXX --give-quantity=XX
Then blocks.py would parse the blockchain and automatically match these orders. If for one reason or other it is unable to do so (i.e order already prefilled or insufficient balance left) then the order is auto cancelled and the NON-BTC asset (i.e XCP) is refunded.
This proposal should give more control to clients and should be able to work alongside with the existing best matched pricing of dex.
[quote author=mtbitcoin link=topic=69.msg373#msg373 date=1392021081]
Would like to add another proposal for allowing users to match specific buy/sell orders vs dex matching based on best price.
This ties back to the idea of reputation and if you look at the block data buyers who have consistently followed up with their BTCPay normally also do so for future orders. It would be quite easy to have an independent 3rd party optional reputation/score based system for specific address
The order command should be something like the following
i.e counterpartyd.py --order --offer_hash=XXXXXXXXXXXXXXXXXX --give-quantity=XX
Then blocks.py would parse the blockchain and automatically match these orders. If for one reason or other it is unable to do so (i.e order already prefilled or insufficient balance left) then the order is auto cancelled and the NON-BTC asset (i.e XCP) is refunded.
This proposal should give more control to clients and should be able to work alongside with the existing best matched pricing of dex.
[/quote]
This sounds like a workable idea, but I’m not sure that the extra complexity is worth it unless there’s no other way. I see what you’re trying to address - that annoying situation where the best price on the DEX is blatently some troll who won’t BTCpay, and due to the simple “best matched price” algorithm you’re powerless to avoid your order being matched against the troll. It’s tempting to think “I wish I could just force my order to match X”. I get it
My objections are fundamental:
* in a properly functional market, you always want the best price.
* an average market user shouldn’t have to think about the complexity of reputation systems and matching specific orders. He just wants the best price, automatically.
* protocols like this need to be as simple and robust as possible
* your suggestion is a hack designed to mitigate damage rather than remove the source of the problem
The real problem here is that trolls are able to afford to match orders and refuse BTCpay in the first place.
If we can fix that, I think you’ll agree, your suggestion is unnecessary. If we can’t fix it then I’ll be the first one to embrace your damage mitigation as very sadly necessary. But it remains to be seen
In any case, it’s important to recognise when we’re going down the path of building a decentralised “craigslist exchange” rather than a decentralised “mtgox exchange” (if you can see the analogy.) One allows individuals to trade, but with the considerable friction of having to find matches and assess reputation themselves. The other is a real market, which simply lets people buy and sell for the best price automatically with zero friction. We should strive for the latter, and only settle for the former when the latter has proved impossible.
[quote author=gacrux link=topic=69.msg364#msg364 date=1392003178]
(Have you considered publishing the latest counterpartyd version in a feed, and having counterpartyd warn the user if it spots a larger number in the feed that its own internal version number?)
[/quote]
I’m going to do this soon.
Take a look at this commit, https://github.com/PhantomPhreak/counterpartyd/commit/34a871fac50dad6a784766d4ba6de06bfccd5689, which now has all order fees specified as a percentage of the BTC to be transacted, and defaults to 1%.