CIP10 - Compress asset amounts?

Development Counterparty
1

@chiguireitor @deweller

The asset amounts are encoded as 8 byte integers. Eyeballing the hex representation, there are plenty of padded zeros, e.g. 1 XCP = x0000000001000000.

Would it not be a straight forward optimization to specify the number of padded zeros? In the above example there are 9 zeros, equal to 4.5 bytes. At $0.01 per byte in btc fee, this is significant.

If the amount is one of a non-divisible token, the hex amount is x0000000000000001 so here the potential saving is even bigger.

The only times there are no padded zeros is when the asset amount is near the max possible amount - in practice almost never.

To specify the amount of padded zeros, it would take 4 bits only. Could be done for each send amount separately, or perhaps better once for the entire list (then can specify the one send with the least padding and partially pad the others so they keep consistent byte length)

2

On CIP10 i suggested a compression idea, but was ultimately removed because LZMA compression on messages has a similar compression ratio and is easier to debug/use. However, the only downside is that lzma has a C/C++ dependency which can crash outside python code, so this idea is worth revisiting.

My originally idea of compressed ints was to add an additional bit to each 16 bit run to indicate if it is all 0s (bit on) or if it is a literal 16 bit nibble. Best case, a 64 bit 0 is represented by just 4 bits, worst case a random 64bit int will be represented by 68 bits.

3

Cool. But this idea was shelved?

Another idea;
How about using 6 byte integer for MPMA send amounts
The highest possible amount would be:
256^6 = 281,474,976,710,656

For divisible assets this is 2.814.749 units. This is more than the entire XCP supply. For PEPECASH this is $45,000 worth.

The gain is 2 bytes per send, which is significant, isn’t it?

In extreme cases where a larger amount should be sent, they can always fall back on a regular send type.

4

LZMA decoder is simple. But PPMd decoder is complex. LZMA2 is better than LZMA.LZMA2 compression does not replace (supersede) LZMA compression, but LZMA2 is merely an additional “wrapper” around LZMA. With LZMA2, data is split into blocks, but each block is still compressed by “normal” LZMA. Because individula blocks are compressed separately, processing the blocks can be parallelized, which allows for multi-threading. LZMA2 also allows “uncompressed” blocks, to better deal with “already compressed” inputs.