Enhanced Asset Information Specification

Development Counterparty
1

The enhanced asset info spec we have from the founders is good, but is very basic.

I have been meaning to write a CIP to extend this asset information to allow for additional information to be specified in a standardized way, but have never got around to it.

Now that we are at a point where users are abusing the ‘description’ field on xchain.io and using it to insert audio and video players and other random html, I feel defining the spec is a must. It is clear people want to be able to use audio/video and other content in their asset JSON, but we have no standards defined to do so.

Below I have come up with a basic schema which allows asset owners to specify much more information about their token, including owner and contact information, social media, as well as a way to specify audio, video, images, files, and dns in a standardized format.

Proposed v2.0.0 JSON Schema

Proposed v2.0.0 JSON Example

This spec has been discussed in the Counterparty Dev channel on telegram, and suggestions from those chats have been incorporated into the above proposed spec.

1 Like
2

@JPJA had some good feedback on how the spec should include support for sha256 hashes for audio/video/images/files in the JSON schema. He also suggested adding the sha256 hash to the asset description to make validation easier for those who wish to validate JSON data.

JPs suggestions have been incorporated into this new proposed spec.

His thread is Immutable JSON / CIP25

3

I feel like this is investing further in a kludge. The “description” field is not the best way to persist info about associated media, is it? Isn’t this approach likely to need continual revisions over time as platforms change? I’m expecting imgur to shut down any time TBH.

Media descriptors are such an important property of a token that they deserves to be a first class citizens IMO, and have a database field dedicated to it. Side benefit: we can use the description field for, say, actual descriptions! (or hashes for the nerds)

4

The “description” field is not the best way to persist info about associated media, is it?

I believe it is, since using a single description field we can point to a JSON file which can have an infinite amount of information associated with it.

Isn’t this approach likely to need continual revisions over time as platforms change?

Yes, in the future this spec may change to add some additional fields to allow specifying even more information, but the spec that I have defined allows for MUCH more information now than the current 1.0.0 spec. This 2.0.0 spec should work for many years into the future, as it supports image, audio, video, files, and DNS.

Media descriptors are such an important property of a token that they deserves to be a first class citizens IMO, and have a database field dedicated to it.

I disagree as I believe this would unnecessarily bloat the CP database by storing more information in the database than is necessary. The counterparty database needs to stay as small and fast as possible, which means only including necessary information on transactions, not storing all sorts of additional meta-data for an asset.

The JSON meta-data can be cached to make it just as highly available as the Counterparty database, without the need to unnecessarily boat the CP database by storing meta-data.

5

Good work! Only a few comments;

  • For images, should the types standard, large, hires have defined acceptable ranges of height, width, MB?
  • JSON format with Hash. Make it clear that the the hash must be of the JSON file itself; 64-character sha256 hash of the json file.
  • Make https:// optional for URLs in description fields.
  • Also allow a shortened hash format for json; domain.com/info.json;HASH_SHORT. I suggest sha256 encoded with base64 and truncated at 16 chars (96 bits).

The basis for the last two points is to fit op_return. The max description length that fits is 52, and even one more char is significantly more expensive (not a big deal now perhaps, but we should prepare for future higher fees). A quirk with asset issuances is that the description must be included in every subsequent issuance (like transfer ownership, issue additional), thus a long description makes all later issuance transactions more expensive too.

6

This project has gone its whole duration without upgrading the json information. I do not see why there should be a sudden rush to get this new standard finalized without adequate input. While the community reviews each other’s work before publishing, it would be prudent to set the due date for responses to this proposal at least 60 days in the future. I know I would personally like at least 30 days (from OP) to gather my research and experiments into a formalized proposal, outlined with explanations and example code.

Due to the nature of the current setup it is possible that someone can create whatever JSON structure they like for any token. Furthermore they can load this json data to their own webpage. Telling users how they can and cannot use the platform is absurd. In this case more than most. However it is important for artists, publishers, and developers alike to be on the same page. This should not be an edict, but something that is agreed upon by those who use it. While there may currently be only one explorer, it does not belong to the community. When new explorers are developed (as some are, currently), they should be given the option to conform to established standards, instead of encouraging their users to practice proprietary methods.

Developers that are currently busy building things for the protocol should have a say on what this new standard covers. It is important that current development is set forth cooperatively so that newcomers in the future can be easily familiar with common practices. Mostly it is important for developers to be able to take into consideration the various metadata structures that can be supported by the protocol, when creating something for users.

Frankly the work arounds of hard linking three centralized hosting services should not be an official standard. Suggesting use of these brands is tantamount to sponsorship or free advertising. AFAIK, Imgur was originally used with freeport because the developer claimed they did not want to curate content, and relied upon this free service for such. Youtube has a reputation for deleting content. Soundcloud offers a free limited space and requires users to pay for more. None of these services are based on peer to peer technology, none of them are open source and all of them are corporate endeavors who do not sponsor counterparty.

Food for thought on a matter of perception. The description value of each Counterparty token has a history. Each entry can be used to store and access data. For example, the sha256 of the main file of the token can be the first description. The next would be an IPFS CID. In this way, the asset itself can have the minimum required metadata on the protocol itself. These kinds of variables would most likely never change, and this information can be aggregated. The practice of limiting character/byte limit to 45 characters or less is usually followed as to avoid btc change issues in the wallet. The offset to that disadvantage is that doing so assists in permanence of cryptoart without reliance on clearnet webhosting and domains in perpetuity.

{“ipfs”:“bafybeide4mourj5zicsbz5b6hqernud6va7y3rzvuwampsjz4xpypasa24”}
70 characters.

For that matter an often underutilized feature has not been brought into consideration TTBOMK. Using the broadcast feature allows the publisher to include any metadata on the protocol with a fee that increases with length. Broadcasting the metadata for a token, then using the token description to connect via tx# will allow for permanent storage of the metadata within the same system as the token. This guarantees that as long as the token is accessible, so is its essential metadata.

The above option is to be considered for permanent and most important aspects of the metadata. Other metadata, that may change over time, or is less vital can be stored with existing json linkage via URLS. Variables such as a description for visually impared, caption at the bottom of the image, alternative sizes of the file, the thumbnail image with less priority can be fetched and edited via linked description file. This can take into consideration the other utilization of history to have one description of a token link directly to the json data in a broadcast, followed by the json data linked in the next description update.

I do have a concern that the more complicated the JSON structure gets, the more difficult it will be to program around. It would be the best case if all explorers could have the proper functionality built in so that all enhancements do not break if one value is empty. It is also troubling to see “undefined” displayed over and over again. In order to look competent, the publisher currently needs to include a series of empty values in each file. With a dozen extra empty variables per token, that is quite a bit of data bloat on the side of the json web host. Not to mention more internet traffic which would add up once the number of bloated json assets gets to the tens of thousands or more.

7

I do not see why there should be a sudden rush to get this new standard finalized without adequate input.

There is no rush to get anything finalized… in case you missed it the CIP status is “DRAFT” and will remain so for a long time to give people a chance to give feedback and improve this standard.

Due to the nature of the current setup it is possible that someone can create whatever JSON structure they like for any token. Furthermore they can load this json data to their own webpage. Telling users how they can and cannot use the platform is absurd.

Not sure where your getting this idea that anyone is telling users how they can and cannot use the platform… we are simply standardizing fields in a JSON file so that people can put data into the JSON file in a standardized format and display the content in a standardized format on blockchain explorers, etc. Nowhere is anyone saying this is the only JSON structure that will be used, people are still free to create their own JSON files containing whatever info they want and point their asset to them.

Developers that are currently busy building things for the protocol should have a say on what this new standard covers

Again, see my first comment where I mention that the CIP is in DRAFT mode (I encourage you to read CIP1 and how the CIP process works, and specifically the various statuses and what they mean). I believe your misinterpreting a CIP being in the repo as it being finalized and for some reason have the mindset that something is trying to be forced through here, which is not the case.

Frankly the work arounds of hard linking three centralized hosting services should not be an official standard.

Huh? What are you talking about? CIP25 is about standardizing the JSON file format to support additional information and documenting asset formats which already work (again, I think your misreading / misinterpreting CIP25)… Nowhere does it say that we should standardize hard linking of centralized hosting services (in fact, the CIP25 is proposed so that we can put more information in the JSON file in a standardized way and get AWAY from using funky formats for centralized services.

If your issue is that we should not be linking any assets to imgur, soundcloud, or youtube in the past… well, one of the only reasons why XCP has NFTs and is able to easily link images to NFT tokens is BECAUSE I came up with the “hack” a few years ago to support this custom format in asset descriptions on xchain.io… this was never meant to be standardized as how people should link images forever… and was just a way for users to easily integrate their NFT image with an NFT token immediately… wether you agree with this “hack” formatting on xchain or not, there is no arguing that without these hacks, we wouldn’t have anywhere near the level of NFTs we have on Counterparty (5000+ NFTs link to imgur images)

For that matter an often underutilized feature has not been brought into consideration TTBOMK. Using the broadcast feature allows the publisher to include any metadata on the protocol with a fee that increases with length.

If you would like to put forth a proposal for your TTBOMK formatting idea, go for it. There are other proposals for storing meta-data on Counterparty as well such as the BVAM method. Please take the time to write up your proposal into a separate CIP for consideration.

Broadcasting the metadata for a token, then using the token description to connect via tx# will allow for permanent storage of the metadata within the same system as the token.

Yes, and the CP database was meant for storing transactions related to tokens, it was never meant to be a general storage for meta-data, as that would unnecessarily bloat the database with meta-data, slowing down database queries. Also what happens when someone changes their meta-data, we now have to bloat the database even further to store that new meta-data, and we are now storing a bunch of OLD meta-data as well. The counterparty database IS NOT the place to store asset metadata, as the database should be as small/fast as possible and only contain information on transactions (there is a reason why the original founders didn’t create a meta-data file and instead came up with the point asset to JSON with enhanced info method)

Meta-data can very easily be cached on servers to make it instantly available without having to make slower external calls to get JSON files from their original hosts (many are missing content-allow-remote-origin headers, so the API calls to get JSON fail).

Here is a simple tool I wrote up last week to start caching all JSON locally on xchain. The general idea is to get the JSON data for an asset available immediately in a single API call without the need to make external calls to get the remote JSON.

As you can see, caching the JSON metadata in an external database and using that database to return JSON can be very fast, much faster than if the meta-data was stored in the CP database… The solution to asset meta-data is NOT to bloat the CP database to allow storing this information (there is a reason the founders didn’t do this, and instead came up with a way to point assets as JSON files to get more meta-data… had they wanted the meta-data stored inside CP, it would have been trivial to do so, but a poor design decision)

I do have a concern that the more complicated the JSON structure gets, the more difficult it will be to program around.

All the new fields are optional. You can continue to use the existing spec as you do now, so not sure what is more complicated other than figuring out “oh, I want to show a video, that should go in the video array”… or “I want to show an image, put the image in the image array”. The JSON files can contain as much or as little data as the user wants.

It would be the best case if all explorers could have the proper functionality built in so that all enhancements do not break if one value is empty.

umm… this is EXACTLY what the CIP25 spec is doing, standardizing the fields in JSON files so that all block explorers can treat the data the exact same.

It is also troubling to see “undefined” displayed over and over again.

  1. if your going to mention seeing something, mention WHERE your seeing it
  2. the only reason that we see “undefined” on xchain is because there is no standard JSON spec defined… CIP25 solves this issue… so, your complaining about something, while not understanding that this CIP standardizes the fields that can be used, which will in turn get rid of the “undefined” message on xchain after I get a chance to update to start using CIP25 spec.
8
  • For images, should the types standard, large, hires have defined acceptable ranges of height, width, MB?

I have the optional “size” field in the images array where people can specify the size/dimension of their images, so i’m not sure if we need acceptable dimension ranges… I kinda prefer to leave it open and let people specify their image size if they want.

As far as limiting file size, I am torn on that issue, as I don’t think we can really do any enforcement of file size referenced in JSON file. About all we could do is define a max file size for each of these image types in the CIP, and then any developer who is using the JSON data (explorers, etc) would need to do their own limitation of loading of images that are over X size… which might not be a bad idea.

Do you have any file size suggestions for the different types? icon, standard, large, hires?

Thanks for this feedback JP. I’ll be sure to work these updates into the next CIP update :slight_smile:

9

No, would need to research this further. Maybe some other NFT protocol uses standard sizes? Can borrow from them.

The archiver I made aborts the download once an upper limit is reached. 30 MB or so.

10

We should also add support for “pdf” to the “files” array… and support showing PDFs on xchain.io

1 Like
11

Are we reading the same proposal?

Supported Asset Description Formats

Below are a number of formats which should be recognized by block explorers.

12

It would be nice to have a section dedicated to public blockchains - so for example the user could include a Samourai wallet paynym, or their lightning node address.

13

This is to support legacy formats which have already been being used for a few years. The CIP25 spec is being defined so that we can get away from needing to rely on 3rd party hosting services… but, unless your wanting block explorers to break and not display NFT images which were working previously, we need to support previous commonly used formats to remain backwards compatible.

14

it is difficult to have a constructive conversation with someone who ignores input and regurgitates logical falacies. “dont be a dick” is a good rule for everyone to follow, even those who like to gas light.

Please re-read what you are responding to. None of my concerns have been addressed for this point. It is not the responsibility of the entire community to assume into standardization of the project a mistake (quick-hack) just because there has been a situation created from previous irresponisible usage.

Lets get realisitc please, and not hyper inflate the discussion to the point of absurdity just to try to get our way by deflecting any logic. These highly defended and centralized services very well may not be around in 10 or 30 years. Who is to blame for leading hundreds of noobs down the path of destruction? Why cause more of a problem later, when a best practice can be established now?

15

It is quite frequent that pdf files are loaded with malware. I do not see any explorer with merit chosing to take on the liability of spreading questionable pdfs.

16

I have now integrated CIP25 support into xchain.io and returned the JSON description field to a text only field on XChain, and will no longer support displaying HTML in the description field after 30 days (giving services a bit of time to migrate to CIP25 with minimal interruption)

Loading user-supplied HTML content is generally an unsafe practice and is frowned upon, as the HTML/javascript code could do nefarious things. (For example, Someone could write up some code to dump a private key from a wallet and make an external call to pass the private key to a server, redirect you to some other site, install a CPU miner on your machine, or worse)

In making these changes to how the description field is treated on XChain, it became clear to me that there is a desire in the community to be able to push the limits of what is possible with tokens, and that we should provide some way to safely load/display user-provided HTML content.

Here are a few examples of tokens which are doing unique things by injecting HTML into an iframe

Rare Pepe Puzzle
https://xchain.io/asset/FAKEPUZZLE

Interactive / Augmented Reality Artwork
https://xchain.io/asset/PEPESTREAM
https://xchain.io/asset/NEWPEPEDESU

Artwork that changes based on the time of day
https://xchain.io/asset/FOURPEPAS

I propose we update CIP25 to add a generic html field where users can pass a chunk of HTML code to be optionally displayed.

I plan to update XChain to support this new “html” field and will allow loading of user-supplied HTML content after displaying a warning message and clicking a ‘Load’ button.

17

I have submitted a PR request for CIP25 to add support for the ipfs format where the user can set the asset description to ipfs:CID where the CID is a the unique content identifier.

This ipfs format only will work for JSON files

The ipfs format is automatically translated to the URL https://ipfs.io/ipfs/CID

1 Like
18

arweave:CID (hash) already supported?

19

arweave:CID is not currently supported. I suggest we add it to the spec in a similar format to ipfs, where the hash always points to a JSON file with additional asset information.

Thoughts?

20

Just submitted a PR to add ordinal inscription support

2 Likes