CIP Proposal 18 - Configuration flag to remove API query limit defaults

This is the discussion thread for the proposed CIP 18 to remove API query limits by configuration flags.

Link to the CIP: https://github.com/CounterpartyXCP/cips/blob/cip18/cip-0018.md

Please, discuss.

Overall I think making API limit a configurable option is a good idea.

Adding an API_LIMIT_1000 flag in config.py, making it configurable from the counterparty-cli code, and acting on its setting in api.py.

Rather than a boolean flag to indicate a limit of 1000 or no limit, I would prefer a API_LIMIT flag which can be configured to any integer, with 0 indicating no limit .

This could make sense, would there be any issue limiting queries below 1000? wouldn’t that be backwards incompatible? (software should be aware then that there’s a limit different than expected, maybe get that info on get_running_info call?)

Changed the CIP to account for @jdogresorg suggestion, which seems reasonable.

@chiguireitor ch What would the default API_LIMIT_ROWS be in the config file?
You mentioned in the CIP that care needs to be take so that no default configuration makes the node susceptible to DDOS attacks

IMO the default should be 1000 (same as it is now). The new feature is the ability to customize that limit or disable it entirely (set value to 0).

1 Like

Indeed, default should be 1000 to allow non-configured-for-this-feature instances to continue working the same

A CIP bounty address has been requested for CIP-18.

Address 13CoTjbVaYzg9kQ2arjKcCmzT5WZmRfgdq has been provisioned to hold donated funds for CIP-18

1 Like

Initial commits for CIP18 support

Testing underway

What should be the expected behaviour for a server with a limit lower than 1000 with a non-limited call? silently send the limited amount and pray for the best or fail fast so developers get informed about the non-standard limits (current coded behaviour)?

PRs merged to master:

Milestone #1 achieved (although merged to master, won’t ask milestone #3 after testing #2) https://github.com/CounterpartyXCP/cips/blob/master/cip-0018.md

Installing mainnet server for the test atm. Sync up to block ~423k right now

Test server with on mainnet has been deployed and synced up (finally)

http://mg.contimita.com:4000

Username/Password: rpc/1234

Current api limit rows: 0

Limit should be specified explicitly in calls to prevent standard default behaviour (if not specified in the call, a limit of 1000 is applied).

Because of the size of some calls, unbounded queries over a non-local uplink will take a good time to complete, be sure about what you want to query before putting “limit”: 0

Also, just to note… some calls like “get_issuances” with limit 0 are gona DoS the server for a very long time while replying stuff… if it happens hit me up here or in the telegram channel (same nick as here)

I did a few simple tests using get_issuances and get_balances and passing 0 or a limit over 1000 returns more than 1000 results as expected.

get_issuances API call with 0 limit returns over 1000 results

get_balances API call with 0 limit returns over 1000 results

Ok, restarting the xcp server with limit set to 100

Default behavior is to error out because of the lower limit.

Found a bug: Setting limit to 0 on a limited or default server returns the whole set.

1 Like

Bugfix pushed to master

Seems like the feature is done, slated for release on the next milestone (you can also pull from master to get the latest).