How can I download all ledger blocks as a JSON dump?

Question

I'm trying to download a relatively up-to-date dataset of all blocks in JSON format, but haven't found a good solution:

• cleos can only fetch blocks one by one ($ cleos get block n) which is way too slow;

• nodeos saves the log as a binary log, and the mongodb and sql plugins from B1 don't seem to be usable in the latest release;

• EOSTracker has an API, but (understandably) it throttles frequency of connections/number of blocks per page, and sometimes I get invalid responses (not sure if the HTTP API is documented anywhere)

Any other ideas? Other structured format than JSON is also welcome as long as it can be processed relatively fast with non-EOSIO tools (eg. SQL dump is also fine)

Answer 1

The best way I found to download the ledger in a structured format is to use eosio::mongo_db_plugin on latest EOSIO release, run $ nodeos --mongo-uri mongodb://127.0.0.1:27017 --replay-blockchain, and then query using mongo or something like pymongo.

$ mongo
...
connecting to: mongodb://127.0.0.1:27017
...
> use EOS
switched to db EOS
> show collections
accounts
actions
block_states
blocks
transaction_traces
transactions
> db.accounts.find({"name": "eosio"})

Answer 2

Firstly create a json file:touch ledger.json

Then you can try this request to where nodeos running: curl --request POST \ --url http://127.0.0.1:8888/v1/chain/get_table_rows > ledger.json click for detail

Or you can use cleos --url http://127.0.0.1:8888 get table <contract> <scope> <table> > ledger.json click for detail