Imagine situation when I need to save a big data In smart contract (for example 1000 000 000 row of information - or just 1 TB). In case of ETH it's almost impossible because gas price goes to hell. How is the situation in EOS? I know that concept of gas doesn't exist here, but here we have RAM limit that depends on our stake, more stake more limit. but how much stake will be enough? Is it common sense to save that much data it EOS smart contract? what's the solution?
Add a comment
|
2 Answers
Same situation as in other blockchains: large data should not be stored on the blockchain. Keep in mind that besides block producers, all validation nodes / non-producing full nodes have to be able download the chain if they want, so hosting large data is not encouraged. Consequently, the costs for doing this are equally prohibitive.
If you want to store large data, the solution recommended by BlockOne at the moment is Demux:
Demux solves these problems by off-loading storage and queries to your preferred database (MongoDB, Postgres etc). As blockchain events happen, your chosen database is updated by Updater functions, which deterministically process an array of Action objects. The database can then be queried by your front-end DAPP through a suitable API (for example, REST or GraphQL), reducing the amount of direct interaction with the blockchain in order to retrieve data for your application. This also can help developers to limit the amount of data that needs to be stored in RAM, lowering applications’ operating costs whilst also improving their architectural efficiency.
Otherwise, storing the hash of the file or integrating your application with IPFS is another recommended path, and BlockOne published a paper on an unreleased EOSIO storage solution that integrates with IPFS:
Abstract: EOS.IO Storage is a proposed decentralized file system designed to give everyone the ability to permanently store and host files accessible by any web browser. Unlike some other proposed alternatives, there would be no upfront fee or ongoing charge for storage or bandwidth on EOS.IO Storage aside from a completely refundable deposit. Users must hold tokens while they need storage and bandwidth and may sell tokens when storage and bandwidth is no longer required. Built on the InterPlanetary File System (IPFS) and the EOS.IO software, EOS.IO Storage will be a service provided by block producers for those who hold tokens on a blockchain that adopts the EOS.IO software. The block producers would be incentivized to replicate and host these files, allowing anyone with an Internet browser to access them.
answered Aug 22, 2018 at 16:27
-
If you save data on IPFS, or centralized database, then what's the purpose of smart contract at all? Smart contract and blockchain comes in place so that you don't hide anything. My app would better be written in mysql, but the whole purpose is to show everyone that you don't hide it. :) Commented Aug 22, 2018 at 16:32
-
store on chain important data that users need on the chain, and off chain less critical data. if you have 1tb of data that needs to be on chain, consider that public blockchains may not be appropriate for your usecase Commented Aug 22, 2018 at 16:36
-
What about saving them in logs? People say for big data, logs are the solution. Commented Aug 22, 2018 at 16:39
-
you can do that in theory, but you need to stake EOS for NET, you can't query the log from the contract, and there is no guarantee anyone will be hosting that part of the log after a period of time as people will prune the chain. I don't think the log is meant to be a solution for big data at all Commented Aug 22, 2018 at 16:42
-
Can we have a little talk in private maybe telegram or skype ? I also got some ideas and I'd love to discuss it with you . let me know Commented Aug 22, 2018 at 16:44
Storing data on blockchain is extremely expensive. It is also problem for network (data transfer) and other users (storage).
Instead you should consider storing data in one of dedicated solutions, centralized (AWS, Google...) or decentralized (IPFS, Sia...) and store on blockchain only information where to find data and signature to guarantee it's correctness.
-
If you save data on IPFS, or centralized database, then what's the purpose of smart contract at all? Smart contract and blockchain comes in place so that you don't hide anything. My app would better be written in mysql, but the whole purpose is to show everyone that you don't hide it. :) Commented Aug 22, 2018 at 16:32
-