I run a non-producing node with 8GB RAM on Ubuntu 18.04 and 4 CPU cores, no Docker(yet). Overnight mongod crashed, not sure why. I just had to start mongod from the /opt folder again and it worked.
But now the sync process is super slow. It's at 20860100 of 22222260 blocks Example: time per: 562566, size: 1, time: 562566

Mongod takes ~ 150% CPU usage and 41% RAM nodeos: 20-45% CPU and ~50%

Is this normal? I'm thinking of stopping the node and upgrading the server to 16GB RAM.
Strange enough my API does not access the full mongoDB, only the parts after the restart of mongod.
1) Is it possible that mongod crashes because of lack of RAM?
2) RAM usage normal?
3) Stopping the node with best practices and restarting does not force me to --mongodb-wipe?

my config.ini:

blocks-dir = "/mnt/xfs/mainnet/blocks" 

bnet-endpoint =
bnet-follow-irreversible = 0
bnet-no-trx = false
read-mode = read-only
validation-mode = light
mongodb-uri = mongodb://
mongodb-queue-size = 2048
mongodb-abi-cache-size = 2048
mongodb-block-start = 1
mongodb-store-blocks = 0
mongodb-store-transactions = 0
mongodb-store-transaction-traces = 0
mongodb-store-action-traces = 1
mongodb-filter-on = *
mongodb-filter-out = eosio:onblock:
mongodb-filter-out = gu2tembqgage::
mongodb-filter-out = blocktwitter::

wasm-runtime = wabt
p2p-max-nodes-per-host = 1
http-validate-host = false
https-client-validate-peers = 1
abi-serializer-max-time-ms = 5000
chain-state-db-size-mb = 32000
reversible-blocks-db-size-mb = 340
contracts-console = false
allowed-connection = any
max-clients = 100
network-version-match = 0
sync-fetch-span = 500
connection-cleanup-period = 30
max-implicit-request = 1500
access-control-allow-origin = *
access-control-allow-headers = *
access-control-allow-credentials = false
verbose-http-errors = true

plugin = eosio::chain_plugin
plugin = eosio::chain_api_plugin
plugin = eosio::bnet_plugin
plugin = eosio::mongo_db_plugin
plugin = eosio::http_plugin  

Your situation is most likely normal.

Are you running on a bare-metal server or a virtual/cloud server? A bare-metal helps a lot but 8GB does not seem plenty.

nodeos is very CPU intensive during replay/initial sync as it's single threaded, and mongod is both CPU intensive and RAM intensive.

In my experience, it takes about a day to initial sync a nodeos without mongo_db_plugin, about 2 days to replay (previously downloaded from https://eosnode.tools/blocks) local block files with mongo_db_plugin enabled, on a Intel Xeon 3.8GHz 4 Core + 64GB DDR4-2666 bare-metal. I tried to initial sync with mongo_db_plugin enabled, but it took forever so I just gave up.

For your questions,

1) Yes. It's fairly often for mongod to crash when RAM is not enough.

2) Yes. Mongo will use a large portion of RAM. According to mongodb doc (https://docs.mongodb.com/manual/core/wiredtiger/),

Starting in 3.4, the WiredTiger internal cache, by default, will use the larger of either:

  • 50% of (RAM - 1 GB), or
  • 256 MB.

3) It should be fine to stop nodeos gracefully (like with Ctrl+C) and then start without replay or --mongodb-wipe.

Btw, it will save you a lot of time and storage space by filtering out spammer accounts. Please refer to https://github.com/eosbean/eos-spammer-list and add filter-out lines to your config.ini .

  • I was looking for a spammer list! Thanks!! Still think you cannot gracefully shutdown nodeos without a replay. Even when using stop.sh – Marcel Blockchain Nov 23 '18 at 16:21

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.