How can I verify if my node is synced up to the head of the blockchain?

Question

I noticed today that one of my telegram bots on a RAM channel was giving wrong data. I had to switch the bot from using the local EOS node to a BP's public node. When I compared the two nodes (my local and BP's node) using cleos, I noticed for example:

$ cleos get table eosio eosio rammarket
{
  "rows": [{
      "supply": "10000000000.0000 RAMCORE",
      "base": {
        "balance": "15799128628 RAM",
        "weight": "0.50000000000000000"
      },
      "quote": {
        "balance": "4349655.6116 EOS",
        "weight": "0.50000000000000000"
      }
    }
  ],
  "more": false
}

$ cleos -u https://api.eosnewyork.io get table eosio eosio rammarket
{
  "rows": [{
      "supply": "10000000000.0000 RAMCORE",
      "base": {
        "balance": "48916058380 RAM",
        "weight": "0.50000000000000000"
      },
      "quote": {
        "balance": "3168032.3075 EOS",
        "weight": "0.50000000000000000"
      }
    }
  ],
  "more": false
}

So it seems like my node is out of sync. The data does not match for the rammarket table. How can I ensure or verify that my local node is synced?

Answer 1

I decided to write a quick PHP script to see how far off my local node and how long will it take to catch up. Here is an example run of the script:

$ php how_far_apart.php

10:55:54 remote: 29,689,570 local 3,134,586 diff: 29,689,570.  @ 279 blk/sec  est. sync: 30 hrs
10:55:59 remote: 29,689,581 local 3,136,789 diff: 29,689,581.  @ 440 blk/sec  est. sync: 19 hrs

[removed for brevity]

10:56:41 remote: 29,689,666 local 3,162,111 diff: 29,689,666.  @ 923 blk/sec  est. sync:  9 hrs
Based on 10 samples, at average sync @ 578.4 blk/sec est. sync in 34 hours

Here is the script:

    <?php

date_default_timezone_set('America/Los_Angeles');

define('CLEOS_CMD',       '/var/production/eos/build/programs/cleos/cleos');
define('LOCAL_ENDPOINT',  'http://localhost:8888');
define('REMOTE_ENDPOINT', 'https://api.eosnewyork.io');


$interval  = 5;
$max_tests = 2 * $interval;
$testing   = 0;
$stats     = [];

$local_head_block_num  = get_head_block_number(LOCAL_ENDPOINT);

while($testing < $max_tests) {

   $now = date('h:i:s');
   $remote_head_block_num = get_head_block_number(REMOTE_ENDPOINT);
   $diff                  = $remote_head_block_num - $local_head_block_num;

   if ($diff <  0)
       die("Local node is ahead of the remote.");

    sleep($interval); // nap for interval seconds

    $last                      = $local_head_block_num;
    $local_head_block_num      = get_head_block_number(LOCAL_ENDPOINT);
    $diff2                     = $local_head_block_num - $last;
    $sync_rate                 = $diff2 / $interval;

    $diff                      = $remote_head_block_num - $new_local_head_block_num;
    $est_time_to_sync          = $sync_rate > 0 ? ($diff / $sync_rate) / 3600 : '0';

    printf("$now remote: %s local %s diff: %s. ", number_format($remote_head_block_num),
                                                  number_format($local_head_block_num),
                                                  number_format($diff));
    printf(" @ %03d blk/sec  est. sync: %2d hrs\n", $sync_rate, number_format($est_time_to_sync,0));



    $stats['sync_rate'] += $sync_rate;
    $stats['eta']       += $est_time_to_sync;
    $testing++;
}

$avrg_sync_rate = $stats['sync_rate'] / $max_tests;
$avrg_eta       = number_format($stats['eta'] / $max_tests, 0);

echo "Based on $max_tests samples, at average sync @ $avrg_sync_rate blk/sec est. sync in $avrg_eta hours\n";

exit;

function get_head_block_number($url) {

   $cmd = CLEOS_CMD . ' -u ' . $url . ' get info';

   if (!$output = shell_exec($cmd))
       return false;

   $info           = json_decode($output);
   return $info->head_block_num;

}

Answer 2

Without using an external service like a blockchain explorer to check the latest block you can do the following.

Check the timestamp of the last block your node has. If it too far back in the past (5 minutes) you can assume your node has not received new blocks and is out of sync.