The get_account_history API call is based on a number for each transaction. So for my account with 130k transactions I can, for example, make a call for a group of transactions starting from number 130000 and going back to 129000 (it goes backwards).

However, these numbers can jump around. If this happens then when I call the next group of transactions (129000 to 128000) transaction 129000 may not be the same as in the previous call.

This could result in a history overlap (and duplication issues - less of a problem) or a gap (and missed transactions - more of a problem).

This is rare on the accounts that I've tested on Hive. On my account it doesn't happen at all. Similarly for accounts up to 500k transactions it doesn't seem to happen. It looks like it may be more of a problem on larger accounts, particularly very large accounts.

To prevent the loading issues described above I chain each batch of transactions together using the block number (plus more detailed transaction id info). So if transaction 129000 (the anchor transaction) is a different transaction in each of the two API calls the check fails.

At this point the load is stopped. My assumption has been that the jumping is temporary and would revert after a short delay, allowing the load to be restarted. However as the problem comes and goes it's difficult to test explicitly.

The issue is further complicated by 'empty' call returns which are caused by long periods with no financial transactions in the account history (lots of json for example). In this situation there is no anchor transaction and the 129000 number becomes more important. Jumping around at this point becomes more problematic.

It's something on my list to improve. I should be able to solve it eventually! @blocktrades team is currently working on the account_history side of things so this is going to lead to some changes that may make things easier.

Posted Using LeoFinance ^Beta