Stuck at block 6709554 when parsing HE chain

[forkyishere]

Not related with #25, but since its implementation needed a full replay, when I have attempted a full history rebuild using node version bellow, I can't get past HE block 6709554.

5|prod-hivenghist-parser-bkp  | parsing block #6709554
5|prod-hivenghist-parser-bkp  | TypeError: Cannot read properties of undefined (reading 'data')
5|prod-hivenghist-parser-bkp  |     at parseTransferOperation (/prod/ssc_tokens_history_backup/contracts/tokens.js:23:16)
5|prod-hivenghist-parser-bkp  |     at parseNftBuy (/prod/ssc_tokens_history_backup/contracts/nftmarket.js:75:11)
5|prod-hivenghist-parser-bkp  |     at runMicrotasks (<anonymous>)
5|prod-hivenghist-parser-bkp  |     at processTicksAndRejections (node:internal/process/task_queues:96:5)
5|prod-hivenghist-parser-bkp  |     at async parseNftMarketContract (/prod/ssc_tokens_history_backup/contracts/nftmarket.js:219:7)
5|prod-hivenghist-parser-bkp  |     at async parseTx (/prod/ssc_tokens_history_backup/history_builder.js:102:5)
5|prod-hivenghist-parser-bkp  |     at async parseBlock (/prod/ssc_tokens_history_backup/history_builder.js:140:5)
5|prod-hivenghist-parser-bkp  |     at async parseSSCChain (/prod/ssc_tokens_history_backup/history_builder.js:168:7)
5|prod-hivenghist-parser-bkp  | Using SSC node: http://127.0.0.1:5000

Node: v16.17.1
Mongo: 4.4.21
Ubuntu: 20.04 LTS

[Rishi556]

That's a new one. I don't think it's related to #25 since the node I'm syncing with that code has made it to 8960380 before I manually stopped it. I am running on Node 18, mongo 6 and ubuntu 22 though. It looks like an issue with the NFT stuff, which @primersion worked on.

[forkyishere]

Probably a node version problem. Will try with 18 to see if that sorts it out. If so, then we might need to bump node min requirement.

[forkyishere]

Tested with node v18.16.0, same problem.

Looking at the block 6709554 itself, makes me believe this transaction (d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0) is what's causing the parsing to fail:

{
  "refHiveBlockNumber": 53601938,
  "transactionId": "d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0",
  "sender": "themightyvolcano",
  "contract": "market",
  "action": "sell",
  "payload": "{\"symbol\":\"SIM\",\"quantity\":\"12000\",\"price\":\"0.0015\",\"isSignedWithActiveKey\":true}",
  "executedCodeHash": "34a62f39ce73ea129970433dba3c9799ef35d7e068a7a421a63173f11d4839666a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d766a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d766a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d76",
  "hash": "a7785e379e251ae645bcaf5289aa0e745847db0bc9a5eccda6f26c0f6cb92d9e",
  "databaseHash": "de7f3375d5db7ba7f2c3c1259bce889295ff27f79c15f01367b1b2d604674955",
  "logs": "{\"events\":[{\"contract\":\"tokens\",\"event\":\"transferToContract\",\"data\":{\"from\":\"themightyvolcano\",\"to\":\"market\",\"symbol\":\"SIM\",\"quantity\":\"12000\"}},{\"contract\":\"tokens\",\"event\":\"transferFromContract\",\"data\":{\"from\":\"market\",\"to\":\"d-pend\",\"symbol\":\"SIM\",\"quantity\":\"12000.000\"}},{\"contract\":\"tokens\",\"event\":\"transferFromContract\",\"data\":{\"from\":\"market\",\"to\":\"themightyvolcano\",\"symbol\":\"SWAP.HIVE\",\"quantity\":\"18.00000000\"}},{\"contract\":\"market\",\"event\":\"orderClosed\",\"data\":{\"account\":\"themightyvolcano\",\"type\":\"sell\",\"txId\":\"d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0\"}}]}",
  "beautifiedPayload": "@themightyvolcano created a sell order for 12000 SIM at 0.0015 SWAP.HIVE",
  "parsedPayload": {
    "symbol": "SIM",
    "quantity": "12000",
    "price": "0.0015",
    "isSignedWithActiveKey": true
  }
}

• logs field has multiple data fields for several contracts

EDIT (18/01/2024): The block above is problematic but looking at the HIVE data is not possible to understand why. One will need to look at the HE chain data where the error is coming from.

[Rishi556]

Can you try adding a log for logEvents right before this line? https://github.com/hive-engine/ssc_tokens_history/blob/hive/contracts/tokens.js#L18 Wondering what it says there

[forkyishere]

Is this the same you have:

parsing block #6709554
This is what is inside logEvent: {"contract":"tokens","event":"transfer","data":{"from":"hamsterino","to":"risingstargame","symbol":"STARBITS","quantity":"188"}}
This is what is inside logEvent: undefined
TypeError: Cannot read properties of undefined (reading 'data')

[Rishi556]

https://he.dtools.dev/b/6709554 yeah looks like the right block. logEvent being undefined is weird. It looks like it's failing on this particular transaction: d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0

[Rishi556]

Hmm, it should definitely be defined according to what I see in the transaction, can you try logging events just before this line? https://github.com/hive-engine/ssc_tokens_history/blob/hive/contracts/nftmarket.js#LL75

[forkyishere]

Is it because there is the assumption that each action only has 1 event?

parsing block #6709554
This is what is inside logEvent: {"contract":"tokens","event":"transfer","data":{"from":"hamsterino","to":"risingstargame","symbol":"STARBITS","quantity":"188"}}
This is what is inside events: [{"contract":"tokens","event":"transfer","data":{"from":"hamsterino","to":"risingstargame","symbol":"STARBITS","quantity":"188"}}]
This is what is inside logEvent: undefined
TypeError: Cannot read properties of undefined (reading 'data')

[Rishi556]

Interesting, it looks like there should be at least 2 actions, but you only seem to have 1. Can you grab the entirety of tx d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0 from your node and paste that here?

[forkyishere]

Its my public node...

RPC: he.atexoras.com:2083
History: he.atexoras.com:8443

{
   "_id": 6709554,
   "blockNumber": 6709554,
   "refHiveBlockNumber": 53601938,
   "refHiveBlockId": "0331e6922280ddd938d6fe6e5b04886a5057473d",
   "prevRefHiveBlockId": "0331e69147b1b9b3acf81f734f7b0e688fa73311",
   "previousHash": "f03baa1e3d873537ce027d99ef9c791476b8242b540da3fabbd52f7ed9a34563",
   "previousDatabaseHash": "21524407aecdb5ace88d37b70f71e5ed97c4f205bccc6df879f9c6d484d06e2a",
   "timestamp": "2021-05-04T18:39:00",
   "transactions": [
      {
         "refHiveBlockNumber": 53601938,
         "transactionId": "63002c925da6f3a585f531bec353d45359c84f69",
         "sender": "hamsterino",
         "contract": "nftmarket",
         "action": "buy",
         "payload": "{\"symbol\":\"STAR\",\"nfts\":[\"214654\"],\"marketAccount\":\"hamsterino\",\"isSignedWithActiveKey\":true}",
         "executedCodeHash": "cfc4ac3913026ec74f527b112f424db5bdea8c818e56a0195dd4ae96bc03ea736a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d766a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d76",
         "hash": "994c4c840d6ee1c943e6a17212013651c53502fc7046ab282ef90040ca4f9d13",
         "databaseHash": "715d1a8c85e1d548f1317da106b04be27f70ade2d54cd852c2f98b674671fdae",
         "logs": "{\"errors\":[\"cannot transfer to self\",\"unable to transfer agent fees\"],\"events\":[{\"contract\":\"tokens\",\"event\":\"transfer\",\"data\":{\"from\":\"hamsterino\",\"to\":\"risingstargame\",\"symbol\":\"STARBITS\",\"quantity\":\"188\"}}]}"
      },
      {
         "refHiveBlockNumber": 53601938,
         "transactionId": "d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0",
         "sender": "themightyvolcano",
         "contract": "market",
         "action": "sell",
         "payload": "{\"symbol\":\"SIM\",\"quantity\":\"12000\",\"price\":\"0.0015\",\"isSignedWithActiveKey\":true}",
         "executedCodeHash": "34a62f39ce73ea129970433dba3c9799ef35d7e068a7a421a63173f11d4839666a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d766a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d766a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d76",
         "hash": "a7785e379e251ae645bcaf5289aa0e745847db0bc9a5eccda6f26c0f6cb92d9e",
         "databaseHash": "de7f3375d5db7ba7f2c3c1259bce889295ff27f79c15f01367b1b2d604674955",
         "logs": "{\"events\":[{\"contract\":\"tokens\",\"event\":\"transferToContract\",\"data\":{\"from\":\"themightyvolcano\",\"to\":\"market\",\"symbol\":\"SIM\",\"quantity\":\"12000\"}},{\"contract\":\"tokens\",\"event\":\"transferFromContract\",\"data\":{\"from\":\"market\",\"to\":\"d-pend\",\"symbol\":\"SIM\",\"quantity\":\"12000.000\"}},{\"contract\":\"tokens\",\"event\":\"transferFromContract\",\"data\":{\"from\":\"market\",\"to\":\"themightyvolcano\",\"symbol\":\"SWAP.HIVE\",\"quantity\":\"18.00000000\"}},{\"contract\":\"market\",\"event\":\"orderClosed\",\"data\":{\"account\":\"themightyvolcano\",\"type\":\"sell\",\"txId\":\"d1fcccd9315c9c9e4a8b27d715c9a557fd4b54f0\"}}]}"
      },
      {
         "refHiveBlockNumber": 53601938,
         "transactionId": "907a9758fb549b8f33bacc389e02364c2306be08",
         "sender": "lifesavings",
         "contract": "market",
         "action": "cancel",
         "payload": "{\"type\":\"buy\",\"id\":\"0372b00b7e7a2b5ff592f8bea6d8af58dd8a0456\",\"isSignedWithActiveKey\":true}",
         "executedCodeHash": "34a62f39ce73ea129970433dba3c9799ef35d7e068a7a421a63173f11d4839666a67140199f5787fe068c92846e4beff0e2988ee61b98849b72b588ea4681d76",
         "hash": "256ce64b2352b43bcbebc63f3fe52871d5fe676225ce7cb580132fd04514d99f",
         "databaseHash": "089fa8e1783e48cbc264b108ed736b343bdc9e462cd21af59a83c71fe1ada174",
         "logs": "{\"events\":[{\"contract\":\"tokens\",\"event\":\"transferFromContract\",\"data\":{\"from\":\"market\",\"to\":\"lifesavings\",\"symbol\":\"SWAP.HIVE\",\"quantity\":\"3.20000000\"}}]}"
      },
      {
         "refHiveBlockNumber": 53601938,
         "transactionId": "4616b04bcaa239d2fa766ad9b7366d39c20502ce",
         "sender": "dcitybids",
         "contract": "nft",
         "action": "transfer",
         "payload": "{\"to\":\"gerber\",\"toType\":\"user\",\"nfts\":[{\"symbol\":\"CITY\",\"ids\":[\"1245435\"]}],\"memo\":{\"payment\":60,\"symbol\":\"SIM\",\"cards_load\":[],\"c_id\":\"c_wors_a\"},\"isSignedWithActiveKey\":true}",
         "executedCodeHash": "c712cbcb2afe485777f366dafcad8f1110ad1b2e9e37e92a6cf7a852820fdbf6",
         "hash": "6f9a2054d1a61e96639f8a758a782c0cbabdf615759712a5eb418bcc89a4dc57",
         "databaseHash": "d6a348f924d9eb7b5a934fc2bced6c8473a918efe11d31c81f03d80f823c96b5",
         "logs": "{\"events\":[{\"contract\":\"nft\",\"event\":\"transfer\",\"data\":{\"from\":\"dcitybids\",\"fromType\":\"u\",\"to\":\"gerber\",\"toType\":\"u\",\"symbol\":\"CITY\",\"id\":\"1245435\"}}]}"
      }
   ],
   "virtualTransactions": [],
   "hash": "e3302640263278c436b1a7f2063a6ce5156aaec5d38dd11ecd8630e14fbf5a16",
   "databaseHash": "b74acba170cd54d30cc0ff0ad44708361e7c0f43e7cfcc17130f6bc247dcf1ac",
   "merkleRoot": "b44eb14e68410496409684ed50824e4a63d2ddde2817ef09b084d95f8370e979",
   "round": 171966,
   "roundHash": "a584d99cea0898c861be1a7829c4b6c8ead4a7298117cf5d3e7a9750f4743611",
   "witness": "blervin",
   "signingKey": "STM5cTb5t9HXanrLADYqUD2mfn9xVru4dmbAy8Gxc8kEtuiHNnaLL",
   "roundSignature": "20241952cc3863510113529bd9e1c07251303971e1619144828a48c53e5cb636f16f88bfa05cf79e911910a655cf4a96bfbfaf24bd5c1cf7b981eb800982f6889e"
}

[Rishi556]

Hmm, I'm getting the same issue now,

0|HistoryParser  | parsing block #6709554
0|HistoryParser  | TypeError: Cannot read properties of undefined (reading 'data')
0|HistoryParser  |     at parseTransferOperation (/root/ssc_tokens_history/contracts/tokens.js:23:16)
0|HistoryParser  |     at parseNftBuy (/root/ssc_tokens_history/contracts/nftmarket.js:75:11)
0|HistoryParser  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
0|HistoryParser  |     at async parseNftMarketContract (/root/ssc_tokens_history/contracts/nftmarket.js:219:7)
0|HistoryParser  |     at async parseTx (/root/ssc_tokens_history/history_builder.js:102:5)
0|HistoryParser  |     at async parseBlock (/root/ssc_tokens_history/history_builder.js:140:5)
0|HistoryParser  |     at async parseSSCChain (/root/ssc_tokens_history/history_builder.js:168:7)
0|HistoryParser  | Using SSC node: http://localhost:6000

wonder what is causing this and how it worked on one machine, but failed on another. Did you ever get past it @forkyishere?

[Rishi556]

Looks like it happens again on block #6709577.

[forkyishere]

No, never got passed that, and been relying on other snapshots.

I am going to do full syncs this weekend (still on the benchmarks phase) and document if I get still stuck.

What version of node? (don't think this has to do with mongo, but I am running 7 now as well on the new env)

[Rishi556]

Again @ 9536651

Node @ v18.19.0 and Mongo @ 7

[Rishi556]

Again @ 9536695

[forkyishere]

But wait, you are getting different blocks every time you go again, or you are saying that you found other blocks ahead with the same problem?

[Rishi556]

Again @ 13486273

[Rishi556]

But wait, you are getting different blocks every time you go again, or you are saying that you found other blocks ahead with the same problem?

Another block ahead with the same issue. I've been skipping these bad blocks for now till I have time to figure out what causes it and fix it(or hopefully someone else fixes it before).

[forkyishere]

Yep, just got the same block. So still happening. I am using ubuntu 23.10, mongo 7.0.5 and node v18.19.0. Will try to document the error this week. I think I know what it means, and should not be a concern (just a situation that was not predicted).

[forkyishere]

Ok, I think I am on something here. But I might need help with the fix.

So, what I have discovered so far is that all these blocks have one (at least for blocks here presented) transaction where in the logs field, there is an error. The transaction usually consists on a buy action of an NFT of the same account (the same that is selling that NFT).

The logs field show: "{"errors":["cannot transfer to self","unable to transfer agent fees"],…"

That looks like where the error is coming from. Because the transfer actually fails, due to not being able to get agent fees.

Next step is to try understand why the parsing of these blocks fail...

[forkyishere]

Is it because its expecting an "events" field with another "data" field inside and the first thing that it gets its an "errors" one, which does not have the data field?

Example for HE block 9536695:
"logs": "{"errors":["cannot transfer to self","unable to transfer agent fees"],"events":[{"contract":"tokens","event":"transfer","data":{"from":"pinkfloyd878","to":"risingstargame","symbol":"STARBITS","quantity":"134"}}]}"

This would not be a big deal, but it means that the parsing needs to contemplate not just expecting "events" from the "logs" field.

If this is correct then the function parseTransferOperation inside ssc_tokens_history/contracts/tokens.js needs to contemplate the scenario of having "errors" in the "logs" field.

[forkyishere]

Ok, I think I know what happened. Back then the contract would still allow for the fees event to happen... when someone was trying to buy something they own already.

Example: https://he.dtools.dev/tx/824d596fc1692c290d6457a9c16631223261f07a

My question here is... did that event of the fees in this transaction really happened? did the user lost those fees?

If so, we need to adapt the parsing to intelligently understand what is parsing as an event.

The problem specifically on these blocks is on ssc_tokens_history/contracts/nftmarket.js file... at the parseNftBuy function. Because on the above situation, there is only 1 event (fees) for a nftbuy action.

Hence the this line will fail:

await parseTransferOperation(collection, insertTx, events[1], payloadObj);

I don't know if there are other situations... so I am testing with this (full replay):

diff --git a/contracts/nftmarket.js b/contracts/nftmarket.js
index 6212ca6..1bbb87c 100644
--- a/contracts/nftmarket.js
+++ b/contracts/nftmarket.js
@@ -64,7 +64,7 @@ async function parseNftChangePrice(collection, nftCollection, sender, contract,
 }

 async function parseNftBuy(collection, nftCollection, sender, contract, action, tx, events, payloadObj) {
-  if (events && events.length > 0) {
+  if (events && events.length > 1) {
     const insertTx = {
       ...tx,
     };

There is another file that does not have the protection, over the parseLotteryOperation function, but we are not using it for now.

[forkyishere]

Ok, I have now passed the first block that was problematic. Still going... no errors.

[primersion]

Hmm did the token transfer actually happen at all? If it did not happen I think the nft-market buy operation could be ignored completely, in case there is an error. But in that case I would say that the hive-engine nodes should not even be logging the token transfer? Maybe that's also a limitation of thive-engine nodes..
In any case we should check if an actual transfer did occur and if not, either

• use your solution, which actually also just ignores the transaction, as the processing of events ignores the transfer and does nothing else
• ignore the nft-market buy operation completely if there are errors

[forkyishere]

Hmm did the token transfer actually happen at all?

In this case, I don't think it HAPPENS anymore... but I can't be sure if the past it did happen. And this one is a corner case because the NFT is being bought by the owner itself. So, I don't think there is a record of any transaction. Hence I would default, that it didn't happen at all.

But I don't know if I can generalize that for all other combinations of errors. And that's where my lack of understanding goes.

I have decided to specifically ignore that event amount combination with NFT purchases only. So far... no errors. But the reality is if there are more "number" of events, that rule does nothing anymore (and we get to the same problem again). Unless indentifying the event type before deciding to parse. Which is my question... is that too costly or simply does not make sense.

I am on 23507060 block already... (taking forever...) but no errors. Anyone trying to use the old code, those transactions will not be parsable... and there is a retry and error situation, by stopping and restarting the parsing server, that will roll over them eventually.

[forkyishere]

Completed sync without problems.

[forkyishere]

Added this to the MR: #32