Tracking: Add ETL to HashingStages & HistoryStages

[joshieDo]

Feature

We should use the ETL Collector on these stages as to decrease both first sync duration and initial write-amplification. These stages use hashes as keys which lead to performance/storage degradation when using tx.insert(current) vs tx.append (with etl). More here.

The flow is very straightforward.

• Insert all data into a Collector.
  • commit_thresholdnow becomes the maximum chunk size to hold in memory. More here.. This also means that we commit all data in one go.
• Iterate collector and:
  • if first sync: use tx.append
  • else: use tx.insert

Example usage on:
TransactionLookupStage (took it from >5h to 20-30min and 157 GiB to 107GiB)
HeaderStage

Stages:

• HashingStages: feat: add ETL to Hashing Stages #7030
• HistoryStages: feat: add ETL to HistoryStages #7249

Additional context

No response

[github-actions]

This issue is stale because it has been open for 21 days with no activity.