Breaking changes (#5191)

Co-authored-by: Bjerg <[email protected]>
Co-authored-by: Roman Krasiuk <[email protected]>
Co-authored-by: joshieDo <[email protected]>
Co-authored-by: joshieDo <[email protected]>
Co-authored-by: Matthias Seitz <[email protected]>
Co-authored-by: Oliver Nordbjerg <[email protected]>
Co-authored-by: Thomas Coratger <[email protected]>

.config/nextest.toml

@@ -4,4 +4,4 @@ slow-timeout = { period = "30s", terminate-after = 4 }
 
 [[profile.default.overrides]]
 filter = "test(general_state_tests)"
-slow-timeout = { period = "1m", terminate-after = 4 }
+slow-timeout = { period = "1m", terminate-after = 10 }

CODEOWNERS

@@ -19,5 +19,5 @@ crates/metrics              @onbjerg
 crates/tracing              @onbjerg
 crates/tasks                @mattsse
 crates/prune                @shekhirin @joshieDo
-crates/snapshot             @joshieDo
+crates/static-file          @joshieDo @shekhirin
 .github/                    @onbjerg @gakonst @DaniPopes

Cargo.toml

@@ -8,6 +8,7 @@ members = [
     "crates/consensus/beacon-core/",
     "crates/consensus/common/",
     "crates/ethereum-forks/",
+    "crates/etl",
     "crates/interfaces/",
     "crates/metrics/",
     "crates/metrics/metrics-derive/",
@@ -41,8 +42,8 @@ members = [
     "crates/node-optimism/",
     "crates/node-core/",
     "crates/node-api/",
-    "crates/snapshot/",
     "crates/stages/",
+    "crates/static-file/",
     "crates/storage/codecs/",
     "crates/storage/codecs/derive/",
     "crates/storage/db/",
@@ -137,6 +138,7 @@ reth-ecies = { path = "crates/net/ecies" }
 reth-eth-wire = { path = "crates/net/eth-wire" }
 reth-ethereum-forks = { path = "crates/ethereum-forks" }
 reth-ethereum-payload-builder = { path = "crates/payload/ethereum" }
+reth-etl = { path = "crates/etl" }
 reth-optimism-payload-builder = { path = "crates/payload/optimism" }
 reth-interfaces = { path = "crates/interfaces" }
 reth-ipc = { path = "crates/rpc/ipc" }
@@ -162,8 +164,8 @@ reth-rpc-builder = { path = "crates/rpc/rpc-builder" }
 reth-rpc-engine-api = { path = "crates/rpc/rpc-engine-api" }
 reth-rpc-types = { path = "crates/rpc/rpc-types" }
 reth-rpc-types-compat = { path = "crates/rpc/rpc-types-compat" }
-reth-snapshot = { path = "crates/snapshot" }
 reth-stages = { path = "crates/stages" }
+reth-static-file = { path = "crates/static-file" }
 reth-tasks = { path = "crates/tasks" }
 reth-tokio-util = { path = "crates/tokio-util" }
 reth-tracing = { path = "crates/tracing" }

README.md

@@ -1,4 +1,4 @@
-# reth 
+# reth
 
 [![CI status](https://github.com/paradigmxyz/reth/workflows/ci/badge.svg)][gh-ci]
 [![cargo-deny status](https://github.com/paradigmxyz/reth/workflows/deny/badge.svg)][gh-deny]
@@ -14,7 +14,7 @@
 | [Developer Docs](./docs)
 | [Crate Docs](https://paradigmxyz.github.io/reth/docs)
 
-*The project is still work in progress, see the [disclaimer below](#status).*
+_The project is still work in progress, see the [disclaimer below](#status)._
 
 [codecov]: https://app.codecov.io/gh/paradigmxyz/reth
 [gh-ci]: https://github.com/paradigmxyz/reth/actions/workflows/ci.yml
@@ -50,6 +50,14 @@ We will be updating the documentation with the completion status of each compone
 
 We appreciate your patience until we get there. Until then, we are happy to answer all questions in the Telegram link above.
 
+### Database compatibility
+
+Reth [v0.2.0-beta.1](https://github.com/paradigmxyz/reth/releases/tag/v0.2.0-beta.1) includes
+a [set of breaking database changes](https://github.com/paradigmxyz/reth/pull/5191) that makes it impossible to use database files produced by earlier versions.
+
+If you had a database produced by alpha versions of Reth, you need to drop it with `reth db drop`
+(using the same arguments such as `--config` or `--datadir` that you passed to `reth node`), and resync using the same `reth node` command you've used before.
+
 ## For Users
 
 See the [Reth Book](https://paradigmxyz.github.io/reth) for instructions on how to install and run Reth.
@@ -105,15 +113,15 @@ cargo test --workspace --features geth-tests
 # With Ethereum Foundation tests
 #
 # Note: Requires cloning https://github.com/ethereum/tests
-# 
+#
 #   cd testing/ef-tests && git clone https://github.com/ethereum/tests ethereum-tests
 cargo test -p ef-tests --features ef-tests
 ```
 
 We recommend using [`cargo nextest`](https://nexte.st/) to speed up testing. With nextest installed, simply substitute `cargo test` with `cargo nextest run`.
 
 > **Note**
-> 
+>
 > Some tests use random number generators to generate test data. If you want to use a deterministic seed, you can set the `SEED` environment variable.
 
 ## Getting Help
@@ -135,9 +143,10 @@ See [`SECURITY.md`](./SECURITY.md).
 Reth is a new implementation of the Ethereum protocol. In the process of developing the node we investigated the design decisions other nodes have made to understand what is done well, what is not, and where we can improve the status quo.
 
 None of this would have been possible without them, so big shoutout to the teams below:
-* [Geth](https://github.com/ethereum/go-ethereum/): We would like to express our heartfelt gratitude to the go-ethereum team for their outstanding contributions to Ethereum over the years. Their tireless efforts and dedication have helped to shape the Ethereum ecosystem and make it the vibrant and innovative community it is today. Thank you for your hard work and commitment to the project.
-* [Erigon](https://github.com/ledgerwatch/erigon) (fka Turbo-Geth): Erigon pioneered the ["Staged Sync" architecture](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) that Reth is using, as well as [introduced MDBX](https://github.com/ledgerwatch/erigon/wiki/Choice-of-storage-engine) as the database of choice. We thank Erigon for pushing the state of the art research on the performance limits of Ethereum nodes.
-* [Akula](https://github.com/akula-bft/akula/): Reth uses forks of the Apache versions of Akula's [MDBX Bindings](https://github.com/paradigmxyz/reth/pull/132), [FastRLP](https://github.com/paradigmxyz/reth/pull/63) and [ECIES](https://github.com/paradigmxyz/reth/pull/80) . Given that these packages were already released under the Apache License, and they implement standardized solutions, we decided not to reimplement them to iterate faster. We thank the Akula team for their contributions to the Rust Ethereum ecosystem and for publishing these packages.
+
+- [Geth](https://github.com/ethereum/go-ethereum/): We would like to express our heartfelt gratitude to the go-ethereum team for their outstanding contributions to Ethereum over the years. Their tireless efforts and dedication have helped to shape the Ethereum ecosystem and make it the vibrant and innovative community it is today. Thank you for your hard work and commitment to the project.
+- [Erigon](https://github.com/ledgerwatch/erigon) (fka Turbo-Geth): Erigon pioneered the ["Staged Sync" architecture](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) that Reth is using, as well as [introduced MDBX](https://github.com/ledgerwatch/erigon/wiki/Choice-of-storage-engine) as the database of choice. We thank Erigon for pushing the state of the art research on the performance limits of Ethereum nodes.
+- [Akula](https://github.com/akula-bft/akula/): Reth uses forks of the Apache versions of Akula's [MDBX Bindings](https://github.com/paradigmxyz/reth/pull/132), [FastRLP](https://github.com/paradigmxyz/reth/pull/63) and [ECIES](https://github.com/paradigmxyz/reth/pull/80) . Given that these packages were already released under the Apache License, and they implement standardized solutions, we decided not to reimplement them to iterate faster. We thank the Akula team for their contributions to the Rust Ethereum ecosystem and for publishing these packages.
 
 [book]: https://paradigmxyz.github.io/reth/
 [tg-url]: https://t.me/paradigm_reth

bin/reth/Cargo.toml

@@ -50,7 +50,7 @@ reth-payload-validator.workspace = true
 reth-basic-payload-builder.workspace = true
 reth-discv4.workspace = true
 reth-prune.workspace = true
-reth-snapshot = { workspace = true, features = ["clap"] }
+reth-static-file = { workspace = true, features = ["clap"] }
 reth-trie.workspace = true
 reth-nippy-jar.workspace = true
 reth-node-api.workspace = true