Zebra 1.9.0

This release includes deployment of NU6 on Testnet, configurable funding streams on custom Testnets, and updates Zebra's end-of-support (EoS) from 16 weeks to 10 weeks so that it will panic before the expected activation height of NU6 on Mainnet.

It also replaces the shielded-scan compilation feature with a new zebra-scanner binary, adds a TrustedChainSync module for replicating Zebra's best chain state, and a gRPC server in zebra-rpc as steps towards zcashd deprecation.

Recovering after finalizing a block from a chain fork

Zebra doesn't enforce an end-of-support height on Testnet, and previous versions of Zebra could mine or follow a chain fork that does not
activate NU6 on Testnet at height 2976000. Once a block from a fork is finalized in Zebra's state cache, updating to a version of Zebra that
does expect NU6 activation at that height will result in Zebra getting stuck, as it cannot rollback its finalized state. This can be resolved
by syncing Zebra from scratch, or by using the copy-state command to create a new state cache up to height 2975999. To use the copy-state
command, first make a copy Zebra's Testnet configuration with a different cache directory path, for example, if Zebra's configuration is at the
default path, by running cp ~/.config/zebrad.toml ./zebrad-copy-target.toml, then opening the new configuration file and editing the
cache_dir path in the state section. Once there's a copy of Zebra's configuration with the new state cache directory path, run:
zebrad copy-state --target-config-path "./zebrad-copy-target.toml" --max-source-height "2975999", and then update the original
Zebra configuration to use the new state cache directory.

Added

• A zebra-scanner binary replacing the shielded-scan compilation feature in zebrad (#8608)
• Adds a TrustedChainSync module for keeping up with Zebra's non-finalized best chain from a separate process (#8596)
• Add a tonic server in zebra-rpc with a chain_tip_change() method that notifies clients when Zebra's best chain tip changes (#8674)
• NU6 network upgrade variant, minimum protocol version, and Testnet activation height (#8693, 8733, #8804)
• Configurable NU6 activation height on Regtest (#8700)
• Configurable Testnet funding streams (#8718)
• Post-NU6 funding streams, including a lockbox funding stream (#8694)
• Add value pool balances to getblockchaininfo RPC method response (#8769)
• Add NU6 lockbox funding stream information to getblocksubsidy RPC method response (#8742)

Changed

• Reduce the end-of-support halt time from 16 weeks to 10 weeks (#8734)
• Bump the current protocol version from 170100 to 170110 (#8804)
• Track the balance of the deferred chain value pool (#8732, #8729)
• Require that coinbase transactions balance exactly after NU6 activation (#8727)
• Support in-place disk format upgrades for major database version bumps (#8748)

Fixed

• Return full network upgrade activation list in getblockchaininfo method (#8699)
• Update documentation for the new zebra-scanner binary (#8675)
• Update documentation for using Zebra with lightwalletd (#8714)
• Reduce debug output for Network on Regtest and default Testnet (#8760)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @oxarbitrage, @therealyingtong and @upbqdn

Zebra 1.8.0

Zebra 1.8.0 - 2024-07-02

• Zebra now uses a default unpaid actions limit of 0, dropping transactions with
  any unpaid actions from the mempool and excluding them when selecting
  transactions from the mempool during block template construction.
• The zebrad binary no longer contains the scanner of shielded transactions.
  This means zebrad no longer contains users' viewing keys.
• Support for custom Testnets and Regtest is greatly enhanced.
• Windows is now back in the second tier of supported platforms.
• The end-of-support time interval is set to match zcashd's 16 weeks.
• The RPC serialization of empty treestates matches zcashd.

Added

• Add an init function for a standalone ReadStateService (#8595)
• Allow configuring more parameters on custom Testnets and an NU5 activation height on Regtest (#8477, #8518, #8524, #8528, #8636)
• Add default constructions for several RPC method responses (#8616, #8505)
• Support constructing Canopy proposal blocks from block templates (#8505)

Docs

• Document custom Testnets, Regtest and how they compare to Mainnet and the default Testnet in the Zebra book (#8526, #8636)

Changed

• Use a default unpaid action limit of 0 so that transactions with unpaid actions are excluded from the mempool and block templates by default (#8638)
• Lower the mandatory checkpoint height from immediately above the ZIP-212 grace period to immediately below Canopy activation (#8629)
• Use the new zcash_script callback API in zebra-script, allowing Zebra's ECC dependencies to be upgraded independently of zcash_script and zcashd (#8566)
• Put Windows in Tier 2 of supported platforms (#8637)
• Remove experimental support for starting the zebra-scan in the zebrad process as a step towards moving the scanner to its own process (#8594)
• Reduce the end of support time from 20 weeks to 16 weeks (#8530)
• Restore parts of the experimental internal-miner feature for use on Regtest (#8506)

Fixed

• Allow square brackets in network config listen addr and external addr without explicitly configuring a port in the listen addr (#8504)
• Fix general conditional compilation attributes (#8602)
• Update median_timespan() method to align with zcashd implementation (#8491)
  upbqdn marked this conversation as resolved.
• Allow custom Testnets to make peer connections (#8528)
• Refactor and simplify the serialization of empty treestates to fix an incorrect serialization of empty note commitment trees for RPCs (#8533)
• Fix port conflict issue in some tests and re-enable those tests on Windows where those issues were especially problematic (#8624)
• Fix a bug with trailing characters in the OpenAPI spec method descriptions (#8597)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @gustavovalverde, @oxarbitrage and @upbqdn

Zebra 1.7.0

Zebra 1.7.0 - 2024-05-07

In this release we introduce Regtest functionality to Zebra and restored Windows support. Also adjusted our Zebra release interval from 2 weeks to 6 weeks approximately.

Added

• Preparing for upstream zcash_client_backend API changes (#8425)
• Regtest support (#8383, #8421, #8368, #8413, #8474, #8475)
• Allow Zebra users to contribute to the P2P network even if behind NAT or firewall (#8488)

Changed

• Adjust estimated release interval to once every 6 weeks and the end of support from 16 to 20 weeks (#8429)

Fixed

• Bump zcash script v0.1.15 and restore Windows support (#8393)
• Avoid possibly returning data from different blocks in z_get_treestate RPC method (#8460)
• Zebra panics with all features and no elasticsearch server available (#8409)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @oxarbitrage and @upbqdn

Zebra 1.6.1

This release adds an OpenAPI specification for Zebra's RPC methods and startup logs about Zebra's storage usage and other database information.

It also includes:

• Bug fixes and improved error messages for some zebra-scan gRPC methods
• A performance improvement in Zebra's getblock RPC method

Added

• Log database information such as storage usage on startup and shutdown (#8336, #8389)
• OpenAPI specification for Zebra's RPC methods (#8342)
• Add block times to output of getblock RPC method when called with verbosity = 2 (#8384)

Changed

• Removed Copy trait impl for Network (#8354)
• Refactored code for network consensus parameters to Network methods (#8340)

Fixed

• Improve zebra-scan gRPC method errors and add timeout to scan service to avoid hanging (#8318)
• Await state service requests in getblock method in parallel (#8376)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @elijahhampton, @gustavovalverde, @idky137, @mpguerra, @oxarbitrage, @upbqdn and @zancas

Zebra 1.6.0

This release exposes the shielded scanning functionality through an initial version of a gRPC server, documented in the Zebra Book.

Note

Building Zebra now depends on protoc. See the Build Instructions for more details.

Added

• Add docker-compose file to run CI locally (#8209)
• Allow users to use Zebra + LWD with persistent states (#8215)

Scanner

• Add a new zebra-grpc crate (#8167)
• Start scanner gRPC server with zebrad (#8241)
• Add gRPC server reflection and document how to use the gRPC server (#8288)
• Add the GetInfo gRPC method (#8178)
• Add the GetResults gRPC method (#8255)
• Add the Scan gRPC method (#8268, #8303)
• Add the RegisterKeys gRPC method (#8266)
• Add the ClearResults and DeleteKeys gRPC methods (#8237)
• Add snapshot tests for new gRPCs (#8277)
• Add unit tests for new gRPCs (#8293)
• Create a tower Service in zebra-scan (#8185)
• Implement the SubscribeResults scan service request (#8253)
• Implement the ClearResults scan service request (#8219)
• Implement the DeleteKeys scan service request (#8217)
• Implement the RegisterKeys scan service request (#8251)
• Implement the Results scan service request (#8224)
• Test the RegisterKeys scan service request (#8281)
• Add ViewingKey type in zebra-chain (#8198)
• Handle RegisterKeys messages in scan task (#8222)

Changed

• Remove rfc.md file (#8228)
• Update Debian from Bullseye to Bookworm in Docker (#8273)
• Remove Zebra RFCs from CONTRIBUTING.md (#8304)
• Publish fewer tags in Docker Hub (#8300)
• Add Zebra crate versions to dev-dependencies and remove circular dev-dependencies (#8171)
• Update docs for building Zebra (#8315)

Fixed

• Set log rotation to avoid docker bugs (#8269)
• Improve error message in non_blocking_logger test (#8276)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @bishopcheckmate, @chairulakmal, @gustavovalverde, @mpguerra, @oxarbitrage and @upbqdn.

Zebra 1.5.2

Zebra 1.5.2 - 2024 - 01 - 23

This release serves as a hotfix for version 1.5.1, addressing issues encountered after its initial release. For more information about version 1.5.1, refer to this link.

Following the release on GitHub, we identified difficulties in publishing the zebra-chain crate to crates.io. Detailed information is available in issue #8180 and its references.

Unfortunately, to resolve this challenge, which involves an unpublished dependency, we had to temporarily remove the internal miner support introduced in version 1.5.1.

In our efforts to reinstate this feature, we've opened a tracking ticket to monitor the progress of the required code that must be merged into the equihash dependency. You can follow the developments in issue #8183, which will only be closed once the feature is successfully restored.

Breaking Changes

• Temporally remove the internal miner functionality (#8184)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@oxarbitrage

Zebra 1.5.1

Zebra 1.5.1 - 2024-01-18

This release:

• Adds a utility for reading scanning results, and finalizes the MVP features of the scanner.
• Adds an experimental internal-miner feature, which mines blocks within zebrad. This feature is only supported on testnet. Use a more efficient GPU or ASIC for mainnet mining.
• Contains many documentation improvements.

Added

• Add an internal Zcash miner to Zebra (#8136, #8150)
• Blockchain scanner new features:
  • Don't scan and log if we are below sapling height (#8121)
  • Restart scanning where left (#8080)
  • Add scanning result reader utility (#8104, #8157)
• Note default path to config in docs (#8143)
• Document how to add a column family (#8149)

Changed

• Make sure scanner database is accessed using the correct types (#8112)
• Move history tree and value balance to typed column families (#8115)
• Refactor the user documentation for scanning (#8124)
• Refactor user & dev documentation (#8145)
• Improve feature flag docs (#8114)
• Allow opening the database in a read-only mode (#8079)
• Send all zebrad logs to the journal under systemd (#7965)

Fixed

• Point to a manually created list of Zebra crates in docs (#8160)
• Add shielded-scan.md to the index (#8095)
• Elasticsearch feature, make bulk size the same for testnet and mainnet (#8127)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @bishopcheckmate, @gustavovalverde, @oxarbitrage, @sandakersmann, @teor2345 and @upbqdn

Zebra 1.5.0

This release:

• fixes a panic that was introduced in Zebra v1.4.0, which happens in rare circumstances when reading cached sprout or history trees.
• further improves how Zebra recovers from network interruptions and prevents potential network hangs.
• limits the ability of synthetic nodes to spread throughout the network through Zebra to address some of the Ziggurat red team report.

As of this release, Zebra requires Rust 1.73 to build.

Finally, we've added an experimental "shielded-scan" feature and the zebra-scan crate as steps
towards supporting shielded scanning in Zebra. This feature has known security issues.
It is for experimental use only. Ongoing development is tracked in issue #7728.

Important Security Warning

Do not use regular or sensitive viewing keys with Zebra's experimental scanning feature. Do not use this
feature on a shared machine. We suggest generating new keys for experimental use.

Security

• security(net): Stop sending peer addresses from version messages directly to the address book (#7977)
• security(net): Limit how many addresses are sent directly to the address book for a single peer address message (#7952)
• security(net): Rate-limit GetAddr responses to avoid sharing the entire address book over a short period (#7955)

Added

• feat(config): Add config field for the viewing keys used by zebra-scan (#7949)
• feat(scan): Add on-disk database to store keys and scan results (#7942, #8036)
• feat(scan): Spawn zebra-scan task from zebrad with configured viewing keys (#7989)
• feat(scan): Create a scan_block function to use across scanning tasks (#7994)
• feat(scan): Scan blocks with Sapling keys and write the results to the database (#8040)
• poc(scan): Proof of concept for shielded scanning (#7758)
• add(docker): Add ldb RocksDB query tool to the Dockerfile (#8074)

Changed

• change(state): Expose ZebraDb methods that can create different kinds of databases (#8002)
• change(state): Make the types for finalized blocks consistent (#7923)
• change(scan): Create a scanner storage database (#8031)
• change(scan): Store scanned TXIDs in "display order" (#8057)
• change(scan): Create a function that scans one block by height, and stores the results in the database (#8045)
• change(scan): Store one transaction ID per database row, to make queries easier (#8062)
• change(log): Silence verbose failed connection logs (#8072)

Fixed

• fix(db): Fix a sprout/history tree read panic in Zebra v1.4.0, which only happens before the 25.3.0 state upgrade completes (#7972)
• fix(net): Fix potential network hangs, and reduce code complexity (#7859)
• fix(scan): Start scanning task only if there are keys to scan (#8059)
• fix(rpc): Make the verbose argument of the getrawtransaction RPC optional (#8076)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @oxarbitrage, @teor2345 and @upbqdn

Zebra 1.4.0

Zebra 1.4.0 - 2023-11-07

Zebra's mining RPCs are now available in release builds. Our Docker images are significantly
smaller, because the smaller Zcash verification parameters are now built into the zebrad binary.
Zebra has updated to the shared Rust dependencies from the zcashd 5.7.0 release.

Zebra recovers better from brief network interruptions, and avoids some network and verification
denial of service and performance issues. We have restored our macOS tests in CI, and now support
macOS on a best-effort basis.

We have changed our documentation website URL, and we are considering deprecating some Docker image
tags in release 1.5.0 and later.

Deprecation Warnings

This release has the following deprecation warnings:

Warning: Deprecation of DockerHub Image Tags in a future release

Zebra currently publishes 11 DockerHub tags for each new release.
We want to reduce the number of DockerHub tags we publish in a future minor Zebra release.

Based on usage and user feedback, we could stop publishing:

• The 1 tag, which updates each release until NU6
• The 1.x tag, which updates each patch release until the next minor release
• The 1.x.y tag, which is the same as v1.x.y
• The sha-xxxxxxx tag, which is the same as v1.x.y (for production releases)

We also want to standardise experimental image tags to -experimental, rather than .experimental.

So for release 1.5.0, we might only publish these tags:

• latest
• latest-experimental (a new tag)
• v1.5.0
• v1.5.0-experimental

Please let us know if you need any other tags by opening a GitHub ticket.

We recommend using the latest tag to always get the most recent Zebra release.

Warning: Documentation Website URL Change

We have replaced the API documentation on the doc.zebra.zfnd.org
website with docs.rs. All links have been updated.

Zebra's API documentation can be found on:

• docs.rs, which renders documentation for the
  public API of the latest crate releases;
• doc-internal.zebra.zfnd.org, which renders
  documentation for the internal API on the main branch.

doc.zebra.zfnd.org stopped being updated a few days before this release,
and it will soon be shut down.

Significant Changes

This release contains the following significant changes:

Mining RPCs in Production Builds

Zebra's mining RPCs are now available in release builds (#7740). Any Zebra instance can be used
by a solo miner or mining pool. This stabilises 12 RPCs, including getblocktemplate, submitblock,
getmininginfo, getnetworksolps, [z_]validateaddress and getblocksubsidy. For more information,
read our mining blog post.

Please let us know
if your mining pool needs extra RPC methods or fields.

Zcash Parameters in zebrad Binary

zebrad now bundles zk-SNARK parameters directly into its binary. This increases the binary size
by a few megabytes, but reduces the size of the Docker image by around 600 MB because
the parameters don't contain the Sprout proving key anymore. The zebrad download
command does nothing, so it has been removed.

Previously, parameters were stored by default in these locations:

• ~/.zcash-params (on Linux); or
• ~/Library/Application Support/ZcashParams (on Mac); or
• C:\Users\Username\AppData\Roaming\ZcashParams (on Windows)

If you have upgraded zebrad to 1.4.0 or later, and zcashd to 5.7.0 or later, you can delete the
parameter files in these directories to save approximately 700 MB disk space.

zcashd have deprecated their fetch-params.sh script,
so it can't be used to retry failed downloads in zebrad 1.3.0 and earlier.

We recommend upgrading to the latest Zebra release to avoid download issues in new installs.

macOS Support

macOS x86_64 is now supported on a best-effort basis. macOS builds and some tests run in Zebra's CI.

Security

• Reconnect with peers after brief network interruption (#7853)
• Add outer timeouts for critical network operations to avoid hangs (#7869)
• Set iterator read bounds where possible in DiskDb, to avoid a known RocksDB denial of service issue (#7731, #7732)
• Fix concurrency issues in tree key formats, and CPU usage in genesis tree roots (#7392)

Removed

• Remove the zebrad download command, because it no longer does anything (#7819)

Added

• Enable mining RPCs by default in production builds (#7740)
• Re-enable macOS builds and tests in CI (#7834)
• Make macOS x86_64 a tier 2 supported platform in the docs (#7843)
• Add macOS M1 as a tier 3 supported platform (#7851)

Changed

• Build Sprout and Sapling parameters into the zebrad binary, so a download server isn't needed (#7800, #7844)
• Bump ECC dependencies for zcashd 5.7.0 (#7784)
• Refactor the installation instructions for the s-nomp mining pool software (#7835)

Fixed

• Make the latest Docker tag point to the production build, rather than the build with experimental features (#7817)
• Fix an incorrect consensus-critical ZIP 212 comment (#7774)
• Fix broken links to zebra_network and zebra_state Config structs on doc-internal.zebra.zfnd.org (#7838)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @mpguerra, @oxarbitrage, @rex4539, @teor2345, @upbqdn, and @vuittont60.

Zebra 1.3.0

This release adds RPC methods for the "Spend before Sync" light wallet feature,
and fixes performance issues and bugs in the mining solution rate RPCs. Progress
bars can now be enabled using a config, please help us test them!

It contains the following updates:

User Testing: Progress Bars

Zebra has progress bars! When progress bars are enabled, you can see Zebra's blocks,
transactions, and peer connections in your terminal. We're asking Zebra users to test this
feature, and give us feedback on the forums.

To show progress bars while running Zebra, add these lines to your zebrad.toml:

[tracing]
progress_bar = "summary"

For more details, including a known issue with time estimates,
read our progress bars blog post.

Security

• Fix database concurrency bugs that could have led to panics or incorrect history tree data (#7590, #7663)

Added

• Zebra's progress bars can now be enabled using a zebrad.toml config (#7615)
• Add missing elasticsearch flag feature to lib docs (#7568)
• Add missing Docker variables and examples (#7552)
• Check database format is valid on startup and shutdown (#7566, #7606). We expect to catch almost all database validity errors in CI (#7602, #7627), so users are unlikely to see them on startup or shutdown.

Spend before Sync Support

• Add state requests and support code for the z_getsubtreesbyindex RPC (#7408, #7734)
• Implement the z_getsubtreesbyindex RPC (#7436)
• Test the z_getsubtreesbyindex RPC (#7515, #7521, #7566, #7514, #7628)
• Format subtree roots in little-endian order (#7466)
• Add note subtree indexes for new and existing blocks (#7437)
• Upgrade subtrees from the tip backwards, for compatibility with wallet syncing (#7531)
• Handle a subtree comparison edge case correctly (#7587)

Changed

• Return errors instead of panicking in methods for Heights (#7591)
• Update tests for compatibility with the ECC's lightwalletd fork (#7349)

Fixed

• Refactor docs for feature flags (#7567)
• Match zcashd's getblockchaininfo capitalisation for NU5 (#7454)
• Fix bugs and performance of getnetworksolps & getnetworkhashps RPCs (#7647)

Contributors

Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @rex4539, @teor2345 and @upbqdn.