Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert to LevelDB #4

Open
Willtech opened this issue Jun 9, 2024 · 1 comment
Open

Convert to LevelDB #4

Willtech opened this issue Jun 9, 2024 · 1 comment

Comments

@Willtech
Copy link
Owner

Willtech commented Jun 9, 2024

Looking at the use of a tuple to store the computed Prime and to load them from storage, it seems apparent that the program operation would benefit through increased speed from the use of LevelDB. The use of a tuple is faster than a list however, as the size of the tuple grows past 10,000 Prime the speed slows from 6 seconds to many days for computing 10,000,000 Prime. LevelDB allows for lookup and storage in memory of only the relevant Prime needed for computation an optimisation designed to increase the speed average of searching for more Prime.

LevelDB Features:

  • Key-Value Store: Data is stored as key-value pairs, where both keys and values are arbitrary byte arrays.
  • Ordered Data: The data is stored sorted by key, which allows for efficient range queries.
  • Atomic Batches: Multiple changes can be made in one atomic batch, ensuring data integrity.
  • Compression: Supports compression of the data via Google's Snappy compression library, reducing storage space. (although we should not use it)
  • Simple API: Offers basic operations like Get, Put, and Delete, making it easy to use.
@Willtech
Copy link
Owner Author

Willtech commented Jun 9, 2024

LevelDB (30/04/2024)
https://github.com/google/leveldb

plyvel 1.5.1
https://pypi.org/project/plyvel/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant