Chunkfs is a file system that can be used to benchmark different chunking algorithms, utilizing different hashing algorithms and storage types.
Chunkfs is currently under active development, breaking changes can always happen.
To use different chunking algorithms with the file system, they must implement Chunker
trait, which has the
following definition:
pub trait Chunker {
fn chunk_data(&mut self, data: &[u8], empty: Vec<Chunk>) -> Vec<Chunk>;
fn rest(&self) -> &[u8];
fn estimate_chunk_count(&self, data: &[u8]) -> usize;
}
Comments for each method are provided in lib.rs.
Add the following dependency to your Cargo.toml
:
[dependencies]
chunkfs = { git = "https://github.com/Piletskii-Oleg/chunkfs.git" }
To use provided chunkers and hashers, use the corresponding features:
[dependencies]
chunkfs = { git = "https://github.com/Piletskii-Oleg/chunkfs.git", features = ["chunkers", "hashers"] }
extern crate chunkfs;
use std::io;
use chunkfs::base::HashMapBase;
use chunkfs::chunker::LeapChunker;
use chunkfs::FileSystem;
use chunkfs::hasher::SimpleHasher;
fn main() -> io::Result<()> {
let base = HashMapBase::default();
let mut fs = FileSystem::new(base, SimpleHasher);
let mut file = fs.create_file("file".to_string(), LeapChunker::default(), true)?;
let data = vec![10; 1024 * 1024];
fs.write_to_file(&mut file, &data)?;
let measurements = fs.close_file(file)?;
println!("{:?}", measurements);
let mut file = fs.open_file("file", LeapChunker::default())?;
let read = fs.read_from_file(&mut file)?;
assert_eq!(read.len(), 1024 * 1024);
assert_eq!(read, data);
Ok(())
}