go-ipfs-chunker implements data Splitters for go-ipfs.
👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/chunker. 🏎️ Good news! There is tooling and documentation to expedite a switch in your repo.
📚 Learn more, including how to take the maintainership mantle or ask questions, here.
go-ipfs-chunker
provides the Splitter
interface. IPFS splitters read data from a reader an create "chunks". These chunks are used to build the ipfs DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses to address content.
The package provides a SizeSplitter
which creates chunks of equal size and it is used by default in most cases, and a rabin
fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.
go-ipfs-chunker
works like a regular Go module:
> go get github.com/ipfs/go-ipfs-chunker
import "github.com/ipfs/go-ipfs-chunker"
Check the GoDoc documentation
MIT © Protocol Labs, Inc.