You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While a []bool is great for simplicity, I suspect it takes much more space in memory than a "compact" Bitset. This would be caused by the memory layout of the underlying array, where each bool is individually addressable.
This is important because Bloom filters are designed specifically to be space-efficient, storing only the conflated Bitset instead all the elements themselves, or even all the hash values.
Go has no built-in Bitset type, but it seems that a common and efficient ready-to-use Bitset implementation is *big.Int. (I created an idiomatic snippet here).
It would be interesting to run a "memory usage" benchmark of []bool vs *big.Int. The speed performance results would be interesting as well (I have no idea which one wins on speed!). Fanis, what do you think?
The text was updated successfully, but these errors were encountered:
Hello @Deleplace. Yes, I think []bool is not the most space efficient model for that. If you check some other implementations of Bloom Filters they use either the @willf one:
While a
[]bool
is great for simplicity, I suspect it takes much more space in memory than a "compact" Bitset. This would be caused by the memory layout of the underlying array, where each bool is individually addressable.This is important because Bloom filters are designed specifically to be space-efficient, storing only the conflated Bitset instead all the elements themselves, or even all the hash values.
Go has no built-in Bitset type, but it seems that a common and efficient ready-to-use Bitset implementation is *big.Int. (I created an idiomatic snippet here).
It would be interesting to run a "memory usage" benchmark of
[]bool
vs*big.Int
. The speed performance results would be interesting as well (I have no idea which one wins on speed!). Fanis, what do you think?The text was updated successfully, but these errors were encountered: