When importing more than 1 bucket via BucketBytes, the full buckets cannot be mapped to a BitArray because the byte array to import is too long (by 1 byte). The problem already arises in FilterMemory.ExportToBytes(), because the last 7 bits of the full bucket are mapped to the 268435456th byte of the result array, which is 1 byte more than can be mapped to the BitArray in void Import(IList<byte[]> bucketBytes).
Example code to reproduce the issue:
long expectedElements = 100_000_000;
double errorRate = 0.0000001;
var hashMethod = HashMethod.XXHash128;
var bf = (FilterMemory)FilterBuilder.Build(expectedElements, errorRate, hashMethod);
var bytes = bf.ExportToBytes();
var bf2 = (FilterMemory)FilterBuilder.Build(new FilterMemoryOptions()
{
Method = hashMethod,
ExpectedElements = expectedElements,
ErrorRate = errorRate,
BucketBytes = bytes
});To fix this issue, Filter.MaxInt could be set to 2147483640 (see pull request #15).
Another fix could be to handle this special case by creating the BitArray at max size, copy all but the last byte into it and and map the last 7 bits manually.
When importing more than 1 bucket via BucketBytes, the full buckets cannot be mapped to a BitArray because the byte array to import is too long (by 1 byte). The problem already arises in
FilterMemory.ExportToBytes(), because the last 7 bits of the full bucket are mapped to the 268435456th byte of the result array, which is 1 byte more than can be mapped to the BitArray invoid Import(IList<byte[]> bucketBytes).Example code to reproduce the issue:
To fix this issue,
Filter.MaxIntcould be set to 2147483640 (see pull request #15).Another fix could be to handle this special case by creating the
BitArrayat max size, copy all but the last byte into it and and map the last 7 bits manually.