Re: [DISCUSSION] Page Level Bloom Filter

Posted by ravipesala on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Page-Level-Bloom-Filter-tp85720p87426.html

Hi Manhua,

 Even at page level the row count will not be available probably from the
next version. It would be decided as per size, not per count. Already code
got merged and we are keeping the count based page configuration temporarily
for backward compatibility.

 So at any place, we will not get the cardinality of the column beforehand.
Either we need to estimate the count from the history or take the
approximate value from the user.

And one more thing is the generation of bloom should follow the datamap
interfaces, not in the min/max generation flow. But we can change datamap
interfaces to add the generated datamap index to carbondata file and index
file instead of separate files. Otherwise, we will loose the index interface
capabilities and makes our code complex. This was already discussed earlier
with Jacky. @Jacky please comment on it.


Regards,
Ravindra.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/