Re: [DISCUSSION] Page Level Bloom Filter

Posted by Manhua-2 on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Page-Level-Bloom-Filter-tp85720p86157.html

Hi xuchuanyin,
Thanks for you inputs.
Table properties will not be used while querying. We will apply page level bloom filter only when both following two conditions are met.
1. query runs into IncludeFilterExecuter for a column
2. page bloom of that column is set/stored in carbondata file

Automatically choosing column to build page bloom is a good idea, but Carbon should design to collect info from usage history first.


On 2019/11/12 01:30:44, xuchuanyin <[hidden email]> wrote:

> +1 for this feature.
>
> Additionally, It is described in your draft that "specify the bloom columns
> using table properties", I do recommend that for the first phrase, we should
> not use this information from table properties while querying.
>
> We can store the index information in blocklet（or page) level and use it
> when we process this blocklet(or page).
> After this, later we can automatically decide to build the corresponding
> index for some columns or some pages on the fly without the user's help.
>
>
>
> --
> Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>