[DISCUSSION] Support new feature: bitmap encode

[simafengyun]

Hi all,

 Let's start the discussion about bitmap encode.

below is the high level design document
https://raw.githubusercontent.com/mayunSaicmotor/saic-carbondata/branch-0.2-saic/examples/src/main/resources/carbondata-bitmap-encode.xlsx

Please feel free to discuss and give your suggestion.

Thanks
Jarck

[simafengyun]

upload the design doc screenshots

[simafengyun]

sub-task list of  bitmap encode Feature:

1 data loading for bitmap encode columns
        generate bitmap encoded data and store to data page
       
2 query bitmap encode data
        read bitmap encode data from data page  to dataChunk2
       
3 apply filter query on bitmap encode column
        equal / not equal filter
        range filter
        and / or query
        include / exclude filter

Any suggestion?

Thanks
Jack  Ma

[David CaiQiang]

Hi,

In my opinion, we need the following points as least.

1. create table ddl can specify a bitmap option

2. add a new Encoding: BITMAP