[DISCUSSION] About partition table query performance

[lionel061201]

Hi dev,
Partition feature is now available on master and I just created a guidance
doc in
https://github.com/apache/carbondata/pull/1258

I added some tips about partition table query performance. Any performance
test and discussion are welcomed here.

AFAIK according to the new datamap feature, when a query has filters in
where clause, it will prune blocklets with B tree index first and then
choose matched partitions in block level. My question is, would it be
helpful if we optimized the filters tree, and scan only the B tree index of
the target partition(s)?

Thanks & Regards,
CaoLu

[Liang Chen]

Hi

+1.Very nice feature, Thanks for your good contribution.
Look forward to seeing the test report.

Regards
Liang

lionel061201 wrote

Hi dev,
Partition feature is now available on master and I just created a guidance
doc in
https://github.com/apache/carbondata/pull/1258

I added some tips about partition table query performance. Any performance
test and discussion are welcomed here.

AFAIK according to the new datamap feature, when a query has filters in
where clause, it will prune blocklets with B tree index first and then
choose matched partitions in block level. My question is, would it be
helpful if we optimized the filters tree, and scan only the B tree index of
the target partition(s)?

Thanks & Regards,
CaoLu