Posted by
Jihong Ma on
Sep 28, 2016; 12:42am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-option-to-disable-multi-layered-index-scan-and-use-full-table-scan-tp1526p1529.html
Ideally this should be an internal improvement, not necessarily exposing it as an config option, Carbon should be able to smartly figure out if leveraging index is beneficial or straightly going for a file scan (just as Parquet).
Regards.
Jihong
-----Original Message-----
From: Liang Big data [mailto:
[hidden email]]
Sent: Tuesday, September 27, 2016 4:43 PM
To:
[hidden email]
Subject: Re: [Discussion]: option to disable multi-layered index scan and use full table scan
Hi
good suggestion!
Add one configurable option to disable index for no filter and small table
scenarios.
One comment : you are suggesting only disable MDK; How about other
index(inverted index,and MINMAX index)?
+1 for this feature
Regards
Liang
2016-09-28 6:49 GMT+08:00 Aniket Adnaik <
[hidden email]>:
> Carbondata can provide some way to disable usage of multi-layered index and
> provide full table scan.
> This may help in following cases;
> 1. Small tables occupying only few number of blocks are probably better of
> using full table scan.
> 2. Queries with large number of projections with no filter may benefit from
> using full table scan.
> 3. Testing different scenarios and comparing with multiple HDFS file
> formats
> that do not provide multi layer index will be easier.
>
> Also, Carbondata scan internally should be smart enough to detect this
> based
> on query, data size, etc.
>
> Any comments?
>
>
>
>
> --
> View this message in context:
http://apache-carbondata-> mailing-list-archive.1130556.n5.nabble.com/Discussion-
> option-to-disable-multi-layered-index-scan-and-use-
> full-table-scan-tp1526.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>
--
Regards
Liang