Posted by
VenuReddy on
Jul 09, 2020; 9:16am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-Do-we-still-need-to-support-carbon-merge-index-in-segment-property-tp97209.html
Dear Community.!
Have recently encountered a problem of Segment directory and segment file in
metadata directory are not created for partitioned table when
'carbon.merge.index.in.segment' property is set to 'false'. And actual index
files which were present in respective partition's '.tmp' directory are also
deleted without moving them out to respective partition directory where its
'.carbondata' file exist. Thus queries throw exception while reading index
files. Please refer jira issue -
https://issues.apache.org/jira/browse/CARBONDATA-3834<
https://issues.apache.org/jira/browse/CARBONDATA-3834>
To address this issue, we have 2 options to go with -
1. Either fix it to work for 'carbon.merge.index.in.segment' set to false
case. There is an open PR
https://github.com/apache/carbondata/pull/3776<
https://github.com/apache/carbondata/pull/3776> for it.
2. Or Deprecate the 'carbon.merge.index.in.segment' property itself. As the
query performance is better when merge index files are in use when compared
to normal index files, and default behavior is to generate merge index
files, probably it is not necessary to support
'carbon.merge.index.in.segment' anymore.
What do you think about it ? Please let me know your opinion.
Thanks,
Venu
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/