[
https://issues.apache.org/jira/browse/CARBONDATA-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Venugopal Reddy K resolved CARBONDATA-3834.
-------------------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> Segment directory and segment file in metadata are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to false.
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-3834
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-3834> Project: CarbonData
> Issue Type: Bug
> Components: hadoop-integration, spark-integration
> Affects Versions: 2.0.0
> Reporter: Venugopal Reddy K
> Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> *[Issue]*
> With the latest version of Carbon, Segment directory and segment file in metadata directory are not created for partitioned table when 'carbon.merge.index.in.segment' property is set to 'false'. And actual index files which were present in respective partition's '.tmp' directory are also deleted without moving them out to respective partition directory where its '.carbondata' file exist. Thus all the queries fails due to this problem.
> {color:#FF0000}Exception in thread "main" java.lang.NullPointerExceptionException in thread "main" java.lang.NullPointerException at org.apache.carbondata.core.readcommitter.TableStatusReadCommittedScope.getCommittedIndexFile(TableStatusReadCommittedScope.java:90) at org.apache.carbondata.core.index.Segment.getCommittedIndexFile(Segment.java:183) at org.apache.carbondata.core.util.BlockletIndexUtil.getTableBlockUniqueIdentifiers(BlockletIndexUtil.java:204) at {color}
>
> This issue was introduced from the resolution of an older optimization issue -CARBONDATA-3641-+[Should improve data loading performance for partition table]+
> i.e., with [
https://github.com/apache/carbondata/pull/3535]
>
> *[Modification Suggestion]*
>
> If 'carbon.merge.index.in.segment' property is false, we can create the segment directory and segment file, and move the index file from respective partition's temp directory to partition directory where the .carbondata file exists.
> Note: This need to be done before the respective partition's .tmp directory is deleted. Otherwise, we loose the index files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
