minor compact throw err 'IndexBuilderException'

Posted by Li Peng on Jan 05, 2017; 7:24am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/minor-compact-throw-err-IndexBuilderException-tp5551.html

Hello:
  in spark shell with carbondata 0.2.0, minor compact, throw err:

WARN  05-01 15:04:06,964 - Lost task 0.0 in stage 0.0 (TID 1, dpnode08): org.apache.carbondata.core.carbon.datastore.exception.IndexBuilderException:
        at org.apache.carbondata.integration.spark.merger.CarbonCompactionUtil.createDataFileFooterMappingForSegments(CarbonCompactionUtil.java:127)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD$$anon$1.<init>(CarbonMergerRDD.scala:121)
        at org.apache.carbondata.spark.rdd.CarbonMergerRDD.compute(CarbonMergerRDD.scala:70)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:313)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:277)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
        at org.apache.spark.scheduler.Task.run(Task.scala:89)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.carbondata.core.util.CarbonUtilException: Problem while reading the file metadata
        at org.apache.carbondata.core.util.CarbonUtil.readMetadatFile(CarbonUtil.java:1063)
        at org.apache.carbondata.integration.spark.merger.CarbonCompactionUtil.createDataFileFooterMappingForSegments(CarbonCompactionUtil.java:123)
        ... 10 more
Caused by: java.io.IOException: It doesn't set the offset properly
        at org.apache.carbondata.core.reader.ThriftReader.setReadOffset(ThriftReader.java:88)
        at org.apache.carbondata.core.reader.CarbonFooterReader.readFooter(CarbonFooterReader.java:55)
        at org.apache.carbondata.core.util.DataFileFooterConverter.readDataFileFooter(DataFileFooterConverter.java:148)
        at org.apache.carbondata.core.util.CarbonUtil.readMetadatFile(CarbonUtil.java:1061)
        ... 11 more