[jira] [Resolved] (CARBONDATA-3552) dataload fails for column added in 1.1 which is a sort column in sort step in new version of carbondata

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacky Li resolved CARBONDATA-3552.
----------------------------------
    Fix Version/s: 2.0.0
       Resolution: Fixed

> dataload fails for column added in 1.1 which is a sort column in sort step in new version of carbondata
> -------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-3552
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3552
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Akash R Nilugal
>            Assignee: Akash R Nilugal
>            Priority: Major
>             Fix For: 2.0.0
>
>          Time Spent: 2h
>  Remaining Estimate: 0h
>
> Problem:
> dataload fails for column added in 1.1 which is a sort column in sort step in new version of carbondata
> In carbon 1.1 version, create a column with Dictionary exclude column, then add a timestamp column, By default it will be a sort column. Upgrade to latest version and load data. Data load fails with below exception
>  org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
>         at org.apache.carbondata.processing.loading.sort.impl.ParallelReadMergeSorterImpl.sort(ParallelReadMergeSorterImpl.java:120)
>         at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:55)
>         at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:109)
>         at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:52)
>         at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD$$anon$1.<init>(NewCarbonDataLoadRDD.scala:150)
>         at org.apache.carbondata.spark.rdd.NewCarbonDataLoadRDD.internalCompute(NewCarbonDataLoadRDD.scala:124)
>         at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:82)
>         at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
>         at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
>         at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
>         at org.apache.spark.scheduler.Task.run(Task.scala:99)
>         at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger.checkFailure(SingleThreadFinalSortFilesMerger.java:226)
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger.startSorting(SingleThreadFinalSortFilesMerger.java:217)
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger.startFinalMerge(SingleThreadFinalSortFilesMerger.java:113)
>         at org.apache.carbondata.processing.loading.sort.impl.ParallelReadMergeSorterImpl.sort(ParallelReadMergeSorterImpl.java:118)
>         ... 14 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.ClassCastException: [B cannot be cast to java.lang.Long
>         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger.checkFailure(SingleThreadFinalSortFilesMerger.java:224)
>         ... 17 more
> Caused by: java.lang.ClassCastException: [B cannot be cast to java.lang.Long
>         at org.apache.carbondata.core.util.comparator.LongSerializableComparator.compare(Comparator.java:180)
>         at org.apache.carbondata.processing.sort.sortdata.IntermediateSortTempRowComparator.compare(IntermediateSortTempRowComparator.java:64)
>         at org.apache.carbondata.processing.sort.sortdata.IntermediateSortTempRowComparator.compare(IntermediateSortTempRowComparator.java:31)
>         at org.apache.carbondata.processing.sort.sortdata.SortTempFileChunkHolder.compareTo(SortTempFileChunkHolder.java:301)
>         at org.apache.carbondata.processing.sort.sortdata.SortTempFileChunkHolder.compareTo(SortTempFileChunkHolder.java:43)
>         at java.util.PriorityQueue.siftUpComparable(PriorityQueue.java:656)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:647)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:344)
>         at java.util.PriorityQueue.add(PriorityQueue.java:321)
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger$2.call(SingleThreadFinalSortFilesMerger.java:204)
>         at org.apache.carbondata.processing.sort.sortdata.SingleThreadFinalSortFilesMerger$2.call(SingleThreadFinalSortFilesMerger.java:190)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         ... 3 more

--
This message was sent by Atlassian Jira
(v8.3.4#803005)