[jira] [Created] (CARBONDATA-1798) class cast exception when loading data from a hive table to carbondata table

[Akash R Nilugal (Jira)]

anubhav tarar created CARBONDATA-1798:
-----------------------------------------

             Summary: class cast exception when loading data from a hive table to carbondata table
                 Key: CARBONDATA-1798
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1798
             Project: CarbonData
          Issue Type: Bug
          Components: spark-integration
    Affects Versions: 1.3.0
         Environment: spark2.1
            Reporter: anubhav tarar
            Assignee: anubhav tarar


1. i created the hive tpch table named lineitem with around 10 gb data

CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2)  ,L_EXTENDEDPRICE DECIMAL(15,2)  ,L_DISCOUNT DECIMAL(15,2)  ,L_TAX DECIMAL(15,2)  ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored as orc;

2.tried to insert the data from that hive table into carbondata table

CREATE TABLE LINEITEM ( L_ORDERKEY INT ,L_PARTKEY INT ,L_SUPPKEY INT ,L_LINENUMBER INT ,L_QUANTITY DECIMAL(15,2)  ,L_EXTENDEDPRICE DECIMAL(15,2)  ,L_DISCOUNT DECIMAL(15,2)  ,L_TAX DECIMAL(15,2)  ,L_RETURNFLAG STRING ,L_LINESTATUS STRING ,L_SHIPDATE Date ,L_COMMITDATE Date ,L_RECEIPTDATE Date ,L_SHIPINSTRUCT STRING ,L_SHIPMODE STRING ,L_COMMENT STRING) stored by 'carbondata';

then i used INSERT INTO LINEITEM SELECT * FROM LINEITEM_ORC;


here are error logs

17/11/22 16:56:40 INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on slave1:41401 (size: 25.2 KB, free: 3.0 GB)
17/11/22 16:59:12 WARN TaskSetManager: Lost task 0.0 in stage 15.0 (TID 20, hadoop-master, executor 2): org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
        at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:114)
        at org.apache.carbondata.processing.loading.steps.SortProcessorStepImpl.execute(SortProcessorStepImpl.java:62)
        at org.apache.carbondata.processing.loading.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:87)
        at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:50)
        at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD$$anon$2.<init>(NewCarbonDataLoadRDD.scala:391)
        at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD.internalCompute(NewCarbonDataLoadRDD.scala:354)
        at org.apache.carbondata.spark.rdd.CarbonRDD.compute(CarbonRDD.scala:60)
        at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
        at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
        at org.apache.spark.scheduler.Task.run(Task.scala:99)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:188)
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.finish(UnsafeIntermediateMerger.java:171)
        at org.apache.carbondata.processing.loading.sort.impl.UnsafeParallelReadMergeSorterImpl.sort(UnsafeParallelReadMergeSorterImpl.java:107)
        ... 14 more
Caused by: java.util.concurrent.ExecutionException: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateMerger.checkForFailure(UnsafeIntermediateMerger.java:185)
        ... 16 more
Caused by: org.apache.carbondata.processing.sort.exception.CarbonSortKeyAndGroupByException:
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:142)
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:45)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 more
Caused by: java.lang.ClassCastException: java.math.BigDecimal cannot be cast to [B
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.writeDataTofile(UnsafeIntermediateFileMerger.java:333)
        at org.apache.carbondata.processing.loading.sort.unsafe.merger.UnsafeIntermediateFileMerger.call(UnsafeIntermediateFileMerger.java:113)
        ... 5 more

17/11/22 16:59:12 INFO TaskSetManager: Starting task 0.1 in stage 15.0 (TID 22, hadoop-master, executor 2, partition 0, NODE_LOCAL, 7696 bytes)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)