dev
hi spark 2.1.1 carbondata 1.1.1 hadoop 2.7.1 cc.sql("insert into e_carbon.prod_inst_c select prod_inst_id,owner_cust_id,'2' mvcc,latn_id,product_id,acc_prod_inst_id,address_id,payment_mode_cd,product_password,important_level,area_code,acc_nbr,exch_id,common_region_id,remark,pay_cycle,begin_rent_time,stop_rent_time,finish_time,stop_status,status_cd,create_date,status_date,update_date,proc_serial,use_cust_id,ext_prod_inst_id,address_desc,area_id,update_staff,create_staff,rec_update_date,account,ext_acc_prod_inst_id,distributor_id,eff_flg from e_carbon.prod_inst where latn_id=593").show; org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:125) at org.apache.carbondata.processing.newflow.DataLoadExecutor.execute(DataLoadExecutor.java:48) at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD$$anon$2.<init>(NewCarbonDataLoadRDD.scala:434) at org.apache.carbondata.spark.rdd.NewDataFrameLoaderRDD.compute(NewCarbonDataLoadRDD.scala:398) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.carbondata.processing.newflow.exception.CarbonDataLoadingException: unable to generate the mdkey at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl.java:181) at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.execute(DataWriterProcessorStepImpl.java:111) ... 11 more Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@23e0197a rejected from java.util.concurrent.ThreadPoolExecutor@64c083fa[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 19] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.addDataToStore(CarbonFactDataHandlerColumnar.java:466) at org.apache.carbondata.processing.newflow.steps.DataWriterProcessorStepImpl.processBatch(DataWriterProcessorStepImpl.java:178) ... 12 more yixu2001 |
hi,
From the log, I just known some issue happened during writing data file step in executor side. To locate this issue, I need more executor log. David
Best Regards
David Cai |
Free forum by Nabble | Edit this page |