[jira] [Comment Edited] (CARBONDATA-2548) When using presto to query a table on carbondata, the "Could not read blocklet details" exception is reported

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505927#comment-16505927 ]

Kevin Kong edited comment on CARBONDATA-2548 at 6/8/18 11:18 AM:
-----------------------------------------------------------------

Data demo

CSV format:

 

1,31726ff0-3a3a-4cdf-af7b-9e049b112817,3,4,5,PRO_TYPE_6,7,8,9,10,11,110,0.40,44.00,2018-06-08 07:19:34,16,DELIVERY_ADDRESS_17,RECIPIENTS_18,13800000019,2018-06-08,RECIPIENTS_21

 

 

JSON format:

 

{"delivery_address":"DELIVERY_ADDRESS_17","comments":"RECIPIENTS_21","exchange_rate":8,"delivery_channel":16,"pro_type":"PRO_TYPE_6","unit_cost_price":9,"order_time":"2018-06-08 07:17:42","order_inputer":5,"unit_selling_price":10,"order_code":"3ae1d29e-3f2c-4235-82c3-acf1f8d6471e","order_account_amount":44.00,"delivery_date":"2018-06-08","sales_area_id":3,"order_discount":0.40,"recipients":"RECIPIENTS_18","order_amount":110,"contact":"13800000019","currency":7,"id":1,"order_num":11,"sales_id":4}

 


was (Author: kevintop):
Data demo

JSON format:

 

{"delivery_address":"DELIVERY_ADDRESS_17","comments":"RECIPIENTS_21","exchange_rate":8,"delivery_channel":16,"pro_type":"PRO_TYPE_6","unit_cost_price":9,"order_time":"2018-06-08 07:17:42","order_inputer":5,"unit_selling_price":10,"order_code":"3ae1d29e-3f2c-4235-82c3-acf1f8d6471e","order_account_amount":44.00,"delivery_date":"2018-06-08","sales_area_id":3,"order_discount":0.40,"recipients":"RECIPIENTS_18","order_amount":110,"contact":"13800000019","currency":7,"id":1,"order_num":11,"sales_id":4}

 

> When using presto to query a table on carbondata, the "Could not read blocklet details" exception is reported
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-2548
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2548
>             Project: CarbonData
>          Issue Type: Bug
>          Components: presto-integration
>    Affects Versions: 1.4.0
>            Reporter: Kevin Kong
>            Priority: Blocker
>
> I think it may be line 373 in CarbonTableInputFormat.java. When the CarbonInputSplit object was constructed, the detailInfo property was not initialized
>  
>                   while (((double) bytesRemaining) / splitSize > 1.1) {
>                     int blkIndex = getBlockIndex(blkLocations, length - bytesRemaining);
>                     splits.add(makeSplit(segment.getSegmentNo(), path, length - bytesRemaining,
>                         splitSize, blkLocations[blkIndex].getHosts(),
>                         blkLocations[blkIndex].getCachedHosts(), FileFormat.ROW_V1));
>  
>  
>  
> The detailed exception is as follows：
>  
> 2018-05-28T00:43:23.903+0800 DEBUG query-execution-11 com.facebook.presto.execution.QueryStateMachine Query 20180527_164323_00008_2xnww failed
> java.lang.RuntimeException: Could not read blocklet details
>  at org.apache.carbondata.presto.impl.CarbonLocalInputSplit.convertSplit(CarbonLocalInputSplit.java:131)
>  at org.apache.carbondata.presto.CarbondataPageSourceProvider.createQueryModel(CarbondataPageSourceProvider.java:139)
>  at org.apache.carbondata.presto.CarbondataPageSourceProvider.createReader(CarbondataPageSourceProvider.java:98)
>  at org.apache.carbondata.presto.CarbondataPageSourceProvider.createPageSource(CarbondataPageSourceProvider.java:79)
>  at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorPageSourceProvider.createPageSource(ClassLoaderSafeConnectorPageSourceProvider.java:44)
>  at com.facebook.presto.split.PageSourceManager.createPageSource(PageSourceManager.java:56)
>  at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:259)
>  at com.facebook.presto.operator.Driver.processInternal(Driver.java:337)
>  at com.facebook.presto.operator.Driver.lambda$processFor$6(Driver.java:241)
>  at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:614)
>  at com.facebook.presto.operator.Driver.processFor(Driver.java:235)
>  at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:622)
>  at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
>  at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
>  at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:485)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)