Posted by
Akash R Nilugal (Jira) on
Sep 08, 2016; 6:03am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Commented-CARBONDATA-222-Query-issue-for-all-dimensions-are-no-dictionary-columns-tp887.html
[
https://issues.apache.org/jira/browse/CARBONDATA-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472899#comment-15472899 ]
ASF GitHub Bot commented on CARBONDATA-222:
-------------------------------------------
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/137#discussion_r77950155
--- Diff: core/src/main/java/org/apache/carbondata/core/carbon/datastore/impl/btree/BTreeDataRefNodeFinder.java ---
@@ -240,9 +240,16 @@ private int compareIndexes(IndexKey first, IndexKey second) {
firstNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset + SHORT_SIZE_IN_BYTES);
secondNodeDictionaryLength =
secondNoDictionaryKeyBuffer.getShort(nonDictionaryKeyOffset + SHORT_SIZE_IN_BYTES);
- compareResult = ByteUtil.UnsafeComparer.INSTANCE
- .compareTo(first.getNoDictionaryKeys(), actualOffset, firstNoDcitionaryLength,
- second.getNoDictionaryKeys(), actualOffset, secondNodeDictionaryLength);
+ int minLength = Math.min(firstNoDcitionaryLength, secondNodeDictionaryLength);
--- End diff --
@kumarvishal09 please comment on this. I am not getting how actualOffset is same for both keybuffers.
Also as I know length can be calculated by nextOffset-currentOffset. Please give the structure of data stored and how this function works
> Query issue for all dimensions are no dictionary columns
> --------------------------------------------------------
>
> Key: CARBONDATA-222
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-222> Project: CarbonData
> Issue Type: Bug
> Reporter: Gin-zhj
> Assignee: Gin-zhj
> Priority: Minor
>
> step 1:
> CREATE TABLE uniqdata_no (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_EXCLUDE'='CUST_NAME,ACTIVE_EMUI_VERSION');
> step 2:
> LOAD DATA INPATH 'D:/download/3lakh_3.csv' into table uniqdata_no OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
> step 3:
> select * from uniqdata_no limit 5;
> the fact file is:
> ,,,,,,,,,,,0
> query failed, catch exception:
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
> at org.apache.carbondata.core.util.ByteUtil$UnsafeComparer.compareTo(ByteUtil.java:197)
> at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.compareIndexes(BTreeDataRefNodeFinder.java:243)
> at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstLeafNode(BTreeDataRefNodeFinder.java:121)
> at org.apache.carbondata.core.carbon.datastore.impl.btree.BTreeDataRefNodeFinder.findFirstDataBlock(BTreeDataRefNodeFinder.java:80)
> at org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfIndex(CarbonInputFormat.java:546)
> at org.apache.carbondata.hadoop.CarbonInputFormat.getDataBlocksOfSegment(CarbonInputFormat.java:473)
> at org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:342)
> at org.apache.carbondata.hadoop.CarbonInputFormat.getSplitsNonFilter(CarbonInputFormat.java:304)
> at org.apache.carbondata.hadoop.CarbonInputFormat.getSplits(CarbonInputFormat.java:277)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)