[jira] [Created] (CARBONDATA-2787) Data load on table with single pass true having preaggregate datamaps without precreated dictionary throws incorrect error message

[Akash R Nilugal (Jira)]

Chetan Bhat created CARBONDATA-2787:
---------------------------------------

             Summary: Data load on table with single pass true having preaggregate datamaps without precreated dictionary throws incorrect error message
                 Key: CARBONDATA-2787
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2787
             Project: CarbonData
          Issue Type: Bug
          Components: data-load
    Affects Versions: 1.4.1
         Environment: Spark 2.2
            Reporter: Chetan Bhat


Steps :

Create table .

Create 5 datamaps.

load data in table with single pass true without precreated dictionary configuration.

0: jdbc:hive2://10.18.98.136:23040/default> CREATE TABLE uniqdata_decimal(CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1', 'SORT_SCOPE'='GLOBAL_SORT');
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.21 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> create datamap uniqdata_agg_decimal on table uniqdata_decimal using 'preaggregate' as select cust_name, avg(DECIMAL_COLUMN1) from uniqdata_decimal group by DECIMAL_COLUMN1, cust_name;
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.557 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> create datamap uniqdata_agg_sum_decimal on table uniqdata_decimal using 'preaggregate' as select cust_name, sum(DECIMAL_COLUMN1) from uniqdata_decimal group by DECIMAL_COLUMN1, cust_name;
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.515 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> create datamap uniqdata_agg_count_decimal on table uniqdata_decimal using 'preaggregate' as select cust_name, count(DECIMAL_COLUMN1) from uniqdata_decimal group by DECIMAL_COLUMN1, cust_name;
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.492 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> create datamap uniqdata_agg_min_decimal on table uniqdata_decimal using 'preaggregate' as select cust_name, min(DECIMAL_COLUMN1) from uniqdata_decimal group by DECIMAL_COLUMN1, cust_name;
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.571 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> create datamap uniqdata_agg_max_decimal on table uniqdata_decimal using 'preaggregate' as select cust_name, max(DECIMAL_COLUMN1) from uniqdata_decimal group by DECIMAL_COLUMN1, cust_name;
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.471 seconds)
0: jdbc:hive2://10.18.98.136:23040/default> show datamap on table uniqdata_decimal;
+-----------------------------+---------------+---------------------------------------------------+---------------------+--+
| DataMapName | ClassName | Associated Table | DataMap Properties |
+-----------------------------+---------------+---------------------------------------------------+---------------------+--+
| uniqdata_agg_decimal | preaggregate | test.uniqdata_decimal_uniqdata_agg_decimal | |
| uniqdata_agg_sum_decimal | preaggregate | test.uniqdata_decimal_uniqdata_agg_sum_decimal | |
| uniqdata_agg_count_decimal | preaggregate | test.uniqdata_decimal_uniqdata_agg_count_decimal | |
| uniqdata_agg_min_decimal | preaggregate | test.uniqdata_decimal_uniqdata_agg_min_decimal | |
| uniqdata_agg_max_decimal | preaggregate | test.uniqdata_decimal_uniqdata_agg_max_decimal | |
+-----------------------------+---------------+---------------------------------------------------+---------------------+--+
5 rows selected (0.035 seconds)

0: jdbc:hive2://10.18.98.136:23040/default> LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table uniqdata_decimal OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1', *'SINGLE_PASS'='true')*;

 

Issue : Load fails with incorrect error message as shown below.

0: jdbc:hive2://10.18.98.136:23040/default> LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table uniqdata_decimal OPTIONS('DELIMITER'=',', 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1', *'SINGLE_PASS'='true'*);
*Error: java.lang.Exception: DataLoad failure (state=,code=0)*

Expected : The error message should be displayed on load failure as single pass should not be configured without precreated dictionary.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)