Posted by
Akash R Nilugal (Jira) on
Nov 09, 2016; 2:26pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Created-CARBONDATA-399-Bad-Records-Data-Load-is-not-FAILED-even-bad-records-action-FAIL-tp2770.html
Babulal created CARBONDATA-399:
----------------------------------
Summary: [Bad Records] Data Load is not FAILED even bad_records_action="FAIL" .
Key: CARBONDATA-399
URL:
https://issues.apache.org/jira/browse/CARBONDATA-399 Project: CarbonData
Issue Type: Bug
Components: data-load
Affects Versions: 0.1.0-incubating
Environment: SUSE 11 SP4
YARN HA
3 Nodes
Reporter: Babulal
Priority: Minor
Data Load is not FAILED when string data are loaded in the int column .
1. Create table defect_5 (imei string ,deviceInformationId int,mac string,productdate timestamp,updatetime timestamp,gamePointId double,contractNumber double) stored by 'carbondata' TBLPROPERTIES('DICTIONARY_INCLUDE'='deviceInformationId') ;
deviceInformationId is int ( it will handled as dimension). Now load the data
2. 0: jdbc:hive2://ha-cluster/default> LOAD DATA inpath 'hdfs://hacluster/tmp/100_default_date_11_header_2.csv' into table defect_5 options('DELIMITER'=',', 'bad_records_action'='FAIL', 'QUOTECHAR'='"','FILEHEADER'='imei,deviceinformationid,mac,productdate,updatetime,gamepointid,contractnumber');
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.969 seconds)
3. Data
imei,deviceinformationid,mac,productdate,updatetime,gamepointid,contractnumber
1AA1,babu,Mikaa1,2015-01-01 11:00:00,2015-01-01 13:00:00,10,260
1AA2,3,Mikaa2,2015-01-02 12:00:00,2015-01-01 14:00:00,278,230
1AA3,1,Mikaa1,2015-01-03 13:00:00,2015-01-01 15:00:00,2556,1
1AA4,10,Mikaa2,2015-01-04 14:00:00,2015-01-01 16:00:00,640,254
1AA5,10,Mikaa,2015-01-05 15:00:00,2015-01-01 17:00:00,980,256
1AA6,10,Mikaa,2015-01-06 16:00:00,2015-01-01 18:00:00,1,2378
1AA7,10,Mikaa,2015-01-07 17:00:00,2015-01-01 19:00:00,96,234
1AA8,9,max,2015-01-08 18:00:00,2015-01-01 20:00:00,89,236
1AA9,10,max,2015-01-09 19:00:00,2015-01-01 21:00:00,198.36,239.2
Expect Outoput:- Data Load should FAIL
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)