White Space does not trim in CarbonData

Posted by vin7149 on May 16, 2017; 12:39pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/White-Space-does-not-trim-in-CarbonData-tp12736.html

Hi Community,

I have created a table in hive and same in CarbonData, In my CSV there is whitespace after the data which is working fine in hive table whereas it shows null in CarbonData table.

In Hive

1: Table created in Hive

create table my_user(id int, name string, address1 string, address2 string,salary double, incentive int)ROW FORMAT DELIMITED FIELDS TERMINATED BY ",";

2: Load Data in table

LOAD DATA LOCAL INPATH '/home/vinod/Downloads/my_user2.csv' OVERWRITE INTO TABLE my_user;

3: Select Query on hive

0: jdbc:hive2://localhost:10000> select * from my_user;
+-----+-----------+-----------+-----------+----------+------------+--+
| id  |   name    | address1  | address2  |  salary  | incentive  |
+-----+-----------+-----------+-----------+----------+------------+--+
| 1   | 'katy'    | 'london'  | 'UK'      | 5000.0   | 5000       |
| 2   | 'John'    | 'goa'     | 'goa'     | 12000.0  | 500        |
| 3   | 'Rachel'  | 'goa'     | 'goa'     | 10000.0  | 1000       |
+-----+-----------+-----------+-----------+----------+------------+--+
3 rows selected (0.122 seconds)

In Carbon

1: Create Table in Carbon

create table my_user_c(id int, name string, address1 string, address2 string,salary double, incentive int) stored by 'carbondata';

2: Load Data in table

load data inpath 'hdfs://localhost:54310/my_user2.csv' into table my_user_c;

3: Select Query on carbon

0: jdbc:hive2://localhost:10000> select * from my_user_c;
+-----+-----------+-----------+-----------+----------+------------+--+
| id  |   name    | address1  | address2  |  salary  | incentive  |
+-----+-----------+-----------+-----------+----------+------------+--+
| 2   | 'John'    | 'goa'     | 'goa'     | 12000.0  | 500        |
| 3   | 'Rachel'  | 'goa'     | 'goa'     | 10000.0  | NULL      |
| 1   | 'katy'    | 'london'  | 'UK'      | 5000.0   | 5000       |
+-----+-----------+-----------+-----------+----------+------------+--+
3 rows selected (0.193 seconds)


it should display 1000 instead of NULL value in CarbonData.