Unable to load CSV using differenent delimiter like "|"(Pipe) and ";" (Semicolon) etc

[Harmeet]

Hey Team,

I am trying to load data from csv using different delimiters like "|"(Pipe) and ";" (Semicolon) etc. But the system gave me an different errors according to delimiter. The example as below:

create table one (name string, description string, salary double, age int, dob timestamp) stored by 'carbondata';

1. "|" (Pipe Delimeter)

deliandquote.csv File >>

name, description, salary, age, dob
tammy| 'my name'| 900000| 22| 19/10/2019

0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/deliandquote.csv' into table one OPTIONS("DELIMITER"="|", 'QUOTECHAR'="'");

Error: java.lang.Exception: DataLoad failure: CSV File provided is not proper. Column names in schema and csv header are not same. CSVFile Name : deliandquote.csv (state=,code=0)

2. ";" (Semicolon)

semiandquote.csv File >>

0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"=";", 'QUOTECHAR'="'");

Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0)

I am not getting, why the different errors are shown, if we are not supporting other characters. Or this is the defect of carbondata ?

[杰]

hi, Harmeet


  For problem 1:
    u need to keep that column name is also delimited by '|',  
    that is ,  name|description|salary|age|dob
 For problem 2:
   in Load command, change "DELIMITER"=";" to "DELIMITER"="\;"  ,
   because ";" is usually means the end,  so need to be escaped.


thanks
Jay




------------------ Original ------------------
From:  "Harmeet";<[hidden email]>;
Date:  Fri, Oct 21, 2016 02:23 PM
To:  "dev"<[hidden email]>;

Subject:  Unable to load CSV using differenent delimiter like "|"(Pipe) and";" (Semicolon) etc



Hey Team,

I am trying to load data from csv using different delimiters like "|"(Pipe)
and ";" (Semicolon) etc. But the system gave me an different errors
according to delimiter. The example as below:

*create table one (name string, description string, salary double, age int,
dob timestamp) stored by 'carbondata';*

*1. "|" (Pipe Delimeter)*

deliandquote.csv File >>

name, description, salary, age, dob
tammy| 'my name'| 900000| 22| 19/10/2019

0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/deliandquote.csv' into table one
OPTIONS("DELIMITER"="|", 'QUOTECHAR'="'");

Error: java.lang.Exception: DataLoad failure: CSV File provided is not
proper. Column names in schema and csv header are not same. CSVFile Name :
deliandquote.csv (state=,code=0)

*2. ";" (Semicolon)*

semiandquote.csv File >>

0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
OPTIONS("DELIMITER"=";", 'QUOTECHAR'="'");

Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near
'one'; line 1 pos 93 (state=,code=0)

I am not getting, why the different errors are shown, if we are not
supporting other characters. Or this is the defect of carbondata ?




--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.

[Harmeet]

Hey Jay, Thanks for your reply.

First case with Pipe symbols working fine, But still in second case i am getting the same error. Below is my query :

csv file >>
name; description; salary; age; dob
tammy; 'my name'; 900000; 22; 19/10/2019


0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'");

Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0)

Please correnct if something getting wrong.

[杰]

hi, Harmeet


  not reproduce. pls provide more details.


reagrds
Jay


------------------ Original ------------------
From:  "Harmeet";<[hidden email]>;
Date:  Fri, Oct 21, 2016 02:51 PM
To:  "dev"<[hidden email]>;

Subject:  Re: Unable to load CSV using differenent delimiter like "|"(Pipe)and";" (Semicolon) etc



Hey Jay, Thanks for your reply.

First case with Pipe symbols working fine, But still in second case i am
getting the same error. Below is my query :

csv file >>
name; description; salary; age; dob
tammy; 'my name'; 900000; 22; 19/10/2019


0: jdbc:hive2://127.0.0.1:10000> load data local inpath
'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one
OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'");

Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near
'one'; line 1 pos 93 (state=,code=0)

Please correnct if something getting wrong.



--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140p2144.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.

[Harmeet]

Hey Jay,

I am getting still same error. My code is up to date. I am creating new database and new table for testing. Following are details:

>> Create database
0: jdbc:hive2://127.0.0.1:10000> create database needb;
+---------+--+
| result  |
+---------+--+
+---------+--+
No rows selected (6.227 seconds)

>> use database
0: jdbc:hive2://127.0.0.1:10000> use needb;
+---------+--+
| result  |
+---------+--+
+---------+--+
No rows selected (0.081 seconds)

>> Create table
0: jdbc:hive2://127.0.0.1:10000> create table one(name string, description string, salary double, age int, dob timestamp) stored by 'carbondata';
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (1.297 seconds)

>> CSV File Data
name; description; salary; age; dob
tammy; my name; 900000; 22; 19/10/2019

>> Load Data
0: jdbc:hive2://127.0.0.1:10000> load data inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one options('DELIMITER'="\;");
Error: org.apache.spark.sql.AnalysisException: missing EOF at 'options' near 'one'; line 1 pos 87 (state=,code=0)

I am still not figure out, what is missing ?