Hey Team,
I am trying to load data from csv using different delimiters like "|"(Pipe) and ";" (Semicolon) etc. But the system gave me an different errors according to delimiter. The example as below: create table one (name string, description string, salary double, age int, dob timestamp) stored by 'carbondata'; 1. "|" (Pipe Delimeter) deliandquote.csv File >> name, description, salary, age, dob tammy| 'my name'| 900000| 22| 19/10/2019 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/deliandquote.csv' into table one OPTIONS("DELIMITER"="|", 'QUOTECHAR'="'"); Error: java.lang.Exception: DataLoad failure: CSV File provided is not proper. Column names in schema and csv header are not same. CSVFile Name : deliandquote.csv (state=,code=0) 2. ";" (Semicolon) semiandquote.csv File >> 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"=";", 'QUOTECHAR'="'"); Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0) I am not getting, why the different errors are shown, if we are not supporting other characters. Or this is the defect of carbondata ? |
hi, Harmeet
For problem 1: u need to keep that column name is also delimited by '|', that is , name|description|salary|age|dob For problem 2: in Load command, change "DELIMITER"=";" to "DELIMITER"="\;" , because ";" is usually means the end, so need to be escaped. thanks Jay ------------------ Original ------------------ From: "Harmeet";<[hidden email]>; Date: Fri, Oct 21, 2016 02:23 PM To: "dev"<[hidden email]>; Subject: Unable to load CSV using differenent delimiter like "|"(Pipe) and";" (Semicolon) etc Hey Team, I am trying to load data from csv using different delimiters like "|"(Pipe) and ";" (Semicolon) etc. But the system gave me an different errors according to delimiter. The example as below: *create table one (name string, description string, salary double, age int, dob timestamp) stored by 'carbondata';* *1. "|" (Pipe Delimeter)* deliandquote.csv File >> name, description, salary, age, dob tammy| 'my name'| 900000| 22| 19/10/2019 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/deliandquote.csv' into table one OPTIONS("DELIMITER"="|", 'QUOTECHAR'="'"); Error: java.lang.Exception: DataLoad failure: CSV File provided is not proper. Column names in schema and csv header are not same. CSVFile Name : deliandquote.csv (state=,code=0) *2. ";" (Semicolon)* semiandquote.csv File >> 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"=";", 'QUOTECHAR'="'"); Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0) I am not getting, why the different errors are shown, if we are not supporting other characters. Or this is the defect of carbondata ? -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com. |
Hey Jay, Thanks for your reply.
First case with Pipe symbols working fine, But still in second case i am getting the same error. Below is my query : csv file >> name; description; salary; age; dob tammy; 'my name'; 900000; 22; 19/10/2019 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'"); Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0) Please correnct if something getting wrong. |
hi, Harmeet
not reproduce. pls provide more details. reagrds Jay ------------------ Original ------------------ From: "Harmeet";<[hidden email]>; Date: Fri, Oct 21, 2016 02:51 PM To: "dev"<[hidden email]>; Subject: Re: Unable to load CSV using differenent delimiter like "|"(Pipe)and";" (Semicolon) etc Hey Jay, Thanks for your reply. First case with Pipe symbols working fine, But still in second case i am getting the same error. Below is my query : csv file >> name; description; salary; age; dob tammy; 'my name'; 900000; 22; 19/10/2019 0: jdbc:hive2://127.0.0.1:10000> load data local inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one OPTIONS("DELIMITER"="\;", 'QUOTECHAR'="'"); Error: org.apache.spark.sql.AnalysisException: missing EOF at 'OPTIONS' near 'one'; line 1 pos 93 (state=,code=0) Please correnct if something getting wrong. -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Unable-to-load-CSV-using-differenent-delimiter-like-Pipe-and-Semicolon-etc-tp2140p2144.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com. |
Hey Jay,
I am getting still same error. My code is up to date. I am creating new database and new table for testing. Following are details: >> Create database 0: jdbc:hive2://127.0.0.1:10000> create database needb; +---------+--+ | result | +---------+--+ +---------+--+ No rows selected (6.227 seconds) >> use database 0: jdbc:hive2://127.0.0.1:10000> use needb; +---------+--+ | result | +---------+--+ +---------+--+ No rows selected (0.081 seconds) >> Create table 0: jdbc:hive2://127.0.0.1:10000> create table one(name string, description string, salary double, age int, dob timestamp) stored by 'carbondata'; +---------+--+ | Result | +---------+--+ +---------+--+ No rows selected (1.297 seconds) >> CSV File Data name; description; salary; age; dob tammy; my name; 900000; 22; 19/10/2019 >> Load Data 0: jdbc:hive2://127.0.0.1:10000> load data inpath 'hdfs://localhost:54310/home/harmeet/semiandquote.csv' into table one options('DELIMITER'="\;"); Error: org.apache.spark.sql.AnalysisException: missing EOF at 'options' near 'one'; line 1 pos 87 (state=,code=0) I am still not figure out, what is missing ? |
Free forum by Nabble | Edit this page |