[jira] [Created] (CARBONDATA-405) Data load fail if dataframe is created with LONG datatype column .

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (CARBONDATA-405) Data load fail if dataframe is created with LONG datatype column .

Akash R Nilugal (Jira)
Babulal created CARBONDATA-405:
----------------------------------

             Summary: Data load fail if dataframe is created with LONG datatype column .
                 Key: CARBONDATA-405
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-405
             Project: CarbonData
          Issue Type: Bug
          Components: data-load
    Affects Versions: 0.1.0-incubating
            Reporter: Babulal


Loading data from dataframe having LONG data type is failling with below error .

Follow the below steps
scala> val mydf = cc.read.format("json").load("/opt/data/people.json")

scala> mydf.printSchema
root
 |-- age: long (nullable = true)
 |-- name: string (nullable = true)


scala> mydf.write.format("carbondata").option("tableName","mycarbon3").save();
INFO  10-11 23:33:28,872 - main Query [
          CREATE TABLE IF NOT EXISTS DEFAULT.MYCARBON3
          (AGE LONG, NAME STRING)
          STORED BY 'ORG.APACHE.CARBONDATA.FORMAT'
      ]
INFO  10-11 23:33:28,877 - Parsing command:
          CREATE TABLE IF NOT EXISTS default.mycarbon3
          (age LONG, name STRING)
          STORED BY 'org.apache.carbondata.format'

NoViableAltException(162@[])
        at org.apache.hadoop.hive.ql.parse.HiveParser.type(HiveParser.java:38610)
        at org.apache.hadoop.hive.ql.parse.HiveParser.colType(HiveParser.java:38367)
        at org.apache.hadoop.hive.ql.parse.HiveParser.columnNameType(HiveParser.java:38051)



Expected:- long data type is not supported so it should be converted to supported data type ( Long to bigint)


Json file values

{"name":"Michael"}
{"name":"Andy", "age":30}
{"name":"Justin", "age":19}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)