[jira] [Resolved] (CARBONDATA-404) Data loading from DataFrame to carbon table is FAILED

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacky Li resolved CARBONDATA-404.
---------------------------------
       Resolution: Fixed
         Assignee: Ravindra Pesala
    Fix Version/s: 0.3.0-incubating

> Data loading from DataFrame to carbon table is FAILED
> -----------------------------------------------------
>
>                 Key: CARBONDATA-404
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-404
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>    Affects Versions: 0.1.0-incubating
>            Reporter: Babulal
>            Assignee: Ravindra Pesala
>             Fix For: 0.3.0-incubating
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Data loading FAILED when   Loading data from DataFrame with tempCSV option =true    (Default option ) in 3 Node cluster .
> Steps
>  val customSchema = StructType(Array(    StructField("imei", StringType, true),    StructField("deviceInformationId", IntegerType, true),    StructField("mac", StringType, true),    StructField("productdate", TimestampType , true),    StructField("updatetime", TimestampType, true),    StructField("gamePointId", DoubleType, true),    StructField("contractNumber", DoubleType, true)       ));
> val df = cc.read.format("com.databricks.spark.csv").option("header", "false").schema(customSchema).load("/opt/data/xyz/100_default_date_11_header.csv");
> Start data loading
> scala> df.write.format("carbondata").option("tableName","mycarbon2").save();
> INFO  10-11 23:24:35,970 - main Query [
>           CREATE TABLE IF NOT EXISTS DEFAULT.MYCARBON2
>           (IMEI STRING, DEVICEINFORMATIONID INT, MAC STRING, PRODUCTDATE TIMESTAMP, UPDATETIME TIMESTAMP, GAMEPOINTID DOUBLE, CONTRACTNUMBER DOUBLE)
>           STORED BY 'ORG.APACHE.CARBONDATA.FORMAT'
>       ]
> INFO  10-11 23:24:35,977 - Parsing command:
>           CREATE TABLE IF NOT EXISTS default.mycarbon2
>           (imei STRING, deviceInformationId INT, mac STRING, productdate TIMESTAMP, updatetime TIMESTAMP, gamePointId DOUBLE, contractNumber DOUBLE)
>           STORED BY 'org.apache.carbondata.format'
> INFO  10-11 23:24:35,978 - Parse Completed
> INFO  10-11 23:24:36,227 - main Query [
>           LOAD DATA INPATH './TEMPCSV'
>           INTO TABLE DEFAULT.MYCARBON2
>           OPTIONS ('FILEHEADER' = 'IMEI,DEVICEINFORMATIONID,MAC,PRODUCTDATE,UPDATETIME,GAMEPOINTID,CONTRACTNUMBER')
>       ]
> INFO  10-11 23:24:36,233 - Successfully able to get the table metadata file lock
> AUDIT 10-11 23:24:36,234 - [BLR1000007781][root][Thread-1]Dataload failed for default.mycarbon2. The input file does not exist: ./tempCSV
> INFO  10-11 23:24:36,234 - main Successfully deleted the lock file /tmp/default/mycarbon2/meta.lock
> INFO  10-11 23:24:36,234 - Table MetaData Unlocked Successfully after data load
> org.apache.carbondata.processing.etl.DataLoadingException: The input file does not exist: ./tempCSV
>         at org.apache.spark.util.FileUtils$$anonfun$getPaths$1.apply$mcVI$sp(FileUtils.scala:66)
> CSV DATA
> 1AA1,1,Mikaa1,2015-01-01 11:00:00,2015-01-01 13:00:00,198,260
> 1AA2,3,Mikaa2,2015-01-02 12:00:00,2015-01-01 14:00:00,278,230
> 1AA3,1,Mikaa1,2015-01-03 13:00:00,2015-01-01 15:00:00,2556,1
> 1AA4,10,Mikaa2,2015-01-04 14:00:00,2015-01-01 16:00:00,640,254
> 1AA5,10,Mikaa,2015-01-05 15:00:00,2015-01-01 17:00:00,980,256
> 1AA6,10,Mikaa,2015-01-06 16:00:00,2015-01-01 18:00:00,1,2378
> 1AA7,10,Mikaa,2015-01-07 17:00:00,2015-01-01 19:00:00,96,234
> 1AA8,9,max,2015-01-08 18:00:00,2015-01-01 20:00:00,89,236

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)