Posted by
李寅威 on
Dec 19, 2016; 12:24pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/InvalidInputException-when-loading-data-to-table-tp4673.html
Hi all,
I'm now learning how to getting started with carbondata according to the tutorial:
https://cwiki.apache.org/confluence/display/CARBONDATA/Quick+Start.
I created a file named sample.csv under the path /home/hadoop/carbondata at the master node, and when I run the script:
scala>val dataFilePath = new File("../carbondata/sample.csv").getCanonicalPath
scala>cc.sql(s"load data inpath '$dataFilePath' into table test_table")
it turns out a "InvalidInputException" while the file is acctually exist, here is the scripts and logs:
scala> val dataFilePath = new File("../carbondata/sample.csv").getCanonicalPath
dataFilePath: String = /home/hadoop/carbondata/sample.csv
scala> cc.sql(s"load data inpath '$dataFilePath' into table test_table")
INFO 19-12 20:18:22,991 - main Query [LOAD DATA INPATH '/HOME/HADOOP/CARBONDATA/SAMPLE.CSV' INTO TABLE TEST_TABLE]
INFO 19-12 20:18:23,271 - Successfully able to get the table metadata file lock
INFO 19-12 20:18:23,276 - main Initiating Direct Load for the Table : (default.test_table)
INFO 19-12 20:18:23,279 - main Generate global dictionary from source data files!
INFO 19-12 20:18:23,296 - main [Block Distribution]
INFO 19-12 20:18:23,297 - main totalInputSpaceConsumed: 74 , defaultParallelism: 28
INFO 19-12 20:18:23,297 - main mapreduce.input.fileinputformat.split.maxsize: 16777216
INFO 19-12 20:18:23,380 - Block broadcast_0 stored as values in memory (estimated size 137.1 KB, free 137.1 KB)
INFO 19-12 20:18:23,397 - Block broadcast_0_piece0 stored as bytes in memory (estimated size 15.0 KB, free 152.1 KB)
INFO 19-12 20:18:23,398 - Added broadcast_0_piece0 in memory on 172.17.195.12:46335 (size: 15.0 KB, free: 511.1 MB)
INFO 19-12 20:18:23,399 - Created broadcast 0 from NewHadoopRDD at CarbonTextFile.scala:73
ERROR 19-12 20:18:23,431 - main generate global dictionary failed
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: /home/hadoop/carbondata/sample.csv
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:285)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:340)
at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:113)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
...
If any of you have met the same problem, would you tell me why this happen, looking forward to your replay, thx~