Posted by
李寅威 on
Dec 15, 2016; 11:45am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Some-questions-about-compiling-carbondata-tp4498.html
Hi all,
I've tried the following two ways to compile carbondata, but they finally all failed, if anyone could help me:
1. with the latest verson carbondata at github and spark 2.0.0, use the command
mvn -DskipTests -Pspark-2.0 -Dspark.version=2.0.0 clean package
and it compiled successly.
but when I run the script ./bin/carbon-spark-shell , it turns out:
ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory
ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory
./bin/carbon-spark-shell: line 78: /bin/spark-submit: No such file or directory
and when I run the script ./bin/carbon-spark-sql, it turns out:
$SPARK_HOME is not set
ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory
ls: cannot access /home/hadoop/incubator-carbondata/assembly/target/scala-2.10: No such file or directory
./bin/carbon-spark-sql: line 79: cmd: command not found
./bin/carbon-spark-sql: line 89: /bin/spark-submit: No such file or directory
./bin/carbon-spark-sql: line 89: exec: /bin/spark-submit: cannot execute: No such file or directory
2.with the carbondata-0.2.0 and spark 1.5.0, use the command
mvn -DskipTests -Pspark-1.5 -X -Dspark.version=1.5.0 clean package
and it compiled failed. the error message is :
[ERROR] /home/hadoop/carbondata-0.2.0/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonHiveMetadataUtil.scala:46: error: not found: value SqlParser
[INFO] val tableIdent = SqlParser.parseTableIdentifier(tableWithDb)
[INFO] ^
[WARNING] one warning found
[ERROR] one error found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache CarbonData :: Parent ........................ SUCCESS [ 0.971 s]
[INFO] Apache CarbonData :: Common ........................ SUCCESS [ 1.702 s]
[INFO] Apache CarbonData :: Core .......................... SUCCESS [ 4.428 s]
[INFO] Apache CarbonData :: Processing .................... SUCCESS [ 1.587 s]
[INFO] Apache CarbonData :: Hadoop ........................ SUCCESS [ 1.104 s]
[INFO] Apache CarbonData :: Spark ......................... FAILURE [ 11.456 s]
[INFO] Apache CarbonData :: Assembly ...................... SKIPPED
[INFO] Apache CarbonData :: Examples ...................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.432 s
[INFO] Finished at: 2016-12-15T19:41:29+08:00
[INFO] Final Memory: 69M/995M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project carbondata-spark: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project carbondata-spark: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136)
at org_scala_tools_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:80)
at org_scala_tools_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:124)
at org_scala_tools_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:80)
at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342)
... 22 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :carbondata-spark