Unable to run Carbon Shell on Spark 2.0

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Unable to run Carbon Shell on Spark 2.0

Harmeet Singh
I am trying to run Carbondata with Carbon shell because Thrift server support is still no there. When i am running the shell script, i am getting following error:

java.lang.ClassNotFoundException: org.apache.spark.repl.carbon.Main
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)


I am looking into spark2.0 module, the org.apache.spark.repl.carbon.Main class is not there. I am trying to create new Main class as like previous spark version but when i build the Carbondata i am getting following error:

[ERROR] /home/harmeet/CarbonData/integration/spark2/src/main/scala/org/apache/spark/repl/carbon/Main.scala:31: error: type mismatch;
[INFO]  found   : Array[String]
[INFO]  required: scala.tools.nsc.Settings
[INFO]     _interp.process(args)


Without Carbonshell or Thrift Server i am not able to pursue with Carbondata with Spark 2.0. Please provide some valid solution ASAP.
Reply | Threaded
Open this post in threaded view
|

Re: Unable to run Carbon Shell on Spark 2.0

Jacky Li
Hi Harmeet,

Now ThriftServer that uses CarbonSession has been merged into master branch, please try the latest master. Thanks.

Regards,
Jacky
Reply | Threaded
Open this post in threaded view
|

Re: Unable to run Carbon Shell on Spark 2.0

Harmeet Singh
Hello Jacky,

Yes, thrift server running successfully on spark 2.0.