Operation not allowed: STORED BY (from Spark Dataframe save)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Operation not allowed: STORED BY (from Spark Dataframe save)

yannv
Hello,

I am trying to create a carbon data table from a Spark Data Frame, however I
am getting an error with the (automatic create table statement)

I run this code on spark-shell (passing the carbon data assembly jar file
for 1.4.0 as well as master branch), on Azure HDInsight cluster with spark
2.2.1.

Code used :




This is the error I am getting :




I tried various constructors for the carbon object without success.
Note : I can create a Carbondata table and insert data from CSV file
successfully, but it looks like when the save is executed it tries to create
the (new) table and I get this error on "Stored by"...



Regards,
Yann



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Operation not allowed: STORED BY (from Spark Dataframe save)

xuchuanyin
Hi, did you create the dataframe through SparkSession?
If it is so, you do better to create that with CarbonSession, which extends
the SparkSession.

You may need to refer to
https://github.com/apache/carbondata/blob/master/docs/quick-start-guide.md




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Operation not allowed: STORED BY (from Spark Dataframe save)

yannv
Hi xuchuanyin

Thank you for your reply.

Raghu found the problem with my code.

I had to remove import spark.implicits._ ad use import carbon.implicits._
instead.

Thanks,
Yann



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/