Re: [jira] [Created] (CARBONDATA-562) Carbon Context initialization is failed with spark 1.6.3

Posted by Liang Chen on Dec 25, 2016; 6:35am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Created-CARBONDATA-562-Carbon-Context-initialization-is-failed-with-spark-1-6-3-tp4969p4970.html

Hi Babulal

Spark didn't support spark 1.6.3 ,you can try spark 1.6.1 and 1.6.2.
Please refer to :
https://cwiki.apache.org/confluence/display/CARBONDATA/Building+CarbonData+And+IDE+Configuration

Regards
Liang

2016-12-25 13:51 GMT+08:00 Babulal (JIRA) <[hidden email]>:

> Babulal created CARBONDATA-562:
> ----------------------------------
>
>              Summary: Carbon Context initialization is failed with spark
> 1.6.3
>                  Key: CARBONDATA-562
>                  URL: https://issues.apache.org/jira/browse/CARBONDATA-562
>              Project: CarbonData
>           Issue Type: Bug
>           Components: core
>     Affects Versions: 0.2.0-incubating
>             Reporter: Babulal
>
>
> Carbon is compiled with below command
> mvn clean -P spark-1.6,provided -Dspark.version=1.6.3 -DskipTests package
>
> start carbon context in yarn client mode
>
>  bin/spark-shell --master yarn-client --executor-memory 2G
> --executor-cores 2 --driver-memory 2G  --num-executors 3
>
>
>   ____              __
>      / __/__  ___ _____/ /__
>     _\ \/ _ \/ _ `/ __/  '_/
>    /___/ .__/\_,_/_/ /_/\_\   version 1.6.3
>       /_/
>
> Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java
> 1.8.0_111)
> Type in expressions to have them evaluated.
>
>
> scala> import org.apache.spark.sql.CarbonContext;
> import org.apache.spark.sql.CarbonContext
>
> scala> val cc=new CarbonContext(sc,"hdfs://hacluser/opt/CarbonStore");
> e of dependencies)
> scala.tools.reflect.ToolBoxError: reflective compilation has failed:
>
> not enough arguments for constructor Optimizer: (conf:
> org.apache.spark.sql.catalyst.CatalystConf)org.apache.spark.
> sql.catalyst.optimizer.Optimizer.
> Unspecified value parameter conf.
>         at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$
> ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314)
> at org.apache.spark.sql.CodeGenerateFactory.<init>(
> CodeGenerateFactory.scala:32)
>         at org.apache.spark.sql.CodeGenerateFactory$.init(
> CodeGenerateFactory.scala:67)
>         at org.apache.spark.sql.CarbonContext.<init>(
> CarbonContext.scala:58)
>         at org.apache.spark.sql.CarbonContext.<init>(
> CarbonContext.scala:52)
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

--
Regards
Liang