Babulal created CARBONDATA-562:
---------------------------------- Summary: Carbon Context initialization is failed with spark 1.6.3 Key: CARBONDATA-562 URL: https://issues.apache.org/jira/browse/CARBONDATA-562 Project: CarbonData Issue Type: Bug Components: core Affects Versions: 0.2.0-incubating Reporter: Babulal Carbon is compiled with below command mvn clean -P spark-1.6,provided -Dspark.version=1.6.3 -DskipTests package start carbon context in yarn client mode bin/spark-shell --master yarn-client --executor-memory 2G --executor-cores 2 --driver-memory 2G --num-executors 3 ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 1.6.3 /_/ Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111) Type in expressions to have them evaluated. scala> import org.apache.spark.sql.CarbonContext; import org.apache.spark.sql.CarbonContext scala> val cc=new CarbonContext(sc,"hdfs://hacluser/opt/CarbonStore"); e of dependencies) scala.tools.reflect.ToolBoxError: reflective compilation has failed: not enough arguments for constructor Optimizer: (conf: org.apache.spark.sql.catalyst.CatalystConf)org.apache.spark.sql.catalyst.optimizer.Optimizer. Unspecified value parameter conf. at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314) at org.apache.spark.sql.CodeGenerateFactory.<init>(CodeGenerateFactory.scala:32) at org.apache.spark.sql.CodeGenerateFactory$.init(CodeGenerateFactory.scala:67) at org.apache.spark.sql.CarbonContext.<init>(CarbonContext.scala:58) at org.apache.spark.sql.CarbonContext.<init>(CarbonContext.scala:52) -- This message was sent by Atlassian JIRA (v6.3.4#6332) |
Administrator
|
Hi Babulal
Spark didn't support spark 1.6.3 ,you can try spark 1.6.1 and 1.6.2. Please refer to : https://cwiki.apache.org/confluence/display/CARBONDATA/Building+CarbonData+And+IDE+Configuration Regards Liang 2016-12-25 13:51 GMT+08:00 Babulal (JIRA) <[hidden email]>: > Babulal created CARBONDATA-562: > ---------------------------------- > > Summary: Carbon Context initialization is failed with spark > 1.6.3 > Key: CARBONDATA-562 > URL: https://issues.apache.org/jira/browse/CARBONDATA-562 > Project: CarbonData > Issue Type: Bug > Components: core > Affects Versions: 0.2.0-incubating > Reporter: Babulal > > > Carbon is compiled with below command > mvn clean -P spark-1.6,provided -Dspark.version=1.6.3 -DskipTests package > > start carbon context in yarn client mode > > bin/spark-shell --master yarn-client --executor-memory 2G > --executor-cores 2 --driver-memory 2G --num-executors 3 > > > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 1.6.3 > /_/ > > Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java > 1.8.0_111) > Type in expressions to have them evaluated. > > > scala> import org.apache.spark.sql.CarbonContext; > import org.apache.spark.sql.CarbonContext > > scala> val cc=new CarbonContext(sc,"hdfs://hacluser/opt/CarbonStore"); > e of dependencies) > scala.tools.reflect.ToolBoxError: reflective compilation has failed: > > not enough arguments for constructor Optimizer: (conf: > org.apache.spark.sql.catalyst.CatalystConf)org.apache.spark. > sql.catalyst.optimizer.Optimizer. > Unspecified value parameter conf. > at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ > ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314) > at org.apache.spark.sql.CodeGenerateFactory.<init>( > CodeGenerateFactory.scala:32) > at org.apache.spark.sql.CodeGenerateFactory$.init( > CodeGenerateFactory.scala:67) > at org.apache.spark.sql.CarbonContext.<init>( > CarbonContext.scala:58) > at org.apache.spark.sql.CarbonContext.<init>( > CarbonContext.scala:52) > > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > -- Regards Liang |
Free forum by Nabble | Edit this page |