[jira] [Updated] (CARBONDATA-3128) HiveExample has some exception

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

xubo245 updated CARBONDATA-3128:
--------------------------------
    Description:
HiveExample has some exception:
org.apache.carbondata.hiveexample.HiveExample

Version:

  -Pspark.version=2.1.1


{code:java}
2018-11-26 15:06:01 INFO  CarbonProperties:1512 - Considered value for min max byte limit for string is: 200
Exception in thread "main" java.lang.ClassNotFoundException: org.apache.spark.sql.hive.CarbonSessionStateBuilder
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
        at org.apache.spark.util.CarbonReflectionUtils$.createObject(CarbonReflectionUtils.scala:322)
        at org.apache.spark.util.CarbonReflectionUtils$.getSessionState(CarbonReflectionUtils.scala:218)
        at org.apache.spark.sql.CarbonSession.sessionState$lzycompute(CarbonSession.scala:56)
        at org.apache.spark.sql.CarbonSession.sessionState(CarbonSession.scala:55)
        at org.apache.spark.sql.CarbonSession$CarbonBuilder$$anonfun$getOrCreateCarbonSession$2.apply(CarbonSession.scala:258)
        at org.apache.spark.sql.CarbonSession$CarbonBuilder$$anonfun$getOrCreateCarbonSession$2.apply(CarbonSession.scala:258)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
        at org.apache.spark.sql.CarbonSession$CarbonBuilder.getOrCreateCarbonSession(CarbonSession.scala:258)
        at org.apache.carbondata.hiveexample.HiveExample$.main(HiveExample.scala:51)
        at org.apache.carbondata.hiveexample.HiveExample.main(HiveExample.scala)
{code}

Spark2.3:

{code:java}
2018-11-26 15:07:58 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.internal.config.package$.SHUFFLE_SPILL_NUM_ELEMENTS_FORCE_SPILL_THRESHOLD()Lorg/apache/spark/internal/config/ConfigEntry;
        at org.apache.spark.sql.internal.SQLConf$.<init>(SQLConf.scala:1011)
        at org.apache.spark.sql.internal.SQLConf$.<clinit>(SQLConf.scala)
        at org.apache.spark.sql.internal.StaticSQLConf$.<init>(StaticSQLConf.scala:31)
        at org.apache.spark.sql.internal.StaticSQLConf$.<clinit>(StaticSQLConf.scala)
        at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:867)
        at org.apache.carbondata.hiveexample.HiveExample$.main(HiveExample.scala:50)
        at org.apache.carbondata.hiveexample.HiveExample.main(HiveExample.scala)

{code}

Please test spark 2.2 too
We should fix it.

  was:
HiveExample has some exception:
org.apache.carbondata.hiveexample.HiveExample

Version:

  -Pspark.version=2.1.1

{code:java}
2018-11-24 10:22:53 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.internal.config.package$.SHUFFLE_SPILL_NUM_ELEMENTS_FORCE_SPILL_THRESHOLD()Lorg/apache/spark/internal/config/ConfigEntry;
        at org.apache.spark.sql.internal.SQLConf$.<init>(SQLConf.scala:1011)
        at org.apache.spark.sql.internal.SQLConf$.<clinit>(SQLConf.scala)
        at org.apache.spark.sql.internal.StaticSQLConf$.<init>(StaticSQLConf.scala:31)
        at org.apache.spark.sql.internal.StaticSQLConf$.<clinit>(StaticSQLConf.scala)
        at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:867)
        at org.apache.carbondata.hiveexample.HiveExample$.main(HiveExample.scala:50)
        at org.apache.carbondata.hiveexample.HiveExample.main(HiveExample.scala)
{code}

We should fix it.

> HiveExample has some exception
> ------------------------------
>
>                 Key: CARBONDATA-3128
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3128
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 1.5.1
>            Reporter: xubo245
>            Priority: Major
>
> HiveExample has some exception:
> org.apache.carbondata.hiveexample.HiveExample
> Version:
>   -Pspark.version=2.1.1
> {code:java}
> 2018-11-26 15:06:01 INFO  CarbonProperties:1512 - Considered value for min max byte limit for string is: 200
> Exception in thread "main" java.lang.ClassNotFoundException: org.apache.spark.sql.hive.CarbonSessionStateBuilder
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
> at org.apache.spark.util.CarbonReflectionUtils$.createObject(CarbonReflectionUtils.scala:322)
> at org.apache.spark.util.CarbonReflectionUtils$.getSessionState(CarbonReflectionUtils.scala:218)
> at org.apache.spark.sql.CarbonSession.sessionState$lzycompute(CarbonSession.scala:56)
> at org.apache.spark.sql.CarbonSession.sessionState(CarbonSession.scala:55)
> at org.apache.spark.sql.CarbonSession$CarbonBuilder$$anonfun$getOrCreateCarbonSession$2.apply(CarbonSession.scala:258)
> at org.apache.spark.sql.CarbonSession$CarbonBuilder$$anonfun$getOrCreateCarbonSession$2.apply(CarbonSession.scala:258)
> at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
> at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
> at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
> at org.apache.spark.sql.CarbonSession$CarbonBuilder.getOrCreateCarbonSession(CarbonSession.scala:258)
> at org.apache.carbondata.hiveexample.HiveExample$.main(HiveExample.scala:51)
> at org.apache.carbondata.hiveexample.HiveExample.main(HiveExample.scala)
> {code}
> Spark2.3:
> {code:java}
> 2018-11-26 15:07:58 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
> Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.internal.config.package$.SHUFFLE_SPILL_NUM_ELEMENTS_FORCE_SPILL_THRESHOLD()Lorg/apache/spark/internal/config/ConfigEntry;
> at org.apache.spark.sql.internal.SQLConf$.<init>(SQLConf.scala:1011)
> at org.apache.spark.sql.internal.SQLConf$.<clinit>(SQLConf.scala)
> at org.apache.spark.sql.internal.StaticSQLConf$.<init>(StaticSQLConf.scala:31)
> at org.apache.spark.sql.internal.StaticSQLConf$.<clinit>(StaticSQLConf.scala)
> at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:867)
> at org.apache.carbondata.hiveexample.HiveExample$.main(HiveExample.scala:50)
> at org.apache.carbondata.hiveexample.HiveExample.main(HiveExample.scala)
> {code}
> Please test spark 2.2 too
> We should fix it.

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)