Posted by
冉明轩 on
Jul 11, 2018; 5:31am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Got-kerberos-login-error-while-starting-CarbonData-Thrift-Server-tp55387.html
Hello all,
I am new to carbondata and just about to start a CarbonData Thrift Server to have a try on its through put.
Environment:
Cluster: Cloudera CDH5.5.3 with kerberos installed as principle + keytab style
Spark: 2.2.1
Carbondata: apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar
Process:
1. build carbondata by following
https://github.com/apache/carbondata/blob/master/build/README.md2. Install and configure carbondata on Spark on Yarn Cluster by following
http://carbondata.apache.org/installation-guide.html3. create file /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh:
#!/bin/bash
/home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/spark-submit \
--class org.apache.carbondata.spark.thriftserver.CarbonThriftServer \
--principal mingxuan/
[hidden email] \
--keytab /home/mingxuan/mingxuan.keytab \
--queue root.mingxuan \
--master yarn \
--deploy-mode client \
--driver-memory 8g \
--executor-cores 4 \
--executor-memory 10g \
--num-executors 10 \
/home/mingxuan/spark-2.2.1-bin-hadoop2.7/carbonlib/apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar
hdfs://nsha/user/hive/warehouse/ranmx.db/carbondata
4. cd to spark dir and run command: sh /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh
Error Log:
18/07/10 16:08:33 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is /user/hive/warehouse
Exception in thread "main" org.apache.hive.service.ServiceException: Unable to login to kerberos with given principal/keytab
at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:58)
at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79)
at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.initCompositeService(HiveThriftServer2.scala:272)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.init(HiveThriftServer2.scala:292)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64)
at org.apache.carbondata.spark.thriftserver.CarbonThriftServer$.main(CarbonThriftServer.scala:93)
at org.apache.carbondata.spark.thriftserver.CarbonThriftServer.main(CarbonThriftServer.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.io.IOException: HiveServer2 Kerberos principal or keytab is not correctly configured
at org.apache.hive.service.auth.HiveAuthFactory.loginFromKeytab(HiveAuthFactory.java:197)
at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:53)
... 21 more
Does anybody has any idea about this exception? Does hadoop 2.6 matters?
Ran Mingxuan