Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev
 I submit a spark application in mode yarn cluster to a cluster with kerberos. In this application, it will successfully query a hive table, but when it try to query a carbon table, it failed with infomation "Delegation Token can be issued only with kerberos or web authentication".  

If I submit this application in mode yarn client, both hive table and carbon table will both success.

And If I submit this application in mode yarn cluster on another cluster without kerberos, both hive table and carbon table will both success.


yixu2001
Reply | Threaded
Open this post in threaded view
|

Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

Naresh P R
Hi,

Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?

how are you passing the principal in yarn cluster ?

can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R

On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:

> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>
Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev 
 Please refer to the attachment "cluster carbon error2.txt" for the log trace.
In this log, I try 2 query statements:
select * from e_carbon.prod_inst_his       prod_inst_his is a hive table, it success.
select * from e_carbon.prod_inst_his_c     prod_inst_his_c is a carbon table, it failed.

I pass the principal in my start script, please refer to the attachment "testCluster.sh 

".

I have set hive.server2.enable.doAs = false in the above test and I have printed it in the log.

yixu2001
 
Date: 2017-11-01 19:40
Subject: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi,
 
Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?
 
how are you passing the principal in yarn cluster ?
 
can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R
 
On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
 
> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>
Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

Naresh P R
Hi yixu,

I am not able to see any attachment in your previous mail.
---
Regards,
Naresh P R

On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
dev 
 Please refer to the attachment "cluster carbon error2.txt" for the log trace.
In this log, I try 2 query statements:
select * from e_carbon.prod_inst_his       prod_inst_his is a hive table, it success.
select * from e_carbon.prod_inst_his_c     prod_inst_his_c is a carbon table, it failed.

I pass the principal in my start script, please refer to the attachment "testCluster.sh 

".

I have set hive.server2.enable.doAs = false in the above test and I have printed it in the log.

yixu2001
 
Date: 2017-11-01 19:40
Subject: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi,
 
Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?
 
how are you passing the principal in yarn cluster ?
 
can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R
 
On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
 
> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev 
 Our platform is installed with HDP 2.4, but spark 2.1 is not included in HDP 2.4, we using spark 2.1 with additional installed of apache version.

yixu2001
 
Date: 2017-11-02 22:02
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu,

I am not able to see any attachment in your previous mail.
---
Regards,
Naresh P R

On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
dev 
 Please refer to the attachment "cluster carbon error2.txt" for the log trace.
In this log, I try 2 query statements:
select * from e_carbon.prod_inst_his       prod_inst_his is a hive table, it success.
select * from e_carbon.prod_inst_his_c     prod_inst_his_c is a carbon table, it failed.

I pass the principal in my start script, please refer to the attachment "testCluster.sh 

".

I have set hive.server2.enable.doAs = false in the above test and I have printed it in the log.

yixu2001
 
Date: 2017-11-01 19:40
Subject: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi,
 
Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?
 
how are you passing the principal in yarn cluster ?
 
can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R
 
On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
 
> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
In reply to this post by Naresh P R
Naresh P R: 
 For the attachments can not be uploaded in maillist,I have add the attachments to the mail for you, please check it. 
 Our platform is installed with HDP 2.4, but spark 2.1 is not included in HDP 2.4, we using spark 2.1 with additional installed of apache version.

yixu2001
 
Date: 2017-11-02 22:02
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu,

I am not able to see any attachment in your previous mail.
---
Regards,
Naresh P R

On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
dev 
 Please refer to the attachment "cluster carbon error2.txt" for the log trace.
In this log, I try 2 query statements:
select * from e_carbon.prod_inst_his       prod_inst_his is a hive table, it success.
select * from e_carbon.prod_inst_his_c     prod_inst_his_c is a carbon table, it failed.

I pass the principal in my start script, please refer to the attachment "testCluster.sh 

".

I have set hive.server2.enable.doAs = false in the above test and I have printed it in the log.

yixu2001
 
Date: 2017-11-01 19:40
Subject: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi,
 
Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?
 
how are you passing the principal in yarn cluster ?
 
can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R
 
On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
 
> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>

Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
In reply to this post by Naresh P R
prnaresh.naresh, dev 
 
 The carbon jar I used does not include hadoop classes & core-site.xml.
The attachment include the jar list while submtting spark job, please confirm it. 

yixu2001
 
Date: 2017-11-03 16:07
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu2001,

Are you using carbon shaded jar with hadoop classes & core-site.xml included in carbon jar ?

If so, can you try to use carbondata individual component jars while submtting spark job?

As per my understanding, this happens if client core-site.xml has hadoop.security.authentication=simple & hdfs is kerberized.

You can also enable verbose to see hadoop jars used in the error trace while querying carbon tables.

Also i am not sure whether CarbonData is tested in HDP kerberos cluster.
---
Regards,
Naresh P R


On Fri, Nov 3, 2017 at 8:36 AM, yixu2001 <[hidden email]> wrote:
Naresh P R: 
 For the attachments can not be uploaded in maillist,I have add the attachments to the mail for you, please check it. 
 Our platform is installed with HDP 2.4, but spark 2.1 is not included in HDP 2.4, we using spark 2.1 with additional installed of apache version.

yixu2001
 
Date: 2017-11-02 22:02
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu,

I am not able to see any attachment in your previous mail.
---
Regards,
Naresh P R

On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
dev 
 Please refer to the attachment "cluster carbon error2.txt" for the log trace.
In this log, I try 2 query statements:
select * from e_carbon.prod_inst_his       prod_inst_his is a hive table, it success.
select * from e_carbon.prod_inst_his_c     prod_inst_his_c is a carbon table, it failed.

I pass the principal in my start script, please refer to the attachment "testCluster.sh 

".

I have set hive.server2.enable.doAs = false in the above test and I have printed it in the log.

yixu2001
 
Date: 2017-11-01 19:40
Subject: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi,
 
Ideally kerberos authentication should work with carbon table, Can you
share us log trace to analyze further more?
 
how are you passing the principal in yarn cluster ?
 
can you try to set hive.server2.enable.doAs = false & run query on carbon
table ?
----
Regards,
Naresh P R
 
On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
 
> dev
>  I submit a spark application in mode yarn cluster to a cluster with
> kerberos. In this application, it will successfully query a hive table, but
> when it try to query a carbon table, it failed with infomation "Delegation
> Token can be issued only with kerberos or web authentication".
>
> If I submit this application in mode yarn client, both hive table and
> carbon table will both success.
>
> And If I submit this application in mode yarn cluster on another cluster
> without kerberos, both hive table and carbon table will both success.
>
>
> yixu2001
>


Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

Naresh P R
Hi yixu2001,

From hadoop code, i could see IOException("Delegation Token can be issued
only with kerberos or web authentication") is thrown only if authentication
method is set as "SIMPLE".

private boolean isAllowedDelegationTokenOp() throws IOException {
    AuthenticationMethod authMethod = this.getConnectionAuthenticationMethod();
    return !UserGroupInformation.isSecurityEnabled() || authMethod ==
AuthenticationMethod.KERBEROS || authMethod ==
AuthenticationMethod.KERBEROS_SSL || authMethod ==
AuthenticationMethod.CERTIFICATE;
}

Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
throws IOException {
....
        if (!this.isAllowedDelegationTokenOp()) {
            throw new IOException("Delegation Token can be issued only
with kerberos or web authentication");
        }

....

}

Can you try to execute queries by copying core-site.xml from hadoop-conf
folder to spark2 conf folder & classpath of spark-submit?

From the provided logs, i could
see carbondata_2.11-1.1.1-bdd-hadoop2.7.2.jar size is 9607344bytes, please
make sure it has only carbondata classes.

I could see Carbon explicitly calling
"TokenCache.obtainTokensForNamenodes" which
is throwing this exception.

If above mentioned steps dint work, you can raise a JIRA to investigate
further on this.
----
Regards,
Naresh P R

On Fri, Nov 3, 2017 at 3:10 PM, yixu2001 <[hidden email]> wrote:

> prnaresh.naresh, dev
>
>  The carbon jar I used does not include hadoop classes & core-site.xml.
> The attachment include the jar list while submtting
> spark job, please confirm it.
> ------------------------------
> yixu2001
>
>
> *From:* Naresh P R <[hidden email]>
> *Date:* 2017-11-03 16:07
> *To:* yixu2001 <[hidden email]>
> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
> web authentication" will occur in yarn cluster
> Hi yixu2001,
>
> Are you using carbon shaded jar with hadoop classes & core-site.xml
> included in carbon jar ?
>
> If so, can you try to use carbondata individual component jars while
> submtting spark job?
>
> As per my understanding, this happens if client core-site.xml has
> hadoop.security.authentication=simple & hdfs is kerberized.
>
> You can also enable verbose to see hadoop jars used in the error trace
> while querying carbon tables.
>
> Also i am not sure whether CarbonData is tested in HDP kerberos cluster.
> ---
> Regards,
> Naresh P R
>
>
> On Fri, Nov 3, 2017 at 8:36 AM, yixu2001 <[hidden email]> wrote:
>
>> Naresh P R:
>>  For the attachments can not be uploaded in maillist,I have
>> add the attachments to the mail for you, please check it.
>>  Our platform is installed with HDP 2.4, but spark 2.1 is
>> not included in HDP 2.4, we using spark 2.1 with additiona
>> l installed of apache version.
>> ------------------------------
>> yixu2001
>>
>>
>> *From:* Naresh P R <[hidden email]>
>> *Date:* 2017-11-02 22:02
>> *To:* dev <[hidden email]>
>> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
>> web authentication" will occur in yarn cluster
>> Hi yixu,
>>
>> I am not able to see any attachment in your previous mail.
>> ---
>> Regards,
>> Naresh P R
>>
>> On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
>>
>>> dev
>>>  Please refer to the attachment "cluster carbon error2.txt"
>>> for the log trace.
>>> In this log, I try 2 query statements:
>>> select * from e_carbon.prod_inst_his       prod_inst_his is
>>> a hive table, it success.
>>> select * from e_carbon.prod_inst_his_c     prod_inst_his_c i
>>> s a carbon table, it failed.
>>>
>>> I pass the principal in my start script, please refer to the
>>>  attachment "testCluster.sh
>>>
>>> ".
>>>
>>> I have set hive.server2.enable.doAs = false in the above tes
>>> t and I have printed it in the log.
>>> ------------------------------
>>> yixu2001
>>>
>>>
>>> *From:* Naresh P R <[hidden email]>
>>> *Date:* 2017-11-01 19:40
>>> *To:* dev <[hidden email]>
>>> *Subject:* Re: Delegation Token can be issued only with kerberos or web
>>> authentication" will occur in yarn cluster
>>> Hi,
>>>
>>> Ideally kerberos authentication should work with carbon table, Can you
>>> share us log trace to analyze further more?
>>>
>>> how are you passing the principal in yarn cluster ?
>>>
>>> can you try to set hive.server2.enable.doAs = false & run query on carbon
>>> table ?
>>> ----
>>> Regards,
>>> Naresh P R
>>>
>>> On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
>>>
>>> > dev
>>> >  I submit a spark application in mode yarn cluster to a cluster with
>>> > kerberos. In this application, it will successfully query a hive
>>> table, but
>>> > when it try to query a carbon table, it failed with infomation
>>> "Delegation
>>> > Token can be issued only with kerberos or web authentication".
>>> >
>>> > If I submit this application in mode yarn client, both hive table and
>>> > carbon table will both success.
>>> >
>>> > And If I submit this application in mode yarn cluster on another
>>> cluster
>>> > without kerberos, both hive table and carbon table will both success.
>>> >
>>> >
>>> > yixu2001
>>> >
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev
 I have copied core-site.xml to spark2 conf folder, it does not work.
I will share you the jar I am using(please get the jar file from link  

download link :https://pan.baidu.com/s/1b6AG5S    password:rww2
 whould you help me to confirm whether the jar is suitable?
If the jar is not suitable, could you please send me a suitable one to try?


yixu2001
 
From: Naresh P R
Date: 2017-11-04 01:26
To: dev
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu2001,
 
From hadoop code, i could see IOException("Delegation Token can be issued
only with kerberos or web authentication") is thrown only if authentication
method is set as "SIMPLE".
 
private boolean isAllowedDelegationTokenOp() throws IOException {
    AuthenticationMethod authMethod = this.getConnectionAuthenticationMethod();
    return !UserGroupInformation.isSecurityEnabled() || authMethod ==
AuthenticationMethod.KERBEROS || authMethod ==
AuthenticationMethod.KERBEROS_SSL || authMethod ==
AuthenticationMethod.CERTIFICATE;
}
 
Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
throws IOException {
....
        if (!this.isAllowedDelegationTokenOp()) {
            throw new IOException("Delegation Token can be issued only
with kerberos or web authentication");
        }
 
....
 
}
 
Can you try to execute queries by copying core-site.xml from hadoop-conf
folder to spark2 conf folder & classpath of spark-submit?
 
From the provided logs, i could
see carbondata_2.11-1.1.1-bdd-hadoop2.7.2.jar size is 9607344bytes, please
make sure it has only carbondata classes.
 
I could see Carbon explicitly calling
"TokenCache.obtainTokensForNamenodes" which
is throwing this exception.
 
If above mentioned steps dint work, you can raise a JIRA to investigate
further on this.
----
Regards,
Naresh P R
 
On Fri, Nov 3, 2017 at 3:10 PM, yixu2001 <[hidden email]> wrote:
 

> prnaresh.naresh, dev
>
>  The carbon jar I used does not include hadoop classes & core-site.xml.
> The attachment include the jar list while submtting
> spark job, please confirm it.
> ------------------------------
> yixu2001
>
>
> *From:* Naresh P R <[hidden email]>
> *Date:* 2017-11-03 16:07
> *To:* yixu2001 <[hidden email]>
> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
> web authentication" will occur in yarn cluster
> Hi yixu2001,
>
> Are you using carbon shaded jar with hadoop classes & core-site.xml
> included in carbon jar ?
>
> If so, can you try to use carbondata individual component jars while
> submtting spark job?
>
> As per my understanding, this happens if client core-site.xml has
> hadoop.security.authentication=simple & hdfs is kerberized.
>
> You can also enable verbose to see hadoop jars used in the error trace
> while querying carbon tables.
>
> Also i am not sure whether CarbonData is tested in HDP kerberos cluster.
> ---
> Regards,
> Naresh P R
>
>
> On Fri, Nov 3, 2017 at 8:36 AM, yixu2001 <[hidden email]> wrote:
>
>> Naresh P R:
>>  For the attachments can not be uploaded in maillist,I have
>> add the attachments to the mail for you, please check it.
>>  Our platform is installed with HDP 2.4, but spark 2.1 is
>> not included in HDP 2.4, we using spark 2.1 with additiona
>> l installed of apache version.
>> ------------------------------
>> yixu2001
>>
>>
>> *From:* Naresh P R <[hidden email]>
>> *Date:* 2017-11-02 22:02
>> *To:* dev <[hidden email]>
>> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
>> web authentication" will occur in yarn cluster
>> Hi yixu,
>>
>> I am not able to see any attachment in your previous mail.
>> ---
>> Regards,
>> Naresh P R
>>
>> On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
>>
>>> dev
>>>  Please refer to the attachment "cluster carbon error2.txt"
>>> for the log trace.
>>> In this log, I try 2 query statements:
>>> select * from e_carbon.prod_inst_his       prod_inst_his is
>>> a hive table, it success.
>>> select * from e_carbon.prod_inst_his_c     prod_inst_his_c i
>>> s a carbon table, it failed.
>>>
>>> I pass the principal in my start script, please refer to the
>>>  attachment "testCluster.sh
>>>
>>> ".
>>>
>>> I have set hive.server2.enable.doAs = false in the above tes
>>> t and I have printed it in the log.
>>> ------------------------------
>>> yixu2001
>>>
>>>
>>> *From:* Naresh P R <[hidden email]>
>>> *Date:* 2017-11-01 19:40
>>> *To:* dev <[hidden email]>
>>> *Subject:* Re: Delegation Token can be issued only with kerberos or web
>>> authentication" will occur in yarn cluster
>>> Hi,
>>>
>>> Ideally kerberos authentication should work with carbon table, Can you
>>> share us log trace to analyze further more?
>>>
>>> how are you passing the principal in yarn cluster ?
>>>
>>> can you try to set hive.server2.enable.doAs = false & run query on carbon
>>> table ?
>>> ----
>>> Regards,
>>> Naresh P R
>>>
>>> On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
>>>
>>> > dev
>>> >  I submit a spark application in mode yarn cluster to a cluster with
>>> > kerberos. In this application, it will successfully query a hive
>>> table, but
>>> > when it try to query a carbon table, it failed with infomation
>>> "Delegation
>>> > Token can be issued only with kerberos or web authentication".
>>> >
>>> > If I submit this application in mode yarn client, both hive table and
>>> > carbon table will both success.
>>> >
>>> > And If I submit this application in mode yarn cluster on another
>>> cluster
>>> > without kerberos, both hive table and carbon table will both success.
>>> >
>>> >
>>> > yixu2001
>>> >
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev
 hello there is no recent progress


yixu2001
 
From: yixu2001
Date: 2017-11-06 16:43
To: dev
CC: prnaresh.naresh; 郭海涛
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
dev
 I have copied core-site.xml to spark2 conf folder, it does not work.
I will share you the jar I am using(please get the jar file from link  

download link :https://pan.baidu.com/s/1b6AG5S    password:rww2
 whould you help me to confirm whether the jar is suitable?
If the jar is not suitable, could you please send me a suitable one to try?


yixu2001
 
From: Naresh P R
Date: 2017-11-04 01:26
To: dev
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu2001,
 
From hadoop code, i could see IOException("Delegation Token can be issued
only with kerberos or web authentication") is thrown only if authentication
method is set as "SIMPLE".
 
private boolean isAllowedDelegationTokenOp() throws IOException {
    AuthenticationMethod authMethod = this.getConnectionAuthenticationMethod();
    return !UserGroupInformation.isSecurityEnabled() || authMethod ==
AuthenticationMethod.KERBEROS || authMethod ==
AuthenticationMethod.KERBEROS_SSL || authMethod ==
AuthenticationMethod.CERTIFICATE;
}
 
Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
throws IOException {
....
        if (!this.isAllowedDelegationTokenOp()) {
            throw new IOException("Delegation Token can be issued only
with kerberos or web authentication");
        }
 
....
 
}
 
Can you try to execute queries by copying core-site.xml from hadoop-conf
folder to spark2 conf folder & classpath of spark-submit?
 
From the provided logs, i could
see carbondata_2.11-1.1.1-bdd-hadoop2.7.2.jar size is 9607344bytes, please
make sure it has only carbondata classes.
 
I could see Carbon explicitly calling
"TokenCache.obtainTokensForNamenodes" which
is throwing this exception.
 
If above mentioned steps dint work, you can raise a JIRA to investigate
further on this.
----
Regards,
Naresh P R
 
On Fri, Nov 3, 2017 at 3:10 PM, yixu2001 <[hidden email]> wrote:
 

> prnaresh.naresh, dev
>
>  The carbon jar I used does not include hadoop classes & core-site.xml.
> The attachment include the jar list while submtting
> spark job, please confirm it.
> ------------------------------
> yixu2001
>
>
> *From:* Naresh P R <[hidden email]>
> *Date:* 2017-11-03 16:07
> *To:* yixu2001 <[hidden email]>
> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
> web authentication" will occur in yarn cluster
> Hi yixu2001,
>
> Are you using carbon shaded jar with hadoop classes & core-site.xml
> included in carbon jar ?
>
> If so, can you try to use carbondata individual component jars while
> submtting spark job?
>
> As per my understanding, this happens if client core-site.xml has
> hadoop.security.authentication=simple & hdfs is kerberized.
>
> You can also enable verbose to see hadoop jars used in the error trace
> while querying carbon tables.
>
> Also i am not sure whether CarbonData is tested in HDP kerberos cluster.
> ---
> Regards,
> Naresh P R
>
>
> On Fri, Nov 3, 2017 at 8:36 AM, yixu2001 <[hidden email]> wrote:
>
>> Naresh P R:
>>  For the attachments can not be uploaded in maillist,I have
>> add the attachments to the mail for you, please check it.
>>  Our platform is installed with HDP 2.4, but spark 2.1 is
>> not included in HDP 2.4, we using spark 2.1 with additiona
>> l installed of apache version.
>> ------------------------------
>> yixu2001
>>
>>
>> *From:* Naresh P R <[hidden email]>
>> *Date:* 2017-11-02 22:02
>> *To:* dev <[hidden email]>
>> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
>> web authentication" will occur in yarn cluster
>> Hi yixu,
>>
>> I am not able to see any attachment in your previous mail.
>> ---
>> Regards,
>> Naresh P R
>>
>> On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
>>
>>> dev
>>>  Please refer to the attachment "cluster carbon error2.txt"
>>> for the log trace.
>>> In this log, I try 2 query statements:
>>> select * from e_carbon.prod_inst_his       prod_inst_his is
>>> a hive table, it success.
>>> select * from e_carbon.prod_inst_his_c     prod_inst_his_c i
>>> s a carbon table, it failed.
>>>
>>> I pass the principal in my start script, please refer to the
>>>  attachment "testCluster.sh
>>>
>>> ".
>>>
>>> I have set hive.server2.enable.doAs = false in the above tes
>>> t and I have printed it in the log.
>>> ------------------------------
>>> yixu2001
>>>
>>>
>>> *From:* Naresh P R <[hidden email]>
>>> *Date:* 2017-11-01 19:40
>>> *To:* dev <[hidden email]>
>>> *Subject:* Re: Delegation Token can be issued only with kerberos or web
>>> authentication" will occur in yarn cluster
>>> Hi,
>>>
>>> Ideally kerberos authentication should work with carbon table, Can you
>>> share us log trace to analyze further more?
>>>
>>> how are you passing the principal in yarn cluster ?
>>>
>>> can you try to set hive.server2.enable.doAs = false & run query on carbon
>>> table ?
>>> ----
>>> Regards,
>>> Naresh P R
>>>
>>> On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
>>>
>>> > dev
>>> >  I submit a spark application in mode yarn cluster to a cluster with
>>> > kerberos. In this application, it will successfully query a hive
>>> table, but
>>> > when it try to query a carbon table, it failed with infomation
>>> "Delegation
>>> > Token can be issued only with kerberos or web authentication".
>>> >
>>> > If I submit this application in mode yarn client, both hive table and
>>> > carbon table will both success.
>>> >
>>> > And If I submit this application in mode yarn cluster on another
>>> cluster
>>> > without kerberos, both hive table and carbon table will both success.
>>> >
>>> >
>>> > yixu2001
>>> >
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster

yixu2001
dev
 What time is expected to release a patch, we can test


yixu2001
 
From: yixu2001
Date: 2017-11-07 15:49
To: dev
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
dev
 hello there is no recent progress


yixu2001
 
From: yixu2001
Date: 2017-11-06 16:43
To: dev
CC: prnaresh.naresh; 郭海涛
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
dev
 I have copied core-site.xml to spark2 conf folder, it does not work.
I will share you the jar I am using(please get the jar file from link  

download link :https://pan.baidu.com/s/1b6AG5S    password:rww2
 whould you help me to confirm whether the jar is suitable?
If the jar is not suitable, could you please send me a suitable one to try?


yixu2001
 
From: Naresh P R
Date: 2017-11-04 01:26
To: dev
Subject: Re: Re: Delegation Token can be issued only with kerberos or web authentication" will occur in yarn cluster
Hi yixu2001,
 
From hadoop code, i could see IOException("Delegation Token can be issued
only with kerberos or web authentication") is thrown only if authentication
method is set as "SIMPLE".
 
private boolean isAllowedDelegationTokenOp() throws IOException {
    AuthenticationMethod authMethod = this.getConnectionAuthenticationMethod();
    return !UserGroupInformation.isSecurityEnabled() || authMethod ==
AuthenticationMethod.KERBEROS || authMethod ==
AuthenticationMethod.KERBEROS_SSL || authMethod ==
AuthenticationMethod.CERTIFICATE;
}
 
Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
throws IOException {
....
        if (!this.isAllowedDelegationTokenOp()) {
            throw new IOException("Delegation Token can be issued only
with kerberos or web authentication");
        }
 
....
 
}
 
Can you try to execute queries by copying core-site.xml from hadoop-conf
folder to spark2 conf folder & classpath of spark-submit?
 
From the provided logs, i could
see carbondata_2.11-1.1.1-bdd-hadoop2.7.2.jar size is 9607344bytes, please
make sure it has only carbondata classes.
 
I could see Carbon explicitly calling
"TokenCache.obtainTokensForNamenodes" which
is throwing this exception.
 
If above mentioned steps dint work, you can raise a JIRA to investigate
further on this.
----
Regards,
Naresh P R
 
On Fri, Nov 3, 2017 at 3:10 PM, yixu2001 <[hidden email]> wrote:
 

> prnaresh.naresh, dev
>
>  The carbon jar I used does not include hadoop classes & core-site.xml.
> The attachment include the jar list while submtting
> spark job, please confirm it.
> ------------------------------
> yixu2001
>
>
> *From:* Naresh P R <[hidden email]>
> *Date:* 2017-11-03 16:07
> *To:* yixu2001 <[hidden email]>
> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
> web authentication" will occur in yarn cluster
> Hi yixu2001,
>
> Are you using carbon shaded jar with hadoop classes & core-site.xml
> included in carbon jar ?
>
> If so, can you try to use carbondata individual component jars while
> submtting spark job?
>
> As per my understanding, this happens if client core-site.xml has
> hadoop.security.authentication=simple & hdfs is kerberized.
>
> You can also enable verbose to see hadoop jars used in the error trace
> while querying carbon tables.
>
> Also i am not sure whether CarbonData is tested in HDP kerberos cluster.
> ---
> Regards,
> Naresh P R
>
>
> On Fri, Nov 3, 2017 at 8:36 AM, yixu2001 <[hidden email]> wrote:
>
>> Naresh P R:
>>  For the attachments can not be uploaded in maillist,I have
>> add the attachments to the mail for you, please check it.
>>  Our platform is installed with HDP 2.4, but spark 2.1 is
>> not included in HDP 2.4, we using spark 2.1 with additiona
>> l installed of apache version.
>> ------------------------------
>> yixu2001
>>
>>
>> *From:* Naresh P R <[hidden email]>
>> *Date:* 2017-11-02 22:02
>> *To:* dev <[hidden email]>
>> *Subject:* Re: Re: Delegation Token can be issued only with kerberos or
>> web authentication" will occur in yarn cluster
>> Hi yixu,
>>
>> I am not able to see any attachment in your previous mail.
>> ---
>> Regards,
>> Naresh P R
>>
>> On Thu, Nov 2, 2017 at 4:40 PM, yixu2001 <[hidden email]> wrote:
>>
>>> dev
>>>  Please refer to the attachment "cluster carbon error2.txt"
>>> for the log trace.
>>> In this log, I try 2 query statements:
>>> select * from e_carbon.prod_inst_his       prod_inst_his is
>>> a hive table, it success.
>>> select * from e_carbon.prod_inst_his_c     prod_inst_his_c i
>>> s a carbon table, it failed.
>>>
>>> I pass the principal in my start script, please refer to the
>>>  attachment "testCluster.sh
>>>
>>> ".
>>>
>>> I have set hive.server2.enable.doAs = false in the above tes
>>> t and I have printed it in the log.
>>> ------------------------------
>>> yixu2001
>>>
>>>
>>> *From:* Naresh P R <[hidden email]>
>>> *Date:* 2017-11-01 19:40
>>> *To:* dev <[hidden email]>
>>> *Subject:* Re: Delegation Token can be issued only with kerberos or web
>>> authentication" will occur in yarn cluster
>>> Hi,
>>>
>>> Ideally kerberos authentication should work with carbon table, Can you
>>> share us log trace to analyze further more?
>>>
>>> how are you passing the principal in yarn cluster ?
>>>
>>> can you try to set hive.server2.enable.doAs = false & run query on carbon
>>> table ?
>>> ----
>>> Regards,
>>> Naresh P R
>>>
>>> On Wed, Nov 1, 2017 at 3:33 PM, yixu2001 <[hidden email]> wrote:
>>>
>>> > dev
>>> >  I submit a spark application in mode yarn cluster to a cluster with
>>> > kerberos. In this application, it will successfully query a hive
>>> table, but
>>> > when it try to query a carbon table, it failed with infomation
>>> "Delegation
>>> > Token can be issued only with kerberos or web authentication".
>>> >
>>> > If I submit this application in mode yarn client, both hive table and
>>> > carbon table will both success.
>>> >
>>> > And If I submit this application in mode yarn cluster on another
>>> cluster
>>> > without kerberos, both hive table and carbon table will both success.
>>> >
>>> >
>>> > yixu2001
>>> >
>>>
>>>
>>
>