Re: user yarn needs the hdfs access when loading data?

Posted by lionel061201 on Dec 29, 2016; 3:56am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/user-yarn-needs-the-hdfs-access-when-loading-data-tp5082p5220.html

Hi,
You can refer to this ticket, I met the same issue :
https://issues.apache.org/jira/browse/CARBONDATA-559
check your kettle home settings in carbon.properties on executor side.

Thanks,
Lionel

On Wed, Dec 28, 2016 at 10:50 AM, Li Peng <[hidden email]> wrote:

> Hi,
>    1. I create carbon table in spark shell with user 'spark', the hdfs
> access of table store location  in hdfs is 755.
>        Run a spark streaming application in yarn-cluster with user 'spark'.
>        The application store dataframe to carbon table,  and
> 'carbon.ddl.base.hdfs.url' in carbon.properties is '/user/spark'.
>
>
>    2. use carbondata 0.2.0 release.
>
>
>    Why user 'yarn' must write access in  loading data?  I must use 'yarn'
> to
> create table and submit app now.
>
>
> Thanks.
>
>
>
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/user-yarn-needs-
> the-hdfs-access-when-loading-data-tp5082p5144.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>