Posted by
刘feng on
Sep 19, 2017; 3:35am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/insert-carbondata-table-failed-tp22756p22762.html
Sorry，
A total of 4 nodes . of which 3 as datanode and snn on one of the datanodes.
Version:
Carbondata 1.1.0
Spark 1.6.0
Hadoop :2.7.2
Thank you for your help , I'm trying again
=========================
Liu feng
-----邮件原件-----
发件人: ravipesala [mailto:
[hidden email]]
发送时间: 2017年9月19日 11:23
收件人:
[hidden email]
主题: Re: insert carbondata table failed
Hello,
I don't get much from the logs but the error seems related to memory issue
from Spark. From your old emails I get that you are using 3 node cluster. Is
that all 3 node has nodemanager and datanodes?
So better give only less number of executors and provide more memory to it
like below. While data loading it is recommended to use one executor per
nodemanager.
spark-shell --master yarn-client --driver-memory 10G --executor-cores 4
--num-executors 3 --executor-memory 25G
And also if any configuration gives any error please provide the executor
log.
Thank you,
Ravindra.
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------