Re:Re: Re:Re:Re:Re:Re:Re: insert into carbon table failed

Posted by a on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/insert-into-carbon-table-failed-tp9609p9763.html

Thank you very much!


I have divided 2 billions data into 4 pieces and loaded in the table 。


The three paramaters carbon.graph.rowset.size、 carbon.sort.size 、carbon.number.of.cores.while.loading may be also effect。


Best regards！

At 2017-03-27 13:53:58, "Liang Chen" <[hidden email]> wrote:

>Hi
>
>1.Use your current test environment (CarbonData 1.0 + Spark1.6), Please
>divide 2 billions data into 4 pieces(each is 0.5 billion), load data again.
>
>2.For CarbonData 1.0 + Spark1.6 with kettle for loading data, please
>configure the bellow 3 parameters in carbon.properties(note: please copy the
>latest carbon.properties to all nodes)
>
>carbon.graph.rowset.size=10000   (by default is 100000, please set to 1/10
>for reducing Rowset size exchanged between data load graph)
>carbon.number.of.cores.while.loading=5 (because your machine has 5 cores)
>carbon.sort.size=50000 ( by default is 500000, please set to 1/10 for
>reducing temp intermediate files)
>
>
>Regards
>Liang
>
>
>
>--
>View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/insert-into-carbon-table-failed-tp9609p9688.html
>Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.