Re: Concurrent data loading issues

Posted by xuchuanyin on Nov 12, 2019; 12:23pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Concurrent-data-loading-issues-tp86108p86196.html

Hi, concurrent load will not cause the problem and I've tried that months
ago.

Seen from the log, it seems that the problem lies in Compation that
automatically triggered after loading.

To solve the problem, I think you can：
1. firstly turn off auto-compaction to increase loading performance, and you
can fire concurrent loading here.
2. trigger compaction at some period for the performance of future query. Be
sure not to fire multiple compaction concurrently which I'm not sure whether
it has problems or not. But I think you can fire compaction along with the
loading （which is not tested by me yet.）



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/