Data Load performance degrade when number of segment increase

Posted by Chin Wei on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Data-Load-performance-degrade-when-number-of-segment-increase-tp86031.html

Hi Community,

I notice that when the number of segments increased, the time taken to load
data increase as well.
After checking, whenever we load 1 csv file, it call readLoadMetadata 9
times. For a table with 10,000 segments each readLoadMetadata call took
50ms.

Is there any plan to improve this or any area that I can look at to improve
it.

Regards,
Chin Wei



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/