Re: Size control of minot compaction

Posted by Ajantha Bhat on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Size-control-of-minor-compaction-tp103414p103464.html

Hi Zhangshunyu, Thanks for providing more details on the problem.

If it is just for skipping history segments during auto minor compaction,
Adding a size threshold for minor compaction should be fine.
We can have a table level, dynamically configurable threshold.
If it is not configured, consider all the segments for merging. If
configured, consider the segments within the threshold value.

Thanks,
Ajantha

On Mon, Nov 23, 2020 at 5:26 PM Zhangshunyu <[hidden email]> wrote:

> Yes, we need to support auto load merge for major compaction or size
> threshold limit for minor compaction.
> In many cases, the user use the minor compaction only want to merge small
> segments by time series (the num of segment is generated intime series),
> they dont want to merge big segment which is large enough.
>
>
>
> -----
> My English name is Sunday
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>