Posted by
Liang Chen on
Oct 26, 2017; 10:32am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-support-user-specified-segments-in-major-compation-tp24540p24908.html
Hi Jin Zhou
Thanks for starting this discussion.
1. For your first proposal : Currently , segment is the system internal
concept, not expose to outside.
Can you provide what exact problems do you encounter? we can find the
alternative solution for your problems.
----------------------------------------------------------------------------------------
1) we can precisely control which part of table to be merged when table is
very large.
2. For your second proposal, my comment is +1, agree. can you please create
an apache jira for this ?
We would like to invite you to participate in implementing this feature
together :)
-----------------------------------------------------------------------------------------
2) each table can has its own compaction strategy which controlled by user
app.
Regards
Liang
Jin Zhou wrote
> Hi community,
> Carbondata currently support two types of compaction: Minor and Major
> compaction.
> CarbonData will do major compaction according to the user defined segment
> size. But which segments to be merged are transparent to users.
> We plan to extend major compaction to support user specified segments,
> this
> will be useful in cases below:
> 1) we can precisely control which part of table to be merged when table is
> very large.
> 2) each table can has its own compaction strategy which controlled by user
> app.
>
> the proposed syntax:
> ALTER TABLE [db_name].table_name COMPACT [SEGMENT seg_id1,seg_id2] 'MAJOR'
> in which [SEGMENT seg_id1,seg_id2] is optional and compatible with
> original
> syntax.
>
>
>
> --
> Sent from:
>
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/