Posted by
akashrn5 on
Apr 01, 2019; 6:32am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Add-new-compaction-type-for-compacting-delta-data-file-tp76597p76633.html
hi,
Thanks for clearing the doubt.
So according to my understanding, bascially you want to merge all the delete
delta files and base carbondtaa files and write a new segment. basically
this helps to reduce IO right?
So here i have some questions regarding that
1. are you planning for a new DDL for this operation? if you are, then DDL
structure?
2. how about the concurrency will handled with this? like update and delete,
compaction to table when this compaction is progress? if concurrent
operations are blocked well and good, else how the segment mapping wil be
maintained?
3. As jacky said, i agee with him, this will be costly operation as you
will be writing the whole segment again and time consuming, how this will be
handled so that user wont be blocked for query or other operation. or is it
recommended to do this operation in offpeak hours?
I suggest, can you please add the design document and create a JIRA for
this, it would be helpful.
Thanks.
Regards,
Akash R
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/