Login  Register

[DISCUSSION] Parallel compaction and update

Posted by Nihal on Sep 14, 2020; 2:31pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Parallel-compaction-and-update-tp100338.html

Dear community,

This mail is regarding the parallel compaction and update.
Current behavior: Currently we are not supporting concurrent compaction and
update because It may cause data inconsistency or incorrect result.
We take the compaction and update lock before any of these operations.
Because of this behavior if one is executing then others have to wait and
sometimes this waiting time is very long.

To come out with this problem we are planning to support parallel compaction
and update. And here I have proposed one of the solutions to implement this
feature.
Paraller_Compaction_And_Update.pdf
<http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t443/Paraller_Compaction_And_Update.pdf>  
Please go through this solution document and provide your input if this
approach is ok or any drawback is there.


Thanks & Regards
Nihal kumar ojha



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/