[Discussion] Refactor Segment Management Interface.
Posted by
ravipesala on
Aug 04, 2018; 4:14am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-Refactor-Segment-Management-Interface-tp58926.html
Hi,
*Carbon uses tablestatus file to record segment status and details of each
segment during each load. This tablestatus enables carbon to support
concurrent loads and reads without data inconsistency or corruption.So it
is a very important feature of carbondata and we should have clean
interfaces to maintain it. Current tablestatus updation is shattered to
multiple places and there is no clean interface, so I am proposing to
refactor the current SegmentStatusManager interface and bringing all
tablestatus operations to a single interface. This new interface allows
adding table status to any other storage like DB. This is needed for S3
type object stores as these are eventually consistent. *
Please check the attached design in the jira
https://issues.apache.org/jira/browse/CARBONDATA-2827Please share your ideas on it.
--
Thanks & Regards,
Ravi