Posted by
Yahui Liu on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Display-the-segment-ID-when-carbondata-load-is-successful-tp105403p106312.html
Hi,
I think after load, only return the segment id which data is loaded to is
enough no matter auto load merge is enable or not. I will add one more
reason apart from @areyouokfreejoe metioned:
1. Because user alredy cares about each load, so mostly in their application
logic, auto load merge is disabled, user will hanlde compaction by
themselves. Auto load merge only base on segment no., not base on any
business relation between the segments. So if they enable auto load merge,
several segments which has no any relation just the segment_id is close will
be compacted. After this kind of compaction, all the information in the
segment before compaction will be lost, this is not what user wants. If any
load is special, in order to not lost any information after compaction, this
load should only merge with the segment which has the same special point
which is only known by the application, carbon currently has no place to
store this information. So only user can control which segments will be
compacted by trigger custom compaction with the segment ids which those
segments have the same special point.
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/