Posted by
dhatchayani on
Feb 15, 2019; 4:53pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-Support-Incremental-load-in-datamap-and-other-MV-datamap-enhancement-tp75160p75168.html
Hi Akash,
It is good to have this feature and expecting to get good understanding of
the design and solution from the design document.
Please clarify the below points.
(1) How are we planning support lazy loads? Is there any one-to-one mapping
between the segments of the main table and the datamaps?
(2) In case if one-to-one mapping to be maintained, lets take pre-aggregate
datamaps, even though 'n' number of segments are loaded to the main table,
after 'n' loads if the user is trying to create a pre-aggregate datamap,
current pre-aggregate implementation creates only one segment, here the
one-to-one mapping is broken. Old store is in this way. What is the plan to
handle the old store?
(3) If the datamap is not updated/inline with the main table loads, the
system will fall back to the main table pruning? If so, is this a time
consuming process to check for the status of the datamaps for all the
queries, if needed to hit datamaps?
Hope all these points will be covered in the design document.
Thanks,
Dhatchayani
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/