Re: In every query, carbondata has to scan all the segment file. This may takes too much time.

Posted by areyouokfreejoe on Jul 14, 2021; 9:56am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/In-every-query-carbondata-has-to-scan-all-the-segment-file-This-may-takes-too-much-time-tp108997p108999.html

I think there can be a file named LAST_MODIFY.
It contains the last update time of the segment file.
When carbon try to refresh the segment cache, if found that the update time in LAST_MODIFY time is the same with the cache, then there is no need to refresh all segment file.