Posted by
manishgupta88 on
Dec 04, 2016; 5:01pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/B-Tree-LRU-cache-New-Feature-tp2366p3640.html
Hi Sujith,
I agree with your point. We can always send a list of invalid segments to
the executors in the query model that needs to be cleared from the cache.
But there are few cases where clearing B-tree cache cannot be ensured like:
1. Table is dropped
2. Execution of clean table DML command.
In these cases we cannot ensure that invalid objects from cache are cleared
from all the executors. Removal only from driver can be ensured.
To handle these cases each executor should have a mechanism to decide for
the invalid segments/block/dictionary cache.
Regards
Manish Gupta
On Sun, Dec 4, 2016 at 10:14 PM, sujith chacko <
[hidden email]>
wrote:
> Hi Shahid,
>
> its a well explained document, just need few clarifications,
>
> a) once compaction is done the segments and its blocks will be invalidated,
> LRU's scope is to evict the unused objects from memory or least recently
> used objects from memory, but after compaction the segment itself becomes
> invalid,So is it really require to hold such objects in LRU cache and wait
> for eviction till its memory size gets full?
>
> Thanks,
> Sujith
>
> On Wed, Nov 23, 2016 at 6:30 PM, mohdshahidkhan <
>
[hidden email]> wrote:
>
> > Please find Design document for B-Tree LRU cache
> >
https://drive.google.com/file/d/0B8sQb--59vO7bWxVeWs1ajBiMG8/view?usp=> > sharing
> >
> >
> >
> > --
> > View this message in context:
http://apache-carbondata-> > mailing-list-archive.1130556.n5.nabble.com/B-Tree-LRU-
> > cache-New-Feature-tp2366p3130.html
> > Sent from the Apache CarbonData Mailing List archive mailing list archive
> > at Nabble.com.
> >
>