Login  Register

Re: B-Tree LRU cache (New Feature)

Posted by Venkata Gollamudi on Dec 09, 2016; 3:32am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/B-Tree-LRU-cache-New-Feature-tp2366p3971.html

Hi Shahid,
Introduce CacheClient who is the owner for proper increment and decrement
of access count, if objects being used and not used. Other wise access
count handling becomes complicated as we add more features to system.
Regards,
Ramana

On Sun, Dec 4, 2016, 10:31 PM manish gupta <[hidden email]>
wrote:

> Hi Sujith,
>
> I agree with your point. We can always send a list of invalid segments to
> the executors in the query model that needs to be cleared from the cache.
> But there are few cases where clearing B-tree cache cannot be ensured like:
> 1. Table is dropped
> 2. Execution of clean table DML command.
>
> In these cases we cannot ensure that invalid objects from cache are cleared
> from all the executors. Removal only from driver can be ensured.
> To handle these cases each executor should have a mechanism to decide for
> the invalid segments/block/dictionary cache.
>
> Regards
> Manish Gupta
>
> On Sun, Dec 4, 2016 at 10:14 PM, sujith chacko <
> [hidden email]>
> wrote:
>
> > Hi Shahid,
> >
> >    its a well explained document, just need few clarifications,
> >
> > a) once compaction is done the segments and its blocks will be
> invalidated,
> > LRU's scope is to evict the unused objects from memory or  least recently
> > used objects from memory, but after compaction the segment itself becomes
> > invalid,So is it really require to hold such objects in LRU cache and
> wait
> > for eviction  till its memory size gets full?
> >
> > Thanks,
> > Sujith
> >
> > On Wed, Nov 23, 2016 at 6:30 PM, mohdshahidkhan <
> > [hidden email]> wrote:
> >
> > > Please find Design document for B-Tree LRU cache
> > > https://drive.google.com/file/d/0B8sQb--59vO7bWxVeWs1ajBiMG8/view?usp=
> > > sharing
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-carbondata-
> > > mailing-list-archive.1130556.n5.nabble.com/B-Tree-LRU-
> > > cache-New-Feature-tp2366p3130.html
> > > Sent from the Apache CarbonData Mailing List archive mailing list
> archive
> > > at Nabble.com.
> > >
> >
>