[
https://issues.apache.org/jira/browse/CARBONDATA-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633257#comment-15633257 ]
ASF GitHub Bot commented on CARBONDATA-375:
-------------------------------------------
GitHub user manishgupta88 opened a pull request:
https://github.com/apache/incubator-carbondata/pull/292 [CARBONDATA-375] Dictionary cache not getting cleared after task completion in dictionary decoder
Problem: Dictionary cache not getting cleared after task completion in dictionary decoder
Analysis: Currently LRU cache eviction policy is based on dictionary access count. For cache to remove a entry its access count must be 0. In dictionary decoder after conversion of surrogate key to actual value the access count for dictionary columns in query is not getting decremented due to which it will never be cleared from memory when LRU cache size is configured.
Fix: Add a task completion listener which will take care of clearing the dictionary in case of both success and failure
Impact area: LRU cache eviction policy which can lead to query and data load failure
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/manishgupta88/incubator-carbondata dictionary_decoder_clear_dictionary
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/292.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #292
----
commit b305f34e1014267b3706c287cef7070189fc3c28
Author: manishgupta88 <
[hidden email]>
Date: 2016-11-03T15:48:03Z
Problem: Dictionary cache not getting cleared after task completion in dictionary decoder
Analysis: Currently LRU cache eviction policy is based on dictionary access count. For cache to remove a entry its access count must be 0. In dictionary decoder after conversion of surrogate key to actual value the access count for dictionary columns in query is not getting decremented due to which it will never be cleared from memory when LRU cache size is configured.
Fix: Add a task completion listener which will take care of clearing the dictionary in case of both success and failure
Impact area: LRU cache eviction policy which can lead to query and data load failure
----
> Dictionary cache not getting cleared after task completion in dictionary decoder
> --------------------------------------------------------------------------------
>
> Key: CARBONDATA-375
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-375> Project: CarbonData
> Issue Type: Bug
> Reporter: Manish Gupta
> Assignee: Manish Gupta
> Priority: Minor
> Fix For: 0.2.0-incubating
>
>
> Currently LRU cache eviction policy is based on dictionary access count. For cache to remove a entry its access count must be 0. In dictionary decoder after conversion of surrogate key to actual value the access count for dictionary columns in query is not getting decremented due to which it will never be cleared from memory when LRU cache size is configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
