[
https://issues.apache.org/jira/browse/CARBONDATA-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manish Gupta updated CARBONDATA-2433:
-------------------------------------
Description:
While seraching using lucene it creates a PriorityQueue to hold the documents. As size is not specified by default the PriorityQueue size is equal to the number of lucene documents. As the docuemnts start getting added to the heap the GC time increases and after some time task fails due to excessive GC and executor OOM occurs.
Reference blog:
*
http://lucene.472066.n3.nabble.com/Optimization-of-memory-usage-in-PriorityQueue-td590355.html*> Executor OOM because of GC when blocklet pruning is done using Lucene datamap
> -----------------------------------------------------------------------------
>
> Key: CARBONDATA-2433
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-2433> Project: CarbonData
> Issue Type: Sub-task
> Affects Versions: 1.4.0
> Reporter: Manish Gupta
> Assignee: Manish Gupta
> Priority: Major
>
> While seraching using lucene it creates a PriorityQueue to hold the documents. As size is not specified by default the PriorityQueue size is equal to the number of lucene documents. As the docuemnts start getting added to the heap the GC time increases and after some time task fails due to excessive GC and executor OOM occurs.
> Reference blog:
> *
http://lucene.472066.n3.nabble.com/Optimization-of-memory-usage-in-PriorityQueue-td590355.html*--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
