Re: [Improvement] Carbon query gc problem

Posted by sraghunandan on Dec 13, 2016; 10:47am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Improvement-Carbon-query-gc-problem-tp4322p4342.html

+1
Good idea to avoid gc overhead.we need to be careful in clearing memory
after use
On Tue, 13 Dec 2016 at 2:17 PM, Kumar Vishal <[hidden email]>
wrote:

> There are lots of gc when carbon is processing more number of records
> during query, which is impacting carbon query performance.To solve this gc
> problem happening when query output is too huge or when more number of
> records are processed, I would like to propose below solution.
>
> Currently we are storing all the data which is read during query from
> carbon data file in heap, when number of query output is huge it is causing
> more gc. Instead of storing in heap we can store this data in offheap and
> will clear when scanning is finished for that query.
>
> Please vote and comment for above proposal.
>
> -Regards
> KUmar Vishal
>