Re: [Improvement] Carbon query gc problem

Posted by Liang Chen on Dec 20, 2016; 3:51am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Improvement-Carbon-query-gc-problem-tp4322p4717.html

Hi +1，Store data in offheap to avoid gc problem , the solution will help performance more.

Kumar Vishal wrote

There are lots of gc when carbon is processing more number of records during query, which is impacting carbon query performance.To solve this gc problem happening when query output is too huge or when more number of records are processed, I would like to propose below solution. Currently we are storing all the data which is read during query from carbon data file in heap, when number of query output is huge it is causing more gc. Instead of storing in heap we can store this data in offheap and will clear when scanning is finished for that query. Please vote and comment for above proposal. -Regards KUmar Vishal