[Improvement] Carbon query gc problem

Posted by kumarvishal09 on Dec 13, 2016; 8:47am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Improvement-Carbon-query-gc-problem-tp4322.html

There are lots of gc when carbon is processing more number of records
during query, which is impacting carbon query performance.To solve this gc
problem happening when query output is too huge or when more number of
records are processed, I would like to propose below solution.

Currently we are storing all the data which is read during query from
carbon data file in heap, when number of query output is huge it is causing
more gc. Instead of storing in heap we can store this data in offheap and
will clear when scanning is finished for that query.

Please vote and comment for above proposal.

-Regards
KUmar Vishal