Disable Adaptive encoding for Double and Float by default

Posted by Ajantha Bhat on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Disable-Adaptive-encoding-for-Double-and-Float-by-default-tp93914.html

Hi all,

I have done insert into flow profiling using JMC with the latest code [with new optimized insert flow]

It seems for 2.5GB carbon to carbon insert, double and float stats collector has used 68.36 GB [25% of TLAB (Thread local allocation buffer)]

The problem is for every value of double and float in every row, we call PrimitivePageStatsCollector.getDecimalCount()
Which makes new objects every time.

So, I want to disable Adaptive encoding for float and double by default. 

I will make this configurable. 

If some user has a well-sorted double or float column and wants to apply adaptive encoding on that, they can enable it to reduce store size.

Thanks,

Ajantha