Re: Discussion(New feature) regarding single pass data loading solution.

Posted by Qingqing Zhou on Oct 13, 2016; 12:54am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-New-feature-regarding-single-pass-data-loading-solution-tp1761p1822.html

On Tue, Oct 11, 2016 at 2:32 AM, Ravindra Pesala <[hidden email]>
wrote:
> Currently data is loading to carbon in 2 pass/jobs
>  1. Generating global dictionary using spark job.

Do we have local dictionaries? If not, what if the column has many
distinct values - will the big dictionary loaded into memory?

Regards,
Qingqing