Questions about Dictionnary Server

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about Dictionnary Server

cenyuhai
Hi, all:
    I have a question, when we should use DictionaryServer?
Reply | Threaded
Open this post in threaded view
|

Re: Questions about Dictionnary Server

ravipesala
Hi,

To generate global dictionary CarbonData first scan all input data and
finds unique data for each column and assign dictionary for each value. So
it is two step process. Irrespective of any new unique dictionary values
are added or not it always need to scan all data to get the dictionary.
To overcome from this issue we introduce this dictionary server. From
second load onwards if there is not much dictionaries are created in the
load then we can choose this option to improve the loading performance. It
just avoids 2 steps process to single step by generating dictionary online
while loading the data.


Regards,
Ravindra.

On Sun, 21 May 2017 at 8:41 PM, Sea <[hidden email]> wrote:

> Hi, all:
>     I have a question, when we should use DictionaryServer?