Login  Register

Is carbondata replace global dictionary in driver site?

classic Classic list List threaded Threaded
2 messages Options Options
Embed post
Permalink
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Is carbondata replace global dictionary in driver site?

carbondata-newuser
9 posts
For example.
Assume column A has a global dictionary  encoding, and the dictionary is
{
      "AAAAA": 1,
      "BBBBB": 2,
      "CCCCC": 3
}
executor 1 return the result [1,2,3]
Finally the driver will replace the 1 to 'AAAAA', 2 to "BBBBB"?
The replace occurs in driver not executor?
If so, I want to ask why?




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Re: Is carbondata replace global dictionary in driver site?

sujith chacko
19 posts
Conversion from dictionary to actual values happens in executor side.
Please refer CarbonDictionaryDecoder.scala class know more details about
conversion logic.

You can also refer the plan by executing explain query.

Thanks,
Sujith

On Fri, 26 Oct 2018 at 3:27 PM, carbondata-newuser <
[hidden email]> wrote:

> For example.
> Assume column A has a global dictionary  encoding, and the dictionary is
> {
>       "AAAAA": 1,
>       "BBBBB": 2,
>       "CCCCC": 3
> }
> executor 1 return the result [1,2,3]
> Finally the driver will replace the 1 to 'AAAAA', 2 to "BBBBB"?
> The replace occurs in driver not executor?
> If so, I want to ask why?
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>