Is carbondata replace global dictionary in driver site?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Is carbondata replace global dictionary in driver site?

carbondata-newuser
For example.
Assume column A has a global dictionary  encoding, and the dictionary is
{
      "AAAAA": 1,
      "BBBBB": 2,
      "CCCCC": 3
}
executor 1 return the result [1,2,3]
Finally the driver will replace the 1 to 'AAAAA', 2 to "BBBBB"?
The replace occurs in driver not executor?
If so, I want to ask why?




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Is carbondata replace global dictionary in driver site?

sujith chacko
Conversion from dictionary to actual values happens in executor side.
Please refer CarbonDictionaryDecoder.scala class know more details about
conversion logic.

You can also refer the plan by executing explain query.

Thanks,
Sujith

On Fri, 26 Oct 2018 at 3:27 PM, carbondata-newuser <
[hidden email]> wrote:

> For example.
> Assume column A has a global dictionary  encoding, and the dictionary is
> {
>       "AAAAA": 1,
>       "BBBBB": 2,
>       "CCCCC": 3
> }
> executor 1 return the result [1,2,3]
> Finally the driver will replace the 1 to 'AAAAA', 2 to "BBBBB"?
> The replace occurs in driver not executor?
> If so, I want to ask why?
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>