[Discussion] Using Lazy Dictionary Decode for Presto Integration

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discussion] Using Lazy Dictionary Decode for Presto Integration

bhavya411
We were trying the Presto with carbon data and in the code currently
Carbondata is decoding the dictionary values into actual values as soon as
the data is read from Carbondata. I think if we do a lazy decode of
dictionary values after aggregation it will make the queries faster.
Please let me know if anybody have some thoughts about decoding it when
calculating the final results.

Thanks and regards
Bhavya
Reply | Threaded
Open this post in threaded view
|

Re: [Discussion] Using Lazy Dictionary Decode for Presto Integration

Liang Chen-2
+1, use the laze decode to utilize carbondata's dictionary, it would
improve aggregation performance.
Please consider adding these code to presto integration module, don't
directly reuse spark module code.

Regards
Liang

2017-07-18 23:46 GMT+08:00 Bhavya Aggarwal <[hidden email]>:

> We were trying the Presto with carbon data and in the code currently
> Carbondata is decoding the dictionary values into actual values as soon as
> the data is read from Carbondata. I think if we do a lazy decode of
> dictionary values after aggregation it will make the queries faster.
> Please let me know if anybody have some thoughts about decoding it when
> calculating the final results.
>
> Thanks and regards
> Bhavya
>