Vectorized reader exceptions

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Vectorized reader exceptions

Swapnil Shinde
Hi All
    I am not sure if this is random exception but this is what I have observed -
Create and load carbondata table from Spark dataframe - Without dictionary_include on two INT columns - Works fine. I can "select *" on it.
Create and load carbondata table from same Spark datafrmae - With dictionary_include on two INT columns - Throws NullPointerException when I run "select *".

Stacktrace throws error here -

I found similar error fixed in this PR but not available in 1.1.0 version - https://github.com/apache/carbondata/pull/970
I kind of tried adding above PR's changes with additional necessary changes to build successfully. (Not sure if it was a good idea)
If I load again and run "select *"  it throws different error -


I am not much familiar with vectorized reader source code so not sure how to fix this. However, due to above error I have to load that dataset without creating dimensions on columns I need for filtration.
As always, your help is very much appreciated!

Thanks
Swapnil