Posted by
manishgupta88 on
May 17, 2017; 4:30am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/White-Space-does-not-trim-in-CarbonData-tp12736p12753.html
Hi Vin,
Value is getting displayed as null because data is not getting trimmed. By
default carbondata will not modify any data. But there is option to verify
your data.
1. While loading the data set the property "
BAD_RECORDS_LOGGER_ENABLE=true". This will create a bad record logger where
in you can see what all records are invalid.
2. Secondly by default trimming data is disabled in the code.
parserSettings.setIgnoreLeadingWhitespaces(false);
parserSettings.setIgnoreTrailingWhitespaces(false);
We can provide an option during data load to trim whitespaces and keep the
default value as false to keep the current behavior intact. Example
load data inpath 'D:/my_user2.csv' into table my_user_c
OPTIONS('TRIM_WHITESPACE'='TRUE')
@Dev team please share your inputs for the 2nd option.
Regards
Manish Gupta
On Wed, May 17, 2017 at 7:47 AM, David Cai <
[hidden email]> wrote:
> I suggest to trim white space before converting a string value to a numeric
> value.
>
>
>
> -----
> Best Regards
> David Cai
> --
> View this message in context:
http://apache-carbondata-dev-> mailing-list-archive.1130556.n5.nabble.com/White-Space-
> does-not-trim-in-CarbonData-tp12736p12751.html
> Sent from the Apache CarbonData Dev Mailing List archive mailing list
> archive at Nabble.com.
>