[
https://issues.apache.org/jira/browse/CARBONDATA-2110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xuchuanyin updated CARBONDATA-2110:
-----------------------------------
Description:
Currently in carbondata, an option named ‘tempCSV’ is available during loading dataframe.
After enabling this option, Carbondata will write the dataframe to a *standard* csv file at first and then load the data files.
The delimiters of the standard csv file, such as field delimiter / escape char/ quote char/ multi-line/ line separator and so on may conflict with the actual field value. For example, if a field contains ',', then it will cause problem to save the tempCSV using ',' as field separator.
So I think it's better to deprecate this option. To make forward compatible, user can still use this option but will get warning about it.
was:
Currently in carbondata, an option named ‘tempCSV’ is available during loading dataframe.
After enabling this option, Carbondata will write the dataframe to a **standard** csv file at first and then load the data files.
The delimiters of the standard csv file, such as field delimiter / escape char/ quote char/ multi-line/ line separator and so on may conflict with the actual field value. For example, if a field contains ',', then it will cause problem to save the tempCSV using ',' as field separator.
So I think it's better to deprecate this option. To make forward compatible, user can still use this option but will get warning about it.
> option of TempCsv should be removed since the default delimiter may conflicts with field value
> ----------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-2110
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-2110> Project: CarbonData
> Issue Type: Bug
> Components: data-load
> Reporter: xuchuanyin
> Priority: Major
>
> Currently in carbondata, an option named ‘tempCSV’ is available during loading dataframe.
>
> After enabling this option, Carbondata will write the dataframe to a *standard* csv file at first and then load the data files.
>
> The delimiters of the standard csv file, such as field delimiter / escape char/ quote char/ multi-line/ line separator and so on may conflict with the actual field value. For example, if a field contains ',', then it will cause problem to save the tempCSV using ',' as field separator.
>
> So I think it's better to deprecate this option. To make forward compatible, user can still use this option but will get warning about it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
