tianli created CARBONDATA-833:
---------------------------------
Summary: load data from dataframe,generater data row may be error when delimiterLevel1 or delimiterLevel2 is special character
Key: CARBONDATA-833
URL:
https://issues.apache.org/jira/browse/CARBONDATA-833 Project: CarbonData
Issue Type: Bug
Components: spark-integration
Affects Versions: 1.0.0-incubating, 1.1.0-incubating
Reporter: tianli
Assignee: tianli
Fix For: 1.1.0-incubating, 1.0.0-incubating
load data from dataframe,generater data row may be error by delimiterLevel1 or delimiterLevel2 is special character
because delimiterLevel1 and delimiterLevel2 when carbonLoadModel is create by CarbonUtil.delimiterConverter(), CarbonScalaUtil.getString direct use carbonLoadModel.getComplexDelimiterLevel1 and carbonLoadModel.getComplexDelimiterLevel2
val delimiter = if (level == 1) {
delimiterLevel1
} else {
delimiterLevel2
}
val builder = new StringBuilder()
s.foreach { x =>
builder.append(getString(x, serializationNullFormat, delimiterLevel1,
delimiterLevel2, timeStampFormat, dateFormat, level + 1)).append(delimiter)
}
make primitive data added a more char \ when datatype is complex
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)