[jira] [Updated] (CARBONDATA-3820) Fix CDC failure when sort columns present in source dataframe

Posted by Akash R Nilugal (Jira) on Jan 03, 2021; 3:39pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Updated-CARBONDATA-3820-Fix-CDC-failure-when-sort-columns-present-in-source-dataframe-tp105235.html


     [ https://issues.apache.org/jira/browse/CARBONDATA-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor updated CARBONDATA-3820:
-------------------------------------
    Issue Type: Bug  (was: New Feature)

> Fix CDC failure when sort columns present in source dataframe
> -------------------------------------------------------------
>
>                 Key: CARBONDATA-3820
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3820
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Xingjun Hao
>            Priority: Major
>             Fix For: 2.1.0
>
>          Time Spent: 4h
>  Remaining Estimate: 0h
>
> If there is GloabalSort table in the CDC Flow. The following exception will
> be throwed:
> Exception in thread "main" java.lang.RuntimeException: column: id specified
> in sort columns does not exist in schema
>         at
> org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildTableSchema(CarbonWriterBuilder.java:828)
>         at
> org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildCarbonTable(CarbonWriterBuilder.java:794)
>         at
> org.apache.carbondata.sdk.file.CarbonWriterBuilder.buildLoadModel(CarbonWriterBuilder.java:720)
>         at
> org.apache.spark.sql.carbondata.execution.datasources.CarbonSparkDataSourceUtil$.prepareLoadModel(CarbonSparkDataSourceUtil.scala:281)
>         at
> org.apache.spark.sql.carbondata.execution.datasources.SparkCarbonFileFormat.prepareWrite(SparkCarbonFileFormat.scala:141)
>         at
> org.apache.spark.sql.execution.command.mutation.merge.CarbonMergeDataSetCommand.processIUD(CarbonMergeDataSetCommand.scala:269)
>         at
> org.apache.spark.sql.execution.command.mutation.merge.CarbonMergeDataSetCommand.processData(CarbonMergeDataSetCommand.scala:152)

--
This message was sent by Atlassian Jira
(v8.3.4#803005)