[jira] [Updated] (CARBONDATA-2293) DataLoad failure for partition table with complex datatype

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

CHENYANG XU updated CARBONDATA-2293:
------------------------------------
    Description:
Error can be reproduced following steps below:

{code:scala}
case class ComplexType(resolution: Resolution)
case class Resolution(height: Int, width: Int)

sparkSession.sql(
      "CREATE TABLE IF NOT EXISTS test_table(" +
        "resolution struct<height:int, width:int>) " +
        "partitioned by (dt string)" +
        "STORED BY 'carbondata'")

val df = sparkSession.sparkContext.parallelize(
      Seq(
        ComplexType(Resolution(1, 2)),
        ComplexType(Resolution(3, 4))
      )
    ).toDF.createOrReplaceTempView("tmp")

sparkSession.sql("insert into table test_table partition(dt='20180323') select resolution from tmp")
sparkSession.sql("select * from test_table").show()
{code}


{panel:title=exception}
java.lang.Exception: DataLoad failure
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadDataWithPartition(CarbonLoadDataCommand.scala:741)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:504)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:280)
        .....

caused by:
[B cannot be cast to java.lang.Integer
        scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
        org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
        org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getInt(rows.scala:194)
        ......
{panel}


  was:
Error can be reproduced following steps below:

{code:scala}
case class ComplexType(resolution: Resolution)
case class Resolution(height: Int, width: Int)

sparkSession.sql(
      "CREATE TABLE IF NOT EXISTS test_table(" +
        "resolution struct<height:int, width:int>) " +
        "partitioned by (dt string)" +
        "STORED BY 'carbondata'")

    val df = sparkSession.sparkContext.parallelize(
      Seq(
        ComplexType(Resolution(1, 2)),
        ComplexType(Resolution(3, 4))
      )
    ).toDF.createOrReplaceTempView("tmp")

    sparkSession.sql("insert into table test_table partition(dt='20180323') select resolution from tmp")
    sparkSession.sql("select * from test_table").show()
{code}


{panel:title=exception}
java.lang.Exception: DataLoad failure
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadDataWithPartition(CarbonLoadDataCommand.scala:741)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:504)
        at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:280)
        .....

caused by:
[B cannot be cast to java.lang.Integer
        scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
        org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
        org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getInt(rows.scala:194)
        ......
{panel}

> DataLoad failure for partition table with complex datatype
> ----------------------------------------------------------
>
>                 Key: CARBONDATA-2293
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2293
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 1.3.1
>            Reporter: CHENYANG XU
>            Priority: Major
>
> Error can be reproduced following steps below:
> {code:scala}
> case class ComplexType(resolution: Resolution)
> case class Resolution(height: Int, width: Int)
> sparkSession.sql(
>       "CREATE TABLE IF NOT EXISTS test_table(" +
>         "resolution struct<height:int, width:int>) " +
>         "partitioned by (dt string)" +
>         "STORED BY 'carbondata'")
> val df = sparkSession.sparkContext.parallelize(
>       Seq(
>         ComplexType(Resolution(1, 2)),
>         ComplexType(Resolution(3, 4))
>       )
>     ).toDF.createOrReplaceTempView("tmp")
> sparkSession.sql("insert into table test_table partition(dt='20180323') select resolution from tmp")
> sparkSession.sql("select * from test_table").show()
> {code}
> {panel:title=exception}
> java.lang.Exception: DataLoad failure
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadDataWithPartition(CarbonLoadDataCommand.scala:741)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:504)
> at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:280)
>         .....
> caused by:
> [B cannot be cast to java.lang.Integer
> scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
> org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getInt(rows.scala:194)
>         ......
> {panel}

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)