CHENYANG XU created CARBONDATA-2293:
---------------------------------------
Summary: DataLoad failure for partition table with complex datatype
Key: CARBONDATA-2293
URL:
https://issues.apache.org/jira/browse/CARBONDATA-2293 Project: CarbonData
Issue Type: Bug
Affects Versions: 1.3.1
Reporter: CHENYANG XU
Error can be reproduced following steps below:
{code:scala}
case class ComplexType(resolution: Resolution)
case class Resolution(height: Int, width: Int)
sparkSession.sql(
"CREATE TABLE IF NOT EXISTS test_table(" +
"resolution struct<height:int, width:int>) " +
"partitioned by (dt string)" +
"STORED BY 'carbondata'")
val df = sparkSession.sparkContext.parallelize(
Seq(
ComplexType(Resolution(1, 2)),
ComplexType(Resolution(3, 4))
)
).toDF.createOrReplaceTempView("tmp")
sparkSession.sql("insert into table test_table partition(dt='20180323') select resolution from tmp")
sparkSession.sql("select * from test_table").show()
{code}
{panel:title=exception}
java.lang.Exception: DataLoad failure
at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadDataWithPartition(CarbonLoadDataCommand.scala:741)
at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.loadData(CarbonLoadDataCommand.scala:504)
at org.apache.spark.sql.execution.command.management.CarbonLoadDataCommand.processData(CarbonLoadDataCommand.scala:280)
.....
caused by:
[B cannot be cast to java.lang.Integer
scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getInt(rows.scala:41)
org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getInt(rows.scala:194)
......
{panel}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
