carbondata unsupported type: StructType when use dataframe load data

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

carbondata unsupported type: StructType when use dataframe load data

dylan
when i load data from dataframe,i get error:
java.lang.RuntimeException: unsupported type:
StructType(StructField(double,DoubleType,false),
StructField(decimal,DecimalType(9,2),false),
StructField(string,StringType,false))
        at scala.sys.package$.error(package.scala:27)
        at
org.apache.carbondata.spark.CarbonDataFrameWriter.org$apache$carbondata$spark$CarbonDataFrameWriter$$convertToCarbonType(CarbonDataFrameWriter.scala:172)
        at
org.apache.carbondata.spark.CarbonDataFrameWriter$$anonfun$4.apply(CarbonDataFrameWriter.scala:182)
        at
org.apache.carbondata.spark.CarbonDataFrameWriter$$anonfun$4.apply(CarbonDataFrameWriter.scala:181)
        at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
        at scala.collection.Iterator$class.foreach(Iterator.scala:727)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at org.apache.spark.sql.types.StructType.foreach(StructType.scala:92)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
        at org.apache.spark.sql.types.StructType.map(StructType.scala:92)
        at
org.apache.carbondata.spark.CarbonDataFrameWriter.makeCreateTableString(CarbonDataFrameWriter.scala:181)
        at
org.apache.carbondata.spark.CarbonDataFrameWriter.saveAsCarbonFile(CarbonDataFrameWriter.scala:39)
        at
org.apache.spark.sql.CarbonSource.createRelation(CarbonDatasourceRelation.scala:109)
        at
org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)
        at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:148)

Looked at the source does not support, may I ask why not support it?



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: carbondata unsupported type: StructType when use dataframe load data

cenyuhai11
I think we should add unit tests for all supported data types... Many data
types are unavailable...



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/