Posted by
Akash R Nilugal (Jira) on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Created-CARBONDATA-738-Able-to-load-dataframe-with-boolean-type-in-a-carbon-table-but-with-nulls-tp8182.html
anubhav tarar created CARBONDATA-738:
----------------------------------------
Summary: Able to load dataframe with boolean type in a carbon table but with null values
Key: CARBONDATA-738
URL:
https://issues.apache.org/jira/browse/CARBONDATA-738 Project: CarbonData
Issue Type: Bug
Components: spark-integration
Affects Versions: 1.0.0-incubating
Environment: spark 1.6,spark 2.1
Reporter: anubhav tarar
Assignee: anubhav tarar
Priority: Trivial
i created a dataframe with boolean type as follows
case class People(name: String, occupation: String, consultant: Boolean)
val people = List(People("sangeeta", "engineer", true), People("pallavi", "consultant", true))
val peopleRDD: RDD[People] = cc.sc.parallelize(people)
val peopleDF: DataFrame = peopleRDD.toDF("name", "occupation", "id")
peopleDF.write
.format("carbondata")
.option("tableName", "carbon2")
.option("compress", "true")
.mode(SaveMode.Overwrite)
.save()
cc.sql("SELECT * FROM carbon2").show()
currently boolean type is not supported in carbon data but table gets created
but it shows me null values
+--------+----------+----+
| name|occupation| id|
+--------+----------+----+
| pallavi|consultant|null|
|sangeeta| engineer|null|
+--------+----------+----+
for boolean type it should throw unsupported type exception
there is problem with carbondataframe writer
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)