[jira] [Resolved] (CARBONDATA-1023) Able to do load from dataframe with byte data type in carbon table

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (CARBONDATA-1023) Able to do load from dataframe with byte data type in carbon table

Akash R Nilugal (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ravindra Pesala resolved CARBONDATA-1023.
-----------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.1.0

> Able to do load from dataframe with byte data type in carbon table
> ------------------------------------------------------------------
>
>                 Key: CARBONDATA-1023
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1023
>             Project: CarbonData
>          Issue Type: Bug
>          Components: sql
>    Affects Versions: 1.0.0-incubating
>         Environment: spark common
>            Reporter: anubhav tarar
>            Assignee: anubhav tarar
>            Priority: Trivial
>             Fix For: 1.1.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> i am able to load data with bytetype from dataframe in a carbon table
> as far as documentation says byte type is not supported
>  val rdd1 = sqlContext.sparkContext.parallelize(
>       Row("byte",1234.toByte) :: Nil)
> val schema1 = StructType(
>       StructField("string",StringType, nullable = false) ::
>       StructField("byte",ByteType, nullable = false) :: Nil)
>    val dataFrame = sqlContext.createDataFrame(rdd1, schema1)
>  dataFrame.write
>       .format("carbondata")
>       .option("tableName", "carbon1")
>       .option("compress", "true")
>       .mode(SaveMode.Overwrite)
>       .save()
>       sql("select * from carbon1").show()
> +------+----+
> |string|byte|
> +------+----+
> |  byte| -46|
> +------+----+
> if we try to create a table with byte type it shows error
>   sql(
>       "CREATE TABLE restructure (empno byte, empname String, designation String, doj Timestamp, " +
>       "workgroupcategory int, workgroupcategoryname String, deptno int, deptname String, " +
>       "projectcode int, projectjoindate Timestamp, projectenddate Timestamp,attendance int," +
>       "utilization int,salary int) STORED BY 'org.apache.carbondata.format'")
> result
> org.apache.carbondata.spark.exception.MalformedCarbonCommandException: Unsupported data type: StructField(empno,ByteType,true).getType



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)