[jira] [Updated] (CARBONDATA-2729) Schema Compatibility problem between version 1.3.0 and 1.4.0

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mohammad Shahid Khan updated CARBONDATA-2729:
---------------------------------------------
    Description:
The table schema created with 1.3 is not compatible with 1.4.0.

Steps:

Do the below operation  on hive meta store.

1.  Create table and load data in 1.3 version.   
{code:java}
spark.sql("create database carbonversion_1_1");
spark.sql("use carbonversion_1_1");
spark.sql("""create table testIntType (c1 string,c2 int) STORED BY 'org.apache.carbondata.format'""")
spark.sql("insert into testIntType select 'a',200"){code}
2. Switch to 1.4 version and create data map on the table created in step 1.
{code:java}
spark.sql("use carbonversion_1_3");
spark.sql("create datamap dm8 on table testinttype3 using 'preaggregate' as select c1, sum(c2) from testinttype3 group by c1")
{code}
 Stack trace:
{code:java}
Exception in thread "main" java.lang.NullPointerException
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalSchemaEvolution(ThriftWrapperSchemaConverterImpl.java:87)
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableSchema(ThriftWrapperSchemaConverterImpl.java:277)
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableInfo(ThriftWrapperSchemaConverterImpl.java:310)
at org.apache.spark.sql.hive.CarbonHiveMetaStore.getThriftTableInfo(CarbonHiveMetaStore.scala:101){code}

  was:
The table schema created with 1.3 is not compatible with 1.4.0.

Steps:

Do the below operation  on hive meta store.

1.  Create table and load data in 1.3 version.   
{code:java}
spark.sql("create database carbonversion_1_1");
spark.sql("use carbonversion_1_1");
spark.sql("""create table testIntType (c1 string,c2 int) STORED BY 'org.apache.carbondata.format'""")
spark.sql("insert into testIntType select 'a',200"){code}
2. Switch to 1.4 version and create data map on the table created in step 1.
{code:java}
spark.sql("use carbonversion_1_3");
spark.sql("create datamap dm8 on table testinttype3 using 'preaggregate' as select c1, sum(c2) from testinttype3 group by c1")
{code}
 stack trace 
{code:java}
stack trace

Exception in thread "main" java.lang.NullPointerException
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalSchemaEvolution(ThriftWrapperSchemaConverterImpl.java:87)
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableSchema(ThriftWrapperSchemaConverterImpl.java:277)
at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableInfo(ThriftWrapperSchemaConverterImpl.java:310)
at org.apache.spark.sql.hive.CarbonHiveMetaStore.getThriftTableInfo(CarbonHiveMetaStore.scala:101){code}

> Schema Compatibility problem between version 1.3.0 and 1.4.0
> ------------------------------------------------------------
>
>                 Key: CARBONDATA-2729
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2729
>             Project: CarbonData
>          Issue Type: Bug
>          Components: file-format
>    Affects Versions: 1.4.0
>            Reporter: Mohammad Shahid Khan
>            Assignee: Mohammad Shahid Khan
>            Priority: Major
>             Fix For: 1.4.1
>
>
> The table schema created with 1.3 is not compatible with 1.4.0.
> Steps:
> Do the below operation  on hive meta store.
> 1.  Create table and load data in 1.3 version.   
> {code:java}
> spark.sql("create database carbonversion_1_1");
> spark.sql("use carbonversion_1_1");
> spark.sql("""create table testIntType (c1 string,c2 int) STORED BY 'org.apache.carbondata.format'""")
> spark.sql("insert into testIntType select 'a',200"){code}
> 2. Switch to 1.4 version and create data map on the table created in step 1.
> {code:java}
> spark.sql("use carbonversion_1_3");
> spark.sql("create datamap dm8 on table testinttype3 using 'preaggregate' as select c1, sum(c2) from testinttype3 group by c1")
> {code}
>  Stack trace:
> {code:java}
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalSchemaEvolution(ThriftWrapperSchemaConverterImpl.java:87)
> at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableSchema(ThriftWrapperSchemaConverterImpl.java:277)
> at org.apache.carbondata.core.metadata.converter.ThriftWrapperSchemaConverterImpl.fromWrapperToExternalTableInfo(ThriftWrapperSchemaConverterImpl.java:310)
> at org.apache.spark.sql.hive.CarbonHiveMetaStore.getThriftTableInfo(CarbonHiveMetaStore.scala:101){code}

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)