[jira] [Updated] (CARBONDATA-2544) [MV] Wrong data displayed with Filter

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Babulal updated CARBONDATA-2544:
--------------------------------
    Description:
spark.sql("drop table if exists test1")
 spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")

spark.sql("insert into test1 select 'name1','USA',12,23")

spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
 spark.sql("rebuild datamap datamv2")
 spark.sql("select country,sum(salary) from test1 group by country").show(200,false)

+--------+----------+
|country|sum(salary)|

+--------+----------+
|USA|23|

+--------+----------+

 

spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)

+--------+----------+
|country|sum(salary)|

+--------+----------+
 +--------+----------+

 

This is because, select query formation is wrong , filter value is changed to lowercase 

2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
 from
 (select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
 from
 test1
 group by test1.`country`) gen_subsumer_0
 where
 (gen_subsumer_0.`country` = 'usa')

 

 

 

  was:
spark.sql("drop table if exists test1")
spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")

spark.sql("insert into test1 select 'name1','USA',12,23")

spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
spark.sql("rebuild datamap datamv2")
spark.sql("select country,sum(salary) from test1 group by country").show(200,false)

+-------+-----------+
|country|sum(salary)|
+-------+-----------+
|USA |23 |
+-------+-----------+

 

spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)

+-------+-----------+
|country|sum(salary)|
+-------+-----------+
+-------+-----------+

 

This is because, select query is wrong filter value is changed to lowercase 

2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
from
 (select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
 from
 test1
 group by test1.`country`) gen_subsumer_0
where
 (gen_subsumer_0.`country` = 'usa')

 

 

 

> [MV] Wrong data displayed with Filter
> --------------------------------------
>
>                 Key: CARBONDATA-2544
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2544
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Babulal
>            Priority: Major
>
> spark.sql("drop table if exists test1")
>  spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")
> spark.sql("insert into test1 select 'name1','USA',12,23")
> spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
>  spark.sql("rebuild datamap datamv2")
>  spark.sql("select country,sum(salary) from test1 group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
> |USA|23|
> +--------+----------+
>  
> spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
>  +--------+----------+
>  
> This is because, select query formation is wrong , filter value is changed to lowercase 
> 2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
>  from
>  (select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
>  from
>  test1
>  group by test1.`country`) gen_subsumer_0
>  where
>  (gen_subsumer_0.`country` = 'usa')
>  
>  
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)