[jira] [Commented] (CARBONDATA-2544) [MV] Wrong data displayed with Filter

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524445#comment-16524445 ]

xubo245 commented on CARBONDATA-2544:
-------------------------------------

It's work fine in cluster：


{code:java}
0: jdbc:hive2://hadoop1:10000> select country,sum(salary) from test20 group by country;
+----------+--------------+--+
| country  | sum(salary)  |
+----------+--------------+--+
| USA      | 23           |
+----------+--------------+--+
1 row selected (1.226 seconds)
0: jdbc:hive2://hadoop1:10000> select country,sum(salary) from test20 where country='USA' group by country;
+----------+--------------+--+
| country  | sum(salary)  |
+----------+--------------+--+
| USA      | 23           |
+----------+--------------+--+
1 row selected (1.655 seconds)
0: jdbc:hive2://hadoop1:10000> select country,sum(salary) from test20 where country='USA' group by country;
+----------+--------------+--+
| country  | sum(salary)  |
+----------+--------------+--+
| USA      | 23           |
+----------+--------------+--+
1 row selected (0.92 seconds)
0: jdbc:hive2://hadoop1:10000> explain select country,sum(salary) from test20 where country='USA' group by country;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                                                                                                                                                                                                                                                                                                                                            plan                                                                                                                                                                                                                                                                                                                                                                                                                            |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| == CarbonData Profiler ==
Table Scan on datamv20_table
 - total blocklets: 1
 - filter: (test20_country <> null and test20_country = USA)
 - pruned by Main DataMap
    - skipped blocklets: 0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| == Physical Plan ==
*HashAggregate(keys=[country#223], functions=[sum(sum(salary)#224L)])
+- Exchange hashpartitioning(country#223, 200)
   +- *HashAggregate(keys=[country#223], functions=[partial_sum(sum(salary)#224L)])
      +- *HashAggregate(keys=[test20_country#103], functions=[sum(sum_salary#104L)])
         +- Exchange hashpartitioning(test20_country#103, 200)
            +- *HashAggregate(keys=[test20_country#103], functions=[partial_sum(sum_salary#104L)])
               +- *BatchedScan CarbonDatasourceHadoopRelation [ Database name :default, Table name :datamv20_table, Schema :Some(StructType(StructField(test20_country,StringType,true), StructField(sum_salary,LongType,true))) ] default.datamv20_table[test20_country#103,sum_salary#104L] PushedFilters: [IsNotNull(test20_country), EqualTo(test20_country,USA)]  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.318 seconds)

{code}

> [MV] Wrong data displayed with Filter
> --------------------------------------
>
>                 Key: CARBONDATA-2544
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2544
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Babulal
>            Assignee: xubo245
>            Priority: Major
>
> spark.sql("drop table if exists test1")
>  spark.sql("create table test1( name string,country string,age int,salary int) stored by 'carbondata' ")
> spark.sql("insert into test1 select 'name1','USA',12,23")
> spark.sql("create datamap datamv2 using 'mv' as select country,sum(salary) from test1 group by country").show()
>  spark.sql("rebuild datamap datamv2")
>  spark.sql("select country,sum(salary) from test1 group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
> |USA|23|
> +--------+----------+
>  
> spark.sql("select country,sum(salary) from test1 where country='USA' group by country").show(200,false)
> +--------+----------+
> |country|sum(salary)|
> +--------+----------+
>  +--------+----------+
>  
> This is because, select query formation is wrong , filter value is changed to lowercase 
> 2018-05-27 00:20:16 INFO CarbonSparkSqlParser:54 - Parsing command: select preAGG() as preAgg, gen_subsumer_0.`country`, gen_subsumer_0.`sum(salary)` as `sum(salary)`
>  from
>  (select test1.`country`, sum(cast(test1.`salary` as bigint)) as `sum(salary)`
>  from
>  test1
>  group by test1.`country`) gen_subsumer_0
>  where
>  (gen_subsumer_0.`country` = 'usa')
>  
>  
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)