[jira] [Comment Edited] (CARBONDATA-2550) [MV] Limit is ignored when data fetched from MV, Query rewrite is Wrong

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523449#comment-16523449 ]

xubo245 edited comment on CARBONDATA-2550 at 6/27/18 1:54 AM:
--------------------------------------------------------------

It's work fine when I test it in local machine today:

{code:java}
No rows selected (0.102 seconds)
0: jdbc:hive2://127.0.0.1:10000> rebuild datamap map1;
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.617 seconds)
0: jdbc:hive2://127.0.0.1:10000> select name,sum(salary) from mvtable1 group by name limit 2;
+-------+--------------+--+
| name  | sum(salary)  |
+-------+--------------+--+
| n3    | 12           |
| n1    | 24           |
+-------+--------------+--+
2 rows selected (0.263 seconds)
0: jdbc:hive2://127.0.0.1:10000> explain select name,sum(salary) from mvtable1 group by name limit 2;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                                                                                                                                                                                            plan                                                                                                                                                                                                                                                                            |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| == CarbonData Profiler ==
Table Scan on map1_table
 - total blocklets: 1
 - filter: none
 - pruned by Main DataMap
    - skipped blocklets: 0
                                                                                                                                                                                                                                                                                                                                                                                                             |
| == Physical Plan ==
CollectLimit 2
+- *HashAggregate(keys=[mvtable1_name#1767], functions=[sum(sum_salary#1768L)])
   +- Exchange hashpartitioning(mvtable1_name#1767, 200)
      +- *HashAggregate(keys=[mvtable1_name#1767], functions=[partial_sum(sum_salary#1768L)])
         +- *BatchedScan CarbonDatasourceHadoopRelation [ Database name :default, Table name :map1_table, Schema :Some(StructType(StructField(mvtable1_name,StringType,true), StructField(sum_salary,LongType,true))) ] default.map1_table[mvtable1_name#1767,sum_salary#1768L]  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.118 seconds)
0: jdbc:hive2://127.0.0.1:10000>

{code}



was (Author: xubo245):
It's work fine when I test it today:

{code:java}
No rows selected (0.102 seconds)
0: jdbc:hive2://127.0.0.1:10000> rebuild datamap map1;
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.617 seconds)
0: jdbc:hive2://127.0.0.1:10000> select name,sum(salary) from mvtable1 group by name limit 2;
+-------+--------------+--+
| name  | sum(salary)  |
+-------+--------------+--+
| n3    | 12           |
| n1    | 24           |
+-------+--------------+--+
2 rows selected (0.263 seconds)
0: jdbc:hive2://127.0.0.1:10000> explain select name,sum(salary) from mvtable1 group by name limit 2;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                                                                                                                                                                                            plan                                                                                                                                                                                                                                                                            |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| == CarbonData Profiler ==
Table Scan on map1_table
 - total blocklets: 1
 - filter: none
 - pruned by Main DataMap
    - skipped blocklets: 0
                                                                                                                                                                                                                                                                                                                                                                                                             |
| == Physical Plan ==
CollectLimit 2
+- *HashAggregate(keys=[mvtable1_name#1767], functions=[sum(sum_salary#1768L)])
   +- Exchange hashpartitioning(mvtable1_name#1767, 200)
      +- *HashAggregate(keys=[mvtable1_name#1767], functions=[partial_sum(sum_salary#1768L)])
         +- *BatchedScan CarbonDatasourceHadoopRelation [ Database name :default, Table name :map1_table, Schema :Some(StructType(StructField(mvtable1_name,StringType,true), StructField(sum_salary,LongType,true))) ] default.map1_table[mvtable1_name#1767,sum_salary#1768L]  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.118 seconds)
0: jdbc:hive2://127.0.0.1:10000>

{code}

> [MV] Limit is ignored when data fetched from MV, Query rewrite is Wrong
> -----------------------------------------------------------------------
>
>                 Key: CARBONDATA-2550
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2550
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Babulal
>            Assignee: xubo245
>            Priority: Major
>
> 0: jdbc:hive2://10.18.222.231:23040> create table mvtable1(name string,age int,salary int) stored by 'carbondata';
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (0.279 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n1',12,12;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (11.973 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n1',12,12;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (9.92 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n3',12,12;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (9.883 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> insert into mvtable1 select 'n4',12,12;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (10.488 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name;
> +-------+--------------+--+
> | name | sum(salary) |
> +-------+--------------+--+
> | n3 | 12 |
> | n1 | 24 |
> | n4 | 12 |
> +-------+--------------+–+
> 0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 2;
> +-------+--------------+--+
> | name | sum(salary) |
> +-------+--------------+--+
> | n3 | 12 |
> | n1 | 24 |
> +-------+--------------+--+
> 2 rows selected (4.175 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> create datamap map1 using 'mv' as select name,sum(salary) from mvtable1 group by name;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (0.396 seconds)
> 0: jdbc:hive2://10.18.222.231:23040> rebuild datamap map1;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (13.246 seconds)
>  
> *0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 2;*
> +----------------+-------------+--+
> | mvtable1_name | sum_salary |
> +----------------+-------------+--+
> | n3 | 12 |
> | n1 | 24 |
> | n4 | 12 |
> +----------------+-------------+--+
> 3 rows selected (2.453 seconds)
> *0: jdbc:hive2://10.18.222.231:23040> select name,sum(salary) from mvtable1 group by name limit 1;*
> +----------------+-------------+--+
> | mvtable1_name | sum_salary |
> +----------------+-------------+--+
> | n3 | 12 |
> | n1 | 24 |
> | n4 | 12 |
> +----------------+-------------+--+
> 3 rows selected (0.347 seconds)
> 0: jdbc:hive2://10.18.222.231:23040>
>  
>  
> Even limit is given MV returns all the records from MV table.
> Cause:- 
> When Rewriting MV query ,limit is ignored,
> 0: jdbc:hive2://10.18.222.231:23040> explain select name,sum(salary) from mvtable1 *group by name limit 2;*
> | plan |
> | == CarbonData Profiler ==
> Table Scan on map1_table
>  - total blocklets: 2
>  - filter: none
>  - pruned by Main DataMap
>  - skipped blocklets: 0
>  |
> | == Physical Plan ==
> *BatchedScan CarbonDatasourceHadoopRelation [ Database name :default, Table name :map1_table, Schema :Some(StructType(StructField(mvtable1_name,StringType,true), StructField(sum_salary,LongType,true))) ] default.map1_table[mvtable1_name#4438,sum_salary#4614L] |
> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
> 2 rows selected (0.36
>  
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)