[GitHub] [carbondata] Indhumathi27 opened a new pull request #3530: [WIP]Fix Select query failure on aggregation of same column on MV

[GitBox]

Indhumathi27 opened a new pull request #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530
 
 
   Be sure to do all of the following checklist to help us incorporate
   your contribution quickly and easily:
   
    - [ ] Any interfaces changed?
   
    - [ ] Any backward compatibility impacted?
   
    - [ ] Document update required?
   
    - [ ] Testing done
           Please provide details on
           - Whether new unit test cases have been added or why no new tests are required?
           - How it is tested? Please attach test report.
           - Is it a performance related change? Please attach the performance test report.
           - Any additional information to help reviewers in testing this change.
         
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568687805
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1260/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568700141
 
 
   Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1270/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568702022
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1281/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568706143
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568713653
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1264/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568725557
 
 
   Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1274/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [WIP]Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-568726615
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1285/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569015674
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1284/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569019533
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1286/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569023410
 
 
   Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1293/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569030953
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1295/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569031786
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1306/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569037150
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1304/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569248356
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1296/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569260178
 
 
   Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1305/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569266788
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1316/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#discussion_r361770549
 
 

 ##########
 File path: datamap/mv/core/src/main/scala/org/apache/carbondata/mv/datamap/MVUtil.scala
 ##########
 @@ -49,23 +49,28 @@ class MVUtil {
       case select: Select =>
         select.children.map {
           case groupBy: GroupBy =>
-            getFieldsFromProject(groupBy.outputList, groupBy.predicateList, logicalRelation)
+            getFieldsFromProject(groupBy.outputList, groupBy.predicateList,
+              logicalRelation, groupBy.flagSpec)
           case _: ModularRelation =>
-            getFieldsFromProject(select.outputList, select.predicateList, logicalRelation)
+            getFieldsFromProject(select.outputList, select.predicateList,
+              logicalRelation, select.flagSpec)
         }.head
       case groupBy: GroupBy =>
         groupBy.child match {
           case select: Select =>
-            getFieldsFromProject(groupBy.outputList, select.predicateList, logicalRelation)
+            getFieldsFromProject(groupBy.outputList, select.predicateList,
+              logicalRelation, select.flagSpec)
           case _: ModularRelation =>
-            getFieldsFromProject(groupBy.outputList, groupBy.predicateList, logicalRelation)
+            getFieldsFromProject(groupBy.outputList, groupBy.predicateList,
+              logicalRelation, groupBy.flagSpec)
         }
     }
   }
 
   def getFieldsFromProject(outputList: Seq[NamedExpression],
       predicateList: Seq[Expression],
-      logicalRelation: Seq[LogicalRelation]): mutable.LinkedHashMap[Field, DataMapField] = {
+      logicalRelation: Seq[LogicalRelation],
+      flagSpec: Seq[Seq[Any]]): mutable.LinkedHashMap[Field, DataMapField] = {
 
 Review comment:
   please add comment to describe parameters

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#discussion_r361770632
 
 

 ##########
 File path: docs/datamap/mv-datamap-guide.md
 ##########
 @@ -91,7 +91,7 @@ EXPLAIN SELECT a, sum(b) from maintable group by a;
    12. NO_INVERTED_INDEX
    13. COLUMN_COMPRESSOR
 
- * All columns of main table at once cannot participate in mv datamap table creation
+ * Creating MV datamap with select query containing only project of all columns of maintable is unsupported
 
 Review comment:
   Can you add an example to the doc for this unsupported case

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3530: [CARBONDATA-3629] Fix Select query failure on aggregation of same column on MV
URL: https://github.com/apache/carbondata/pull/3530#issuecomment-569401396
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1316/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services