[GitHub] [carbondata] Indhumathi27 opened a new pull request #3651: [WIP] Fix Incorrect query results on mv with limit

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-599491757
 
 
   Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/767/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-599493253
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2475/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601539304
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#discussion_r395444257
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAlterTableCompactionCommand.scala
 ##########
 @@ -220,17 +219,23 @@ case class CarbonAlterTableCompactionCommand(
     val LOGGER = LogServiceFactory.getLogService(this.getClass.getName)
     val compactionType = CompactionType.valueOf(alterTableModel.compactionType.toUpperCase)
     val compactionSize = CarbonDataMergerUtil.getCompactionSize(compactionType, carbonLoadModel)
+    val carbonTable = carbonLoadModel.getCarbonDataLoadSchema.getCarbonTable
     if (CompactionType.IUD_UPDDEL_DELTA == compactionType) {
       if (alterTableModel.segmentUpdateStatusManager.isDefined) {
         carbonLoadModel.setSegmentUpdateStatusManager(
           alterTableModel.segmentUpdateStatusManager.get)
         carbonLoadModel.setLoadMetadataDetails(
           alterTableModel.segmentUpdateStatusManager.get.getLoadMetadataDetails.toList.asJava)
+      } else {
+        // segmentUpdateStatusManager will not be defined in case of IUD compaction on materialized
 
 Review comment:
   mention horizontal compaction also, even in descripton

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#discussion_r395444682
 
 

 ##########
 File path: mv/core/src/main/scala/org/apache/carbondata/mv/extension/MVHelper.scala
 ##########
 @@ -85,6 +85,25 @@ object MVHelper {
                                                   "with limit")
       case _ =>
     }
+
+    // Order by columns needs to be present in projection list for creating mv. This is because,
+    // we have to perform order by on all segments during query, which requires the order by column
+    // data
+    queryPlan.transform {
+      case sort@Sort(order, _, _) =>
+        order.map { orderByCol =>
+          orderByCol.child match {
+            case attr: AttributeReference =>
+              if (!queryPlan.output.contains(attr.toAttribute)) {
+                throw new UnsupportedOperationException(
+                  "Order by columns must be present in project columns")
 
 Review comment:
   please give column name also in error

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601567502
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/810/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601568178
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2514/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601569921
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601587463
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2516/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601619408
 
 
   Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/814/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601620708
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2521/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651#issuecomment-601635105
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3651: [CARBONDATA-3733] Fix Incorrect query results on mv with limit
URL: https://github.com/apache/carbondata/pull/3651
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services