[GitHub] [carbondata] Indhumathi27 opened a new pull request #4063: [CARBONDATA-4095] Fix Select Query with SI filter fails, when columnDrift is enabled

[GitBox]

Indhumathi27 opened a new pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063


    ### Why is this PR needed?
   After converting expression to IN Expression for maintable with SI, expression is not processed if  ColumnDrift is enabled. Query fails with NPE during resolveFilter. Exception is added in JIRA
   
    ### What changes were proposed in this PR?
   Process the filter expression after adding implicit expression
       
    ### Does this PR introduce any user interface change?
    - No
   
   
    ### Is any new testcase added?
   
    - Yes
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#issuecomment-749585828


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5230/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#issuecomment-749593991


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3469/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on a change in pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#discussion_r547702404



##########
File path: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##########
@@ -683,6 +683,8 @@ public QueryModel createQueryModel(InputSplit inputSplit, TaskAttemptContext tas
     }
     if (indexFilter != null) {
       checkAndAddImplicitExpression(indexFilter.getExpression(), inputSplit);
+      // process filter expression after adding implicit expression
+      indexFilter.processFilterExpression();

Review comment:
       in `Querymodelbuilder.build()` we call `processFilterExpression` with check of columndrift, so here if we call process, then in case of no column drift , two times will be called, so please refactor to call this only in case of columndrift true




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on a change in pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#discussion_r547708086



##########
File path: hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##########
@@ -683,6 +683,8 @@ public QueryModel createQueryModel(InputSplit inputSplit, TaskAttemptContext tas
     }
     if (indexFilter != null) {
       checkAndAddImplicitExpression(indexFilter.getExpression(), inputSplit);
+      // process filter expression after adding implicit expression
+      indexFilter.processFilterExpression();

Review comment:
       done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#issuecomment-749996165


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3475/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#issuecomment-749996377


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5236/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063#issuecomment-750008809


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

asfgit closed pull request #4063:
URL: https://github.com/apache/carbondata/pull/4063


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]