[GitHub] [carbondata] VenuReddy2103 opened a new pull request #3953: [CARBONDATA-4008]Fixed IN filter on date column is returning 0 results when 'carbon.push.rowfilters.for.vector' is true

[GitBox]

QiangCai commented on a change in pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#discussion_r497268957



##########
File path: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecutorImpl.java
##########
@@ -106,6 +108,11 @@
    */
   boolean isNaturalSorted;
 
+  /**
+   * date direct dictionary generator
+   */
+  private DirectDictionaryGenerator dateDictionaryGenerator;
+
   public RowLevelFilterExecutorImpl(List<DimColumnResolvedFilterInfo> dimColEvaluatorInfoList,

Review comment:
       in my opinion, when we need to use RowLevelFilterExecutorImpl,  better to consider why we pushdown this predicate into the create relation at first.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on a change in pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#discussion_r500783398



##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/filterexpr/TestInFilter.scala
##########
@@ -165,8 +168,27 @@ class TestInFilter extends QueryTest with BeforeAndAfterAll{
       Seq(Row(4, 1.00, 2.00, 3.00)))
   }
 
-  override def afterAll(): Unit = {
+  test("test infilter with date, timestamp columns") {
+    sql("create table test_table(i int, dt date, ts timestamp) stored as carbondata")
+    sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
+    sql("insert into test_table select 2, '2020-07-04', '2020-07-04 14:12:15'")
+    sql("insert into test_table select 3, '2020-09-23', '2020-09-23 12:30:45'")
+
+    checkAnswer(sql("select * from test_table where dt IN ('2020-03-30', '2020-09-23')"),

Review comment:
       can you add a query here with the different values in In() list as you mentioned in description?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

VenuReddy2103 commented on a change in pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#discussion_r503075660



##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/filterexpr/TestInFilter.scala
##########
@@ -165,8 +168,27 @@ class TestInFilter extends QueryTest with BeforeAndAfterAll{
       Seq(Row(4, 1.00, 2.00, 3.00)))
   }
 
-  override def afterAll(): Unit = {
+  test("test infilter with date, timestamp columns") {
+    sql("create table test_table(i int, dt date, ts timestamp) stored as carbondata")
+    sql("insert into test_table select 1, '2020-03-30', '2020-03-30 10:00:00'")
+    sql("insert into test_table select 2, '2020-07-04', '2020-07-04 14:12:15'")
+    sql("insert into test_table select 3, '2020-09-23', '2020-09-23 12:30:45'")
+
+    checkAnswer(sql("select * from test_table where dt IN ('2020-03-30', '2020-09-23')"),

Review comment:
       Testcase is already present in base




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

QiangCai commented on pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#issuecomment-707009265


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

QiangCai commented on pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#issuecomment-707009405


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#issuecomment-707069675


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4383/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#issuecomment-707070681


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2631/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

QiangCai commented on pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953#issuecomment-707073901


   merge it.
   please raise another PR to improve the filter performance.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

asfgit closed pull request #3953:
URL: https://github.com/apache/carbondata/pull/3953


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]