[GitHub] carbondata pull request #1969: [CARBONDATA-2151][Streaming] Fix filter query...

[qiuchenjian-2]

GitHub user QiangCai opened a pull request:

    https://github.com/apache/carbondata/pull/1969

    [CARBONDATA-2151][Streaming] Fix filter query issue on streaming table

    1. Fix filter query issue for timestamp, date, decimal
    2. Add more test case
    dataType:  int, streaming, float, double, decimal, timestamp, date, complex
    operation: =, <>, >=, >, <, <=, in, like, between, is null, is not null
   
   
     - [x] Any interfaces changed?
     no
     - [x] Any backward compatibility impacted?
     no
     - [x] Document update required?
    no
     - [x] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
              enhanced test case
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           
     - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    small changes


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/QiangCai/carbondata streaming_timestamp

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1969.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1969
   
----
commit a318c6773f3fc45a63f34e4dd3c35f893f22c0b9
Author: QiangCai <qiangcai@...>
Date:   2018-02-11T08:06:01Z

    fix filter query issue on streaming table

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3696/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2456/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3717/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2477/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3519/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3829/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2585/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3617/



---

[qiuchenjian-2]

Github user QiangCai commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3836/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2591/



---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1969#discussion_r170179962
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/FilterUtil.java ---
    @@ -1790,6 +1790,9 @@ public static void removeNullValues(DimensionColumnDataChunk dimensionColumnData
       }
     
       public static void updateIndexOfColumnExpression(Expression exp, int dimOridnalMax) {
    +    if (exp == null) {
    +      return;
    --- End diff --
   
    add one comment to explain it can be null


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1969#discussion_r170180010
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java ---
    @@ -145,6 +146,34 @@ public BitSetGroup applyFilter(BlocksChunkHolder blockChunkHolder, boolean useBi
         return applyNoAndDirectFilter(blockChunkHolder);
       }
     
    +  /**
    +   * apply range filter on a row
    --- End diff --
   
    explain the return value


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3840/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2595/



---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1969


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1969
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3624/



---