[GitHub] carbondata pull request #1254: [WIP] Fixed Date range filter with cast not w...

[qiuchenjian-2]

GitHub user ravipesala opened a pull request:

    https://github.com/apache/carbondata/pull/1254

    [WIP] Fixed Date range filter with cast not working

   

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ravipesala/incubator-carbondata datefilter-fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1254.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1254
   
----
commit ebcc6b75e15d09844aa166f6ebb5c5ebe872f742
Author: Ravindra Pesala <[hidden email]>
Date:   2017-08-12T06:12:26Z

    Fixed Date range filter with cast not working

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1254
 
    SDV Build Success with Spark 2.1, Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/192/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1254#discussion_r133364318
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/keygenerator/directdictionary/timestamp/DateDirectDictionaryGenerator.java ---
    @@ -154,14 +147,14 @@ private int generateDirectSurrogateKeyForNonTimestampType(String memberStr) {
       }
     
       private int generateKey(long timeValue) {
    -    long milli = timeValue + threadLocalLocalTimeZone.get().getOffset(timeValue);
    -    return (int) Math.floor((double) milli / MILLIS_PER_DAY) + cutOffDate;
    +    return (int) Math.floor((double) timeValue / MILLIS_PER_DAY) + cutOffDate;
       }
     
       public void initialize() {
         if (simpleDateFormatLocal.get() == null) {
           simpleDateFormatLocal.set(new SimpleDateFormat(dateFormat));
           simpleDateFormatLocal.get().setLenient(false);
    +      simpleDateFormatLocal.get().setTimeZone(TimeZone.getTimeZone("GMT"));
    --- End diff --
   
    Does it mean carbon will process all Date/TimeStamp as GMT?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1254#discussion_r134010107
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/keygenerator/directdictionary/timestamp/DateDirectDictionaryGenerator.java ---
    @@ -154,14 +147,14 @@ private int generateDirectSurrogateKeyForNonTimestampType(String memberStr) {
       }
     
       private int generateKey(long timeValue) {
    -    long milli = timeValue + threadLocalLocalTimeZone.get().getOffset(timeValue);
    -    return (int) Math.floor((double) milli / MILLIS_PER_DAY) + cutOffDate;
    +    return (int) Math.floor((double) timeValue / MILLIS_PER_DAY) + cutOffDate;
       }
     
       public void initialize() {
         if (simpleDateFormatLocal.get() == null) {
           simpleDateFormatLocal.set(new SimpleDateFormat(dateFormat));
           simpleDateFormatLocal.get().setLenient(false);
    +      simpleDateFormatLocal.get().setTimeZone(TimeZone.getTimeZone("GMT"));
    --- End diff --
   
    Loading and querying both uses GMT here only for date, we should not be based on machines timezone as it gives issues


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1254
 
    SDV Build Success with Spark 2.1, Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/241/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1254
 
    retest this please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1254
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/647/



---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/1254
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1254


---