[GitHub] carbondata pull request #1744: [CARBONDATA-1955] Delta DataType calculation ...

[qiuchenjian-2]

GitHub user mohammadshahidkhan opened a pull request:

    https://github.com/apache/carbondata/pull/1744

    [CARBONDATA-1955] Delta DataType calculation is incorrect for long type

    **Problem**:
    In case of Long type, the delta data type is always choosing the Long type.
    But it should choose the datatype based on diff (max-min)  of max and min values.
    **Solution**:
    Corrected to choose the delta data type based on max and min values.
     - [X] Any interfaces changed?
     NA
     - [X] Any backward compatibility impacted?
     **This issue must be synced to 1.2.0 branch**
     - [X] Document update required?
   
     - [X] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           Added unit test cases to test codec selection
     - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    NA

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mohammadshahidkhan/incubator-carbondata compatibility

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1744.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1744
   
----
commit 296e52c5d022df347a607decc86aa1eacfe8f30b
Author: mohammadshahidkhan <mohdshahidkhan1987@...>
Date:   2017-12-28T12:42:37Z

    [CARBONDATA-1955] Delta DataType calculation is incorrect for long type

----


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2637/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1235/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2459/



---

[qiuchenjian-2]

Github user mohammadshahidkhan commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2485/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1261/



---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1744#discussion_r159467557
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/DefaultEncodingFactory.java ---
    @@ -216,8 +216,13 @@ private static DataType fitDelta(DataType dataType, Object max, Object min) {
         } else if (dataType == DataTypes.INT) {
           value = (long) (int) max - (long) (int) min;
    --- End diff --
   
    Even for all other datatypes has same problem right, I mean there is a chance of giving bigger datatype than original type.


---

[qiuchenjian-2]

Github user mohammadshahidkhan commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1744#discussion_r159654010
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/DefaultEncodingFactory.java ---
    @@ -216,8 +216,13 @@ private static DataType fitDelta(DataType dataType, Object max, Object min) {
         } else if (dataType == DataTypes.INT) {
           value = (long) (int) max - (long) (int) min;
    --- End diff --
   
    As per our discussion for other dataTypes overflow handling not required.


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1744
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1744


---