[GitHub] carbondata pull request #1302: [CARBONDATA-1435] Fix reader backward compati...

[qiuchenjian-2]

GitHub user jackylk opened a pull request:

    https://github.com/apache/carbondata/pull/1302

    [CARBONDATA-1435] Fix reader backward compatible issue

    Using master code, it is having NPE when reading old carbondata file. This PR fixed it

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jackylk/incubator-carbondata fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1302.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1302
   
----
commit 1859e52a79df5967009bbdc8660b609806b76b4c
Author: Jacky Li <[hidden email]>
Date:   2017-08-30T15:56:12Z

    wip

commit 720caa7d4a7871666cd195daed1cd631e9c765de
Author: Jacky Li <[hidden email]>
Date:   2017-08-30T16:15:37Z

    fix compatibility

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/1302
 
    retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1302#discussion_r136224665
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingStrategy.java ---
    @@ -112,10 +112,41 @@ public ColumnPageDecoder createDecoderLegacy(ValueEncoderMeta metadata) {
           case SHORT:
           case INT:
           case LONG:
    -        return DefaultEncodingStrategy.selectCodecByAlgorithmForIntegral(stats).createDecoder(null);
    +        // create the codec based on algorithm and create decoder by recovering the metadata
    +        ColumnPageCodec codec = DefaultEncodingStrategy.selectCodecByAlgorithmForIntegral(stats);
    +        if (codec instanceof AdaptiveIntegralCodec) {
    +          AdaptiveIntegralCodec adaptiveCodec = (AdaptiveIntegralCodec) codec;
    +          AdaptiveIntegralEncoderMeta meta = new AdaptiveIntegralEncoderMeta(
    --- End diff --
   
    The parameter sequence of `DataType`, `stats`,`compressorName` are different from that in `AdaptiveDeltaIntegralEncoderMeta` and `DirectCompressorEncoderMeta`. It would be better to keep them same.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1302
 
    SDV Build Success with Spark 2.1, Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/337/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1302#discussion_r136251781
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingStrategy.java ---
    @@ -112,10 +112,41 @@ public ColumnPageDecoder createDecoderLegacy(ValueEncoderMeta metadata) {
           case SHORT:
           case INT:
           case LONG:
    -        return DefaultEncodingStrategy.selectCodecByAlgorithmForIntegral(stats).createDecoder(null);
    +        // create the codec based on algorithm and create decoder by recovering the metadata
    +        ColumnPageCodec codec = DefaultEncodingStrategy.selectCodecByAlgorithmForIntegral(stats);
    +        if (codec instanceof AdaptiveIntegralCodec) {
    +          AdaptiveIntegralCodec adaptiveCodec = (AdaptiveIntegralCodec) codec;
    +          AdaptiveIntegralEncoderMeta meta = new AdaptiveIntegralEncoderMeta(
    --- End diff --
   
    It is  because AdaptiveIntegralEncoderMeta constructor is different from other. I will modify this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1302
 
    SDV Build Failed with Spark 2.1, Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/352/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1302


---