[GitHub] carbondata pull request #2499: [CARBONDATA-2648] Fixed NPE issue with legacy...

[qiuchenjian-2]

GitHub user manishgupta88 opened a pull request:

    https://github.com/apache/carbondata/pull/2499

    [CARBONDATA-2648] Fixed NPE issue with legacy store when CACHE_LEVEL is Blocklet

    Things done as part of this PR:
    1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
    2. Added clearing of SegmentProperties cache holder from executor
   
    **Problem 1:**
    Null pointer exception thrown when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
   
    **Analysis:**
    In BlcokletDataMap schema is created to consider blockletInfo while adding to unsafe but in case of legacy store we dont weite the blocklet Info. This lead to Null pointer exception while calculating the size of row during adding to unsafe.
   
    **Solution**
    For legacy store always call super class methods which is BlockDataMap.
   
     - [ ] Any interfaces changed?
     No
     - [ ] Any backward compatibility impacted?
     NA
     - [ ] Document update required?
    No
     - [ ] Testing done
    Manually verified
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    NA


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishgupta88/carbondata configurable_cache_columns

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2499.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2499
   
----
commit edbff70ed7c650c6aa36ac397c1c86c2bad3fb1d
Author: manishgupta88 <tomanishgupta18@...>
Date:   2018-07-12T16:05:45Z

    1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
    2. Added clearing of SegmentProperties cache holder from executor

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7069/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5848/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7078/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5811/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5855/



---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2499#discussion_r202253245
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java ---
    @@ -35,18 +35,34 @@
     
       // holds the reference to CarbonTable
       private CarbonTable carbonTable;
    +  /**
    +   * flag to specify whether to load table block metadata in unsafe or safe. Default value is true
    +   */
    +  private boolean addTableBlockToUnsafe = true;
     
       public TableBlockIndexUniqueIdentifierWrapper(
           TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) {
         this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier;
         this.carbonTable = carbonTable;
       }
     
    +  public TableBlockIndexUniqueIdentifierWrapper(
    --- End diff --
   
    What is the use of this constructor when no body uses it?


---

[qiuchenjian-2]

Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2499#discussion_r202278145
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java ---
    @@ -35,18 +35,34 @@
     
       // holds the reference to CarbonTable
       private CarbonTable carbonTable;
    +  /**
    +   * flag to specify whether to load table block metadata in unsafe or safe. Default value is true
    +   */
    +  private boolean addTableBlockToUnsafe = true;
     
       public TableBlockIndexUniqueIdentifierWrapper(
           TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) {
         this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier;
         this.carbonTable = carbonTable;
       }
     
    +  public TableBlockIndexUniqueIdentifierWrapper(
    --- End diff --
   
    As discussed I have added a Note


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2499
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2499


---