GitHub user manishgupta88 opened a pull request:
https://github.com/apache/carbondata/pull/2499 [CARBONDATA-2648] Fixed NPE issue with legacy store when CACHE_LEVEL is Blocklet Things done as part of this PR: 1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap. 2. Added clearing of SegmentProperties cache holder from executor **Problem 1:** Null pointer exception thrown when store is of <= 1.1 version and DataMap is of type BlockletDataMap. **Analysis:** In BlcokletDataMap schema is created to consider blockletInfo while adding to unsafe but in case of legacy store we dont weite the blocklet Info. This lead to Null pointer exception while calculating the size of row during adding to unsafe. **Solution** For legacy store always call super class methods which is BlockDataMap. - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? NA - [ ] Document update required? No - [ ] Testing done Manually verified - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/manishgupta88/carbondata configurable_cache_columns Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2499.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2499 ---- commit edbff70ed7c650c6aa36ac397c1c86c2bad3fb1d Author: manishgupta88 <tomanishgupta18@...> Date: 2018-07-12T16:05:45Z 1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap. 2. Added clearing of SegmentProperties cache holder from executor ---- --- |
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7069/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5848/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7078/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2499 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5811/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5855/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2499#discussion_r202253245 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java --- @@ -35,18 +35,34 @@ // holds the reference to CarbonTable private CarbonTable carbonTable; + /** + * flag to specify whether to load table block metadata in unsafe or safe. Default value is true + */ + private boolean addTableBlockToUnsafe = true; public TableBlockIndexUniqueIdentifierWrapper( TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) { this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier; this.carbonTable = carbonTable; } + public TableBlockIndexUniqueIdentifierWrapper( --- End diff -- What is the use of this constructor when no body uses it? --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2499#discussion_r202278145 --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java --- @@ -35,18 +35,34 @@ // holds the reference to CarbonTable private CarbonTable carbonTable; + /** + * flag to specify whether to load table block metadata in unsafe or safe. Default value is true + */ + private boolean addTableBlockToUnsafe = true; public TableBlockIndexUniqueIdentifierWrapper( TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) { this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier; this.carbonTable = carbonTable; } + public TableBlockIndexUniqueIdentifierWrapper( --- End diff -- As discussed I have added a Note --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2499 LGTM --- |
In reply to this post by qiuchenjian-2
|
Free forum by Nabble | Edit this page |