Posted by
GitBox on
Mar 12, 2021; 7:31am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-jack86596-opened-a-new-pull-request-4105-CARBONDATA-4148-Reindex-failed-when-SI-hae-tp106768.html
jack86596 opened a new pull request #4105:
URL:
https://github.com/apache/carbondata/pull/4105 ### Why is this PR needed?
Reindex failed when SI has stale carbonindexmerge file, throw exception FileNotFoundException. This is because SegmentFileStore.getIndexFiles stores the mapping of indexfile to indexmergefile, when stale carbon indexmergefile exists, indexmergefile will not be null. During merging index file, new indexmergefile will be created with same name as before in the same location. At the end of CarbonIndexFileMergeWriter.writeMergeIndexFileBasedOnSegmentFile, carbon index file will be deleted. Since indexmergefile is stored in the indexFiles list, newly created indexmergefile will be delete also, which leads to FileNotFoundException.
### What changes were proposed in this PR?
1. SegmentFileStore.getIndexFiles stores the mapping of indexfile to indexmergefile which is redundant.
2. SegmentFileStore.getIndexOrMergeFiles returns both index file and index merge file, so function name is incorrect, rename to getIndexAndMergeFiles.
3. CarbonLoaderUtil.getActiveExecutor actually get active node, so function name is incorrect, rename to getActiveNode, together replace all "executor" with "node" in function assignBlocksByDataLocality.
### Does this PR introduce any user interface change?
- No
### Is any new testcase added?
- Yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[hidden email]