Posted by
GitBox on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-ShreelekhyaG-opened-a-new-pull-request-3988-WIP-Clean-index-files-when-clean-filesd-tp102150p107516.html
ShreelekhyaG commented on a change in pull request #3988:
URL:
https://github.com/apache/carbondata/pull/3988#discussion_r611628742##########
File path: core/src/main/java/org/apache/carbondata/core/writer/CarbonIndexFileMergeWriter.java
##########
@@ -184,30 +185,49 @@ private String mergeCarbonIndexFilesOfSegment(String segmentId,
return indexLocationMap;
}
- private String writeMergeIndexFileBasedOnSegmentFolder(List<String> indexFileNamesTobeAdded,
- boolean readFileFooterFromCarbonDataFile, String segmentPath, CarbonFile[] indexFiles,
- String segmentId, String uuid) throws IOException {
+ public String writeMergeIndexFileBasedOnSegmentFolder(List<String> indexFileNamesTobeAdded,
+ boolean isOldStoreIndexFilesPresent, String segmentPath,
+ String segmentId, String uuid, boolean readBasedOnUUID) throws IOException {
+ CarbonFile[] indexFiles = null;
SegmentIndexFileStore fileStore = new SegmentIndexFileStore();
- if (readFileFooterFromCarbonDataFile) {
+ if (isOldStoreIndexFilesPresent) {
// this case will be used in case of upgrade where old store will not have the blocklet
// info in the index file and therefore blocklet info need to be read from the file footer
// in the carbondata file
- fileStore.readAllIndexAndFillBlockletInfo(segmentPath, uuid);
+ fileStore.readAllIndexAndFillBlockletInfo(segmentPath, null);
Review comment:
`readAllIndexAndFillBlockletInfo` is being used when old store is present. and this old store may not always match with uuid. In addHivePartition flow, we can have index files with different timestamps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[hidden email]