Login  Register

[GitHub] [carbondata] BrooksLI commented on a change in pull request #3917: [CARBONDATA-3978] Clean files refactor and added support for a trash folder where all the carbondata files will be copied to after triggering clean files on Deleted/Compacted segments

Posted by GitBox on Sep 14, 2020; 1:24am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-vikramahuja1001-opened-a-new-pull-request-3917-WIP-clean-files-refactor-tp100002p100286.html


BrooksLI commented on a change in pull request #3917:
URL: https://github.com/apache/carbondata/pull/3917#discussion_r486292381



##########
File path: core/src/main/java/org/apache/carbondata/core/metadata/SegmentFileStore.java
##########
@@ -1106,19 +1107,39 @@ public static void cleanSegments(CarbonTable table, List<PartitionSpec> partitio
    */
   public static void deleteSegment(String tablePath, Segment segment,
       List<PartitionSpec> partitionSpecs,
-      SegmentUpdateStatusManager updateStatusManager) throws Exception {
+      SegmentUpdateStatusManager updateStatusManager, String tableName, String DatabaseName)
+          throws Exception {
     SegmentFileStore fileStore = new SegmentFileStore(tablePath, segment.getSegmentFileName());
     List<String> indexOrMergeFiles = fileStore.readIndexFiles(SegmentStatus.SUCCESS, true,
         FileFactory.getConfiguration());
     Map<String, List<String>> indexFilesMap = fileStore.getIndexFilesMap();
     for (Map.Entry<String, List<String>> entry : indexFilesMap.entrySet()) {
+      // If the file to be deleted is a carbondata file, copy that file to the trash folder.
+      if (entry.getKey().endsWith(".carbondata")) {

Review comment:
       Can we extract the method rather than duplicate it




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]