[GitHub] [carbondata] kunal642 commented on a change in pull request #3719: [CARBONDATA-3754]avoid listing index files during SI rebuild

[GitBox]

kunal642 commented on a change in pull request #3719:
URL: https://github.com/apache/carbondata/pull/3719#discussion_r411152212



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/util/SecondaryIndexUtil.scala
##########
@@ -280,6 +286,23 @@ object SecondaryIndexUtil {
     }
   }
 
+  /**
+   * This method deletes the old index files or merge index file after data files merge
+   */
+  private def deleteOldIndexOrMergeIndexFiles(
+      factTimeStamp: Long,
+      validSegments: util.List[Segment],
+      indexCarbonTable: CarbonTable): Unit = {
+    // delete the index/merge index carbonFile of old data files
+    validSegments.asScala.foreach { segment =>
+      indexCarbonTable.getIndexFilesListForSegment(segment).asScala.filter { indexFile =>
+        DataFileUtil.getTimeStampFromFileName(indexFile).toLong > factTimeStamp
+      }.foreach { indexFile =>

Review comment:
       combine filter and foreach into a single foreach




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]