Login  Register

[GitHub] [carbondata] Karan980 commented on a change in pull request #4099: [CARBONDATA-4141] Index Server is not caching indexes for external tables with sdk segments.

Posted by GitBox on Mar 09, 2021; 12:28pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-Karan980-opened-a-new-pull-request-4099-CARBONDATA-4141-Index-Server-is-not-cachin-tp106516p106624.html


Karan980 commented on a change in pull request #4099:
URL: https://github.com/apache/carbondata/pull/4099#discussion_r590318643



##########
File path: core/src/main/java/org/apache/carbondata/core/index/IndexUtil.java
##########
@@ -112,15 +115,32 @@ public static IndexJob getIndexJob(Configuration configuration) throws IOExcepti
    */
   private static void executeClearIndexJob(IndexJob indexJob,
       CarbonTable carbonTable, String indexToClear) throws IOException {
-    SegmentStatusManager.ValidAndInvalidSegmentsInfo validAndInvalidSegmentsInfo =
-            getValidAndInvalidSegments(carbonTable, FileFactory.getConfiguration());
-    List<String> invalidSegment = new ArrayList<>();
-    for (Segment segment : validAndInvalidSegmentsInfo.getInvalidSegments()) {
-      invalidSegment.add(segment.getSegmentNo());
+    IndexInputFormat indexInputFormat;
+    if (!carbonTable.isTransactionalTable()) {
+      ReadCommittedScope readCommittedScope =
+          new LatestFilesReadCommittedScope(carbonTable.getTablePath(),
+              FileFactory.getConfiguration());
+      LoadMetadataDetails[] loadMetadataDetails = readCommittedScope.getSegmentList();
+      List<Segment> listOfValidSegments = new ArrayList<>();

Review comment:
       Done

##########
File path: core/src/main/java/org/apache/carbondata/core/index/IndexUtil.java
##########
@@ -112,15 +115,32 @@ public static IndexJob getIndexJob(Configuration configuration) throws IOExcepti
    */
   private static void executeClearIndexJob(IndexJob indexJob,
       CarbonTable carbonTable, String indexToClear) throws IOException {
-    SegmentStatusManager.ValidAndInvalidSegmentsInfo validAndInvalidSegmentsInfo =
-            getValidAndInvalidSegments(carbonTable, FileFactory.getConfiguration());
-    List<String> invalidSegment = new ArrayList<>();
-    for (Segment segment : validAndInvalidSegmentsInfo.getInvalidSegments()) {
-      invalidSegment.add(segment.getSegmentNo());
+    IndexInputFormat indexInputFormat;
+    if (!carbonTable.isTransactionalTable()) {
+      ReadCommittedScope readCommittedScope =
+          new LatestFilesReadCommittedScope(carbonTable.getTablePath(),
+              FileFactory.getConfiguration());
+      LoadMetadataDetails[] loadMetadataDetails = readCommittedScope.getSegmentList();
+      List<Segment> listOfValidSegments = new ArrayList<>();
+      for (LoadMetadataDetails segment : loadMetadataDetails) {

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]