[GitHub] [carbondata] manishnalla1994 commented on a change in pull request #3432: [CARBONDATA-3567] Added validation for compacting mixed format segments

[GitBox]

manishnalla1994 commented on a change in pull request #3432: [CARBONDATA-3567] Added validation for compacting mixed format segments
URL: https://github.com/apache/carbondata/pull/3432#discussion_r342448380
 
 

 ##########
 File path: integration/spark-common/src/main/scala/org/apache/carbondata/api/CarbonStore.scala
 ##########
 @@ -120,9 +120,16 @@ object CarbonStore {
               (-1L, -1L)
             }
           } else {
-            // for batch segment, we can get the data size from table status file directly
-            (if (load.getDataSize == null) -1L else load.getDataSize.toLong,
-              if (load.getIndexSize == null) -1L else load.getIndexSize.toLong)
+            // If the added segment is other than carbon segment then we can only display the
+            // data size and not index size and for batch segment, we can get the data size
+            // from table status file directly
+            if (!load.getFileFormat.equals(FileFormat.COLUMNAR_V3) &&
+                !load.getFileFormat.equals(FileFormat.ROW_V1)) {
+              (if (load.getIndexSize == null) -1L else load.getIndexSize.toLong, -1L)
 
 Review comment:
   The index size is calculated as the size of the files present in the location, so here it gives the correct size of the corresponding parquet/orc files.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services