[GitHub] [carbondata] akashrn5 opened a new pull request #3837: [wip]remove compressor name from tupleID

[GitBox]

akashrn5 opened a new pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837


    ### Why is this PR needed?
   
   
    ### What changes were proposed in this PR?
   
       
    ### Does this PR introduce any user interface change?
    - No
    - Yes. (please explain the change and update document)
   
    ### Is any new testcase added?
    - No
    - Yes
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657256684


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1617/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657256916


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3358/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657544654


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657552266


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1632/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657559220


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3373/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657624857


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3378/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-657630762


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1637/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-664178336


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-664184402


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3501/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-664283269


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1759/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-668418007


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on a change in pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#discussion_r464843011



##########
File path: core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
##########
@@ -952,10 +952,22 @@ public static String getSegmentBlockNameKey(String segID, String blockName,
     String blockNameWithOutPart = blockName
         .substring(blockName.indexOf(CarbonCommonConstants.HYPHEN) + 1,
             blockName.lastIndexOf(CarbonTablePath.getCarbonDataExtension()));
+    // to remove compressor name
+    int index = blockNameWithOutPart.lastIndexOf(CarbonCommonConstants.POINT);
     if (isPartitionTable) {
-      return blockNameWithOutPart;
+      if (index != -1) {
+        return blockNameWithOutPart.replace(blockNameWithOutPart.substring(index), "");
+      } else {
+        return blockNameWithOutPart;
+      }
+    }
+    if (index != -1) {
+      String blockNameWithoutCompressorName =
+          blockNameWithOutPart.replace(blockNameWithOutPart.substring(index), "");
+      return segID + CarbonCommonConstants.FILE_SEPARATOR + blockNameWithoutCompressorName;
+    } else {
+      return segID + CarbonCommonConstants.FILE_SEPARATOR + blockNameWithOutPart;

Review comment:
       Please add a unit test to validate this functionality




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-668490005


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3590/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-668490907


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1851/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-673577314


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3711/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-673586193


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1972/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on a change in pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#discussion_r470633975



##########
File path: core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
##########
@@ -952,10 +952,22 @@ public static String getSegmentBlockNameKey(String segID, String blockName,
     String blockNameWithOutPart = blockName
         .substring(blockName.indexOf(CarbonCommonConstants.HYPHEN) + 1,
             blockName.lastIndexOf(CarbonTablePath.getCarbonDataExtension()));
+    // to remove compressor name
+    int index = blockNameWithOutPart.lastIndexOf(CarbonCommonConstants.POINT);
     if (isPartitionTable) {
-      return blockNameWithOutPart;
+      if (index != -1) {
+        return blockNameWithOutPart.replace(blockNameWithOutPart.substring(index), "");
+      } else {
+        return blockNameWithOutPart;
+      }
+    }
+    if (index != -1) {
+      String blockNameWithoutCompressorName =
+          blockNameWithOutPart.replace(blockNameWithOutPart.substring(index), "");
+      return segID + CarbonCommonConstants.FILE_SEPARATOR + blockNameWithoutCompressorName;
+    } else {
+      return segID + CarbonCommonConstants.FILE_SEPARATOR + blockNameWithOutPart;

Review comment:
       done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-674083566


   @kunal642 added test case, please review


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3837:
URL: https://github.com/apache/carbondata/pull/3837#issuecomment-674142591


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3726/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]