[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4096: [CARBONDATA-4133] Concurrent Insert Overwrite with static partition on Index server fails

[GitBox]

ShreelekhyaG opened a new pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096


    ### Why is this PR needed?
    Concurrent Insert Overwrite with static partition on Index server fails. When index server and prepriming are enabled, prepriming is triggered even when load fails as it is in finally block.
   
    ### What changes were proposed in this PR?
   Added a check to trigger prepriming when there is no error thrown.
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No, tested in cluster.
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-781972393


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3331/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-781975135


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5090/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-782214347


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3731/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-782214754


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5495/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784179393


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5505/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784187992


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3740/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784287020


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3742/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784287882


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5507/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784915249


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3745/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784915396


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5510/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-784962351


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-785033429


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5512/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-785039014


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3747/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-785703438


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5516/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-785709079


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3751/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-788591088


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3340/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#issuecomment-788594583


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5099/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on a change in pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#discussion_r589197682



##########
File path: core/src/main/java/org/apache/carbondata/core/indexstore/ExtendedBlockletWrapper.java
##########
@@ -121,8 +121,9 @@ public ExtendedBlockletWrapper(List<ExtendedBlocklet> extendedBlockletList, Stri
     DataOutputStream stream = new DataOutputStream(bos);
     try {
       for (ExtendedBlocklet extendedBlocklet : extendedBlockletList) {
+        boolean isExternalPath = !extendedBlocklet.getFilePath().contains(tablePath);

Review comment:
       what if external segment path starts with table path?

##########
File path: core/src/main/java/org/apache/carbondata/core/index/Segment.java
##########
@@ -378,11 +382,10 @@ public void write(DataOutput out) throws IOException {
       out.writeUTF(segmentString);
     }
     out.writeLong(indexSize);
-    if (segmentPath == null) {
-      out.writeBoolean(false);
-    } else {
+    if (isExternalSegment()) {

Review comment:
       No need to have if check. Just "out..writeBoolean(isExternalSegment)"

##########
File path: core/src/main/java/org/apache/carbondata/core/index/Segment.java
##########
@@ -79,7 +79,9 @@
   /**
    * Path of segment where it exists
    */
-  private String segmentPath;
+  private transient String segmentPath;
+
+  private boolean isExternalSegment = false;

Review comment:
       please add comment




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG commented on a change in pull request #4096:
URL: https://github.com/apache/carbondata/pull/4096#discussion_r589272680



##########
File path: core/src/main/java/org/apache/carbondata/core/index/Segment.java
##########
@@ -79,7 +79,9 @@
   /**
    * Path of segment where it exists
    */
-  private String segmentPath;
+  private transient String segmentPath;
+
+  private boolean isExternalSegment = false;

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]