[GitHub] [carbondata] jackylk opened a new pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error

[GitBox]

jackylk commented on a change in pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#discussion_r364533050
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
 ##########
 @@ -714,4 +714,16 @@ public static String generateBadRecordsPath(String badLogStoreLocation, String s
           CarbonCommonConstants.FILE_SEPARATOR + taskNo;
     }
   }
+
+  /**
+   * Return the parent path of the input file
+   */
+  public static String getParentPath(String dataFilePath) {
+    int endIndex = dataFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
 
 Review comment:
   I updated the comment in this function

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-572369871
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1551/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#discussion_r364570295
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/reader/CarbonIndexFileReader.java
 ##########
 @@ -112,4 +112,5 @@ public void openThriftReader(byte[] fileData) throws IOException {
   public boolean hasNext() throws IOException {
     return thriftReader.hasNext();
   }
+
 
 Review comment:
   please revert this change

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#discussion_r364571797
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
 ##########
 @@ -714,4 +714,18 @@ public static String generateBadRecordsPath(String badLogStoreLocation, String s
           CarbonCommonConstants.FILE_SEPARATOR + taskNo;
     }
   }
+
+  /**
+   * Return the parent path of the input file.
+   * For example, if input file path is /user/warehouse/t1/file.carbondata
+   * then return will be /user/warehouse/t1
+   */
+  public static String getParentPath(String dataFilePath) {
+    int endIndex = dataFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+    if (endIndex > -1) {
 
 Review comment:
   can we replace if else with conditional operator here?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#discussion_r364712733
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
 ##########
 @@ -714,4 +714,18 @@ public static String generateBadRecordsPath(String badLogStoreLocation, String s
           CarbonCommonConstants.FILE_SEPARATOR + taskNo;
     }
   }
+
+  /**
+   * Return the parent path of the input file.
+   * For example, if input file path is /user/warehouse/t1/file.carbondata
+   * then return will be /user/warehouse/t1
+   */
+  public static String getParentPath(String dataFilePath) {
+    int endIndex = dataFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR);
+    if (endIndex > -1) {
 
 Review comment:
   I think this is better, especially for debugging

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#discussion_r364712006
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/reader/CarbonIndexFileReader.java
 ##########
 @@ -112,4 +112,5 @@ public void openThriftReader(byte[] fileData) throws IOException {
   public boolean hasNext() throws IOException {
     return thriftReader.hasNext();
   }
+
 
 Review comment:
   fixed

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-572577165
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1571/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

QiangCai commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-573292476
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-574986035
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-574987391
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1650/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-575228056
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-575265799
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1670/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on issue #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561#issuecomment-577008432
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3561: [HOTFIX] Fix INSERT STAGE footer read error
URL: https://github.com/apache/carbondata/pull/3561
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services