[GitHub] [carbondata] kunal642 opened a new pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

[GitBox]

kunal642 opened a new pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941


    ### Why is this PR needed?
    FileNotFoundException being thrown in hive during insert.
   
    ### What changes were proposed in this PR?
   Avoid calling getAbsolutePath as the path sent as parameters is not present
       
    ### Does this PR introduce any user interface change?
    - No
    - Yes. (please explain the change and update document)
   
    ### Is any new testcase added?
    - No
    - Yes
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695922622






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695927232


   @kunal642 the PR description is not so clear, can you please elaborate little bit for better understanding of the issue and the fix?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695946242


   @akashrn5 done


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695946399


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695949306


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2398/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695949845


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4139/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696040616


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2403/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696051210


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4144/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696060168


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696118453


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4150/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696120884


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2409/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695922622






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695946242






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-695927232


   @kunal642 the PR description is not so clear, can you please elaborate little bit for better understanding of the issue and the fix?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696511140


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on a change in pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#discussion_r492488092



##########
File path: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##########
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration configuration, String pa
     try {
       splitList = carbonInputFormat.getSplits(jobContext);
     } catch (IOException ex) {
-      if (ex.getMessage().contains("No Index files are present in the table location :")) {
+      LOGGER.error("Unable to get splits: ", ex);
+      if (ex.getMessage().contains("No Index files are present in the table location :") ||

Review comment:
       can you add UT for these, so that later if anyone changes message, they should take care here.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

kunal642 commented on a change in pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#discussion_r492497322



##########
File path: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##########
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration configuration, String pa
     try {
       splitList = carbonInputFormat.getSplits(jobContext);
     } catch (IOException ex) {
-      if (ex.getMessage().contains("No Index files are present in the table location :")) {
+      LOGGER.error("Unable to get splits: ", ex);
+      if (ex.getMessage().contains("No Index files are present in the table location :") ||

Review comment:
       This new added scenario is only failing in cluster




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on a change in pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#discussion_r492504326



##########
File path: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##########
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration configuration, String pa
     try {
       splitList = carbonInputFormat.getSplits(jobContext);
     } catch (IOException ex) {
-      if (ex.getMessage().contains("No Index files are present in the table location :")) {
+      LOGGER.error("Unable to get splits: ", ex);
+      if (ex.getMessage().contains("No Index files are present in the table location :") ||
+          ex.getMessage().contains("CarbonData file is not present in the table location")) {

Review comment:
       As i can see from code, `CarbonData file is not present in the table location` is a FIleNotFoundException message, not IO, so this condition will be always false?
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696565124


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4160/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]