[GitHub] carbondata pull request #1739: [CARBONDATA-1834] when running insert overwri...

[qiuchenjian-2]

GitHub user kushalsaha opened a pull request:

    https://github.com/apache/carbondata/pull/1739

    [CARBONDATA-1834] when running insert overwrite task and parallely executing select query task throwing FileNotFoundException so we modify the log message

    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [ ] Any interfaces changed?
     No
     - [ ] Any backward compatibility impacted?
     No
     - [ ] Document update required?
    No
     - [ ] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           Yes
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kushalsaha/carbondata 29Dec_FOFissue

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1739.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1739
   
----
commit 682d8d74d841e6e21f7b0c414e162fee9efe525c
Author: akashrn5 <akashnilugal@...>
Date:   2017-12-28T08:13:15Z

    fix select * issue after compaction, delete and clean files operation

commit f7df897bcc3962dd45a84ecb5492398f633e776c
Author: kushalsaha <kushalsaha1988@...>
Date:   2017-12-29T05:36:46Z

    CARBONDATA-1834 added log message along with FOF exception

----


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2617/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2618/



---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159034285
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
    --- End diff --
   
    use new InterruptedException (msg, e) to add `e` itself to the InterruptedException object


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159034311
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
             }
             ext = ext.getCause();
           }
           throw new InterruptedException(e.getMessage());
    --- End diff --
   
    use new InterruptedException(msg, e) to add e itself to the InterruptedException object


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159034322
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
             }
             ext = ext.getCause();
           }
           throw new InterruptedException(e.getMessage());
         }
    +    catch (Exception e) {
    +      Throwable ext = e;
    +      while (ext != null) {
    +        if (ext instanceof FileNotFoundException) {
    +          LOGGER.error(e);
    +          throw new InterruptedException(
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
    --- End diff --
   
    use new InterruptedException(msg, e) to add e itself to the InterruptedException object


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159034333
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
             }
             ext = ext.getCause();
           }
           throw new InterruptedException(e.getMessage());
         }
    +    catch (Exception e) {
    --- End diff --
   
    move to previous line, please follow this in future


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159034422
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
             }
             ext = ext.getCause();
           }
           throw new InterruptedException(e.getMessage());
         }
    +    catch (Exception e) {
    --- End diff --
   
    This catch handling is the same as previous catch?


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2431/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1205/



---

[qiuchenjian-2]

Github user kushalsaha commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159037847
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
    --- End diff --
   
    InterruptedException takes only one parameter


---

[qiuchenjian-2]

Github user kushalsaha commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1739#discussion_r159039052
 
    --- Diff: integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java ---
    @@ -121,12 +126,24 @@ public VectorizedCarbonRecordReader(QueryModel queryModel, InputMetricsStats inp
           while (ext != null) {
             if (ext instanceof FileNotFoundException) {
               throw new InterruptedException(
    -              e.getMessage() + ". insert overwrite may be in progress.Please check");
    +              "Insert overwrite may be in progress.Please check " + e.getMessage());
             }
             ext = ext.getCause();
           }
           throw new InterruptedException(e.getMessage());
         }
    +    catch (Exception e) {
    --- End diff --
   
    this catch block is for handling any checked exception if its thrown in the form of Runtime exception . In this case FileNotFoundException is thrown by casting as Runtime exception.


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2622/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2438/



---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1739


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1739
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1215/



---