[GitHub] carbondata pull request #1977: [CARBONDATA-2181] Thread Leak during compacti...

[qiuchenjian-2]

GitHub user sounakr opened a pull request:

    https://github.com/apache/carbondata/pull/1977

    [CARBONDATA-2181] Thread Leak during compaction processing on restructured table

    Thread Leak during compaction processing on restructured table. The executor service which is being initialised has to be closed once processing finish or else error encountered.
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [ ] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sounakr/incubator-carbondata threadset

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1977.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1977
   
----
commit c12c9b177f075b1913cbe527ae1e9c72d34c7fbf
Author: sounakr <sounakr@...>
Date:   2018-02-09T11:42:50Z

    [CARBONDATA-2181] Thread Leak during compaction processing on restructured table.

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3738/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2498/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3536/



---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1977#discussion_r170169527
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/DimensionRawColumnChunk.java ---
    @@ -49,7 +49,8 @@ public DimensionRawColumnChunk(int columnIndex, ByteBuffer rawData, long offSet,
        * Convert all raw data with all pages to processed DimensionColumnDataChunk's
        * @return
        */
    -  public DimensionColumnDataChunk[] convertToDimColDataChunks() {
    +  public DimensionColumnDataChunk[] convertToDimColDataChunks()
    +      throws IOException, MemoryException, RuntimeException {
    --- End diff --
   
    wrap to single exception and throw it, don't throw many exceptions


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    @sounakr I don't think it is required to do so many file changes to change exception throwing to fix the issue. I think change in `CompactionResultSortProcessor.java`  would be enough to fix this issue.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3034/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1977
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4278/



---

[qiuchenjian-2]

Github user sounakr closed the pull request at:

    https://github.com/apache/carbondata/pull/1977


---