[GitHub] carbondata pull request #1454: [CARBONDATA-1658] Fixed Thread leak issue in ...

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1454: [CARBONDATA-1658] Fixed Thread leak issue in ...

qiuchenjian-2
GitHub user kumarvishal09 opened a pull request:

    https://github.com/apache/carbondata/pull/1454

    [CARBONDATA-1658] Fixed Thread leak issue in no sort

    **Problem**: In case of no sort executor service is not shutting down in writer step which is causing thread leak. In case of long run it will throwing OOM error
    **Solution:**: Need to shutdown executor service in all the case success and failure


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kumarvishal09/incubator-carbondata insertintothreadleak

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1454.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1454
   
----
commit 3c99905ab1f129d8d9af4f84d25b2345be57a27c
Author: kumarvishal <[hidden email]>
Date:   2017-10-30T15:22:19Z

    Fixed Thread leak issue in no sort

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1454: [CARBONDATA-1658] Fixed Thread leak issue in no sort

qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1454
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/753/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1454: [CARBONDATA-1658] Fixed Thread leak issue in no sort

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1454
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1390/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1454: [CARBONDATA-1658] Fixed Thread leak issue in ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1454#discussion_r147769643
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestNoSortDataLoadUsingInsert.scala ---
    @@ -0,0 +1,54 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.spark.testsuite.dataload
    +
    +import java.io.{BufferedWriter, File, FileWriter, FilenameFilter}
    +
    +import org.apache.spark.sql.Row
    +import org.apache.spark.sql.test.util.QueryTest
    +import org.scalatest.BeforeAndAfterAll
    +
    +import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.util.CarbonProperties
    +
    +class TestNoSortDataLoadUsingInsert extends QueryTest with BeforeAndAfterAll {
    --- End diff --
   
    can you add this in TestSortColumns, so that no new test suite is needed


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1454: [CARBONDATA-1658] Fixed Thread leak issue in ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1454#discussion_r147903166
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestNoSortDataLoadUsingInsert.scala ---
    @@ -0,0 +1,54 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.spark.testsuite.dataload
    +
    +import java.io.{BufferedWriter, File, FileWriter, FilenameFilter}
    +
    +import org.apache.spark.sql.Row
    +import org.apache.spark.sql.test.util.QueryTest
    +import org.scalatest.BeforeAndAfterAll
    +
    +import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.util.CarbonProperties
    +
    +class TestNoSortDataLoadUsingInsert extends QueryTest with BeforeAndAfterAll {
    --- End diff --
   
    ok


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1454: [CARBONDATA-1658] Fixed Thread leak issue in no sort

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1454
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/763/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1454: [CARBONDATA-1658] Fixed Thread leak issue in no sort

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1454
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1400/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1454: [CARBONDATA-1658] Fixed Thread leak issue in ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1454


---