[GitHub] incubator-carbondata pull request #533: [WIP]Fixed offheap Query Crash issue

[qiuchenjian-2]

GitHub user kumarvishal09 opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/533

    [WIP]Fixed offheap Query Crash issue

    **Problem**
    Query is failing sometime when number of 1 block contains more than one blocklet
    **Reason**
     Offheap memory is getting clear before all the records are accessed
    **Solutiion**
    Once all the records are accessed then clear the memory


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kumarvishal09/incubator-carbondata offheapcrashissue

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/533.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #533
   
----
commit 538024c02ed3a7819fef665da873a111005cb67c
Author: kumarvishal <[hidden email]>
Date:   2017-01-13T04:50:26Z

    fixed offheap crash issue

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/533
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/586/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/533
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/589/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/533#discussion_r96003271
 
    --- Diff: core/src/main/java/org/apache/carbondata/scan/executor/impl/AbstractQueryExecutor.java ---
    @@ -470,6 +477,7 @@ private AggregatorInfo getAggregatorInfoForBlock(QueryModel queryModel,
        */
       @Override public void finish() throws QueryExecutionException {
         CarbonUtil.clearBlockCache(queryProperties.dataBlocks);
    +    queryIterator.close();
    --- End diff --
   
    queryIterator is assigned in subclass?
    better to add a if check here for null value


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/533#discussion_r96003724
 
    --- Diff: core/src/main/java/org/apache/carbondata/scan/executor/impl/DetailQueryExecutor.java ---
    @@ -39,11 +39,12 @@
       public CarbonIterator<BatchResult> execute(QueryModel queryModel)
           throws QueryExecutionException, IOException {
         List<BlockExecutionInfo> blockExecutionInfoList = getBlockExecutionInfos(queryModel);
    -    return new DetailQueryResultIterator(
    +    this.queryIterator= new DetailQueryResultIterator(
    --- End diff --
   
    add space before `=`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/533#discussion_r96071986
 
    --- Diff: core/src/main/java/org/apache/carbondata/scan/executor/impl/DetailQueryExecutor.java ---
    @@ -39,11 +39,12 @@
       public CarbonIterator<BatchResult> execute(QueryModel queryModel)
           throws QueryExecutionException, IOException {
         List<BlockExecutionInfo> blockExecutionInfoList = getBlockExecutionInfos(queryModel);
    -    return new DetailQueryResultIterator(
    +    this.queryIterator= new DetailQueryResultIterator(
    --- End diff --
   
    ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/533#discussion_r96072237
 
    --- Diff: core/src/main/java/org/apache/carbondata/scan/executor/impl/AbstractQueryExecutor.java ---
    @@ -470,6 +477,7 @@ private AggregatorInfo getAggregatorInfoForBlock(QueryModel queryModel,
        */
       @Override public void finish() throws QueryExecutionException {
         CarbonUtil.clearBlockCache(queryProperties.dataBlocks);
    +    queryIterator.close();
    --- End diff --
   
    ok


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/533
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/596/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/533
 
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/533


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---