[GitHub] carbondata pull request #1206: [CARBONDATA-1337] fix error of 'Problem while...

[qiuchenjian-2]

GitHub user zzcclp opened a pull request:

    https://github.com/apache/carbondata/pull/1206

    [CARBONDATA-1337] fix error of 'Problem while intermediate merging' w…

    …hen loading data
   
    With PR-1198(CARBONDATA-1281), when loading data, there was error occuring, as following:
   
    `2017-07-28 13:10:49,443 - ERROR - org.apache.carbondata.common.logging.impl.StandardLogService.logErrorMessage(StandardLogService.java:143) - pool-17-thread-1 -pool-17-thread-1 Problem while intermediate merging
    org.apache.carbondata.processing.sortandgroupby.exception.CarbonSortKeyAndGroupByException: Problem while getting the file
            at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.initialize(IntermediateFileMerger.java:168)
            at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.call(IntermediateFileMerger.java:110)
            at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.call(IntermediateFileMerger.java:37)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    Caused by: java.io.FileNotFoundException: [Ljava.lang.String;@54218780/carbondata_hundred_million_pr11987419814281939480.merge (No such file or directory)
            at java.io.FileOutputStream.open0(Native Method)
            at java.io.FileOutputStream.open(FileOutputStream.java:270)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
            at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
            at org.apache.carbondata.processing.sortandgroupby.sortdata.IntermediateFileMerger.initialize(IntermediateFileMerger.java:163)
            ... 6 more`
   
    According to @xuchuanyin 's suggestion, it needs to fix the code of method SortIntermediateFileMerger.startIntermediateMerging.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zzcclp/carbondata CARBONDATA-1337

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1206.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1206
   
----
commit 44a08522416fe04278550b994fe16cfcc4a35365
Author: Zhang Zhichao <[hidden email]>
Date:   2017-07-28T08:13:40Z

    [CARBONDATA-1337] fix error of 'Problem while intermediate merging' when loading data
   
    According to xuchuanyin's suggestion, it needs to fix the code of method SortIntermediateFileMerger.startIntermediateMerging.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/642/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3238/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1206#discussion_r130037736
 
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortIntermediateFileMerger.java ---
    @@ -17,6 +17,7 @@
     package org.apache.carbondata.processing.sortandgroupby.sortdata;
     
     import java.io.File;
    +import java.util.Random;
    --- End diff --
   
    this will cause checkstyle problems
   
    `import` should be organized in alphabetical order.
   
    You can let it IDE import automatic instead of manually.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user zzcclp commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1206#discussion_r130038378
 
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/sortandgroupby/sortdata/SortIntermediateFileMerger.java ---
    @@ -17,6 +17,7 @@
     package org.apache.carbondata.processing.sortandgroupby.sortdata;
     
     import java.io.File;
    +import java.util.Random;
    --- End diff --
   
    Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/644/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3239/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user chenliang613 commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    LGTM， thanks for your contribution. @zzcclp


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1206


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/1206
 
    Thanks, @chenliang613 , @xuchuanyin .


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---