[GitHub] carbondata pull request #2752: [WIP]Even after carbon file is copied to targ...

[qiuchenjian-2]

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2752#discussion_r220173275
 
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java ---
    @@ -270,12 +270,18 @@ protected void commitCurrentFile(boolean copyInCurrentThread) {
         notifyDataMapBlockEnd();
         CarbonUtil.closeStreams(this.fileOutputStream, this.fileChannel);
         if (!enableDirectlyWriteData2Hdfs) {
    -      if (copyInCurrentThread) {
    -        CarbonUtil.copyCarbonDataFileToCarbonStorePath(carbonDataFileTempPath,
    -            model.getCarbonDataDirectoryPath(), fileSizeInBytes);
    -      } else {
    -        executorServiceSubmitList.add(executorService.submit(
    -            new CompleteHdfsBackendThread(carbonDataFileTempPath)));
    +      try {
    +        if (copyInCurrentThread) {
    +          CarbonUtil.copyCarbonDataFileToCarbonStorePath(carbonDataFileTempPath,
    +              model.getCarbonDataDirectoryPath(), fileSizeInBytes);
    +          FileFactory
    +              .deleteFile(carbonDataFileTempPath, FileFactory.getFileType(carbonDataFileTempPath));
    +        } else {
    +          executorServiceSubmitList
    --- End diff --
   
    When copying in different thread there also we need to delete carbondata file, i think this you have missed , please check


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8725/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/478/



---

[qiuchenjian-2]

Github user xuchuanyin commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    LGTM


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8727/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/657/



---

[qiuchenjian-2]

Github user kumarvishal09 commented on the issue:

    https://github.com/apache/carbondata/pull/2752
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2752


---