[GitHub] carbondata pull request #2045: [CARBONDATA-2230]Add a path into table path t...

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3862/



---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4223/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2978/



---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    @jackylk @ravipesala please help to review, thanks


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3870/



---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    retest sdv please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2983/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4228/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4234/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2990/



---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3874/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3876/



---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2045#discussion_r174233092
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/locks/CarbonLockUtil.java ---
    @@ -107,4 +114,36 @@ public static int getLockProperty(String property, int defaultValue) {
         }
       }
     
    +  /**
    +   * Currently the segment lock files are not deleted immediately when unlock,
    +   * so it needs to delete expired lock files before delete loads.
    +   */
    +  public static void deleteExpiredSegmentLockFiles(CarbonTable carbonTable) {
    +    LoadMetadataDetails[] details =
    --- End diff --
   
    I think you can better list the segment lock files from locks folder and check the modified time of it and remove . Otherwise if you do as per tablestatus then namenode calls would be many.


---

[qiuchenjian-2]

Github user zzcclp commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2045#discussion_r174338430
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/locks/CarbonLockUtil.java ---
    @@ -107,4 +114,36 @@ public static int getLockProperty(String property, int defaultValue) {
         }
       }
     
    +  /**
    +   * Currently the segment lock files are not deleted immediately when unlock,
    +   * so it needs to delete expired lock files before delete loads.
    +   */
    +  public static void deleteExpiredSegmentLockFiles(CarbonTable carbonTable) {
    +    LoadMetadataDetails[] details =
    --- End diff --
   
    I think it is better to use this method to check the status and last modified time of the segment at the same time. As the size of the tablestatus file is getting larger, it is a problem, there are many places will scan this file, not just here, we need to solve this problem. I have raise a [topic](http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/The-size-of-the-tablestatus-file-is-getting-larger-does-it-impact-the-performance-of-reading-this-fi-td41941.html) to discuss this on mailling list.


---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    retest this please


---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    retest sdv please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3024/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2045
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4268/



---