[GitHub] carbondata pull request #2391: [CARBONDATA-2625] Optimize the performance of...

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6511/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5429/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5370/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6541/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5445/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6542/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5446/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5371/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6567/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5399/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6574/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5403/



---

[qiuchenjian-2]

Github user KanakaKumar commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2391#discussion_r198715511
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
    @@ -1744,6 +1744,28 @@
       @InterfaceStability.Unstable
       public static final String CARBON_SEARCH_MODE_SCAN_THREAD = "carbon.search.scan.thread";
     
    +  /**
    +   * The size of thread pool used for reading files in Work for CarbonReader. By default,
    +   * it is number of cores in Worker
    +   */
    +  @CarbonProperty
    +  @InterfaceStability.Unstable
    +  public static final String CARBON_READER_THREAD = "carbon.reader.thread";
    +
    +  /**
    +   * Whether it support CarbonReader, default value is false
    +   */
    +  @CarbonProperty
    +  @InterfaceStability.Unstable
    +  public static final String CARBON_READER_SUPPORT = "carbon.reader.support";
    --- End diff --
   
    Does this mean enable SDK reader ? If yes, please change property name to enable.sdk.query.executor.
   



---

[qiuchenjian-2]

Github user KanakaKumar commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2391#discussion_r198716005
 
    --- Diff: integration/spark-common-cluster-test/src/test/scala/org/apache/spark/sql/common/util/QueryTest.scala ---
    @@ -45,6 +45,7 @@ class QueryTest extends PlanTest with Suite {
       Locale.setDefault(Locale.US)
       CarbonProperties.getInstance()
         .addProperty(CarbonCommonConstants.VALIDATE_DIRECT_QUERY_ON_DATAMAP, "false")
    +    .addProperty(CarbonCommonConstants.CARBON_READER_SUPPORT, "false");
    --- End diff --
   
    All these test (DataLoadingTestCase , QueryTest) are not related to SDK flow.  Why they are impacted?
    If its becasue of the flag is not cleared, I think its better to use ThreadLocal to store sdk flag and construct the SDKReader instead of CARBON_READER_SUPPORT parameter.


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2391#discussion_r199081225
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
    @@ -1744,6 +1744,28 @@
       @InterfaceStability.Unstable
       public static final String CARBON_SEARCH_MODE_SCAN_THREAD = "carbon.search.scan.thread";
     
    +  /**
    +   * The size of thread pool used for reading files in Work for CarbonReader. By default,
    +   * it is number of cores in Worker
    +   */
    +  @CarbonProperty
    +  @InterfaceStability.Unstable
    +  public static final String CARBON_READER_THREAD = "carbon.reader.thread";
    +
    +  /**
    +   * Whether it support CarbonReader, default value is false
    +   */
    +  @CarbonProperty
    +  @InterfaceStability.Unstable
    +  public static final String CARBON_READER_SUPPORT = "carbon.reader.support";
    --- End diff --
   
    yeah, ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2391#discussion_r199083856
 
    --- Diff: integration/spark-common-cluster-test/src/test/scala/org/apache/spark/sql/common/util/QueryTest.scala ---
    @@ -45,6 +45,7 @@ class QueryTest extends PlanTest with Suite {
       Locale.setDefault(Locale.US)
       CarbonProperties.getInstance()
         .addProperty(CarbonCommonConstants.VALIDATE_DIRECT_QUERY_ON_DATAMAP, "false")
    +    .addProperty(CarbonCommonConstants.CARBON_READER_SUPPORT, "false");
    --- End diff --
   
    I don't know why  it show SDV fail. It shouldn't clean all.
   
    ok, I changed.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6640/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6642/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2391
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5468/



---