[GitHub] carbondata pull request #2683: [WIP] Print data folder information

[qiuchenjian-2]

Github user xuchuanyin commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    LGTM


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2683#discussion_r217322845
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
    @@ -168,6 +168,65 @@ public static Object getMeasureObjectBasedOnDataType(ColumnPage measurePage, int
         }
       }
     
    +  /**
    +   * Calculate data percentage in [min, max] scope based on data type
    +   * @param data data to calculate the percentage
    +   * @param min min value
    +   * @param max max value
    +   * @param column column schema including data type
    +   * @return result
    +   */
    +  public static double computePercentage(byte[] data, byte[] min, byte[] max, ColumnSchema column) {
    --- End diff --
   
    Bette move this method to CI tool only as it is used only there


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2683#discussion_r217323443
 
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/TestUtil.java ---
    @@ -36,38 +36,34 @@
     import org.apache.avro.generic.GenericDatumReader;
     import org.apache.avro.generic.GenericDatumWriter;
     import org.apache.avro.io.DecoderFactory;
    -import org.apache.avro.io.Encoder;
     import org.apache.avro.io.JsonDecoder;
     import org.apache.hadoop.conf.Configuration;
    -import org.junit.Assert;
     
    [e-mail]("Test")
     public class TestUtil {
     
    -  public static Configuration configuration = new Configuration();
    +  public static final Configuration configuration = new Configuration();
    --- End diff --
   
    Why is this test class changed?  How it is related to CLI tool


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2683#discussion_r217589323
 
    --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/TestUtil.java ---
    @@ -36,38 +36,34 @@
     import org.apache.avro.generic.GenericDatumReader;
     import org.apache.avro.generic.GenericDatumWriter;
     import org.apache.avro.io.DecoderFactory;
    -import org.apache.avro.io.Encoder;
     import org.apache.avro.io.JsonDecoder;
     import org.apache.hadoop.conf.Configuration;
    -import org.junit.Assert;
     
    [e-mail]("Test")
     public class TestUtil {
     
    -  public static Configuration configuration = new Configuration();
    +  public static final Configuration configuration = new Configuration();
    --- End diff --
   
    Because now CLI is a separate module and testcase in CLI need to use TestUtil to write carbonfiles using SDK, so I have to move the TestUtil to src so that CLI testcase can use it. Is there other way?


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2683#discussion_r217589388
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
    @@ -168,6 +168,65 @@ public static Object getMeasureObjectBasedOnDataType(ColumnPage measurePage, int
         }
       }
     
    +  /**
    +   * Calculate data percentage in [min, max] scope based on data type
    +   * @param data data to calculate the percentage
    +   * @param min min value
    +   * @param max max value
    +   * @param column column schema including data type
    +   * @return result
    +   */
    +  public static double computePercentage(byte[] data, byte[] min, byte[] max, ColumnSchema column) {
    --- End diff --
   
    ok



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/280/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8519/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/449/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/281/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8520/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/450/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2683
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2683


---