[GitHub] carbondata pull request #2274: [CARBONDATA-2440] default value of ENABLE_OFF...

classic Classic list List threaded Threaded
71 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2274#discussion_r189755781
 
    --- Diff: docs/sdk-writer-guide.md ---
    @@ -13,25 +13,33 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
     
      import org.apache.carbondata.common.exceptions.sql.InvalidLoadOptionException;
      import org.apache.carbondata.core.metadata.datatype.DataTypes;
    + import org.apache.carbondata.core.util.CarbonProperties;
      import org.apache.carbondata.sdk.file.CarbonWriter;
      import org.apache.carbondata.sdk.file.CarbonWriterBuilder;
      import org.apache.carbondata.sdk.file.Field;
      import org.apache.carbondata.sdk.file.Schema;
     
      public class TestSdk {
    -
    +
    +   // pass true or false while executing the main to use offheap memory or not
        public static void main(String[] args) throws IOException, InvalidLoadOptionException {
    -     testSdkWriter();
    +     if (args.length > 0 && args[0] != null) {
    +       testSdkWriter(args[0]);
    +     } else {
    +       testSdkWriter("true");
    +     }
        }
     
    -   public static void testSdkWriter() throws IOException, InvalidLoadOptionException {
    -     String path = "/home/root1/Documents/ab/temp";
    +   public static void testSdkWriter(String enableOffheap) throws IOException, InvalidLoadOptionException {
    +     String path = "./target/testCSVSdkWriter";
     
          Field[] fields = new Field[2];
          fields[0] = new Field("name", DataTypes.STRING);
          fields[1] = new Field("age", DataTypes.INT);
     
          Schema schema = new Schema(fields);
    +
    +     CarbonProperties.getInstance().addProperty("enable.offheap.sort", enableOffheap);
    --- End diff --
   
    The value of "enable.offheap.sort" will transform to false when args[0] not equal to true, including "false" and other string, like "f","any" and so on.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2274#discussion_r189755954
 
    --- Diff: docs/sdk-writer-guide.md ---
    @@ -334,6 +342,39 @@ public Schema(Field[] fields);
     public static Schema parseJson(String json);
     ```
     
    +### Class org.apache.carbondata.core.util.CarbonProperties
    +
    +```
    +/**
    +* This method will be responsible to get the instance of CarbonProperties class
    +*
    +* @return carbon properties instance
    +*/
    +public static CarbonProperties getInstance();
    +```
    +
    +```
    +/**
    +* This method will be used to add a new property
    +*
    +* @param key
    +* @return CarbonProperties object
    --- End diff --
   
    should add @param value  and describe


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2274#discussion_r189756081
 
    --- Diff: docs/sdk-writer-guide.md ---
    @@ -334,6 +342,39 @@ public Schema(Field[] fields);
     public static Schema parseJson(String json);
     ```
     
    +### Class org.apache.carbondata.core.util.CarbonProperties
    +
    +```
    +/**
    +* This method will be responsible to get the instance of CarbonProperties class
    +*
    +* @return carbon properties instance
    +*/
    +public static CarbonProperties getInstance();
    +```
    +
    +```
    +/**
    +* This method will be used to add a new property
    +*
    +* @param key
    +* @return CarbonProperties object
    +*/
    +public CarbonProperties addProperty(String key, String value);
    +```
    +
    +```
    +/**
    +* This method will be used to get the property value. If property is not
    +* present, then it will return the default value.
    +*
    +* @param key
    +* @return properties value
    --- End diff --
   
    Please add @param defaultValue and related describe.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2274: [CARBONDATA-2440] doc updated to set the property fo...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2274
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5036/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user rahulforallp commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2274#discussion_r189778890
 
    --- Diff: docs/sdk-writer-guide.md ---
    @@ -13,25 +13,33 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
     
      import org.apache.carbondata.common.exceptions.sql.InvalidLoadOptionException;
      import org.apache.carbondata.core.metadata.datatype.DataTypes;
    + import org.apache.carbondata.core.util.CarbonProperties;
      import org.apache.carbondata.sdk.file.CarbonWriter;
      import org.apache.carbondata.sdk.file.CarbonWriterBuilder;
      import org.apache.carbondata.sdk.file.Field;
      import org.apache.carbondata.sdk.file.Schema;
     
      public class TestSdk {
    -
    +
    +   // pass true or false while executing the main to use offheap memory or not
        public static void main(String[] args) throws IOException, InvalidLoadOptionException {
    -     testSdkWriter();
    +     if (args.length > 0 && args[0] != null) {
    +       testSdkWriter(args[0]);
    +     } else {
    +       testSdkWriter("true");
    +     }
        }
     
    -   public static void testSdkWriter() throws IOException, InvalidLoadOptionException {
    -     String path = "/home/root1/Documents/ab/temp";
    +   public static void testSdkWriter(String enableOffheap) throws IOException, InvalidLoadOptionException {
    +     String path = "./target/testCSVSdkWriter";
     
          Field[] fields = new Field[2];
          fields[0] = new Field("name", DataTypes.STRING);
          fields[1] = new Field("age", DataTypes.INT);
     
          Schema schema = new Schema(fields);
    +
    +     CarbonProperties.getInstance().addProperty("enable.offheap.sort", enableOffheap);
    --- End diff --
   
    @xubo245 if args[0] is specified as any other value except true or false then it will log warning message and will set default value.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2274: [CARBONDATA-2440] doc updated to set the property fo...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2274
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4874/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2274#discussion_r189787899
 
    --- Diff: docs/sdk-writer-guide.md ---
    @@ -13,25 +13,33 @@ These SDK writer output contains just a carbondata and carbonindex files. No met
     
      import org.apache.carbondata.common.exceptions.sql.InvalidLoadOptionException;
      import org.apache.carbondata.core.metadata.datatype.DataTypes;
    + import org.apache.carbondata.core.util.CarbonProperties;
      import org.apache.carbondata.sdk.file.CarbonWriter;
      import org.apache.carbondata.sdk.file.CarbonWriterBuilder;
      import org.apache.carbondata.sdk.file.Field;
      import org.apache.carbondata.sdk.file.Schema;
     
      public class TestSdk {
    -
    +
    +   // pass true or false while executing the main to use offheap memory or not
        public static void main(String[] args) throws IOException, InvalidLoadOptionException {
    -     testSdkWriter();
    +     if (args.length > 0 && args[0] != null) {
    +       testSdkWriter(args[0]);
    +     } else {
    +       testSdkWriter("true");
    +     }
        }
     
    -   public static void testSdkWriter() throws IOException, InvalidLoadOptionException {
    -     String path = "/home/root1/Documents/ab/temp";
    +   public static void testSdkWriter(String enableOffheap) throws IOException, InvalidLoadOptionException {
    +     String path = "./target/testCSVSdkWriter";
     
          Field[] fields = new Field[2];
          fields[0] = new Field("name", DataTypes.STRING);
          fields[1] = new Field("age", DataTypes.INT);
     
          Schema schema = new Schema(fields);
    +
    +     CarbonProperties.getInstance().addProperty("enable.offheap.sort", enableOffheap);
    --- End diff --
   
    It's different,validateEnableUnsafeSort method validate "enable.unsafe.sort", not "enable.offheap.sort".


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2274: [CARBONDATA-2440] doc updated to set the property fo...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2274
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6033/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2274: [CARBONDATA-2440] doc updated to set the property fo...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2274
 
    retest this please


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2274: [CARBONDATA-2440] doc updated to set the property fo...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2274
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2274: [CARBONDATA-2440] doc updated to set the prop...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2274


---
1234