[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/277/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/399/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1030/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/402/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/279/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1033/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    @jackylk  CI pass


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1362
 
    @jackylk  please review it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890027
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingFactory.java ---
    @@ -92,6 +90,10 @@ public ColumnPageDecoder createDecoder(List<Encoding> encodings, List<ByteBuffer
           RLEEncoderMeta metadata = new RLEEncoderMeta();
           metadata.readFields(in);
           return new RLECodec().createDecoder(metadata);
    +    } else if (encoding == BOOL_BYTE) {
    +      BooleanEncoderMeta metadata=new BooleanEncoderMeta();
    --- End diff --
   
    ok, I have changed the encoding from snappy to RLE


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890183
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/bool/BooleanConvert.java ---
    @@ -0,0 +1,61 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.core.datastore.page.encoding.bool;
    +
    +/**
    + * convert tools for boolean data type
    + */
    +public class BooleanConvert {
    --- End diff --
   
    I think it should be reserved. There are some place invoke it.


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890228
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -187,6 +179,7 @@ public static ColumnPage newPage(TableSpec.ColumnSpec columnSpec, DataType dataT
             case BYTE:
             case SHORT:
             case SHORT_INT:
    +        case BOOLEAN:
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890311
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -271,6 +267,18 @@ private static ColumnPage newIntPage(TableSpec.ColumnSpec columnSpec, int[] intD
         return columnPage;
       }
     
    +  private static ColumnPage newBooleanPage(TableSpec.ColumnSpec columnSpec, byte[] booleanData) {
    +    ColumnPage columnPage = createPage(columnSpec, BOOLEAN, booleanData.length);
    --- End diff --
   
    ok, I use newBytePage to replace newBooleanPage


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890342
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/bool/BooleanEncoderMeta.java ---
    @@ -0,0 +1,40 @@
    +package org.apache.carbondata.core.datastore.page.encoding.bool;
    +
    +import java.io.DataInput;
    +import java.io.DataOutput;
    +import java.io.IOException;
    +
    +import org.apache.carbondata.core.datastore.TableSpec;
    +import org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoderMeta;
    +import org.apache.carbondata.core.datastore.page.statistics.SimpleStatsResult;
    +import org.apache.carbondata.core.metadata.datatype.DataType;
    +import org.apache.carbondata.core.metadata.schema.table.Writable;
    +
    +public class BooleanEncoderMeta extends ColumnPageEncoderMeta implements Writable {
    +  private String compressorName;
    +
    +  public BooleanEncoderMeta() {
    +  }
    +
    +  public BooleanEncoderMeta(TableSpec.ColumnSpec columnSpec, DataType storeDataType,
    +                            SimpleStatsResult stats, String compressorName) {
    +    super(columnSpec,storeDataType,stats,compressorName);
    +    this.compressorName = compressorName;
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890387
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -436,6 +439,11 @@ public void putData(int rowId, Object value) {
       public abstract void putShortInt(int rowId, int value);
     
       /**
    +   * Set boolean value at rowId
    +   */
    +  public abstract void putBoolean(int rowId, boolean value);
    --- End diff --
   
    ok, I have changed it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890406
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -321,6 +315,11 @@ private static ColumnPage newLVBytesPage(TableSpec.ColumnSpec columnSpec,
       public abstract void setShortIntPage(byte[] shortIntData);
     
       /**
    +   * Set boolean values to page
    +   */
    +  public abstract void setBooleanPage(byte[] booleanData);
    --- End diff --
   
    ok, I have changed it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890419
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -485,6 +496,11 @@ private void putNull(int rowId) {
       public abstract int getShortInt(int rowId);
     
       /**
    +   * Get boolean value at rowId
    +   */
    +  public abstract boolean getBoolean(int rowId);
    --- End diff --
   
    ok, I have changed it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890454
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -321,6 +315,11 @@ private static ColumnPage newLVBytesPage(TableSpec.ColumnSpec columnSpec,
       public abstract void setShortIntPage(byte[] shortIntData);
     
       /**
    +   * Set boolean values to page
    +   */
    +  public abstract void setBooleanPage(byte[] booleanData);
    --- End diff --
   
    ok, I have removed it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890685
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
    @@ -530,6 +546,11 @@ private void putNull(int rowId) {
       public abstract byte[] getShortIntPage();
     
       /**
    +   * Get boolean value page
    +   */
    +  public abstract byte[] getBooleanPage();
    --- End diff --
   
    Ok, I have changed it.


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890703
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingFactory.java ---
    @@ -38,11 +38,7 @@
     import org.apache.carbondata.core.util.CarbonUtil;
     import org.apache.carbondata.format.Encoding;
     
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_DELTA_INTEGRAL;
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_FLOATING;
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_INTEGRAL;
    -import static org.apache.carbondata.format.Encoding.DIRECT_COMPRESS;
    -import static org.apache.carbondata.format.Encoding.RLE_INTEGRAL;
    +import static org.apache.carbondata.format.Encoding.*;
    --- End diff --
   
    Ok, I have changed it.


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1362#discussion_r143890779
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingFactory.java ---
    @@ -38,11 +38,7 @@
     import org.apache.carbondata.core.util.CarbonUtil;
     import org.apache.carbondata.format.Encoding;
     
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_DELTA_INTEGRAL;
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_FLOATING;
    -import static org.apache.carbondata.format.Encoding.ADAPTIVE_INTEGRAL;
    -import static org.apache.carbondata.format.Encoding.DIRECT_COMPRESS;
    -import static org.apache.carbondata.format.Encoding.RLE_INTEGRAL;
    +import static org.apache.carbondata.format.Encoding.*;
    --- End diff --
   
    I also changed similar problem in columnPage


---