[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

classic Classic list List threaded Threaded
96 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129516143
 
    --- Diff: integration/spark-common-test/src/test/resources/partition_data.csv ---
    @@ -0,0 +1,27 @@
    +id,vin,logdate,phonenumber,country,area,salary
    --- End diff --
   
    Oh, this file is copied from example package. Maybe I can reduce them and keep only one.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129533186
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
    @@ -308,6 +308,10 @@
       @CarbonProperty
       public static final String NUM_CORES_COMPACTING = "carbon.number.of.cores.while.compacting";
       /**
    +   * Number of cores to be used while alter partition
    +   */
    +  public static final String NUM_CORES_ALT_PARTITION = "carbon.number.of.cores.while.altPartition";
    +  /**
    --- End diff --
   
    Add spaceline


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129534273
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/PartitionInfo.java ---
    @@ -65,6 +65,31 @@ public PartitionInfo(List<ColumnSchema> columnSchemaList, PartitionType partitio
         this.partitionIds = new ArrayList<>();
       }
     
    +  /**
    +   * add partition means split default partition, add in last directly
    --- End diff --
   
    default partition is 0, so why split partition ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129582111
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java ---
    @@ -0,0 +1,95 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.carbondata.core.scan.result.iterator;
    +
    +import org.apache.carbondata.common.CarbonIterator;
    +import org.apache.carbondata.common.logging.LogService;
    +import org.apache.carbondata.common.logging.LogServiceFactory;
    +import org.apache.carbondata.core.scan.result.BatchResult;
    +
    +
    +public class PartitionSpliterRawResultIterator extends CarbonIterator<Object[]> {
    +
    +  private CarbonIterator<BatchResult> iterator;
    +  private BatchResult batch;
    +  private int counter;
    +
    +  /**
    +   * LOGGER
    +   */
    +  private static final LogService LOGGER =
    +      LogServiceFactory.getLogService(PartitionSpliterRawResultIterator.class.getName());
    +
    +  public PartitionSpliterRawResultIterator(CarbonIterator<BatchResult> iterator) {
    +    this.iterator = iterator;
    +  }
    +
    +
    --- End diff --
   
    delete useless space line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129582258
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java ---
    @@ -0,0 +1,95 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.carbondata.core.scan.result.iterator;
    +
    +import org.apache.carbondata.common.CarbonIterator;
    +import org.apache.carbondata.common.logging.LogService;
    +import org.apache.carbondata.common.logging.LogServiceFactory;
    +import org.apache.carbondata.core.scan.result.BatchResult;
    +
    +
    +public class PartitionSpliterRawResultIterator extends CarbonIterator<Object[]> {
    +
    +  private CarbonIterator<BatchResult> iterator;
    +  private BatchResult batch;
    +  private int counter;
    +
    +  /**
    +   * LOGGER
    +   */
    --- End diff --
   
    I think this is not necessary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129582365
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java ---
    @@ -0,0 +1,95 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.carbondata.core.scan.result.iterator;
    +
    +import org.apache.carbondata.common.CarbonIterator;
    +import org.apache.carbondata.common.logging.LogService;
    +import org.apache.carbondata.common.logging.LogServiceFactory;
    +import org.apache.carbondata.core.scan.result.BatchResult;
    +
    +
    --- End diff --
   
    delete space line.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129583064
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/PartitionInfo.java ---
    @@ -65,6 +65,31 @@ public PartitionInfo(List<ColumnSchema> columnSchemaList, PartitionType partitio
         this.partitionIds = new ArrayList<>();
       }
     
    +  /**
    +   * add partition means split default partition, add in last directly
    --- End diff --
   
    because maybe there is data existed in default partition need to be filled in new partition


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129583246
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java ---
    @@ -0,0 +1,95 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.carbondata.core.scan.result.iterator;
    +
    +import org.apache.carbondata.common.CarbonIterator;
    +import org.apache.carbondata.common.logging.LogService;
    +import org.apache.carbondata.common.logging.LogServiceFactory;
    +import org.apache.carbondata.core.scan.result.BatchResult;
    +
    +
    +public class PartitionSpliterRawResultIterator extends CarbonIterator<Object[]> {
    +
    +  private CarbonIterator<BatchResult> iterator;
    +  private BatchResult batch;
    +  private int counter;
    +
    +  /**
    +   * LOGGER
    +   */
    +  private static final LogService LOGGER =
    +      LogServiceFactory.getLogService(PartitionSpliterRawResultIterator.class.getName());
    +
    +  public PartitionSpliterRawResultIterator(CarbonIterator<BatchResult> iterator) {
    +    this.iterator = iterator;
    +  }
    +
    +
    +  @Override public boolean hasNext() {
    +    if (null == batch || checkBatchEnd(batch)) {
    +      if (iterator.hasNext()) {
    +        batch = iterator.next();
    +        counter = 0;
    +      } else {
    +        return false;
    +      }
    +    }
    +
    +    if (!checkBatchEnd(batch)) {
    +      return true;
    +    } else {
    +      return false;
    +    }
    --- End diff --
   
    use return !checkBatchEnd(batch) instead.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129583765
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/scan/result/iterator/PartitionSpliterRawResultIterator.java ---
    @@ -0,0 +1,95 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.carbondata.core.scan.result.iterator;
    +
    +import org.apache.carbondata.common.CarbonIterator;
    +import org.apache.carbondata.common.logging.LogService;
    +import org.apache.carbondata.common.logging.LogServiceFactory;
    +import org.apache.carbondata.core.scan.result.BatchResult;
    +
    +
    +public class PartitionSpliterRawResultIterator extends CarbonIterator<Object[]> {
    +
    +  private CarbonIterator<BatchResult> iterator;
    +  private BatchResult batch;
    +  private int counter;
    +
    +  /**
    +   * LOGGER
    +   */
    +  private static final LogService LOGGER =
    +      LogServiceFactory.getLogService(PartitionSpliterRawResultIterator.class.getName());
    +
    +  public PartitionSpliterRawResultIterator(CarbonIterator<BatchResult> iterator) {
    +    this.iterator = iterator;
    +  }
    +
    +
    +  @Override public boolean hasNext() {
    +    if (null == batch || checkBatchEnd(batch)) {
    +      if (iterator.hasNext()) {
    +        batch = iterator.next();
    +        counter = 0;
    +      } else {
    +        return false;
    +      }
    +    }
    +
    +    if (!checkBatchEnd(batch)) {
    +      return true;
    +    } else {
    +      return false;
    +    }
    +  }
    +
    +  @Override public Object[] next() {
    +    if (batch == null) {
    +      batch = iterator.next();
    +    }
    +    if (!checkBatchEnd(batch)) {
    +      try {
    +        return batch.getRawRow(counter++);
    +      } catch (Exception e) {
    +        LOGGER.error(e.getMessage());
    +        return null;
    +      }
    +    } else {
    +      batch = iterator.next();
    +      counter = 0;
    +    }
    +    try {
    +      return batch.getRawRow(counter++);
    +    } catch (Exception e) {
    +      LOGGER.error(e.getMessage());
    +      return null;
    --- End diff --
   
    This logical can be optimized.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129584623
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -101,17 +126,40 @@ object CarbonPartitionExample {
         spark.sql("""
            | CREATE TABLE IF NOT EXISTS t5
            | (
    +       | id Int,
            | vin String,
            | logdate Timestamp,
            | phonenumber Long,
    -       | area String
    +       | area String,
    +       | salary Int
            |)
            | PARTITIONED BY (country String)
            | STORED BY 'carbondata'
            | TBLPROPERTIES('PARTITION_TYPE'='LIST',
    -       | 'LIST_INFO'='(China,United States),UK ,japan,(Canada,Russia), South Korea ')
    +       | 'LIST_INFO'='(China, US),UK ,Japan,(Canada,Russia, Good, NotGood), Korea ')
    --- End diff --
   
    add space before ,


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129598531
 
    --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonInputFormat.java ---
    @@ -440,9 +510,16 @@ protected Expression getFilterPredicates(Configuration configuration) {
             for (Map.Entry<SegmentTaskIndexStore.TaskBucketHolder, AbstractIndex> entry :
                 segmentIndexMap.entrySet()) {
               SegmentTaskIndexStore.TaskBucketHolder taskHolder = entry.getKey();
    -          int taskId = CarbonTablePath.DataFileUtil.getTaskIdFromTaskNo(taskHolder.taskNo);
    +          int partitionId = CarbonTablePath.DataFileUtil.getTaskIdFromTaskNo(taskHolder.taskNo);
    +          //oldPartitionIdList is only used in alter table partition command because it change
    +          //partition info first and then read data.
    +          //for other normal query should use newest partitionIdList
    --- End diff --
   
    use /** */  instead


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129598686
 
    --- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonInputFormat.java ---
    @@ -440,9 +510,16 @@ protected Expression getFilterPredicates(Configuration configuration) {
             for (Map.Entry<SegmentTaskIndexStore.TaskBucketHolder, AbstractIndex> entry :
                 segmentIndexMap.entrySet()) {
               SegmentTaskIndexStore.TaskBucketHolder taskHolder = entry.getKey();
    -          int taskId = CarbonTablePath.DataFileUtil.getTaskIdFromTaskNo(taskHolder.taskNo);
    +          int partitionId = CarbonTablePath.DataFileUtil.getTaskIdFromTaskNo(taskHolder.taskNo);
    +          //oldPartitionIdList is only used in alter table partition command because it change
    +          //partition info first and then read data.
    +          //for other normal query should use newest partitionIdList
    --- End diff --
   
    use /** */ instead if multi line notes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129740890
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -101,17 +126,40 @@ object CarbonPartitionExample {
         spark.sql("""
            | CREATE TABLE IF NOT EXISTS t5
            | (
    +       | id Int,
            | vin String,
            | logdate Timestamp,
            | phonenumber Long,
    -       | area String
    +       | area String,
    +       | salary Int
            |)
            | PARTITIONED BY (country String)
            | STORED BY 'carbondata'
            | TBLPROPERTIES('PARTITION_TYPE'='LIST',
    -       | 'LIST_INFO'='(China,United States),UK ,japan,(Canada,Russia), South Korea ')
    +       | 'LIST_INFO'='(China, US),UK ,Japan,(Canada,Russia, Good, NotGood), Korea ')
    --- End diff --
   
    Hi @chenerlu , here in DDL statement, it's designed to leave no space to mock real situation which could happen in customer writing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r129744678
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
    @@ -308,6 +308,10 @@
       @CarbonProperty
       public static final String NUM_CORES_COMPACTING = "carbon.number.of.cores.while.compacting";
       /**
    +   * Number of cores to be used while alter partition
    +   */
    +  public static final String NUM_CORES_ALT_PARTITION = "carbon.number.of.cores.while.altPartition";
    +  /**
    --- End diff --
   
    No space line in other variables here, so keep one style.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1192: [CARBONDATA-940] alter table add/split partition for...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1192
 
    Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/612/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1192: [CARBONDATA-940] alter table add/split partition for...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1192
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3207/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1192: [CARBONDATA-940] alter table add/split partition for...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1192
 
    Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/614/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1192: [CARBONDATA-940] alter table add/split partition for...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1192
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3209/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r130501212
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/PartitionInfo.java ---
    @@ -65,6 +65,31 @@ public PartitionInfo(List<ColumnSchema> columnSchemaList, PartitionType partitio
         this.partitionIds = new ArrayList<>();
       }
     
    +  /**
    +   * add partition means split default partition, add in last directly
    +   */
    +  public void  addPartition(int addPartitionCount) {
    +    for (int i = 0; i < addPartitionCount; i++) {
    +      partitionIds.add(++MAX_PARTITION);
    +      numPartitions++;
    +    }
    +  }
    +
    +  /**
    +   * e.g. original partition[0,1,2,3,4,5]
    +   * split partition 2 to partition 6,7,8 (will not reuse 2)
    +   * then sourcePartitionId is 2, newPartitionNumbers is 3
    +   * @param sourcePartitionIndex
    +   * @param newPartitionNumbers
    +   */
    +  public void splitPartition(int sourcePartitionIndex, int newPartitionNumbers) {
    --- End diff --
   
    When will partition-split action be triggered?
   
    In my memory, there is no “partition-split” action in Hive, etc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1192: [CARBONDATA-940] alter table add/split partit...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1192#discussion_r130501996
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java ---
    @@ -303,6 +303,20 @@ public String getCarbonIndexFilePath(String taskId, String partitionId, String s
         }
       }
     
    +  public String getCarbonIndexFilePath(String taskId, String partitionId, String segmentId,
    +      int batchNo, String bucketNumber, String timeStamp,
    +      ColumnarFormatVersion columnarFormatVersion) {
    +    switch (columnarFormatVersion) {
    +      case V1:
    +      case V2:
    +        return getCarbonIndexFilePath(taskId, partitionId, segmentId, bucketNumber);
    +      default:
    --- End diff --
   
    Whats the different between V1/V2 and default?
   
    Does V3 acts the same as default?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
12345