[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
Locked
12
12
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
 
|
GitHub user ravikiran23 opened a pull request:
https://github.com/apache/carbondata/pull/1007 [CARBONDATA-1146] Handling of the v3 format in delete flow of IUD. problem : delete flow of IUD is not handling the V3 format feature of pages in the blocklet. Solution: Adding the page info inside the delete delta file and handling of the same in the query flow. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravikiran23/incubator-carbondata master-after-iud Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1007.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1007 ---- commit 74f7b55db2e64e1dc596b7333035b743883fd653 Author: ravikiran23 <[hidden email]> Date: 2017-06-02T15:01:57Z Adding the Pages support in the Delete Method. commit c641523280986c49b903db6d6c5f27615addd203 Author: Manohar <[hidden email]> Date: 2017-06-08T09:50:49Z correcting the size of the vector batch excluding the filtered rows. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/162/<h2>Build result: FAILURE</span></h2>[...truncated 80.67 KB...][JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/processing/pom.xml to org.apache.carbondata/carbondata-processing/1.2.0-SNAPSHOT/carbondata-processing-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Processing #161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark-common/pom.xml to org.apache.carbondata/carbondata-spark-common/1.2.0-SNAPSHOT/carbondata-spark-common-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Spark Common #161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/examples/spark/pom.xml to org.apache.carbondata/carbondata-examples-spark/1.2.0-SNAPSHOT/carbondata-examp les-spark-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/hive/pom.xml to org.apache.carbondata/carbondata-hive/1.2.0-SNAPSHOT/carbondata-hive-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/common/pom.xml to org.apache.carbondata/carbondata-common/1.2.0-SNAPSHOT/carbondata-common-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Common #161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/assembly/pom.xml to org.apache.carbondata/carbondata-assembly/1.2.0-SNAPSHOT/carbondata-assembly-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/pom.xml to org.apache.carbondata/carbondata-parent/1.2.0-SNAPSHOT/carbondata-parent-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Parent # 161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/examples/flink/pom.xml to org.apache.carbondata/carbondata-examples-flink/1.2.0-SNAPSHOT/carbondata-examples-flink-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/core/pom.xml to org.apache.carbondata/carbondata-core/1.2.0-SNAPSHOT/carbondata-core-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Core #161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/hadoop/pom.xml to org.apache.carbondata/carbondata-hadoop/1.2.0-SNAPSHOT/carbondata-hadoop-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Hadoop #161 to compare, so performing full copy of artifacts[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/presto/pom .xml to org.apache.carbondata/carbondata-presto/1.2.0-SNAPSHOT/carbondata-presto-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark-common-test/pom.xml to org.apache.carbondata/carbondata-spark-common-test/1.2.0-SNAPSHOT/carbondata-spark-common-test-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark/pom.xml to org.apache.carbondata/carbondata-spark/1.2.0-SNAPSHOT/carbondata-spark-1.2.0-SNAPSHOT.pomNo artifacts from carbondata-pr-spark-1.6 » Apache CarbonData :: Spark #161 to compare, so performing full copy of artifactschannel stoppedSetting status of c641523280986c49b903db6d6c5f27615addd203 to FAILURE with url https://builds.apache.org/job/carbondata-pr-spark-1.6/162/ and message: 'Tests Failed for Spark1.6 'Using context: Jenkins(Spark1.6): mvn clean test -Pspark-1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1007 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2289/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1007 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2293/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1007 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2295/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/168/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1007 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2299/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/172/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
|
|
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1007#discussion_r121077093 --- Diff: core/src/main/java/org/apache/carbondata/core/reader/CarbonDeleteFilesDataReader.java --- @@ -80,8 +79,8 @@ private void initThreadPoolSize() { * @return * @throws Exception */ - public int[] getDeleteDataFromAllFiles(List<String> deltaFiles, String blockletId) - throws Exception { + public Map<String, Integer[]> getDeleteDataFromAllFiles(List<String> deltaFiles, --- End diff -- Please maintain `Map<Integer, int[]>` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
|
|
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1007#discussion_r121077526 --- Diff: core/src/main/java/org/apache/carbondata/core/mutate/DeleteDeltaBlockletDetails.java --- @@ -31,6 +31,8 @@ private static final long serialVersionUID = 1206104914911491724L; private String id; + private String pageId; --- End diff -- Please maintain pageid as int, it is always integer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
|
|
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1007#discussion_r121077736 --- Diff: core/src/main/java/org/apache/carbondata/core/cache/update/BlockletLevelDeleteDeltaDataCache.java --- @@ -17,26 +17,36 @@ package org.apache.carbondata.core.cache.update; +import java.util.HashMap; +import java.util.Map; + +import org.apache.carbondata.core.constants.CarbonCommonConstants; + import org.roaringbitmap.RoaringBitmap; /** * This class maintains delete delta data cache of each blocklet along with the block timestamp */ public class BlockletLevelDeleteDeltaDataCache { - private RoaringBitmap deleteDelataDataCache; + private Map<String, RoaringBitmap> deleteDelataDataCache = --- End diff -- change to `Map<Integer, RoaringBitmap>` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1007 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2343/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/220/<h2>Failed Tests: <span class='status-failure'>1</span></h2><h3><a name='carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test' /><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/220/org.apache.carbondata$carbondata-spark-common-test/testReport'>carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test</a>: <span class='status-failure'>1</span></h3><ul><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/220/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.spark.testsuite.allqueries/InsertIntoCarbonTableTestCase/insert_into_carbon_table_from_carbon_table_union_query/'><strong>org.apache.carbondata.spark.testsuite.allqueries.InsertIntoCarbonTableTestCase.insert into carbon table from carbon table union query</strong></a></li></ul> --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
|
|
In reply to this post by qiuchenjian-2
Github user ravikiran23 closed the pull request at:
https://github.com/apache/carbondata/pull/1007 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata pull request #1007: [CARBONDATA-1146] Handling of the v3 format i...
|
|
In reply to this post by qiuchenjian-2
GitHub user ravikiran23 reopened a pull request:
https://github.com/apache/carbondata/pull/1007 [CARBONDATA-1146] Handling of the v3 format in delete flow of IUD. problem : delete flow of IUD is not handling the V3 format feature of pages in the blocklet. Solution: Adding the page info inside the delete delta file and handling of the same in the query flow. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravikiran23/incubator-carbondata master-after-iud Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/1007.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1007 ---- commit dba02a0ad6a581f8d4659902a7d1767e6c4b70f5 Author: ravikiran23 <[hidden email]> Date: 2017-06-02T15:01:57Z Adding the Pages support in the Delete Method. commit 8ac3caa4d8912bf168e199d1b20be2c6937bf9ef Author: Manohar <[hidden email]> Date: 2017-06-08T09:50:49Z correcting the size of the vector batch excluding the filtered rows. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/221/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
[GitHub] carbondata issue #1007: [CARBONDATA-1146] Handling of the v3 format in delet...
|
|
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:
https://github.com/apache/carbondata/pull/1007 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/carbondata-pr-spark-1.6/222/<h2>Build result: FAILURE</span></h2>[...truncated 78.91 KB...][ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn <goals> -rf :carbondata-spark[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/processing/pom.xml to org.apache.carbondata/carbondata-processing/1.2.0-SNAPSHOT/carbondata-processing-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark-common/pom.xml to org.apache.carbondata/carbondata-spark-common/1.2.0-SNAPSHOT/carbondata-spark-common-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave /workspace/carbondata-pr-spark-1.6/examples/spark/pom.xml to org.apache.carbondata/carbondata-examples-spark/1.2.0-SNAPSHOT/carbondata-examples-spark-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/hive/pom.xml to org.apache.carbondata/carbondata-hive/1.2.0-SNAPSHOT/carbondata-hive-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/common/pom.xml to org.apache.carbondata/carbondata-common/1.2.0-SNAPSHOT/carbondata-common-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/assembly/pom.xml to org.apache.carbondata/carbondata-assembly/1.2.0-SNAPSHOT/carbondata-assembly-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/pom.xml to org.apache.carbondata/carbondata-parent/1.2.0-SNAPSHOT/carbondata-parent-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/ca rbondata-pr-spark-1.6/examples/flink/pom.xml to org.apache.carbondata/carbondata-examples-flink/1.2.0-SNAPSHOT/carbondata-examples-flink-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/core/pom.xml to org.apache.carbondata/carbondata-core/1.2.0-SNAPSHOT/carbondata-core-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/hadoop/pom.xml to org.apache.carbondata/carbondata-hadoop/1.2.0-SNAPSHOT/carbondata-hadoop-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/presto/pom.xml to org.apache.carbondata/carbondata-presto/1.2.0-SNAPSHOT/carbondata-presto-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark-common-test/pom.xml to org.apache.carbondata/carbondata-spark-common-test/1.2.0-SNAPSHOT/carbondata-spark-common-test-1.2.0-SNAPSHOT.pom[JENKINS] Archiving /home/j enkins/jenkins-slave/workspace/carbondata-pr-spark-1.6/integration/spark/pom.xml to org.apache.carbondata/carbondata-spark/1.2.0-SNAPSHOT/carbondata-spark-1.2.0-SNAPSHOT.pomchannel stoppedSetting status of 83097ce368cb35c42d20acba5e38b3a090267103 to FAILURE with url https://builds.apache.org/job/carbondata-pr-spark-1.6/222/ and message: 'Tests Failed for Spark1.6 'Using context: Jenkins(Spark1.6): mvn clean test -Pspark-1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
«
Return to Apache CarbonData JIRA issues
|
1 view|%1 views
| Free forum by Nabble | Edit this page |