Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1112 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/223/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1112 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2803/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124892155 --- Diff: integration/presto/README.md --- @@ -20,14 +20,10 @@ Please follow the below steps to query carbondata in presto ### Config presto server -* Download presto server 0.166 : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/ -* Finish configuration as per https://prestodb.io/docs/current/installation/deployment.html - for example: +* Download presto server ( >= 0.166) : https://repo1.maven.org/maven2/com/facebook/presto/presto-server/ --- End diff -- I tried using presto 0.179 with carbondata 1.2.0, it looks fine. But I am going to change `>=` to `=`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124892557 --- Diff: integration/presto/README.md --- @@ -59,28 +55,50 @@ Please follow the below steps to query carbondata in presto ``` * config carbondata-connector for presto - First:compile carbondata-presto integration module + Firstly: Compile carbondata, including carbondata-presto integration module ``` $ git clone https://github.com/apache/carbondata - $ cd carbondata/integration/presto - $ mvn clean package + $ cd carbondata + $ mvn -DskipTests -P{spark-version} -Dspark.version={spark-version-number} -Dhadoop.version={hadoop-version-number} clean package + ``` + Replace the spark and hadoop version with the version used in your cluster. + For example, if you use Spark2.1.0 and Hadoop 2.7.3, you would like to compile using: --- End diff -- OK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124892745 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableReader.java --- @@ -133,30 +163,54 @@ public boolean updateCarbonFile() { return true; } + /** + * Return the schema names under a schema store path (this.carbonFileList). + * @return + */ public List<String> updateSchemaList() { updateCarbonFile(); if (carbonFileList != null) { - List<String> schemaList = - Stream.of(carbonFileList.listFiles()).map(a -> a.getName()).collect(Collectors.toList()); + /*List<String> schemaList = + Stream.of(carbonFileList.listFiles()).map(a -> a.getName()).collect(Collectors.toList());*/ + List<String> schemaList = new ArrayList<>(); + for (CarbonFile file : carbonFileList.listFiles()) --- End diff -- OK. It is a mistake, I will recover it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124892937 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonLocalInputSplit.java --- @@ -17,19 +17,22 @@ package org.apache.carbondata.presto.impl; -import java.util.List; - import com.fasterxml.jackson.annotation.JsonCreator; import com.fasterxml.jackson.annotation.JsonProperty; +import java.util.List; + +/** + * BHQ: CarbonLocalInputSplit represents a block, it contains a set of blocklet. --- End diff -- OK. I will remove BHQ. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1112 Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/237/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1112 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2817/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user chenliang613 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124933020 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableReader.java --- @@ -520,7 +626,7 @@ private boolean isValidBlockBasedOnUpdateDetails( int blkIndex; for ( bytesRemaining = length; - (double) bytesRemaining / (double) splitSize > 1.1D; + (double) bytesRemaining / (double) splitSize > 1.1D;// BHQ: when there are more than one splits left. bytesRemaining -= splitSize) { --- End diff -- please remove BHQ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1112#discussion_r124961570 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableReader.java --- @@ -520,7 +626,7 @@ private boolean isValidBlockBasedOnUpdateDetails( int blkIndex; for ( bytesRemaining = length; - (double) bytesRemaining / (double) splitSize > 1.1D; + (double) bytesRemaining / (double) splitSize > 1.1D;// BHQ: when there are more than one splits left. bytesRemaining -= splitSize) { --- End diff -- OK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user HoneyQuery closed the pull request at:
https://github.com/apache/carbondata/pull/1112 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
Free forum by Nabble | Edit this page |