Apache CarbonData Dev Mailing List archive › Apache CarbonData JIRA issues

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

Classic

List

Threaded

68 messages Options

1234

qiuchenjian-2

[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2830

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/907/

---

qiuchenjian-2

[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2830

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/914/

---

qiuchenjian-2

[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2830

Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9180/

---

qiuchenjian-2

[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2830

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1122/

---

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kunal642 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226980043

--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala ---
@@ -493,6 +493,23 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser {
showHistory.isDefined)
}

+
+ protected lazy val cli: Parser[LogicalPlan] =
+ (SHOW ~> SUMMARY ~> FOR ~> TABLE) ~ (ident <~ ".").? ~ ident ~
--- End diff --

Change `(SHOW ~> SUMMARY ~> FOR ~> TABLE) ~ (ident <~ ".").?` to `(SHOW ~> SUMMARY ~> FOR ~> TABLE) ~> (ident <~ ".").?` as we dont want the output of show summary for table

---

qiuchenjian-2

[GitHub] carbondata issue #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kumarvishal09 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226987833

--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/CarbonCli.java ---
@@ -40,6 +41,10 @@
@InterfaceStability.Unstable
public class CarbonCli {

+ private static ArrayList<String> outPuts;
--- End diff --

change private static ArrayList<String> outPuts to List<String>

---

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kumarvishal09 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226987983

--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/CarbonCli.java ---
@@ -40,6 +41,10 @@
@InterfaceStability.Unstable
public class CarbonCli {

+ private static ArrayList<String> outPuts;
+
+ private static boolean isPrintInConsole = true;
--- End diff --

please add doc for this variable

---

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kumarvishal09 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226989395

--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/TablePrinter.java ---
@@ -17,26 +17,28 @@

package org.apache.carbondata.tool;

-import java.io.PrintStream;
+import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;

class TablePrinter {
--- End diff --

Now TablePrinter is not printing any details, it is adding to list. Please change the class name, Please check other places if applicable

---

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kumarvishal09 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226991227

--- Diff: tools/cli/src/main/java/org/apache/carbondata/tool/DataSummary.java ---
@@ -226,25 +282,46 @@ private int getColumnIndex(String columnName) {
private boolean collected = false;

private void printColumnStats(String columnName) throws IOException, MemoryException {
- out.println();
- out.println("## Column Statistics for '" + columnName + "'");
+ outPuts.add("");
+ outPuts.add("## Column Statistics for '" + columnName + "'");
collectStats(columnName);

int columnIndex = getColumnIndex(columnName);
String[] header = new String[]{"BLK", "BLKLT", "Meta Size", "Data Size",
- "LocalDict", "DictEntries", "DictSize", "AvgPageSize", "Min%", "Max%"};
+ "LocalDict", "DictEntries", "DictSize", "AvgPageSize", "Min%", "Max%", "Min", "Max"};

- ShardPrinter printer = new ShardPrinter(header);
+ ShardPrinter printer = new ShardPrinter(header, outPuts);
for (Map.Entry<String, DataFile> entry : dataFiles.entrySet()) {
DataFile file = entry.getValue();
for (DataFile.Blocklet blocklet : file.getAllBlocklets()) {
- String min, max;
+ String min, max, minPercent, maxPercent;
--- End diff --

Now min max will not be present if size is more for example varchar string this case is not handled please check

---

qiuchenjian-2

[GitHub] carbondata pull request #2830: [CARBONDATA-3025]Added CLI enhancements

In reply to this post by qiuchenjian-2

Github user kumarvishal09 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2830#discussion_r226987643

--- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonShowSummaryCommand.scala ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.command.management
+
+import java.util
+
+import scala.collection.JavaConverters._
+
+import org.apache.spark.sql.{CarbonEnv, Row, SparkSession}
+import org.apache.spark.sql.catalyst.expressions.{Attribute, AttributeReference}
+import org.apache.spark.sql.execution.command.{Checker, DataCommand}
+import org.apache.spark.sql.types.StringType
+
+import org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException
+import org.apache.carbondata.tool.CarbonCli
+
+case class CarbonShowSummaryCommand(
--- End diff --

Please add class level doc

---

qiuchenjian-2