[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
GitHub user lionelcao opened a pull request:

    https://github.com/apache/carbondata/pull/1173

    [CARBONDATA-1209] add partitionId in show partition result

    add partitionId in show partition result
    change example to avoid truncate of result.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lionelcao/carbondata carbon_910_12

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1173.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1173
   
----
commit 9b697884d6287a2d78ac33aa167af3b4d0393de2
Author: lionelcao <[hidden email]>
Date:   2017-07-14T12:24:24Z

    [CARBONDATA-1209] add partitionId in show partition result

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/493/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3082/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3083/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3089/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/499/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128301440
 
    --- Diff: examples/spark/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -121,18 +121,18 @@ object CarbonPartitionExample {
         cc.sql("alter table hiveDB.t7 add partition (city = 'Shanghai')")
         //  show partitions
         try {
    -      cc.sql("SHOW PARTITIONS t0").show()
    +      cc.sql("SHOW PARTITIONS t0").show(100, false)
         } catch {
    -      case ex: AnalysisException => print(ex.getMessage())
    +      case ex: AnalysisException => print(ex.getMessage() + "\n")
    --- End diff --
   
    Use println instead.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128301570
 
    --- Diff: examples/spark/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -121,18 +121,18 @@ object CarbonPartitionExample {
         cc.sql("alter table hiveDB.t7 add partition (city = 'Shanghai')")
         //  show partitions
         try {
    -      cc.sql("SHOW PARTITIONS t0").show()
    +      cc.sql("SHOW PARTITIONS t0").show(100, false)
    --- End diff --
   
    why use false?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128301910
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -123,7 +123,7 @@ object CarbonPartitionExample {
         try {
           spark.sql(s"DROP TABLE IF EXISTS partitionDB.t9")
         } catch {
    -      case ex: NoSuchDatabaseException => print(ex.getMessage())
    +      case ex: NoSuchDatabaseException => print(ex.getMessage() + "\n")
    --- End diff --
   
    same problem.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128302086
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -144,15 +144,15 @@ object CarbonPartitionExample {
     
         // show partitions
         try {
    -      spark.sql("""SHOW PARTITIONS t0""").show()
    +      spark.sql("""SHOW PARTITIONS t0""").show(100, false)
         } catch {
    -      case ex: AnalysisException => print(ex.getMessage())
    +      case ex: AnalysisException => print(ex.getMessage() + "\n")
    --- End diff --
   
    same problem.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128302516
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/partition/TestShowPartitions.scala ---
    @@ -146,31 +146,31 @@ class TestShowPartition  extends QueryTest with BeforeAndAfterAll {
     
       test("show partition table: hash table") {
         // EqualTo
    -    checkAnswer(sql("show partitions hashTable"), Seq(Row("empno=HASH_NUMBER(3)")))
    +    checkAnswer(sql("show partitions hashTable"), Seq(Row("empno = HASH_NUMBER(3)")))
     
       }
     
       test("show partition table: range partition") {
         // EqualTo
    -    checkAnswer(sql("show partitions rangeTable"), Seq(Row("doj=default"),
    -      Row("doj<01-01-2010"), Row("01-01-2010<=doj<01-01-2015")))
    +    checkAnswer(sql("show partitions rangeTable"), Seq(Row("0, doj = DEFAULT"),
    +      Row("1, doj < 01-01-2010"), Row("2, 01-01-2010 <= doj < 01-01-2015")))
       }
     
       test("show partition table: list partition") {
         // EqualTo
    -    checkAnswer(sql("show partitions listTable"), Seq(Row("workgroupcategory=default"),
    -      Row("workgroupcategory=0"), Row("workgroupcategory=1"), Row("workgroupcategory=2, 3")))
    +    checkAnswer(sql("show partitions listTable"), Seq(Row("0, workgroupcategory = DEFAULT"),
    --- End diff --
   
    May be keep default will be better.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user chenerlu commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128303568
 
    --- Diff: integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala ---
    @@ -585,38 +585,44 @@ object CommonUtil {
         var result = Seq.newBuilder[Row]
         partitionType match {
           case PartitionType.RANGE =>
    -        result.+=(RowFactory.create(columnName + "=default"))
    -        var rangeInfo = partitionInfo.getRangeInfo
    -        var size = rangeInfo.size() - 1
    +        result.+=(RowFactory.create("0" + ", " + columnName + " = DEFAULT"))
    +        val rangeInfo = partitionInfo.getRangeInfo
    +        val size = rangeInfo.size() - 1
             for (index <- 0 to size) {
               if (index == 0) {
    -            result.+=(RowFactory.create(columnName + "<" + rangeInfo.get(index)))
    +            val id = partitionInfo.getPartitionId(index + 1).toString
    +            val desc = columnName + " < " + rangeInfo.get(index)
    +            result.+=(RowFactory.create(id + ", " + desc))
    --- End diff --
   
    Make sure this logic is correct.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128408816
 
    --- Diff: examples/spark/src/main/scala/org/apache/carbondata/examples/CarbonPartitionExample.scala ---
    @@ -121,18 +121,18 @@ object CarbonPartitionExample {
         cc.sql("alter table hiveDB.t7 add partition (city = 'Shanghai')")
         //  show partitions
         try {
    -      cc.sql("SHOW PARTITIONS t0").show()
    +      cc.sql("SHOW PARTITIONS t0").show(100, false)
    --- End diff --
   
    Hi @chenerlu , here we use 'false' to avoid truncating the result


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #1173: [CARBONDATA-1209] add partitionId in show par...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user lionelcao commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1173#discussion_r128408941
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/partition/TestShowPartitions.scala ---
    @@ -146,31 +146,31 @@ class TestShowPartition  extends QueryTest with BeforeAndAfterAll {
     
       test("show partition table: hash table") {
         // EqualTo
    -    checkAnswer(sql("show partitions hashTable"), Seq(Row("empno=HASH_NUMBER(3)")))
    +    checkAnswer(sql("show partitions hashTable"), Seq(Row("empno = HASH_NUMBER(3)")))
     
       }
     
       test("show partition table: range partition") {
         // EqualTo
    -    checkAnswer(sql("show partitions rangeTable"), Seq(Row("doj=default"),
    -      Row("doj<01-01-2010"), Row("01-01-2010<=doj<01-01-2015")))
    +    checkAnswer(sql("show partitions rangeTable"), Seq(Row("0, doj = DEFAULT"),
    +      Row("1, doj < 01-01-2010"), Row("2, 01-01-2010 <= doj < 01-01-2015")))
       }
     
       test("show partition table: list partition") {
         // EqualTo
    -    checkAnswer(sql("show partitions listTable"), Seq(Row("workgroupcategory=default"),
    -      Row("workgroupcategory=0"), Row("workgroupcategory=1"), Row("workgroupcategory=2, 3")))
    +    checkAnswer(sql("show partitions listTable"), Seq(Row("0, workgroupcategory = DEFAULT"),
    --- End diff --
   
    change to upper case is to distinguish from the other partition info value.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/543/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3135/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/554/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #1173: [CARBONDATA-1209] add partitionId in show partition ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1173
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3147/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
12