[GitHub] incubator-carbondata pull request #795: [CARBONDATA-923]fix issue of insertI...

[qiuchenjian-2]

GitHub user QiangCai opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/795

    [CARBONDATA-923]fix issue of insertInto read from OneRowRelation

    Reproduce:
    create table OneRowTable(col1 string, col2 string, col3 int, col4 double) stored by 'carbondata'
    insert into OneRowTable select '0.1', 'a.b', 1, 1.2
   
    Exception:
    org.apache.spark.sql.AnalysisException: cannot resolve '`0.1`' given input columns: [0.1, a.b, 1, 1.2];;
    'Project ['0.1, 'a.b]
    +- Project 0.1 AS 0.1#11, a.b AS a.b#12, 1 AS 1#13, 1.2 AS 1.2#14
    +- OneRowRelation$
   
    Solution:
    rename output attribute of select sub-query

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/QiangCai/incubator-carbondata fixInsertInto

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/795.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #795
   
----
commit 7c2bb5f970155a854f2b807e8aaf32de8bca43d8
Author: QiangCai <[hidden email]>
Date:   2017-04-13T13:09:12Z

    insertInto read from OneRowRelation

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Failed  with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1610/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Failed  with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1638/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1676/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1677/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user QiangCai commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    @gvramana
    I moved test case to spark 2.1. And spark 1.6 doesn't support it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556260
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
    @@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
             )
         }
         if (child.output.size >= relation.carbonRelation.output.size) {
    -      InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      var index = 0
    +      val newChildOutput = child.output.map { column =>
    --- End diff --
   
    use `mapWithIndex` instead


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556346
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
    @@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
             )
         }
         if (child.output.size >= relation.carbonRelation.output.size) {
    -      InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      var index = 0
    +      val newChildOutput = child.output.map { column =>
    +        index = index + 1
    +        column match {
    +          case attr: Alias =>
    +            Alias(attr.child, s"col$index")(attr.exprId)
    +          case attr: Attribute =>
    +            Alias(attr, s"col$index")(NamedExpression.newExprId)
    +          case attr => attr
    +        }
    +      }
    +      if (newChildOutput == child.output) {
    +        InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      } else {
    +        InsertIntoCarbonTable(relation,
    --- End diff --
   
    move `relation` to next line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/795#discussion_r113556768
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
    @@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
             )
         }
         if (child.output.size >= relation.carbonRelation.output.size) {
    -      InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      var index = 0
    +      val newChildOutput = child.output.map { column =>
    +        index = index + 1
    +        column match {
    +          case attr: Alias =>
    +            Alias(attr.child, s"col$index")(attr.exprId)
    +          case attr: Attribute =>
    +            Alias(attr, s"col$index")(NamedExpression.newExprId)
    +          case attr => attr
    +        }
    +      }
    +      if (newChildOutput == child.output) {
    +        InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      } else {
    +        InsertIntoCarbonTable(relation,
    +          p.partition,
    +          Project(newChildOutput, child),
    --- End diff --
   
    I think you can move the creation of `InsertIntoCarbonTable` below else clause, so that it will take care of both if and else case. Only one parameter is different.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1814/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1815/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/795#discussion_r113606332
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
    @@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
             )
         }
         if (child.output.size >= relation.carbonRelation.output.size) {
    -      InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      var index = 0
    +      val newChildOutput = child.output.map { column =>
    --- End diff --
   
    fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/795#discussion_r113606343
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala ---
    @@ -52,7 +53,26 @@ object CarbonPreInsertionCasts extends Rule[LogicalPlan] {
             )
         }
         if (child.output.size >= relation.carbonRelation.output.size) {
    -      InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      var index = 0
    +      val newChildOutput = child.output.map { column =>
    +        index = index + 1
    +        column match {
    +          case attr: Alias =>
    +            Alias(attr.child, s"col$index")(attr.exprId)
    +          case attr: Attribute =>
    +            Alias(attr, s"col$index")(NamedExpression.newExprId)
    +          case attr => attr
    +        }
    +      }
    +      if (newChildOutput == child.output) {
    +        InsertIntoCarbonTable(relation, p.partition, p.child, p.overwrite, p.ifNotExists)
    +      } else {
    +        InsertIntoCarbonTable(relation,
    +          p.partition,
    +          Project(newChildOutput, child),
    --- End diff --
   
    fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/795
 
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/795


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---