maheshrajus commented on a change in pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#discussion_r638867906 ########## File path: integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala ########## @@ -257,6 +258,23 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser { (SET ~> "(" ~> repsep(element, ",") <~ ")") ~ ("=" ~> restInput) <~ opt(";") ^^ { case tab ~ columns ~ rest => + // If update is received for complex data types then throw exception + var finalColumns = List.empty[String] + var updateColumns = new ListBuffer[String]() + columns.foreach { column => + if (column.contains('.')) { + val columnFullName = column.split('.') + if ((tab._3.isDefined && tab._3.get.equals(columnFullName(0))) + || tab._4.table.equals(columnFullName(0))) { + updateColumns += columnFullName(1) Review comment: hi ajantha , i tried with below example. Carbondata not receiving command parse as spark itself throwing exception. Spark is allowing two level update but not three levels. Please help me if anything I am missing over here. ``` CREATE TABLE test_rename (str1 struct<a:int>, str2 struct<a:struct<b:int>>, str3 struct<a:struct<b:struct<c:int>>>, intfield int) STORED AS carbondata update test_rename set(str3.a.b)=(4) ``` ![image](https://user-images.githubusercontent.com/17046058/119519572-2d245800-bd97-11eb-90f1-700a91113999.png) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
kunal642 commented on a change in pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#discussion_r642830546 ########## File path: integration/spark/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala ########## @@ -757,14 +757,99 @@ class TestComplexDataType extends QueryTest with BeforeAndAfterAll { sql("DROP TABLE IF EXISTS test") sql("create table test(id int,a struct<b:int,c:int>,d array<int>) STORED AS carbondata") sql("insert into test values(1, named_struct('b', 2, 'c', 3), array(4))") - val structException = intercept[UnsupportedOperationException]( + val structException = intercept[AnalysisException]( sql("update test set(a.b)=(4) where id=1").collect()) - assertResult("Unsupported operation on Complex data type")(structException.getMessage) + assert(structException.getMessage().contains("Unsupported operation on Complex data type")) val arrayException = intercept[UnsupportedOperationException]( sql("update test set(a)=(4) where id=1").collect()) assertResult("Unsupported operation on Complex data type")(arrayException.getMessage) } + test("check update operation on primitive data types when complex type present in table which " + + "has child name equal to primitive data types") { + sql("drop table if exists update_complex") + sql("create table update_complex (a int, b string, struct1 STRUCT<a:int, c:string>) " + + "stored as carbondata") + sql("insert into update_complex select 1,'c', named_struct('a',4,'b','d')") + sql("update update_complex set (a)=(4)") + checkAnswer(sql("select a from update_complex"), + Seq(Row(4))) + sql("update update_complex set (b)=('y')") Review comment: can you combine these update and select commands into one. update.. set(a) update.. set(b) can be update.. set(a,b) same with select after doing multiple updates, fire 1 select command to verify the changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
kunal642 commented on a change in pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#discussion_r642830546 ########## File path: integration/spark/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala ########## @@ -757,14 +757,99 @@ class TestComplexDataType extends QueryTest with BeforeAndAfterAll { sql("DROP TABLE IF EXISTS test") sql("create table test(id int,a struct<b:int,c:int>,d array<int>) STORED AS carbondata") sql("insert into test values(1, named_struct('b', 2, 'c', 3), array(4))") - val structException = intercept[UnsupportedOperationException]( + val structException = intercept[AnalysisException]( sql("update test set(a.b)=(4) where id=1").collect()) - assertResult("Unsupported operation on Complex data type")(structException.getMessage) + assert(structException.getMessage().contains("Unsupported operation on Complex data type")) val arrayException = intercept[UnsupportedOperationException]( sql("update test set(a)=(4) where id=1").collect()) assertResult("Unsupported operation on Complex data type")(arrayException.getMessage) } + test("check update operation on primitive data types when complex type present in table which " + + "has child name equal to primitive data types") { + sql("drop table if exists update_complex") + sql("create table update_complex (a int, b string, struct1 STRUCT<a:int, c:string>) " + + "stored as carbondata") + sql("insert into update_complex select 1,'c', named_struct('a',4,'b','d')") + sql("update update_complex set (a)=(4)") + checkAnswer(sql("select a from update_complex"), + Seq(Row(4))) + sql("update update_complex set (b)=('y')") Review comment: can you combine these update and select commands into one. update.. set(a) update.. set(b) can be update.. set(a,b) same with select after doing multiple updates, fire 1 select command to verify the changes check the same for other test cases added byyou ########## File path: integration/spark/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala ########## @@ -757,14 +757,99 @@ class TestComplexDataType extends QueryTest with BeforeAndAfterAll { sql("DROP TABLE IF EXISTS test") sql("create table test(id int,a struct<b:int,c:int>,d array<int>) STORED AS carbondata") sql("insert into test values(1, named_struct('b', 2, 'c', 3), array(4))") - val structException = intercept[UnsupportedOperationException]( + val structException = intercept[AnalysisException]( sql("update test set(a.b)=(4) where id=1").collect()) - assertResult("Unsupported operation on Complex data type")(structException.getMessage) + assert(structException.getMessage().contains("Unsupported operation on Complex data type")) val arrayException = intercept[UnsupportedOperationException]( sql("update test set(a)=(4) where id=1").collect()) assertResult("Unsupported operation on Complex data type")(arrayException.getMessage) } + test("check update operation on primitive data types when complex type present in table which " + + "has child name equal to primitive data types") { + sql("drop table if exists update_complex") + sql("create table update_complex (a int, b string, struct1 STRUCT<a:int, c:string>) " + + "stored as carbondata") + sql("insert into update_complex select 1,'c', named_struct('a',4,'b','d')") + sql("update update_complex set (a)=(4)") + checkAnswer(sql("select a from update_complex"), + Seq(Row(4))) + sql("update update_complex set (b)=('y')") Review comment: can you combine these update and select commands into one. update.. set(a) update.. set(b) can be update.. set(a,b) same with select after doing multiple updates, fire 1 select command to verify the changes check the same for other test cases added by you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
maheshrajus commented on a change in pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#discussion_r642947612 ########## File path: integration/spark/src/test/scala/org/apache/carbondata/integration/spark/testsuite/complexType/TestComplexDataType.scala ########## @@ -757,14 +757,99 @@ class TestComplexDataType extends QueryTest with BeforeAndAfterAll { sql("DROP TABLE IF EXISTS test") sql("create table test(id int,a struct<b:int,c:int>,d array<int>) STORED AS carbondata") sql("insert into test values(1, named_struct('b', 2, 'c', 3), array(4))") - val structException = intercept[UnsupportedOperationException]( + val structException = intercept[AnalysisException]( sql("update test set(a.b)=(4) where id=1").collect()) - assertResult("Unsupported operation on Complex data type")(structException.getMessage) + assert(structException.getMessage().contains("Unsupported operation on Complex data type")) val arrayException = intercept[UnsupportedOperationException]( sql("update test set(a)=(4) where id=1").collect()) assertResult("Unsupported operation on Complex data type")(arrayException.getMessage) } + test("check update operation on primitive data types when complex type present in table which " + + "has child name equal to primitive data types") { + sql("drop table if exists update_complex") + sql("create table update_complex (a int, b string, struct1 STRUCT<a:int, c:string>) " + + "stored as carbondata") + sql("insert into update_complex select 1,'c', named_struct('a',4,'b','d')") + sql("update update_complex set (a)=(4)") + checkAnswer(sql("select a from update_complex"), + Seq(Row(4))) + sql("update update_complex set (b)=('y')") Review comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
maheshrajus commented on a change in pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#discussion_r642950598 ########## File path: integration/spark/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala ########## @@ -257,6 +258,23 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser { (SET ~> "(" ~> repsep(element, ",") <~ ")") ~ ("=" ~> restInput) <~ opt(";") ^^ { case tab ~ columns ~ rest => + // If update is received for complex data types then throw exception + var finalColumns = List.empty[String] + var updateColumns = new ListBuffer[String]() + columns.foreach { column => + if (column.contains('.')) { + val columnFullName = column.split('.') + if ((tab._3.isDefined && tab._3.get.equals(columnFullName(0))) + || tab._4.table.equals(columnFullName(0))) { + updateColumns += columnFullName(1) Review comment: @ajantha-bhat Look like the update command has some problem when we give nested complex columns[a.b.c] from the carbon parsing side. I will check and raise the new JIRA as it is the base issue. I have to check all other impacted test cases also regarding this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852048908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
maheshrajus commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852115005 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852191282 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3725/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852191744 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5469/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
maheshrajus commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852281570 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852351028 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3727/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852354126 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5471/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
kunal642 commented on pull request #4139: URL: https://github.com/apache/carbondata/pull/4139#issuecomment-852767340 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
asfgit closed pull request #4139: URL: https://github.com/apache/carbondata/pull/4139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
Free forum by Nabble | Edit this page |