ShreelekhyaG opened a new pull request #4147: URL: https://github.com/apache/carbondata/pull/4147 ### Why is this PR needed? When trying to register a table of old store which has MV, it fails parser error(syntax issue while creating table). It is trying to create table with `relatedmvtablesmap` property which is not valid. ### What changes were proposed in this PR? - Removed `relatedmvtablesmap` from table properties in `RefreshCarbonTableCommand` - After Main table has registered, to register MV made changes to get the schema from the system folder and register. ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-854858593 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3763/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-854902879 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5506/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
ShreelekhyaG commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856489014 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
Indhumathi27 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856514647 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856560360 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5519/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856563176 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3776/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
Indhumathi27 commented on a change in pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#discussion_r647299409 ########## File path: integration/spark/src/test/scala/org/apache/spark/carbondata/register/TestRegisterCarbonTable.scala ########## @@ -231,6 +232,45 @@ class TestRegisterCarbonTable extends QueryTest with BeforeAndAfterEach { } } + test("test register table with mv") { + sql(s"create database carbon location '$dbLocation'") + sql("use carbon") + sql( + """create table carbon.carbontable ( + |c1 string,c2 int,c3 string,c5 string) STORED AS carbondata""".stripMargin) + sql("insert into carbontable select 'a',1,'aa','aaa'") + sql("create materialized view mv1 as select avg(c2),c3 from carbontable group by c3") + backUpData(dbLocation, Some("carbon"), "carbontable") + backUpData(dbLocation, Some("carbon"), "mv1") + val source = dbLocation + CarbonCommonConstants.FILE_SEPARATOR + "_system" + + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema.mv1" + val destination = dbLocation + "_back" + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema.mv1" + backUpMvSchema(source, destination) + sql("drop table carbontable") + if (!CarbonEnv.getInstance(sqlContext.sparkSession).carbonMetaStore.isReadFromHiveMetaStore) { + restoreData(dbLocation, "carbontable") + restoreData(dbLocation, "mv1") + backUpMvSchema(destination, source) + sql("refresh table carbontable") + sql("refresh table mv1") + sql("refresh materialized view mv1") + checkAnswer(sql("select count(*) from carbontable"), Row(1)) + checkAnswer(sql("select c1 from carbontable"), Seq(Row("a"))) + sql("show materialized views on table carbontable").show(false) Review comment: please remove this show -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856654750 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5522/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856656830 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3779/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
ShreelekhyaG commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856665109 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856726384 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3781/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
ShreelekhyaG commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856745833 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856864464 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3783/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-856867436 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5526/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
ShreelekhyaG commented on a change in pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#discussion_r647969173 ########## File path: integration/spark/src/test/scala/org/apache/spark/carbondata/register/TestRegisterCarbonTable.scala ########## @@ -231,6 +232,45 @@ class TestRegisterCarbonTable extends QueryTest with BeforeAndAfterEach { } } + test("test register table with mv") { + sql(s"create database carbon location '$dbLocation'") + sql("use carbon") + sql( + """create table carbon.carbontable ( + |c1 string,c2 int,c3 string,c5 string) STORED AS carbondata""".stripMargin) + sql("insert into carbontable select 'a',1,'aa','aaa'") + sql("create materialized view mv1 as select avg(c2),c3 from carbontable group by c3") + backUpData(dbLocation, Some("carbon"), "carbontable") + backUpData(dbLocation, Some("carbon"), "mv1") + val source = dbLocation + CarbonCommonConstants.FILE_SEPARATOR + "_system" + + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema.mv1" + val destination = dbLocation + "_back" + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema.mv1" + backUpMvSchema(source, destination) + sql("drop table carbontable") + if (!CarbonEnv.getInstance(sqlContext.sparkSession).carbonMetaStore.isReadFromHiveMetaStore) { + restoreData(dbLocation, "carbontable") + restoreData(dbLocation, "mv1") + backUpMvSchema(destination, source) + sql("refresh table carbontable") + sql("refresh table mv1") + sql("refresh materialized view mv1") + checkAnswer(sql("select count(*) from carbontable"), Row(1)) + checkAnswer(sql("select c1 from carbontable"), Seq(Row("a"))) + sql("show materialized views on table carbontable").show(false) Review comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
Indhumathi27 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-858311083 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-858387425 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3792/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
CarbonDataQA2 commented on pull request #4147: URL: https://github.com/apache/carbondata/pull/4147#issuecomment-858404376 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5535/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
In reply to this post by GitBox
asfgit closed pull request #4147: URL: https://github.com/apache/carbondata/pull/4147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [hidden email] |
Free forum by Nabble | Edit this page |