Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1416#discussion_r145422709 --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/events/Events.scala --- @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.events + +import org.apache.carbondata.core.metadata.CarbonTableIdentifier +import org.apache.carbondata.processing.loading.model.CarbonLoadModel + +/** + * event for database operations + */ +trait DatabaseEvent extends Event { + val databaseName: String +} + +/** + * event for table related operations + */ +trait TableEvent extends DatabaseEvent { + val carbonTableIdentifier: CarbonTableIdentifier + override lazy val databaseName: String = carbonTableIdentifier.getDatabaseName +} + +/** + * event for load operations + */ +trait LoadEvent extends TableEvent { --- End diff -- Move this trait to LoadEvents class --- |
In reply to this post by qiuchenjian-2
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1416#discussion_r145424680 --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/events/EventListenerExample.scala --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.events + +import java.io.File + +import org.apache.spark.sql.SparkSession + +import org.apache.carbondata.core.metadata.CarbonTableIdentifier +import org.apache.carbondata.processing.loading.model.CarbonLoadModel + +/** + * Example for event listener + */ +object EventListenerExample extends App { + + ListenerBus.getInstance() + .addListener(LoadTablePreExecutionEvent.getClass.getName, new EventListener { + + override def onEvent(event: Event): Unit = { + println("Test event listener event") + } + }) + + import org.apache.spark.sql.CarbonSession._ + + val rootPath = new File(this.getClass.getResource("/").getPath + + "../../../..").getCanonicalPath + val storeLocation = s"$rootPath/examples/spark2/target/store" + val warehouse = s"$rootPath/examples/spark2/target/warehouse" + val sparkSession = SparkSession + .builder() + .master("local") + .appName("CarbonSessionExample") + .config("spark.sql.warehouse.dir", warehouse) + .config("spark.driver.host", "localhost") + .getOrCreateCarbonSession(storeLocation) + val carbonTableIdentifier = new CarbonTableIdentifier("db1", "tbl1", "tbl1-id") + val carbonLoadModel = new CarbonLoadModel --- End diff -- Example to be used how to subscribe to data load event --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1416 Can you explain more on the intention of this PR? --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1416 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1175/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1416 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/544/ --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/1416 @jackylk ..This PR will provide an event listener interface using which you can register the events based on the need of your operations. For example before dropping a table you need to record the information from that table that can be used for clean up of resources/caches used by that table on executor or driver side. In this case you can register an event PreDropEventListener which will perform the required task and add an event which will triggered just before drop. --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1416#discussion_r146236785 --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/events/Events.scala --- @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.events + +import org.apache.carbondata.core.metadata.CarbonTableIdentifier +import org.apache.carbondata.processing.loading.model.CarbonLoadModel + +/** + * event for database operations + */ +trait DatabaseEvent extends Event { + val databaseName: String +} + +/** + * event for table related operations + */ +trait TableEvent extends DatabaseEvent { + val carbonTableIdentifier: CarbonTableIdentifier + override lazy val databaseName: String = carbonTableIdentifier.getDatabaseName +} + +/** + * event for load operations + */ +trait LoadEvent extends TableEvent { --- End diff -- I think it is better to keep all the possible events in Events.scala class in order to know what all different type of events are possible and keep their concrete implementation in different classes to clearly define their behavior. --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1416 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1284/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1416 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/650/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1416 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/854/ --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 closed the pull request at:
https://github.com/apache/carbondata/pull/1416 --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/1416 Code already merged as part of PR #1473 --- |
Free forum by Nabble | Edit this page |