Posted by
GitBox on
Jan 22, 2021; 6:08am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-Indhumathi27-opened-a-new-pull-request-4076-WIP-Added-mvExists-property-in-MV-facte-tp105379p105547.html
Indhumathi27 commented on a change in pull request #4076:
URL:
https://github.com/apache/carbondata/pull/4076#discussion_r561693339##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/view/CarbonDropMVCommand.scala
##########
@@ -90,6 +95,37 @@ case class CarbonDropMVCommand(
}
}
+ // Update the mvExists and related databases property to mv fact tables
+ schema.getRelatedTables.asScala.foreach { table =>
+ val dbName = table.getDatabaseName
+ val tableName = table.getTableName
+ try {
+ val carbonTable =
+ CarbonEnv.getCarbonTable(Some(dbName), tableName)(session)
+ val relatedMVTablesMap = carbonTable.getMVTablesMap
+ // check if database has materialized views or not
+ val anotherMVExistsInDb = viewManager.getSchemas(databaseName).asScala.exists {
+ mvSchema =>
+ mvSchema.getRelatedTables.asScala.exists(_.getTableName.equalsIgnoreCase(tableName))
+ }
+ if (!anotherMVExistsInDb) {
+ // If database dont have any MV, then remove the database from related tables
+ // property and update table property of fact table
+ relatedMVTablesMap.get(databaseName).remove(name)
+ if (relatedMVTablesMap.get(databaseName).isEmpty) {
+ relatedMVTablesMap.remove(databaseName)
+ }
+ CarbonIndexUtil.addOrModifyTableProperty(carbonTable,
+ Map("relatedMVTablesMap".toLowerCase -> new Gson().toJson(relatedMVTablesMap)),
+ needLock = false)(session)
+ CarbonHiveIndexMetadataUtil.refreshTable(dbName, tableName, session)
+ }
+ } catch {
+ case _: Exception =>
+ // ignore as table is a non-carbon table
Review comment:
This Exception is added, because for parquet and orc, carbon table will not be present.
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -569,14 +568,31 @@ private synchronized void touchMDTFile() throws IOException {
FileFactory.createDirectoryAndSetPermission(this.systemDirectory,
new FsPermission(FsAction.ALL, FsAction.ALL, FsAction.ALL));
}
- CarbonFile schemaIndexFile = FileFactory.getCarbonFile(this.schemaIndexFilePath);
- if (schemaIndexFile.exists()) {
- schemaIndexFile.delete();
+ // two or more JVM process can access this method to update last modified time at same
+ // time causing exception. So take a system level lock on system folder and update
+ // last modified time of schema index file
+ ICarbonLock systemDirLock = CarbonLockFactory
+ .getSystemLevelCarbonLockObj(this.systemDirectory,
+ LockUsage.MATERIALIZED_VIEW_STATUS_LOCK);
+ boolean locked = false;
+ try {
+ locked = systemDirLock.lockWithRetries();
+ if (locked) {
+ CarbonFile schemaIndexFile = FileFactory.getCarbonFile(this.schemaIndexFilePath);
+ if (schemaIndexFile.exists()) {
+ schemaIndexFile.delete();
+ }
+ schemaIndexFile.createNewFile(new FsPermission(FsAction.ALL, FsAction.ALL, FsAction.ALL));
+ this.lastModifiedTime = schemaIndexFile.getLastModifiedTime();
+ } else {
+ LOG.warn("Unable to get Lock to refresh schema index last modified time");
Review comment:
Not needed, As next operation will compare last modified time and update it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[hidden email]