Posted by
Indhumathi on
Jan 20, 2021; 12:02pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Bug-SI-Compatibility-Issue-tp105485.html
Hello all,
Currently, for Secondary Index metadata, we are adding the below two
properties to index table and Fact table on SI creation.
1. indextableexists -> will be set to true, if table has SI. will be null
for SI table.
2. isIndexTable -> to know if the table is SI or not
When CarbonInternalMetastore#refreshIndexInfo is called on lookUpRelation,
we update the above properties to table, based on the
indexMetadata information stored in Hive.
Issue scenario:
For compatibility scenario, where fact table and SI are created in older
versions and if we need to upgrade those tables to latest version,
we execute the below commands.
1. Refresh table -> register the table with metaStore
2. Register Index command
-> loads index information to Hive and set index formation to Fact
table.
Currently, after refresh index table and before register index, we can
perform any operation on index Table(including load,alter, iud,..),
because isIndexTable property will be set only during Register index.
These operations will affect Registering the
index to fact table.
Do we need to block those operations, even before register index Or can
we allow it and check Fact and SI table Sync during register index?
Please let me know your solutions about the same.
Regards,
Indhumathi M
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/