Posted by
GitBox on
Jun 03, 2021; 11:46am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/GitHub-carbondata-akkio-97-opened-a-new-pull-request-4129-WIP-alter-rename-complex-types-tp108015p108618.html
akashrn5 commented on a change in pull request #4129:
URL:
https://github.com/apache/carbondata/pull/4129#discussion_r644723696##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableColRenameDataTypeChangeCommand.scala
##########
@@ -225,12 +264,46 @@ private[sql] case class CarbonAlterTableColRenameDataTypeChangeCommand(
newColumnProperties.put(CarbonCommonConstants.COLUMN_COMMENT, newColumnComment.get)
columnSchema.setColumnProperties(newColumnProperties)
}
- addColumnSchema = columnSchema
+ addedTableColumnSchema = columnSchema
+ } else if (isComplexChild(columnSchema)) {
+ if (alteredColumnNamesMap.contains(columnSchemaName)) {
+ // matches exactly
+ val newComplexChildName = alteredColumnNamesMap(columnSchemaName)
+ columnSchema.setColumn_name(newComplexChildName)
+ isSchemaEntryRequired = true
+ } else {
+ val alteredParent = checkIfParentIsAltered(columnSchemaName)
+ /*
+ * Lets say, if complex schema is: str struct<a: int>
+ * and if parent column is changed from str -> str2
+ * then its child name should also be changed from str.a -> str2.a
+ */
+ if (alteredParent != null) {
+ val newParent = alteredColumnNamesMap(alteredParent)
+ val newComplexChildName = newParent + columnSchemaName
+ .split(alteredParent)(1)
+ columnSchema.setColumn_name(newComplexChildName)
+ isSchemaEntryRequired = true
+ }
+ }
+ }
+ // validate duplicate child columns
+ if (uniqueColumnSet.contains(columnSchema.getColumn_name)) {
+ throw new UnsupportedOperationException("Duplicate columns are present")
+ }
+
+ // make a new schema evolution entry after column rename or datatype change
+ if (isSchemaEntryRequired) {
+ addedColumnsList ++= List(columnSchema)
+ deletedColumnsList ++= List(deletedColumnSchema)
timeStamp = System.currentTimeMillis()
- // make a new schema evolution entry after column rename or datatype change
- schemaEvolutionEntry = AlterTableUtil
- .addNewSchemaEvolutionEntry(timeStamp, addColumnSchema, deletedColumnSchema)
+ schemaEvolutionEntry = AlterTableUtil.addNewSchemaEvolutionEntry(schemaEvolutionEntry,
Review comment:
i don't see any reason to change this API, i think you have changed API and added null checks in `addNewSchemaEvolutionEntry` since in case of complex there can be multiple columns changed like child and parent.
But just sending single column in List and adding null check is not good, here itself make the deleted and added column as List, incase of primitive it will be with single column, in case of complex when both parent and child changes, it will have multiple values. Please do the necessary changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[hidden email]