[GitHub] [carbondata] Karan980 opened a new pull request #4133: [CARBONDATA-4184] alter table Set TBLPROPERTIES for RANGE_COLUMN sets unsupported datatype(complex_datatypes/Binary/Boolean/Decimal) as RANGE_COLUMN.

[GitBox]

Karan980 opened a new pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133


    ### Why is this PR needed?
   Alter table set command was not validating unsupported dataTypes for range column.
   
   
    ### What changes were proposed in this PR?
   Added validation for unsupported dataTypes before setting range column value.
   
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - Yes
   
       
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-839771177


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3604/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-839775031


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5349/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Karan980 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840311696


   retest this please


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on a change in pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#discussion_r631579778



##########
File path: integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##########
@@ -652,6 +652,16 @@ object AlterTableUtil {
         throw new MalformedCarbonCommandException(
           s"Table property ${ CarbonCommonConstants.RANGE_COLUMN }: ${ rangeColumnProp }" +
           s" is not exists in the table")
+      }
+      val dataType = rangeColumn.getDataType.getName;
+      if (DataTypes.BINARY.getName.equalsIgnoreCase(dataType) ||

Review comment:
       This check is same as CarbonParserUtil range column check. can extract to method and reuse

##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestRangeColumnDataLoad.scala
##########
@@ -836,15 +836,52 @@ class TestRangeColumnDataLoad extends QueryTest with BeforeAndAfterEach with Bef
   test("set and unset table property: range_column") {
     sql(
       """
-        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT)
-        | STORED AS carbondata
+        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT, bin binary,
+        |  bool1 boolean, arr1 array<int>, struct1 struct<id1:string,name1:string>,
+        |  map1 map<string,string>, dec1 decimal(10,5)) STORED AS carbondata
         | TBLPROPERTIES('SORT_SCOPE'='LOCAL_SORT', 'SORT_COLUMNS'='name, city')
       """.stripMargin)
 
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='city')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
     sql("ALTER TABLE carbon_range_column6 UNSET TBLPROPERTIES('range_column')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
+
+    var ex = intercept[RuntimeException] {
+      sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='bin')")

Review comment:
       Can move this alter exception validation to method. method could be validateInvalidRangecolumn(rangeColumn: String, dataType: String)




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on a change in pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#discussion_r631582485



##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestRangeColumnDataLoad.scala
##########
@@ -836,15 +836,52 @@ class TestRangeColumnDataLoad extends QueryTest with BeforeAndAfterEach with Bef
   test("set and unset table property: range_column") {
     sql(
       """
-        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT)
-        | STORED AS carbondata
+        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT, bin binary,
+        |  bool1 boolean, arr1 array<int>, struct1 struct<id1:string,name1:string>,
+        |  map1 map<string,string>, dec1 decimal(10,5)) STORED AS carbondata
         | TBLPROPERTIES('SORT_SCOPE'='LOCAL_SORT', 'SORT_COLUMNS'='name, city')
       """.stripMargin)
 
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='city')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
     sql("ALTER TABLE carbon_range_column6 UNSET TBLPROPERTIES('range_column')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
+
+    var ex = intercept[RuntimeException] {
+      sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='bin')")

Review comment:
       Can move this alter exception validation to method. method could be
   def validateInvalidRangeColumn(invalidRangeColumnsMap: Map[String, String]): Unit = {
         invalidRangeColumnsMap.foreach { value =>
           val ex = intercept[RuntimeException] {
             sql(s"ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='${ value._1 }')")
           }
           assertResult("Alter table newProperties operation failed: RANGE_COLUMN doesn't " +
                        s"support ${ value._2 } data type: ${value._1}")(ex.getMessage)
         }
       }
       val invalidRangeColumnsMap = Map("bin" -> "BINARY", "bool1" -> "BOOLEAN", "arr1" -> "ARRAY",
         "struct1" -> "STRUCT", "map1" -> "MAP", "dec1" -> "DECIMAL")
       validateInvalidRangeColumn(invalidRangeColumnsMap)




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Karan980 commented on a change in pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#discussion_r631606623



##########
File path: integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##########
@@ -652,6 +652,16 @@ object AlterTableUtil {
         throw new MalformedCarbonCommandException(
           s"Table property ${ CarbonCommonConstants.RANGE_COLUMN }: ${ rangeColumnProp }" +
           s" is not exists in the table")
+      }
+      val dataType = rangeColumn.getDataType.getName;
+      if (DataTypes.BINARY.getName.equalsIgnoreCase(dataType) ||

Review comment:
       Done
   




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Karan980 commented on a change in pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#discussion_r631610450



##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestRangeColumnDataLoad.scala
##########
@@ -836,15 +836,52 @@ class TestRangeColumnDataLoad extends QueryTest with BeforeAndAfterEach with Bef
   test("set and unset table property: range_column") {
     sql(
       """
-        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT)
-        | STORED AS carbondata
+        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT, bin binary,
+        |  bool1 boolean, arr1 array<int>, struct1 struct<id1:string,name1:string>,
+        |  map1 map<string,string>, dec1 decimal(10,5)) STORED AS carbondata
         | TBLPROPERTIES('SORT_SCOPE'='LOCAL_SORT', 'SORT_COLUMNS'='name, city')
       """.stripMargin)
 
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='city')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
     sql("ALTER TABLE carbon_range_column6 UNSET TBLPROPERTIES('range_column')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
+
+    var ex = intercept[RuntimeException] {
+      sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='bin')")

Review comment:
       Done, with different method signature.




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840363196


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5357/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840364122


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3612/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Karan980 commented on a change in pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#discussion_r631621283



##########
File path: integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/dataload/TestRangeColumnDataLoad.scala
##########
@@ -836,15 +836,52 @@ class TestRangeColumnDataLoad extends QueryTest with BeforeAndAfterEach with Bef
   test("set and unset table property: range_column") {
     sql(
       """
-        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT)
-        | STORED AS carbondata
+        | CREATE TABLE carbon_range_column6(id INT, name STRING, city STRING, age INT, bin binary,
+        |  bool1 boolean, arr1 array<int>, struct1 struct<id1:string,name1:string>,
+        |  map1 map<string,string>, dec1 decimal(10,5)) STORED AS carbondata
         | TBLPROPERTIES('SORT_SCOPE'='LOCAL_SORT', 'SORT_COLUMNS'='name, city')
       """.stripMargin)
 
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='city')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
     sql("ALTER TABLE carbon_range_column6 UNSET TBLPROPERTIES('range_column')")
     sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='name')")
+
+    var ex = intercept[RuntimeException] {
+      sql("ALTER TABLE carbon_range_column6 SET TBLPROPERTIES('range_column'='bin')")

Review comment:
       Done




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840383889


   LGTM


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840407273


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5360/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840414279


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3615/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840423730


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3618/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-840425489


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5363/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-842879938


   retest this please


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-842945088


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5386/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-842946231


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3642/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4133:
URL: https://github.com/apache/carbondata/pull/4133#issuecomment-845290950


   retest this please


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]