[GitHub] [carbondata] nihal0107 opened a new pull request #3647: [CARBONDATA-3348] Fix the case insensitive validation for duplicate …

[GitBox]

nihal0107 opened a new pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647
 
 
   …sort column.
   
   Why is this PR needed?
   Currently the column name with different case(upper and lower)
   is getting considered as different columns for sort properties.
   
       What changes were proposed in this PR?
       Added the case insensitive validation for duplicate sort column.
   
       Does this PR introduce any user interface change?
       No
   
       Is any new testcase added?
       No
   
       
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593211815
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593242779
 
 
   add to whitelist

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate …
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593242999
 
 
   please add one test case

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593260243
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2258/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Zhangshunyu commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593269015
 
 
   the failed testcases need to change the message in assert which has changed with code modification

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593304352
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2264/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593375823
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2266/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593432763
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2267/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386812861
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
-        sortKey.diff(sortKey.distinct).mkString(",") + " or it contains illegal argumnet.")
+          sortKeyLowerCase.diff(sortKeyLowerCase.distinct).mkString(",") +
+          " or it contains illegal argument.")
 
 Review comment:
   `contains` to `contains an`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813002
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
 
 Review comment:
   ```suggestion
           "SORT_COLUMNS Either having duplicate columns : " +
   ```   SORT_COLUMNS Either contains

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813002
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
 
 Review comment:
   ```suggestion
           "SORT_COLUMNS Either having duplicate columns : " +
   ```   SORT_COLUMNS Either contains

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386813122
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
 
 Review comment:
   ```suggestion
           "SORT_COLUMNS Either contains duplicate columns : " +
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

nihal0107 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386817767
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
 
 Review comment:
   Done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

nihal0107 commented on a change in pull request #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#discussion_r386817816
 
 

 ##########
 File path: integration/spark/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
 ##########
 @@ -760,11 +760,13 @@ object CommonUtil {
       fields: Seq[(String, String)],
       varcharCols: Seq[String]
   ): Unit = {
-    if (sortKey.diff(sortKey.distinct).length > 0 ||
-        (sortKey.length > 1 && sortKey.contains(""))) {
+    val sortKeyLowerCase = sortKey.map(_.toLowerCase())
+    if ((sortKeyLowerCase).diff(sortKeyLowerCase.distinct).length > 0 ||
+      (sortKeyLowerCase.length > 1 && sortKeyLowerCase.contains(""))) {
       throw new MalformedCarbonCommandException(
         "SORT_COLUMNS Either having duplicate columns : " +
-        sortKey.diff(sortKey.distinct).mkString(",") + " or it contains illegal argumnet.")
+          sortKeyLowerCase.diff(sortKeyLowerCase.distinct).mkString(",") +
+          " or it contains illegal argument.")
 
 Review comment:
   Done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593814993
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2275/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593838340
 
 
   add to whitelist

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593838402
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593839205
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/575/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

nihal0107 commented on issue #3647: [CARBONDATA-3348]  Fix the case insensitive validation for duplicate sort column.
URL: https://github.com/apache/carbondata/pull/3647#issuecomment-593856540
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services