[GitHub] NamanRastogi commented on a change in pull request #3103: [CARBONDATA-3273] [CARBONDATA-3274] Fix for SORT_SCOPE in CarbonLoadDataCommand

[GitBox]

NamanRastogi commented on a change in pull request #3103: [CARBONDATA-3273] [CARBONDATA-3274] Fix for SORT_SCOPE in CarbonLoadDataCommand
URL: https://github.com/apache/carbondata/pull/3103#discussion_r250977867
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
 ##########
 @@ -209,17 +209,21 @@ case class CarbonLoadDataCommand(
     * 4. Session property CARBON_OPTIONS_SORT_SCOPE
     * 5. Default Sort Scope LOAD_SORT_SCOPE
     */
-    if (tableProperties.get(CarbonCommonConstants.SORT_COLUMNS) != null &&
-        tableProperties.get(CarbonCommonConstants.SORT_SCOPE) == null) {
-      // If there are Sort Columns given for the table and Sort Scope is not specified,
-      // we will take it as whichever sort scope given or LOCAL_SORT as default
-      optionsFinal
-        .put(CarbonCommonConstants.SORT_SCOPE,
-          carbonProperty
-            .getProperty(
-              CarbonLoadOptionConstants.CARBON_TABLE_LOAD_SORT_SCOPE + table.getDatabaseName + "." +
-              table.getTableName, carbonProperty.getProperty(CarbonCommonConstants.LOAD_SORT_SCOPE,
-                SortScopeOptions.getSortScope("LOCAL_SORT").toString)))
+    if (tableProperties.get(CarbonCommonConstants.SORT_COLUMNS) == null ||
+        tableProperties.get(CarbonCommonConstants.SORT_COLUMNS).equals("")) {
+      // If tableProperties.SORT_COLUMNS is null
+      optionsFinal.put(CarbonCommonConstants.SORT_SCOPE, "NO_SORT")
+    }
+    else if (tableProperties.get(CarbonCommonConstants.SORT_SCOPE) == null ||
+             tableProperties.get(CarbonCommonConstants.SORT_SCOPE).equals("")) {
+      // If tableProperties.SORT_COLUMNS is not null
+      // and tableProperties.SORT_SCOPE is null
+      optionsFinal.put(CarbonCommonConstants.SORT_SCOPE,
+        carbonProperty.getProperty(CarbonLoadOptionConstants.CARBON_TABLE_LOAD_SORT_SCOPE +
+          table.getDatabaseName + "." + table.getTableName,
+          carbonProperty.getProperty(CarbonLoadOptionConstants.CARBON_OPTIONS_SORT_SCOPE,
+            carbonProperty.getProperty(CarbonCommonConstants.LOAD_SORT_SCOPE,
+              "LOCAL_SORT"))))
 
 Review comment:
   If the SORT_COLUMNS are set and any SORT_SCOPE is not found, then LOCAL_SORT is used.
   If no SORT_COLUMNS are set, then take the SORT_SCOPE as NO_SORT.
   
   The default behaviour is NO_SORT if SORT_COLUMNS are not provided, and LOCAL_SORT if SORT_COLUMNS are provided explicetely in CREATE TABLE command. If nothing is specified in CREATE TABLE command, it takes SORT_SCOPE as NO_SORT and SORT_COLUMNS as none.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services