[GitHub] [carbondata] ajantha-bhat opened a new pull request #3821: [WIP] Use qualified table name for global sort compaction

[GitBox]

ajantha-bhat opened a new pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821


    ### Why is this PR needed?
   
   
    ### What changes were proposed in this PR?
   
       
    ### Does this PR introduce any user interface change?
    - No
    - Yes. (please explain the change and update document)
   
    ### Is any new testcase added?
    - No
    - Yes
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821#issuecomment-653383759


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3296/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821#issuecomment-653384017


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1559/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ajantha-bhat commented on pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821#issuecomment-653384296


   @jackylk : please check and merge


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

jackylk commented on a change in pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821#discussion_r449408180



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/util/SparkSQLUtil.scala
##########
@@ -165,9 +165,9 @@ object SparkSQLUtil {
      * datatype of column data and corresponding datatype in schema provided to create dataframe.
      * Since carbonScanRDD gives Long data for timestamp column and corresponding column datatype in
      * schema is Timestamp, this validation fails if we use createDataFrame API which takes rdd as
-     * input. Hence, using below API which creates dataframe from tablename.
+     * input. Hence, using below API which creates dataframe from qualified tablename.
      */
-    sparkSession.sqlContext.table(carbonTable.getTableName)
+    sparkSession.sqlContext.table(carbonTable.getDatabaseName + "." + carbonTable.getTableName)

Review comment:
       Is there a utility for this? I guess no need to construct it ourself




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ajantha-bhat commented on a change in pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821#discussion_r449409262



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/util/SparkSQLUtil.scala
##########
@@ -165,9 +165,9 @@ object SparkSQLUtil {
      * datatype of column data and corresponding datatype in schema provided to create dataframe.
      * Since carbonScanRDD gives Long data for timestamp column and corresponding column datatype in
      * schema is Timestamp, this validation fails if we use createDataFrame API which takes rdd as
-     * input. Hence, using below API which creates dataframe from tablename.
+     * input. Hence, using below API which creates dataframe from qualified tablename.
      */
-    sparkSession.sqlContext.table(carbonTable.getTableName)
+    sparkSession.sqlContext.table(carbonTable.getDatabaseName + "." + carbonTable.getTableName)

Review comment:
       qualified name is spark usage, so carbon doesn't have utility class for it. And carbon's uniqueTableName is dbname_tableName, so cannot use it either.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

asfgit closed pull request #3821:
URL: https://github.com/apache/carbondata/pull/3821


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]