[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3537: Metacache issues

[GitBox]

vikramahuja1001 opened a new pull request #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537
 
 
   Be sure to do all of the following checklist to help us incorporate
   your contribution quickly and easily:
   
    - [ ] Any interfaces changed?
   
    - [ ] Any backward compatibility impacted?
   
    - [ ] Document update required?
   
    - [ ] Testing done
           Please provide details on
           - Whether new unit test cases have been added or why no new tests are required?
           - How it is tested? Please attach test report.
           - Is it a performance related change? Please attach the performance test report.
           - Any additional information to help reviewers in testing this change.
         
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569262987
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1299/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569274070
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1320/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569286118
 
 
   Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1308/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569896692
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569898197
 
 
   Build Failed  with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1374/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569900901
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1395/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on issue #3537: Metacache issues
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569908950
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569912856
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1378/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362201899
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
 ##########
 @@ -45,6 +47,8 @@ import org.apache.carbondata.processing.loading.events.LoadEvents.{LoadMetadataE
 import org.apache.carbondata.spark.rdd.SparkReadSupport
 import org.apache.carbondata.spark.readsupport.SparkRowReadSupportImpl
 
+
 
 Review comment:
   revert this change

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362202583
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
 ##########
 @@ -125,6 +130,21 @@ class CarbonEnv {
     LOGGER.info("Initialize CarbonEnv completed...")
   }
 
+
+  def createIndexServerTmp(sparkSession: SparkSession): Unit = {
+    val conf = sparkSession.sparkContext.hadoopConfiguration
+    val fs = FileSystem.get(conf)
+    if (fs.getUri.toString.startsWith("hdfs://")) {
 
 Review comment:
   Use CarbonUtil.checkAndAppendFileSystemURIScheme to add URI to the path

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362203886
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
 ##########
 @@ -45,6 +47,8 @@ import org.apache.carbondata.processing.loading.events.LoadEvents.{LoadMetadataE
 import org.apache.carbondata.spark.rdd.SparkReadSupport
 import org.apache.carbondata.spark.readsupport.SparkRowReadSupportImpl
 
+
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on a change in pull request #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#discussion_r362203892
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
 ##########
 @@ -125,6 +130,21 @@ class CarbonEnv {
     LOGGER.info("Initialize CarbonEnv completed...")
   }
 
+
+  def createIndexServerTmp(sparkSession: SparkSession): Unit = {
+    val conf = sparkSession.sparkContext.hadoopConfiguration
+    val fs = FileSystem.get(conf)
+    if (fs.getUri.toString.startsWith("hdfs://")) {
 
 Review comment:
   done

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569918773
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1388/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569920825
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1400/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

vikramahuja1001 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569923726
 
 
   retest this please
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569924691
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1384/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569927454
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1379/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569936628
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1389/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3537: [CARBONDATA-3646] [CARBONDATA-3647]: 1. Select query * fails when using index server. 2. Filter query failure with Index server when loaded with global_sort_partition = 100000, giving Null pointer exception
URL: https://github.com/apache/carbondata/pull/3537#issuecomment-569941202
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1401/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services