[GitHub] [carbondata] ShreelekhyaG opened a new pull request #4098: [WIP] UT with index server

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853637600


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3736/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853641071


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5480/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853906134


   2.3 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_Manual_2.3_Compile_and_UT/1/
   
   
   
   2.4 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4_Compile_and_UT/65/
   
   
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG edited a comment on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853906134


   2.3 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_Manual_2.3_Compile_and_UT/1/
   
   
   
   2.4 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4_Compile_and_UT/72/
   
   
   
   
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854359673


   LGTM


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854359792


   retest this please


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854408367


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5495/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854408812


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3752/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854420766


   retest this please


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG commented on a change in pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#discussion_r644526559



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/test/SparkTestQueryExecutor.scala
##########
@@ -82,6 +83,15 @@ object SparkTestQueryExecutor {
       copyResourcesifNotExists(hdfsUrl, s"$integrationPath/spark/src/test/resources",
         s"$integrationPath//spark-common-cluster-test/src/test/resources/testdatafileslist.txt")
   }
+  if (System.getProperty("useIndexServer") != null || System.getenv("useIndexServer") != null) {

Review comment:
       ok

##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/test/util/CarbonFunSuite.scala
##########
@@ -44,4 +44,18 @@ private[spark] abstract class CarbonFunSuite extends FunSuite {
     }
   }
 
+  protected def test(testName : scala.Predef.String, ignoreForIndexServer: Boolean,
+                     testTags : org.scalatest.Tag*)
+                    (testFun : => Unit): Unit = {
+    if (ignoreForIndexServer) {
+      if (System.getProperty("useIndexServer") == null) {

Review comment:
       done

##########
File path: integration/spark/src/main/scala/org/apache/carbondata/indexserver/IndexServer.scala
##########
@@ -50,6 +51,7 @@ trait ServerInterface {
   /**
    * Used to prune and cache the index for the table.
    */
+  @throws(classOf[IOException])

Review comment:
       no need to ignore testcase as external carbon segments are pruned by indexserver. non-carbon segments are not pruned. From `distributedPrunedRdd `when async calls to prune are made, `TableIndex.prune` method throws IOException and this is not handled in parent caller - `indexserver.getsplits`




--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853580234






--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853637600






--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ShreelekhyaG edited a comment on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-853906134


   2.3 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_Manual_2.3_Compile_and_UT/1/
   
   
   
   2.4 build and UT with indexserver: http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4_Compile_and_UT/72/
   
   
   
   
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

Indhumathi27 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854359673






--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854489198


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3754/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854491088


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5497/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854617875


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3756/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854674056


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5500/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854801121


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/3761/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA2 commented on pull request #4098:
URL: https://github.com/apache/carbondata/pull/4098#issuecomment-854804780


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/5504/
   


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]