[GitHub] [carbondata] Karan980 opened a new pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] Karan980 opened a new pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox

Karan980 opened a new pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088


    ### Why is this PR needed?
   Prepriming is not working in Index Server. Server.getRemoteUser returns null value in async call of prepriming  which results in NPE and crashes the indexServer application.
   
    ### What changes were proposed in this PR?
   Computed the Server.getRemoteUser value before making the async prepriming call and then used the same value during async call.
   
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-774024419


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5436/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-774026587


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3675/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-775114494


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5441/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-775119897


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3680/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-777596563


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3313/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-777598244


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5071/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] brijoobopanna commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

brijoobopanna commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-778193923


   retest this please
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-778248712


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3699/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-778255658


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5462/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] kunal642 commented on a change in pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

kunal642 commented on a change in pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#discussion_r576143690



##########
File path: integration/spark/src/main/scala/org/apache/carbondata/indexserver/IndexServer.scala
##########
@@ -128,6 +128,10 @@ object IndexServer extends ServerInterface {
   def getCount(request: IndexInputFormat): LongWritable = {
     doAs {
       val sparkSession = SparkSQLUtil.getSparkSession
+      var currentUser: String = null
+      if (!request.isFallbackJob && Server.getRemoteUser != null) {
+        currentUser = Server.getRemoteUser.getShortUserName

Review comment:
       what happens if the RemoteUser is null?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] Karan980 commented on a change in pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

Karan980 commented on a change in pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#discussion_r577845412



##########
File path: integration/spark/src/main/scala/org/apache/carbondata/indexserver/IndexServer.scala
##########
@@ -128,6 +128,10 @@ object IndexServer extends ServerInterface {
   def getCount(request: IndexInputFormat): LongWritable = {
     doAs {
       val sparkSession = SparkSQLUtil.getSparkSession
+      var currentUser: String = null
+      if (!request.isFallbackJob && Server.getRemoteUser != null) {
+        currentUser = Server.getRemoteUser.getShortUserName

Review comment:
       Null check for Server.getRemoteUser removed




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-780816628


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5488/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-780817533


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3725/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] kunal642 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

kunal642 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-781519825


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] asfgit closed pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

asfgit closed pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-781558153


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/3324/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4088: [CARBONDATA-4121] Prepriming is not working in Index Server.

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4088:
URL: https://github.com/apache/carbondata/pull/4088#issuecomment-781561196


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/5081/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]