[GitHub] carbondata pull request #2239: [CARBONDATA-2408] Fix search mode master Sasl...

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5493/



---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184849996
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/store/SparkCarbonStore.scala ---
    @@ -110,8 +110,16 @@ class SparkCarbonStore extends MetaCachedCarbonStore {
       }
     
       def startSearchMode(): Unit = {
    +    LOG.info("Search mode master start service")
         master = new Master(session.sparkContext.getConf)
         master.startService()
    +    var count = 0
    +    while (!master.isStarted()) {
    +      Thread.sleep(10)
    +      count = count + 1;
    +      LOG.info("Sleeping and waiting search mode master finish start " + count)
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184850026
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/store/SparkCarbonStore.scala ---
    @@ -110,8 +110,16 @@ class SparkCarbonStore extends MetaCachedCarbonStore {
       }
     
       def startSearchMode(): Unit = {
    --- End diff --
   
    thanks


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184850068
 
    --- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
    @@ -64,13 +64,29 @@ class Master(sparkConf: SparkConf, port: Int) {
     
       private val scheduler: Scheduler = new Scheduler
     
    +  /**
    +   * flag is the master status, default value is false, which means master not start now.
    +   * if it is true, then search mode master finished start.
    +   */
    +  private var flag: Boolean = false
    +
       def this(sparkConf: SparkConf) = {
         this(sparkConf, CarbonProperties.getSearchMasterPort)
       }
     
    +  /**
    +   * get the status flag info
    +   *
    +   * @return whether master started
    +   */
    +  def isStarted(): Boolean = {
    +    this.flag
    +  }
    +
       /** start service and listen on port passed in constructor */
       def startService(): Unit = {
         if (rpcEnv == null) {
    +      LOG.info("Start search mode  master thread")
    --- End diff --
   
    remove it


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184850096
 
    --- Diff: integration/spark2/src/main/scala/org/apache/carbondata/store/SparkCarbonStore.scala ---
    @@ -110,8 +110,16 @@ class SparkCarbonStore extends MetaCachedCarbonStore {
       }
     
       def startSearchMode(): Unit = {
    +    LOG.info("Search mode master start service")
         master = new Master(session.sparkContext.getConf)
         master.startService()
    +    var count = 0
    +    while (!master.isStarted()) {
    +      Thread.sleep(10)
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184850235
 
    --- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
    @@ -80,9 +96,18 @@ class Master(sparkConf: SparkConf, port: Int) {
               rpcEnv = new NettyRpcEnvFactory().create(config)
               val registryEndpoint: RpcEndpoint = new Registry(rpcEnv, Master.this)
               rpcEnv.setupEndpoint("registry-service", registryEndpoint)
    +          if (!isStarted) {
    --- End diff --
   
    I will try


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2239#discussion_r184850376
 
    --- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
    @@ -80,9 +96,18 @@ class Master(sparkConf: SparkConf, port: Int) {
               rpcEnv = new NettyRpcEnvFactory().create(config)
               val registryEndpoint: RpcEndpoint = new Registry(rpcEnv, Master.this)
               rpcEnv.setupEndpoint("registry-service", registryEndpoint)
    +          if (!isStarted) {
    +            synchronized {
    +              if (!isStarted) {
    +                flag = true
    +              }
    +            }
    +          }
               rpcEnv.awaitTermination()
             }
           }).start()
    +    } else {
    +      LOG.info("rpcEnv exists:" + rpcEnv.address)
    --- End diff --
   
    Search mode master has already existed before.  maybe by this search mode program, maybe by others.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5504/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4339/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4607/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5547/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4384/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4643/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4406/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5568/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4666/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4417/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5579/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    @jackylk Please review it.


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2239
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4673/



---