[GitHub] carbondata pull request #2256: [CARBONDATA-2422] Search mode Master port sho...

[qiuchenjian-2]

GitHub user jackylk opened a pull request:

    https://github.com/apache/carbondata/pull/2256

    [CARBONDATA-2422] Search mode Master port should be dynamic

    In SDV test, sometimes search mode testcase failed because Master port is occupied. This PR adds support for dynamic master port to avoid port binding failure
   
     - [X] Any interfaces changed?
     No
     - [X] Any backward compatibility impacted?
     No
     - [X] Document update required?
    No
     - [X] Testing done
            Please provide details on
            - Whether new unit test cases have been added or why no new tests are required?
            - How it is tested? Please attach test report.
            - Is it a performance related change? Please attach the performance test report.
            - Any additional information to help reviewers in testing this change.
           No
     - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    NA

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jackylk/incubator-carbondata master-port

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2256.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2256
   
----
commit a54ff923443b9abd5ef9decc1d93bd29e4a2a36c
Author: Jacky Li <jacky.likun@...>
Date:   2018-05-01T12:25:16Z

    support dynamic master port

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5559/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4397/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4655/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4425/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5586/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4682/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5596/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4435/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4690/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2256
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2256


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2256#discussion_r186953529
 
    --- Diff: store/search/src/main/scala/org/apache/spark/rpc/Master.scala ---
    @@ -64,22 +64,39 @@ class Master(sparkConf: SparkConf, port: Int) {
     
       private val scheduler: Scheduler = new Scheduler
     
    -  def this(sparkConf: SparkConf) = {
    -    this(sparkConf, CarbonProperties.getSearchMasterPort)
    -  }
    -
       /** start service and listen on port passed in constructor */
       def startService(): Unit = {
         if (rpcEnv == null) {
           new Thread(new Runnable {
             override def run(): Unit = {
               val hostAddress = InetAddress.getLocalHost.getHostAddress
    -          val config = RpcEnvConfig(
    -            sparkConf, "registry-service", hostAddress, "", CarbonProperties.getSearchMasterPort,
    -            new SecurityManager(sparkConf), clientMode = false)
    -          rpcEnv = new NettyRpcEnvFactory().create(config)
    +          var port = CarbonProperties.getSearchMasterPort
    --- End diff --
   
    It's better to ensure there are no intersection between master port and worker port, for example. the default master and worker port is 10100, 10200,  not 10020,10021. Because if master use 10020 fail, master will use 10021, and the worker will failed to use 10021.


---