Howdy Andrew,

I noticed you have a configuration item that we were not aware of:  spark.history.ui.port .  Is that new for 1.0?

Also, we noticed that the Workers and the Drivers were opening up four dynamic ports per application context.  It looks like you were seeing two.  

Everything else looks like it aligns!
Jacob

Jacob D. Eisinger
IBM Emerging Technologies
jeising@us.ibm.com - (512) 286-6075


Inactive hide details for Andrew Ash ---05/23/2014 10:30:58 AM---Hi everyone, I've also been interested in better understandingAndrew Ash ---05/23/2014 10:30:58 AM---Hi everyone, I've also been interested in better understanding what ports are used where

From: Andrew Ash <andrew@andrewash.com>
To: user@spark.apache.org
Date: 05/23/2014 10:30 AM
Subject: Re: Comprehensive Port Configuration reference?





Hi everyone,

I've also been interested in better understanding what ports are used where and the direction the network connections go.  I've observed a running cluster and read through code, and came up with the below documentation addition.

https://github.com/apache/spark/pull/856

Scott and Jacob -- it sounds like you two have pulled together some of this yourselves for writing firewall rules.  Would you mind taking a look at this pull request and confirming that it matches your observations?  Wrong documentation is worse than no documentation, so I'd like to make sure this is right.

Cheers,
Andrew


On Wed, May 7, 2014 at 10:19 AM, Mark Baker <distobj@acm.org> wrote: