spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ☼ R Nair (रविशंकर नायर) <>
Subject Spark production scenario
Date Fri, 09 Mar 2018 03:09:58 GMT
Hi all,

We are going to move to production with an 8 node Spark cluster. Request
some help for below

We are running on YARN cluster manager.That means YARN is installed with
SSH between the nodes. When we run a standalone Spark program with
spark-submit, YARN initializes a resource manager followed by application
master per application. This is allocated randomely with arbitrary port.
So, would we be opening all ports in between the nodes in a production
implementation ?


View raw message