spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yncxcw <>
Subject Re: Spark production scenario
Date Fri, 09 Mar 2018 06:49:19 GMT
hi, Passion

I don't know an exact solution. But yes, the port each executor chosen to
communicate with driver is random.  I am wondering if it's possible that you
can have a node has two ethernet card, configure one card for intranet for
Spark and configure one card for WAN. Then connect the rests nodes using the

And also, I think you might not use WAN for Spark data transfer since the
amount of data during shuffle is huge. You got to have a high-speed switch
for your cluster.

Hopes this answer can help you!


Sent from:

To unsubscribe e-mail:

View raw message