spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Hanson (JIRA)" <>
Subject [jira] [Commented] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip
Date Fri, 24 Feb 2017 02:44:45 GMT


Alex Hanson commented on SPARK-4563:

I have a similar issue where I'm running Spark v1.6.3 and wanted to add to what Danny suggested
above for a Spark 1.6 solution. My Spark cluster is running standalone with nothing else running
on those nodes. My solution uses iptables to forward traffic for the internal Docker IP (which
I can't seem to configure the Spark cluster to not expose, at least as of v1.6) to the Docker
Host IP, which works because the network on which the Docker host resides is a
network, and the Docker internal network is Since there are no other 172.18.*.*
addresses, putting these rules in place on the Spark nodes doesn't collide with anything else
on the host network.

I know I'm not the only one who can't make the jump from 1.6 to 2.0 (or 2.1) quite yet due
to production applications, so I wanted to add my comment here that might help others through
similar challenges.

> Allow spark driver to bind to different ip then advertise ip
> ------------------------------------------------------------
>                 Key: SPARK-4563
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Deploy
>            Reporter: Long Nguyen
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>             Fix For: 2.1.0
> Spark driver bind ip and advertise is not configurable. is only bind
ip. SPARK_PUBLIC_DNS does not work for spark driver. Allow option to set advertised ip/hostname

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message