spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Hanson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4563) Allow spark driver to bind to different ip then advertise ip
Date Fri, 24 Feb 2017 02:44:45 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881819#comment-15881819
] 

Alex Hanson commented on SPARK-4563:
------------------------------------

I have a similar issue where I'm running Spark v1.6.3 and wanted to add to what Danny suggested
above for a Spark 1.6 solution. My Spark cluster is running standalone with nothing else running
on those nodes. My solution uses iptables to forward traffic for the internal Docker IP (which
I can't seem to configure the Spark cluster to not expose, at least as of v1.6) to the Docker
Host IP, which works because the network on which the Docker host resides is a 10.10.0.0/16
network, and the Docker internal network is 172.18.0.0/16. Since there are no other 172.18.*.*
addresses, putting these rules in place on the Spark nodes doesn't collide with anything else
on the host network.

I know I'm not the only one who can't make the jump from 1.6 to 2.0 (or 2.1) quite yet due
to production applications, so I wanted to add my comment here that might help others through
similar challenges.

> Allow spark driver to bind to different ip then advertise ip
> ------------------------------------------------------------
>
>                 Key: SPARK-4563
>                 URL: https://issues.apache.org/jira/browse/SPARK-4563
>             Project: Spark
>          Issue Type: Improvement
>          Components: Deploy
>            Reporter: Long Nguyen
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>             Fix For: 2.1.0
>
>
> Spark driver bind ip and advertise is not configurable. spark.driver.host is only bind
ip. SPARK_PUBLIC_DNS does not work for spark driver. Allow option to set advertised ip/hostname



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message