spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron <aarongm...@gmail.com>
Subject Spark Driver "behind" NAT
Date Mon, 05 Jan 2015 12:41:00 GMT
Hello there, I was wondering if there is a way to have the spark-shell (or
pyspark) sit behind a NAT when talking to the cluster?

Basically, we have OpenStack instances that run with internal IPs, and we
assign floating IPs as needed.  Since the workers make direct TCP
connections back, the spark-shell is binding to the internal IP..not the
"floating."  Our other use case is running Vagrant VMs on our local
machines..but, we don't have those VMs' NICs setup in "bridged" mode..it
too has an "internal" IP.

I tried using the SPARK_LOCAL_IP, and the various --conf spark.driver.host
parameters...but it still get's "angry."

Any thoughts/suggestions?

Currently our work around is to VPNC connection from inside the vagrant VMs
or Openstack instances...but, that doesn't seem like a long term plan.

Thanks in advance!

Cheers,
Aaron

Mime
View raw message