spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Kalyanpur <kalyanpursuni...@gmail.com>
Subject Spark UI to use Marathon assigned port
Date Thu, 07 Sep 2017 16:43:32 GMT
Hello all,

I am running PySpark Job (v2.0.2) with checkpoint enabled in Mesos cluster
and am using Marathon for orchestration.

When the job is restarted using Marathon, Spark UI is not getting started
at the port specified by Marathon. Instead, it is picking port from the
checkpoint.

Is there a way we can make spark job to use the port assigned by Marathon
instead of Spark job picking the configuration from the Checkpoint?

Please let me know if you need any information.

Thank you,
Sunil

Mime
View raw message