spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Han JU <>
Subject Worker re-spawn and dynamic node joining
Date Wed, 14 May 2014 16:22:37 GMT
Hi all,

Just 2 questions:

  1. Is there a way to automatically re-spawn spark workers? We've
situations where executor OOM causes worker process to be DEAD and it does
not came back automatically.

  2. How to dynamically add (or remove) some worker machines to (from) the
cluster? We'd like to leverage the auto-scaling group in EC2 for example.

We're using spark-standalone.

Thanks a lot.

*JU Han*

Data Engineer @

+33 0619608888

View raw message