hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Kumara Subramanian <codin.mart...@gmail.com>
Subject Re: Application Master High Availability
Date Tue, 11 Aug 2020 22:53:13 GMT
Thanks Eric & Wilfred.
To give you some context, I work on Apache Samza and we have streaming as a
service offering on top of YARN.
Ideally, we would like to ensure in the event of AM restarts, the container
it spawned can still be taken over by the new AM.

"Keep containers across application attempts" seems like an option that
might work for us.
Let me investigate and play with the parameter.

Appreciate your quick response.


On Tue, Aug 11, 2020 at 6:31 AM Eric Payne <eric.payne1000@yahoo.com> wrote:

> Bharath,
> while there is no concept of HA AM in YARN, some frameworks do support
> preserving containers across AM restarts.
> In the yarn distributed shell, for example, you can set the
> "-keep_containers_across_application_attempts"
> parameter.
> -Eric
> On Monday, August 10, 2020, 7:18:18 PM CDT, Bharath Kumara Subramanian <
> codin.martial@gmail.com> wrote:
> Hi,
> I am looking for more documentation/information on AM high availability. I
> looked through the documentation and found resources on RM high
> availability but none for AM.
> I understand, YARN has provisions to restart AM in case of failure up to a
> configured number of attempts. However, I wanted to know if YARN has an
> active/standby option for AM. I would like to avoid bringing up the
> processing containers again in the event of AM failure and have the standby
> AM take over the managing my application.
> Thanks,
> Bharath

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message