hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: More than one RM in YARN?
Date Wed, 19 Sep 2012 18:04:18 GMT
Sorry about that. There is some code in the RM that's supposed to be used
to save state but its not used. So neither state preserving restart works
nor failover.

I have been guilty of being tardy on this project but I hope to be working
on the restart piece fairly soon. Please do watch MAPREDUCE-4326 (this
should soon move to the YARN sub-project) for details.


-----Original Message-----
From: Chris Riccomini [mailto:criccomini@linkedin.com]
Sent: Wednesday, September 19, 2012 10:21 AM
To: mapreduce-dev@hadoop.apache.org
Subject: Re: More than one RM in YARN?

Hey Bikas,


Initially, I'd hoped that NMs failed over to appropriate RMs via ZK
notifications. When that didn't seem to be implemented, I hoped that RMs,
at least kept in sync with each other via ZK, so that I could use a round
robin DNS (or HW load balancer) for the RMs. When that didn't seem to be
implemented, I decided to just use DNS and a manual failover.

Simple is best, I suppose.


On 9/19/12 10:18 AM, "Bikas Saha" <bikas@hortonworks.com> wrote:

>If I understand you correctly, your target scenario is failover?
>Its good to know that you have something that works for you right now.
>-----Original Message-----
>From: Chris Riccomini [mailto:criccomini@linkedin.com]
>Sent: Wednesday, September 19, 2012 10:06 AM
>To: mapreduce-dev@hadoop.apache.org
>Subject: Re: More than one RM in YARN?
>Hey Bikas,
>Ideally, I'd like to run them concurrently. What I'm discovering is
>that this is not (currently) possible, so I'm settling for a manual
>fail over using DNS, which should suffice for the time being.
>On 9/19/12 10:05 AM, "Bikas Saha" <bikas@hortonworks.com> wrote:
>>Could you please elaborate a bit on the use case you are targeting? Do
>>you want to run multiple active RM's in the cluster? And they are
>>being used concurrently? Or are you talking about failover scenarios?
>>-----Original Message-----
>>From: Harsh J [mailto:harsh@cloudera.com]
>>Sent: Tuesday, September 18, 2012 9:37 PM
>>To: mapreduce-dev@hadoop.apache.org
>>Subject: Re: More than one RM in YARN?
>>Not yet, it is coming:
>>https://issues.apache.org/jira/browse/MAPREDUCE-4326 should go in
>>first, followed by the HA work after it via
>>https://issues.apache.org/jira/browse/MAPREDUCE-4345 (Latter isn't a
>>dupe of the former exactly).
>>On Wed, Sep 19, 2012 at 6:12 AM, Chris Riccomini
>>> Hey Guys,
>>> Is anyone running more than one RM on a YARN cluster (backed by ZK)?
>>We're looking at this, and I want to know if there's a standard way to
>>do it. Are people doing round robin DNS, load balancers, etc? Is it
>>possible to configure node managers to have a list of RMs rather than
>single one?
>>> Thanks!
>>> Chris
>>Harsh J

View raw message