samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Mangi <r...@chartbeat.com>
Subject Re: Does Samza work with ResourceManager in HA?
Date Tue, 03 Nov 2015 19:55:06 GMT
Nope, we left it out.

Rick


> On Nov 3, 2015, at 11:04 AM, John Tipper <john_tipper@hotmail.com> wrote:
> 
> Thanks Rick, much appreciated.  Did you have to set yarn.resourcemanager.hostname or
can you leave this out?
> 
> Thanks,
> 
> John
> 
> ________________________________________
> From: Rick Mangi <rick@chartbeat.com>
> Sent: 03 November 2015 15:46
> To: dev@samza.apache.org
> Subject: Re: Does Samza work with ResourceManager in HA?
> 
> Hi John,
> 
> We just got this set up last week. I haven’t fully tested the failover but it certainly
works for testing out our samza jobs on a pretty large cluster.
> 
> All of the server names are fqdn in our live yarn-site.xml
> 
> Hope this helps,
> 
> Rick
> 
> 
> <property>
>  <name>yarn.resourcemanager.ha.enabled</name>
>  <value>true</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
>  <value>true</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.cluster-id</name>
>  <value>ops02</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.ha.rm-ids</name>
>  <value>rm1,rm2</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.hostname.rm1</name>
>  <value>yarnmaster01</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.hostname.rm2</name>
>  <value>yarnmaster02</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.webapp.address.rm1</name>
>  <value>yarnmaster01:8088</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.webapp.address.rm2</name>
>  <value>yarnmaster02:8088</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.zk-address</name>
>  <value>zk01:2181,zk02:2181,zk03:2181,zk04:2181,zk06:2181</va
> lue>
> </property>
> <property>
>  <name>yarn.resourcemanager.store.class</name>
>  <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
> </property>
> 
> <property>
>  <name>yarn.resourcemanager.recovery.enabled</name>
>  <value>true</value>
> </property>
> <property>
>  <name>yarn.resourcemanager.zk-state-store.parent-path</name>
>  <value>/yarn_ops02_rmstore</value>
> </property>
> 
> 
> 
>> On Nov 3, 2015, at 6:10 AM, John Tipper <john_tipper@hotmail.com> wrote:
>> 
>> 
>> 
>> Does anyone have Samza working with resource manager in HA? If so, what do I set
yarn.resourcemanager.hostname to in yarn-site.xml?  Can anyone share a working yarn-site.xml
please?
>> 
>> 
>> If I set it to the first of my RMs, the the job submission works ok if I submit the
job from that RM and the RM is the active one. If the RM machine that I run the job submission
from is not active, I get connection refused errors on port 8032.  If I don't set it, I get
errors where run-job.sh tries to submit to 0.0.0.0:8032
>> 
>> 
>> Many thanks,
>> 
>> 
>> John
> 


Mime
View raw message