samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Mangi <r...@chartbeat.com>
Subject Re: Does Samza work with ResourceManager in HA?
Date Tue, 03 Nov 2015 15:46:33 GMT
Hi John,

We just got this set up last week. I haven’t fully tested the failover but it certainly
works for testing out our samza jobs on a pretty large cluster.

All of the server names are fqdn in our live yarn-site.xml

Hope this helps,

Rick


<property>
  <name>yarn.resourcemanager.ha.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.cluster-id</name>
  <value>ops02</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.rm-ids</name>
  <value>rm1,rm2</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm1</name>
  <value>yarnmaster01</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm2</name>
  <value>yarnmaster02</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm1</name>
  <value>yarnmaster01:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm2</name>
  <value>yarnmaster02:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.zk-address</name>
  <value>zk01:2181,zk02:2181,zk03:2181,zk04:2181,zk06:2181</va
lue>
</property>
<property>
  <name>yarn.resourcemanager.store.class</name>
  <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
</property>

<property>
  <name>yarn.resourcemanager.recovery.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.zk-state-store.parent-path</name>
  <value>/yarn_ops02_rmstore</value>
</property>



> On Nov 3, 2015, at 6:10 AM, John Tipper <john_tipper@hotmail.com> wrote:
> 
> 
> 
> Does anyone have Samza working with resource manager in HA? If so, what do I set yarn.resourcemanager.hostname
to in yarn-site.xml?  Can anyone share a working yarn-site.xml please?
> 
> 
> If I set it to the first of my RMs, the the job submission works ok if I submit the job
from that RM and the RM is the active one. If the RM machine that I run the job submission
from is not active, I get connection refused errors on port 8032.  If I don't set it, I get
errors where run-job.sh tries to submit to 0.0.0.0:8032
> 
> 
> Many thanks,
> 
> 
> John


Mime
View raw message