samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxim Logvinenko <mlogvine...@gmail.com>
Subject Re: Review Request 51633: SAMZA-1013: Add YARN Node label support
Date Fri, 07 Oct 2016 00:09:06 GMT


> On Oct. 5, 2016, 6:18 p.m., Jagadish Venkatraman wrote:
> > samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java,
line 187
> > <https://reviews.apache.org/r/51633/diff/1/?file=1491248#file1491248line187>
> >
> >     nit: containerLabel so that we distinguish it from the AM label.

done


> On Oct. 5, 2016, 6:18 p.m., Jagadish Venkatraman wrote:
> > samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala, line 97
> > <https://reviews.apache.org/r/51633/diff/1/?file=1491249#file1491249line97>
> >
> >     nit: rename this to appMasterLabel (to avoid a conflict with containerLabel).

done


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51633/#review151519
-----------------------------------------------------------


On Oct. 7, 2016, 12:08 a.m., Maxim Logvinenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51633/
> -----------------------------------------------------------
> 
> (Updated Oct. 7, 2016, 12:08 a.m.)
> 
> 
> Review request for samza.
> 
> 
> Bugs: SAMZA-1013
>     https://issues.apache.org/jira/browse/SAMZA-1013
> 
> 
> Repository: samza
> 
> 
> Description
> -------
> 
> YARN Node labels were introduced in Hadoop version 2.6, which allows to group nodes with
similar characteristics and allows applications to specify where to run. This patch adds support
for YARN node labels in Samza.
> 
> In this implementation, node labels are defined directly in yarnConfig in YarnClusterResourceManager.
It might be better to have node labels as a part of SamzaResourceRequest and SamzaResource
classes, but org.apache.hadoop.yarn.api.records.Container class doesn't contain node label
and hence we have nothing to pass to the SamzaResource constructor in onContainersAllocated
method of YarnClusterResourceManager class.
> 
> 
> Diffs
> -----
> 
>   samza-yarn/src/main/java/org/apache/samza/config/YarnConfig.java 8f2dc48 
>   samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
96d3d7c 
>   samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala 0998c43 
> 
> Diff: https://reviews.apache.org/r/51633/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Maxim Logvinenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message