samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang Zhang <zhan3...@umn.edu>
Subject Re: Running Samza with YARN Node label support
Date Wed, 18 Dec 2019 02:12:28 GMT
Hello Debraj,

We do not have a formal documentation in open source to describe how yarn
node label is used in general. In contrast, we have an example of using
yarn node label to specify Samza container to run over "HDD" rather than
default "SSD" nodes. Please take a look at the following guide and let us
know whether it can be applied for your use case. Thank you for reporting
this issue!
=================================================Step-by-step guide


   1.

   Add the *yarn.container.label *and* yarn.am.container.label* to the
   job's *app-def* if not already present. The default of an empty string
   will keep the current default behavior of using SSD nodes.
   <?xml version="1.0" encoding="UTF-8"?>
   <application xmlns="urn:com:linkedin:ns:configuration:definition:1.0"
   name="my-application" version="">
       <configuration-definition>
           <property name="yarn.am.container.label" default="" /> <!-- the
   label used for launching the application master -->
           <property name="yarn.container.label" default="" /> <!-- the
   label used for other containers -->
       </configuration-definition>
   </application>
   2. If you had to modify your *app-def* in step 1, you will need to do a
   trigger-build to get the change to take effect.
   3.

   Add the label to *application.src* for your job. The *hdd* label will
   assign your containers to machines with spinning disks instead of solid
   state disks.
   <?xml version="1.0" encoding="UTF-8"?>
   <application xmlns="urn:com:linkedin:ns:configuration:source:1.0"
   name="my-application">
     <configuration-source>
       <property name="yarn.container.label" value="hdd" />
       <property name="yarn.am.container.label" value="hdd" />
     </configuration-source>
   </application>


   4.

   Deploy.

=================================================


Best,

Yang

On Tue, Dec 17, 2019 at 10:13 AM Debraj Manna <subharaj.manna@gmail.com>
wrote:

> Hi
>
> I am seeing running samza with yarn node label is resolved in 0.12.
>
>
> https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel
>
> But I am not able to locate the relevant documentation in samza-yarn
> documentation
> <https://samza.apache.org/learn/documentation/latest/deployment/yarn.html>
>
> Can someone point me to the relevant documentation?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message