samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debraj Manna <subharaj.ma...@gmail.com>
Subject Re: Running Samza with YARN Node label support
Date Wed, 18 Dec 2019 05:58:21 GMT
Thanks, Yang for replaying.

Yes, my use case is almost similar.

Can you let me know which app-def you are referring to? I am not able to
locate yarn.am.container.label in samza-configurations
<http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html>
.
Is there any samza project whose code I can refer to regarding the usage of
these configurations?

On Wed, Dec 18, 2019 at 7:42 AM Yang Zhang <zhan3248@umn.edu> wrote:

> Hello Debraj,
>
> We do not have a formal documentation in open source to describe how yarn
> node label is used in general. In contrast, we have an example of using
> yarn node label to specify Samza container to run over "HDD" rather than
> default "SSD" nodes. Please take a look at the following guide and let us
> know whether it can be applied for your use case. Thank you for reporting
> this issue!
> =================================================Step-by-step guide
>
>
>    1.
>
>    Add the *yarn.container.label *and* yarn.am.container.label* to the
>    job's *app-def* if not already present. The default of an empty string
>    will keep the current default behavior of using SSD nodes.
>    <?xml version="1.0" encoding="UTF-8"?>
>    <application xmlns="urn:com:linkedin:ns:configuration:definition:1.0"
>    name="my-application" version="">
>        <configuration-definition>
>            <property name="yarn.am.container.label" default="" /> <!-- the
>    label used for launching the application master -->
>            <property name="yarn.container.label" default="" /> <!-- the
>    label used for other containers -->
>        </configuration-definition>
>    </application>
>    2. If you had to modify your *app-def* in step 1, you will need to do a
>    trigger-build to get the change to take effect.
>    3.
>
>    Add the label to *application.src* for your job. The *hdd* label will
>    assign your containers to machines with spinning disks instead of solid
>    state disks.
>    <?xml version="1.0" encoding="UTF-8"?>
>    <application xmlns="urn:com:linkedin:ns:configuration:source:1.0"
>    name="my-application">
>      <configuration-source>
>        <property name="yarn.container.label" value="hdd" />
>        <property name="yarn.am.container.label" value="hdd" />
>      </configuration-source>
>    </application>
>
>
>    4.
>
>    Deploy.
>
> =================================================
>
>
> Best,
>
> Yang
>
> On Tue, Dec 17, 2019 at 10:13 AM Debraj Manna <subharaj.manna@gmail.com>
> wrote:
>
> > Hi
> >
> > I am seeing running samza with yarn node label is resolved in 0.12.
> >
> >
> >
> https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel
> >
> > But I am not able to locate the relevant documentation in samza-yarn
> > documentation
> > <
> https://samza.apache.org/learn/documentation/latest/deployment/yarn.html>
> >
> > Can someone point me to the relevant documentation?
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message