samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagadish Venkatraman <jagadish1...@gmail.com>
Subject Re: Can't start container in newly added hosts
Date Mon, 17 Jul 2017 15:17:58 GMT
These log lines are benign, and are at the INFO level.

Host affinity means - Samza will ensure that your containers start up on
the same host each time and re-use the on-disk snapshot of your RocksDb
state (instead of bootstrapping the entire state from Kafka's changelog).
This feature is beneficial for jobs that store a lot of data in RocksDb.

Please refer
https://samza.apache.org/learn/documentation/0.13/yarn/yarn-host-affinity.html
for more information.


On Mon, Jul 17, 2017 at 3:07 AM, 舒琦 <shuqi@eefung.com> wrote:

> Hi Guys,
>
>         After add 3 hosts to yarn cluster, some job can’t startup, then I
> found some informations as shown below:
>
>         2017-07-17 17:37:29.928 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow79.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow6.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> YarnClusterResourceManager [INFO] Container allocated from RM on
> rflow5.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow79.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000007 in the buffer
> for ANY_HOST
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow6.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000008 in the buffer
> for ANY_HOST
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ContainerProcessManager [INFO] Container allocated from RM on
> rflow5.antfact.com
>
>         2017-07-17 17:37:29.929 [AMRM Callback Handler Thread]
> ResourceRequestState [INFO] Host affinity not enabled. Saving the
> samzaResource container_e38_1500040925911_0028_01_000010 in the buffer
> for ANY_HOST
>
>         What does “Host affinity” mean?
>
>         Thanks!
>
> ————————
> Qi Shu
>
>


-- 
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message