samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chen Song <chen.song...@gmail.com>
Subject Re: question on yarn.container.cpu.cores
Date Thu, 19 Nov 2015 04:40:28 GMT
Thanks Navina

So theoretically I can create a thread pool within a container. I know it
is very hacky but it should increase parallelism.

Chen

On Mon, Nov 16, 2015 at 5:49 PM, Navina Ramesh <nramesh@linkedin.com> wrote:

> Hi Chen,
> Samza container is still single threaded. In case of yarn based deployment,
> Samza uses this config value to verify that the cluster has sufficient
> capacity to support running your job.
>
> Apart from this verification, I don't believe we utilize this config value.
> If you set it to > 1, it won't have any effect on the Samza job execution
> itself. However, you may end-up under-utilizing your Yarn cluster
> resources.
>
> HTH!
> Navina
>
> On Mon, Nov 16, 2015 at 2:32 PM, Chen Song <chen.song.82@gmail.com> wrote:
>
> > According to the documentation, each Samza container is single threaded.
> > Why giving yarn.container.cpu.cores as a config option and what is the
> > implication
> > to set this to a value > 1?
> >
> > --
> > Chen Song
> >
>
>
>
> --
> Navina R.
>



-- 
Chen Song

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message