samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sgg <sggraha...@gmail.com>
Subject Re: changing the number of containers in a Samza Job
Date Thu, 12 Dec 2013 03:47:05 GMT
ok thanks for the clarification Chris.  I had hoped it could be done on the fly, but I wasn't
counting on it :-)
On Dec 11, 2013, at 12:32 PM, Chris Riccomini <criccomini@linkedin.com> wrote:

> Hey sgg,
> 
> Yes, you can change the container count to any number where 1 <= container
> count <= max input partitions. The configuration to make this change is:
> 
>  yarn.container.count=1
> 
> All config in Samza is resolved at job start time, and remains immutable
> for the remainder of the job execution, so you can't dynamically change
> the container count. You need to restart the Samza job to update the
> configuration.
> 
> Cheers,
> Chris
> 
> On 12/11/13 8:47 AM, "sgg" <sggraham64@gmail.com> wrote:
> 
>> A part of the documentation
>> (http://samza.incubator.apache.org/learn/documentation/0.7.0/container/tas
>> k-runner.html) suggests it ?might? be possible to change the number of
>> containers allocated to a job:
>> 	"Your Samza job might be unable to keep up with only one container, so
>> you ask for a second YARN container to put some of the StreamTask
>> partitions"
>> 
>> How does one ask for a second YARN container?  Is it a matter of stoping
>> the job, changing the configuration and starting a new job?  or is there
>> some way to do this without starting a job?
>> 
>> sgg
> 


Mime
View raw message