samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordi Blasi Uribarri <jbl...@nextel.es>
Subject RE: Relaunching init
Date Fri, 09 Oct 2015 06:49:11 GMT
At the moment I am using version 0.9.1 and I feel that it will be the solution when I switch
to the next version of my application. I have the project in a "don't touch if not really
needed" status.

Thanks,

	Jordi

-----Mensaje original-----
De: Navina Ramesh [mailto:nramesh@linkedin.com.INVALID] 
Enviado el: jueves, 08 de octubre de 2015 19:31
Para: dev@samza.apache.org
Asunto: Re: Relaunching init

Hi Jordi,

init() method is called only once during container start-up and not called again until the
container is restarted. Semantically, it doesn't seem fit to make "init" callable more than
once.

With Samza 0.10 (that is, the current master branch), you can pass configuration via a broadcast
stream. You can configure one system stream partition to be consumed by all stream tasks.
This way, whenever you write to the broadcast stream, you can reset or modify your config.
I don't think we have the documentation for this on the website, yet.

Here is how you configure a broadcast stream:
Let's say your broadcast stream is a kafka stream called "my-broadcast-stream" and has 1 partition.
You can configure your job with:
task.global.inputs = kafka.my-broadcast-stream#0

Format is $system-name.$stream-name#$partition-number

You can also make multiple partitions broadcast by specifying a range.
Format is $system-name.$stream-name#[$partitionStart-$partitionEnd]

In the above case, if your broadcast stream has 3 partitions that you want to broadcast, your
configuration looks like:
task.global.inputs = kafka.my-broadcast-stream#[0-2]

Data from broadcast stream will be available like any other input stream in the process()
method. You may have to change your app logic to handle the config change. This is one way
for satisfying your use-case.

Let me know if this solution will work for you.

Thanks!
Navina


On Thu, Oct 8, 2015 at 2:45 AM, Jordi Blasi Uribarri <jblasi@nextel.es>
wrote:

> Hi,
>
> Is there a way of forcing the init of a job being rerun without 
> killing the job and restarting it?
>
> I have some configurations loaded in a job that are obtained in the 
> public void init(Config config, TaskContext context) function. When 
> the configuration changes and I need it to be reaplied the only way I 
> find is to completely kill the job and start it. This is a process 
> that takes quite a long. I don’t really know why but until I have the 
> messages flowing again I can stay for several minutes waiting.
>
> Is there a way to do this in any other way? I am thinking of some kind 
> of kill –HUP like in BIND servers (ok, I had a time as sysadmin ☺ ) or 
> maybe a way of programmatically calling a running job to reload.
>
> Thanks ,
>
>   Jordi
> ________________________________
> Jordi Blasi Uribarri
> Área I+D+i
>
> jblasi@nextel.es
> Oficina Bilbao
>
> [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]
>



--
Navina R.
Mime
View raw message