samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navina Ramesh <nram...@linkedin.com.INVALID>
Subject Re: Relaunching init
Date Thu, 08 Oct 2015 17:31:10 GMT
Hi Jordi,

init() method is called only once during container start-up and not called
again until the container is restarted. Semantically, it doesn't seem fit
to make "init" callable more than once.

With Samza 0.10 (that is, the current master branch), you can pass
configuration via a broadcast stream. You can configure one system stream
partition to be consumed by all stream tasks. This way, whenever you write
to the broadcast stream, you can reset or modify your config.
I don't think we have the documentation for this on the website, yet.

Here is how you configure a broadcast stream:
Let's say your broadcast stream is a kafka stream called
"my-broadcast-stream" and has 1 partition. You can configure your job with:
task.global.inputs = kafka.my-broadcast-stream#0

Format is $system-name.$stream-name#$partition-number

You can also make multiple partitions broadcast by specifying a range.
Format is $system-name.$stream-name#[$partitionStart-$partitionEnd]

In the above case, if your broadcast stream has 3 partitions that you want
to broadcast, your configuration looks like:
task.global.inputs = kafka.my-broadcast-stream#[0-2]

Data from broadcast stream will be available like any other input stream in
the process() method. You may have to change your app logic to handle the
config change. This is one way for satisfying your use-case.

Let me know if this solution will work for you.

Thanks!
Navina


On Thu, Oct 8, 2015 at 2:45 AM, Jordi Blasi Uribarri <jblasi@nextel.es>
wrote:

> Hi,
>
> Is there a way of forcing the init of a job being rerun without killing
> the job and restarting it?
>
> I have some configurations loaded in a job that are obtained in the public
> void init(Config config, TaskContext context) function. When the
> configuration changes and I need it to be reaplied the only way I find is
> to completely kill the job and start it. This is a process that takes quite
> a long. I don’t really know why but until I have the messages flowing again
> I can stay for several minutes waiting.
>
> Is there a way to do this in any other way? I am thinking of some kind of
> kill –HUP like in BIND servers (ok, I had a time as sysadmin ☺ ) or maybe a
> way of programmatically calling a running job to reload.
>
> Thanks ,
>
>   Jordi
> ________________________________
> Jordi Blasi Uribarri
> Área I+D+i
>
> jblasi@nextel.es
> Oficina Bilbao
>
> [http://www.nextel.es/wp-content/uploads/Firma_Nextel_2015.png]
>



-- 
Navina R.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message