samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aram Mkrtchyan <aram.mkrtch...@picsart.com.INVALID>
Subject [Multiple producers for one task]
Date Mon, 16 Nov 2015 11:36:10 GMT
Hi guys,

We're processing JSON data from Kafka using Samza, and we'd like to have a
single Samza Job that's able to process and produce the messages to
different systems.

For example, consume messages from kafka, and produce them to Cassandra,
Graphite and other systems, so that the messages are being consumed once.
We want this because the tasks themselves are very simple, and we don't
want to have separate samza jobs for them.

We'd like someone to compare possible approaches.

   1. Having Multiple producer systems for one task.
   2. Having Single producer which has registry of small message handlers,
   which process messages (synchronous/asynchronous)?
   3. Having Multiple Jobs is the only valid way of doing it.

Thanks.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message