samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thunder Stumpges <thunder.stump...@gmail.com>
Subject 0.14 to 1.x Low-Level application migration questions
Date Sat, 01 Jun 2019 18:14:57 GMT
Hey guys,

I'm following the guide here:
http://samza.apache.org/releases/1.0.0

In step 3 it says:
"In Samza 1.0, a Samza application’s input, output, and processing-task
should be specified in code, rather than in config. "

How much IS picked up from config? Will all the configuration of the
systems (consumer and producer properties, buffering, etc) be picked up
from the config properties still? What about stream settings like offset
reset, offset default, etc?

In some of my tasks, I have a dedicated coordinator system. I don't see how
to register that system via the ApplicationDescriptor, nor how to associate
it with the coordinator (config setting `*job.coordinator.system*`). It
seems that a KafkaSystem is only associated with the ApplicationDescriptor
via its Input/Output/Table descriptors. Is this correct?

I would like to keep my config in config, not in code, but it feels like
this is forcing me to move some (or all?) of it into code. I had custom
config re-writers which made this very flexible, but I'm not seeing how to
adapt this to the "new way". The Application/ApplicationDescriptor seems to
have no connection to the Configuration / properties...

One other thing, is that in a few of my jobs, we have dynamic output
SystemStream(s) created based on other runtime state. Is this not going to
be possible anymore?

A little more guidance would be most helpful.

Thanks!
Thunder Stumpges

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message