samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bae, Jae Hyeon" <>
Subject Question on Samza kafka consumer
Date Sat, 13 Dec 2014 00:51:15 GMT

I am almost done in PoC and finally I need to clear my several concerns
regarding resilience problem. Then hopefully, if everything goes well, you
will have the 2nd biggest Samza cluster out of LinkedIn within a year.

Samza is not using high level Kafka consumer but it still have zookeeper
dependency to discovery kafka broker and topic, partitions.

1. How does it handle rebalancing when we add more partitions to the topic?
Do we need to restart the job? Or does it start consuming automatically?

2. When the zookeeper dies, what will happen? If Samza kafka consumer is
not heavily dependent on zookeeper for distribution coordination, I think
that we don't need to restart jobs in an urgent way. ZkClient will try to
re-establish sessions but sometimes it cannot recover at all in the worse
case, we observed a few times so far.

Thank you
Best, Jae

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message