kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: Consumers in the same group but consuming different topics?
Date Thu, 15 Dec 2016 16:33:24 GMT
Hi Avi,

Technically you can, but not without writing some code. If you want to use
consumer groups then you would need to write a custom PartitionAssignor and
configure it in your Consumer Config, like so:
consumerProps.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
YourPartitionAssignor.class.getName());

Alternatively you could use manual partition assignment to temporarily work
around it.

Thanks,
Damian

On Thu, 15 Dec 2016 at 16:24 Avi Flax <avi.flax@parkassist.com> wrote:

> I’m trying to debug something about a Kafka Streams app, and it would be
> helpful to me to start up a new instance of the app that’s only consuming
> from a subset of the topics from which this app consumes. I’m hesitating
> though because I don’t know if the consumer group scheme will support this
> scenario, or if it’ll screw something up.
>
> So: can I run consumer A on machine A, consuming from topics 1,2,3, and
> consumer B on machine B, consuming from topics 4,5,6, but have both
> consumers be in the same group? I know this sounds silly and is counter to
> the whole point of consumer groups, but it’d just be temporary, and just
> for debugging — I need to observe the log output of a consumer instance
> when it’s consuming from a specific set of topics, and this seems like one
> way to do that. Would this break anything though?
>
> Thanks!
> Avi
>
> ————
> Software Architect @ Park Assist
> We’re hiring! http://tech.parkassist.com/jobs/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message