samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liu Bo <diabl...@gmail.com>
Subject Re: No samza consumer group found
Date Wed, 16 Mar 2016 06:10:12 GMT
Hi David

we also have this monitoring requirement

one way I can think of is periodically reading the latest checkpoint
information from the corresponding check_point topic, and then compare it
with the topic offset.

it's not that straight forward comparing to existing tools such as yours,
and requires additional effort to integrate with dashboard tools such as
grafana.

I think this group have better ways to do this ;-)


On 16 March 2016 at 13:51, David Yu <david.yu@optimizely.com> wrote:

> I asked simply because I would like to monitor the topic consumed by our
> samza job using tools like this:
> https://github.com/quantifind/KafkaOffsetMonitor
>
>
>
> On Tue, Mar 15, 2016 at 9:13 PM, Jagadish Venkatraman <
> jagadish1989@gmail.com> wrote:
>
> > Hi David,
> >
> > Samza is using the simple consumer - I'm not aware of a notion of group
> id.
> > Others will comment on that.
> >
> > As far as I know, the client id for a particular consumer instance is
> > defined by a combination of the job name, the current timestamp and a
> > sequence number.
> >
> > val clientId = KafkaUtil.getClientId("samza-consumer", config)
> >
> > KafkaUtil.getClientId has this logic.
> >
> > I'm curious about your usecase as to why you are interested in inspecting
> > this information?
> >
> > Thanks,
> > Jagadish
> >
> > On Tuesday, March 15, 2016, David Yu <david.yu@optimizely.com> wrote:
> >
> > > Our samza job is consuming from a Kafka topic. AFAIU, samza will auto
> > > assign the job a consumer group id and client id. However, I'm not able
> > to
> > > see that showing up under zookeeper. Am I missing something?
> > >
> >
>



-- 
All the best

Liu Bo

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message