kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Palino <tpal...@gmail.com>
Subject Re: API to query cluster metadata on-demand
Date Thu, 03 Sep 2015 18:45:49 GMT
What Gwen said :)

We developed a python web service internally called Menagerie that provides
this functionality for both Kafka and Zookeeper. We use it to drive a web
dashboard for stats, our (old style) lag checking, and some other CLI
tools. Unfortunately it ties into too much internal LinkedIn tooling for us
to open source.

That's one of the reasons we released Burrow (
https://github.com/linkedin/Burrow). The primary use is to do lag checking
for consumers as a service. But I'm also moving functionality from
Menagerie into it. Right now you can use it to fetch topic lists, partition
counts, and broker offsets. You can also get information for consumers (as
long as they are committing offsets to Kafka and not ZK).

If it looks useful and there's some bit of info you'd like it to provide,
you can submit a github issue and I'll take a look at it.

-Todd

On Thursday, September 3, 2015, Andrew Otto <aotto@wikimedia.org> wrote:

> If you don’t mind doing it with a C CLI:
>
> https://github.com/edenhill/kafkacat
>
> $ kafkacat -L -b mybroker
>
> But, uhhh, you probably want a something in the Java API.
>
> :)
>
>
> > On Sep 3, 2015, at 13:58, Gwen Shapira <gwen@confluent.io <javascript:;>>
> wrote:
> >
> > Ah, I wish.
> >
> > We are working on it :)
> >
> > On Thu, Sep 3, 2015 at 9:10 AM, Simon Cooper <
> > simon.cooper@featurespace.co.uk <javascript:;>> wrote:
> >
> >> Is there a basic interface in the new client APIs to get the list of
> >> topics on a cluster, and get information on the topics (offsets, sizes,
> >> etc), without having to deal with a producer or consumer? I just want a
> >> basic synchronous API to query the metadata as-is. Does this exist in
> some
> >> form?
> >>
> >> Thanks,
> >> Simon
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message