kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Anouncing Kafka Offset Monitor 0.1
Date Fri, 07 Mar 2014 18:59:47 GMT
This is really useful! I added it to the ecosystem page:
https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem

-Jay


On Fri, Mar 7, 2014 at 10:49 AM, Pierre Andrews <pierre@quantifind.com>wrote:

> Hello everyone,
>
> at Quantifind, we are big users of Kafka and we like it a lot!
> In a few use cases, we had to figure out if a queue was growing and how its
> consumers were behaving. There are a few command-line tools to try to
> figure out what's going on, but it's not always easy to debug and to see
> what has happened while everyone was sleeping.
>
> To be able to monitor our kafka queues and consumers, we thus developed a
> tiny web app that could tell us the log size of each topic in the brokers
> and the offsets of each consumers. That's very similar to what the kafka
> ConsumerOffsetChecker tool* is doing, but instead of having a one off
> snapshot, our app keeps an history and displays a nice graph of what's
> going on.
>
> You can find screenshots and more details here:
> http://quantifind.github.io/KafkaOffsetMonitor/
>
> the code is on github:
> https://github.com/quantifind/KafkaOffsetMonitor
>
> If you have kafka 0.8 setup, it's very easy to use:
>
>   1- download the current jar
>
> http://quantifind.github.io/KafkaOffsetMonitor/dist/KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar
>   2- run it, pointing at your kafka brokers:
>     java -cp KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar \
>      com.quantifind.kafka.offsetapp.OffsetGetterWeb \
>      --zk zk-server1,zk-server2 \
>      --port 8080 \
>      --refresh 10.seconds \
>      --retain 2.days
>    3- open your browser and point it to localhost:8080
>
> You can run it locally or host it on a server if you prefer.
>
> It's all open source and we'll be happy to receive issue report and pull
> requests.
>
> I hope that you like it and that it can find some uses in the rest of the
> Kafka community.
>
> Best
>
> Pierre
>
> PS: we are aware of https://github.com/claudemamo/kafka-web-console but
> are
> currently offering slightly different features. Hopefully we can merge the
> projects in the future.
>
> * here:
>
> https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/ConsumerOffsetChecker.scala
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message