kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Morin <steve.mo...@gmail.com>
Subject Re: Anouncing Kafka Offset Monitor 0.1
Date Fri, 07 Mar 2014 20:08:18 GMT
Very nice

> On Mar 7, 2014, at 11:55, Pierre Andrews <pierre@quantifind.com> wrote:
> 
> Claude, we should join forces ;)
> 
> 
>> On Fri, Mar 7, 2014 at 4:45 PM, Claude Mamo <claude.mamo@gmail.com> wrote:
>> 
>> Awesome!!! ;-)
>> 
>> Claude
>> 
>> 
>> On Fri, Mar 7, 2014 at 4:03 PM, Pierre Andrews <pierre@quantifind.com
>>> wrote:
>> 
>>> Great! Thanks!
>>> 
>>> 
>>>> On Fri, Mar 7, 2014 at 3:59 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
>>>> 
>>>> This is really useful! I added it to the ecosystem page:
>>>> https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem
>>>> 
>>>> -Jay
>>>> 
>>>> 
>>>> On Fri, Mar 7, 2014 at 10:49 AM, Pierre Andrews <pierre@quantifind.com
>>>>> wrote:
>>>> 
>>>>> Hello everyone,
>>>>> 
>>>>> at Quantifind, we are big users of Kafka and we like it a lot!
>>>>> In a few use cases, we had to figure out if a queue was growing and
>> how
>>>> its
>>>>> consumers were behaving. There are a few command-line tools to try to
>>>>> figure out what's going on, but it's not always easy to debug and to
>>> see
>>>>> what has happened while everyone was sleeping.
>>>>> 
>>>>> To be able to monitor our kafka queues and consumers, we thus
>>> developed a
>>>>> tiny web app that could tell us the log size of each topic in the
>>> brokers
>>>>> and the offsets of each consumers. That's very similar to what the
>>> kafka
>>>>> ConsumerOffsetChecker tool* is doing, but instead of having a one off
>>>>> snapshot, our app keeps an history and displays a nice graph of
>> what's
>>>>> going on.
>>>>> 
>>>>> You can find screenshots and more details here:
>>>>> http://quantifind.github.io/KafkaOffsetMonitor/
>>>>> 
>>>>> the code is on github:
>>>>> https://github.com/quantifind/KafkaOffsetMonitor
>>>>> 
>>>>> If you have kafka 0.8 setup, it's very easy to use:
>>>>> 
>>>>>  1- download the current jar
>> http://quantifind.github.io/KafkaOffsetMonitor/dist/KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar
>>>>>  2- run it, pointing at your kafka brokers:
>>>>>    java -cp KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar \
>>>>>     com.quantifind.kafka.offsetapp.OffsetGetterWeb \
>>>>>     --zk zk-server1,zk-server2 \
>>>>>     --port 8080 \
>>>>>     --refresh 10.seconds \
>>>>>     --retain 2.days
>>>>>   3- open your browser and point it to localhost:8080
>>>>> 
>>>>> You can run it locally or host it on a server if you prefer.
>>>>> 
>>>>> It's all open source and we'll be happy to receive issue report and
>>> pull
>>>>> requests.
>>>>> 
>>>>> I hope that you like it and that it can find some uses in the rest of
>>> the
>>>>> Kafka community.
>>>>> 
>>>>> Best
>>>>> 
>>>>> Pierre
>>>>> 
>>>>> PS: we are aware of
>> https://github.com/claudemamo/kafka-web-consolebut
>>>>> are
>>>>> currently offering slightly different features. Hopefully we can
>> merge
>>>> the
>>>>> projects in the future.
>>>>> 
>>>>> * here:
>> https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/ConsumerOffsetChecker.scala
>> 

Mime
View raw message