kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barto Rael <abcduser...@gmail.com>
Subject Re: count of bytes stored in kafka
Date Sat, 28 Dec 2013 05:27:10 GMT
Hi Otis,

Thanks for response!

We are using kafka 0.8.0. I was looking for some jmx metrix that can give
count of messages/bytes stored in kafka.

If we look into data stores like hbase we can see  how many messages are
stored in hbase or if we look into search engine like elasticsearch then
also we can see  how many messages are stored/indexed in elasticsearch. In
similar way i was expecting that there should be direct way to see count
of  messages or bytes stored in kafka without using any external tool.

Is there any direct way to see count of messages or bytes stored in kafka ?




On Sat, Dec 28, 2013 at 3:32 AM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:

> Hi,
>
> Not a direct way to see this, but we use Kafka 0.7.2 and monitor the disk
> space on it, so that's one way to see *roughly* what is happening.
> If you use something like SPM to monitor your Kafka nodes (see
>
> http://blog.sematext.com/2013/10/16/announcement-spm-performance-monitoring-for-kafka/
> )
> and if broker stores data on the disk partition that nothing else
> writes
> to, then you can look at disk usage and read/write rates for that specific
> partition and thus get the more precise numbers.
>
> If there's a more direct way to see/get this number, I'm all eyeballs!
>
> Otis
> --
> Performance Monitoring * Log Analytics * Search Analytics
> Solr & Elasticsearch Support * http://sematext.com/
>
>
> On Thu, Dec 26, 2013 at 11:37 PM, Barto Rael <abcduser786@gmail.com>
> wrote:
>
> > Hi All,
> >
> > Can we see count of messages or bytes of a topic present in kafka
> > using jconsole or any other way ?
> >
> >
> > Jmx metric "BytesInperSec" doesn't show data stored in kafka.
> > As per my understanding:
> > "BytesInperSec" is count of bytes in a topic that have been received
> > by the broker since the broker process started. It is just counter
> > value that get incremented each time a new byte is received by broker.
> > The value is reset to 0 when broker is restarted.
> > Also the Byte count will not be decremented if any bytes are deleted
> > from broker.
> >
> > Is there any way to check count of bytes stored in kafka even after
> > kafka is restarted?
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message