kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: perf testing
Date Mon, 24 Oct 2011 20:12:43 GMT
Yeah that is a good point can you update the wiki. Sounds like we need
iostat.

-Jay

On Mon, Oct 24, 2011 at 12:27 PM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> Thanks for starting this discussion Jay !
>
> >> dstat - This is the existing perf tool to get read/write/fs stats
>
> My only concern about dstat is that it doesn't seem to give some
> important IO stats like
>
> avgrq-sz - size of the write request after merging several writes
> together into one big write ( indicates how sequential the IO is)
> wrqm_s - write requests merged per sec (also, could indicate how much
> of the total IO is sequential and can be grouped into one write on the
> disk)
> w/s - write IO requests per sec (indicates how many flushes the kafka
> broker is doing per sec)
> % busy - indicates how busy the disks are.
>
> At the very least, %busy is what we would like to know. Maybe I
> couldn't pass the right options to dstat.
>
> Does anyone have more ideas about dstat or some other perf tool,
> preferably one that would work on Linux ?
>
> Thanks,
> Neha
>
> On Mon, Oct 24, 2011 at 12:06 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
> > Neha and I have been doing so work on perf testing. My past experience
> with
> > these perf and integration testing frameworks is that
> >
> >   1. This kind of testing is extremely important. At least as important
> as
> >   unit testing. A lot of the bugs that are caught in production could be
> >   caught be good integration tests but likely will never be caught be
> unit
> >   tests.
> >   2. It is hard to get all your pieces scripted up so that you can fully
> >   automate the perf analysis you want to do and run this every day.
> >   3. It requires dedicated hardware that doesn't change from day to day.
> >   4. One of the biggest problems is that perf code is always kind of an
> >   afterthought. As a result one never gets to a framework that is good
> enough
> >   to use. Instead you keep re-writing the same test harnesses over and
> over
> >   but with little tweaks for each new test you need to run, then throwing
> that
> >   code away because it is so specific it can't be reused.
> >
> > To hopefully help I started a wiki where we could work out some ideas for
> > this. The idea is basically just to dump out all the stats we have now to
> > CSV and do some analysis in R. Then script this up in a way that we can
> > craft "test scenarios" and run a bunch of these different configurations.
> >
> > Wiki here:
> > https://cwiki.apache.org/confluence/display/KAFKA/Performance+testing
> >
> > Would love to hear people's thoughts.
> >
> > -Jay
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message