kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: some operational details
Date Fri, 09 Mar 2012 22:25:00 GMT
This is for a single topic. For all topics combined we do about 8 billion
messages/day.

-Jay

On Fri, Mar 2, 2012 at 3:48 PM, Andrew Psaltis <Andrew.Psaltis@webtrends.com
> wrote:

> Neha,
> That makes sense, and the totals at the top are the24 hour total.
>
> Thanks,
> Andrew
>
> -----Original Message-----
> From: Neha Narkhede [mailto:neha.narkhede@gmail.com]
> Sent: Friday, March 02, 2012 4:30 PM
> To: kafka-users@incubator.apache.org
> Subject: Re: some operational details
>
> Well, each of the data points in that graph represents the number of
> messages produced for a 10 minute time period.
>
> Thanks,
> Neha
>
> On Tue, Feb 14, 2012 at 6:16 PM, Andrew Psaltis <
> Andrew.Psaltis@webtrends.com> wrote:
> > And does this screenshot:
> https://issues.apache.org/jira/secure/attachment/12513709/Picture+18.pngcorrectly represent
the amount of messages that are processed in a 24hr
> period?
> >
> > -----Original Message-----
> > From: Jay Kreps [mailto:jay.kreps@gmail.com]
> > Sent: Tuesday, February 14, 2012 5:52 PM
> > To: kafka-users@incubator.apache.org
> > Subject: Re: some operational details
> >
> > Yeah technically that is not 100% correct. We have tuned about 10 topics
> by adding more partitions to add parallelism.
> >
> > -Jay
> >
> > On Mon, Feb 13, 2012 at 1:03 AM, Pierre-Yves Ritschard <pyr@spootnik.org
> >wrote:
> >
> >> one partition only ? so the key here is that you start as many
> >> brokers as there are consumers ?
> >>
> >> On Sat, Feb 11, 2012 at 9:29 PM, Andrew Psaltis
> >> <Andrew.Psaltis@webtrends.com> wrote:
> >> > Jay and Neha,
> >> > Is there information you can provide to help put the hardware and
> >> > config
> >> settings in context? Perhaps, the number of topics, messages per day
> >> or per second, and anything else that may help understand the
> >> performance you get with the given configuration. I know there are
> >> many other variables at play that would preclude one, from making the
> >> leap that given the same setup we get the same performance.
> >> >
> >> > Thanks,
> >> > Andrew
> >> >
> >> > -----Original Message-----
> >> > From: Neha Narkhede [mailto:neha.narkhede@gmail.com]
> >> > Sent: Friday, February 10, 2012 10:48 PM
> >> > To: kafka-users@incubator.apache.org
> >> > Subject: Re: some operational details
> >> >
> >> > This is very useful. I will add details to the zookeeper section soon.
> >> >
> >> > Thanks,
> >> > Neha
> >> >
> >> > On Friday, February 10, 2012, Jay Kreps <jay.kreps@gmail.com> wrote:
> >> >> Hey All,
> >> >>
> >> >> Various people have asked for details on our configuration and
> >> >> monitoring for kafka. I thought it might be helpful to start
> >> >> writing it
> >> up.
> >> >>  https://cwiki.apache.org/confluence/display/KAFKA/Operations
> >> >>
> >> >> If anyone else has tips to share, please feel free to add to that
> wiki.
> >> >>
> >> >> -Jay
> >> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message