kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anishek Agarwal <anis...@gmail.com>
Subject Re: kafka 0.8.2 broker behaviour
Date Tue, 24 May 2016 09:34:45 GMT
yeh mostly they are up always, sometimes when we do new deployment they
might be down for about 15 mins or so. seems to have become slightly better
now, i changed the replica socket buffer size and also the number of
replica fetcher threads.



On Mon, May 23, 2016 at 6:24 PM, Gerard Klijs <gerard.klijs@dizzit.com>
wrote:

> Are you sure consumers are always up, when they are behind they could
> generate a lot of traffic in a small amount of time?
>
> On Mon, May 23, 2016 at 9:11 AM Anishek Agarwal <anishek@gmail.com> wrote:
>
> > additionally all the read / writes are happening via storm topologies.
> >
> > On Mon, May 23, 2016 at 12:17 PM, Anishek Agarwal <anishek@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > we are using 4 kafka machines in production with 4 topics and each
> topic
> > > either 16/32 partitions and replication factor of 2. each machine has 3
> > > disks for kafka logs.
> > >
> > > we see a strange behaviour where we see high disk usage spikes on one
> of
> > > the disks on all machines. it varies over time, with different disks
> > > showing spikes over time. is this normal .. i thought the partition per
> > > topic is distributed per disk and hence unless traffic is such that its
> > > hitting one partition this should not happen.
> > >
> > > any suggestions on how to verify what is happening will be helpful.
> > >
> > > Thanks
> > > anishek
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message