kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: new log.dirs property (as opposed to log.dir)
Date Thu, 15 Aug 2013 17:04:56 GMT
Thanks Jay, I'll do some testing with this and report back.

Jason


On Thu, Aug 15, 2013 at 7:10 AM, Jay Kreps <jay.kreps@gmail.com> wrote:

> I believe either should work. The broker has a record of what it should
> have in zk and will recreate any missing logs. Try it to make sure though.
>
> Sent from my iPhone
>
> On Aug 15, 2013, at 12:52 AM, Jason Rosenberg <jbr@squareup.com> wrote:
>
> > Ok, that makes sense that the broker will shut itself down.
> >
> > If we bring it back up, can this be with an altered set of log.dirs?
>  Will
> > the destroyed partitions get rebuilt on a new log.dir?  Or do we have to
> > bring it back up with a new or repaired disk, matching the old log.dir,
> in
> > order for those replicas to be rebuilt?
> >
> > Jason
> >
> >
> > On Wed, Aug 14, 2013 at 4:16 PM, Jay Kreps <jay.kreps@gmail.com> wrote:
> >
> >> If you get a disk error that results in an IOException the broker will
> >> shut itself down. You would then have the option of replacing the disk
> or
> >> deleting that data directory from the list. When the broker is brought
> back
> >> up the intact partitions will quickly catch up and be online; the
> destroyed
> >> partitions will have to fully rebuild off the other replicas and will
> take
> >> a little longer but will automatically come back online once they have
> >> restored off the replicas.
> >>
> >> -jay
> >>
> >> Sent from my iPhone
> >>
> >> On Aug 14, 2013, at 1:49 PM, Jason Rosenberg <jbr@squareup.com> wrote:
> >>
> >>> I'm getting ready to try out this configuration (use multiple disks, no
> >>> RAID, per broker).  One concern is the procedure for recovering if
> there
> >> is
> >>> a disk failure.
> >>>
> >>> If a disk fails, will the broker go offline, or will it continue
> serving
> >>> partitions on its remaining good disks?  And if so, is there a
> procedure
> >>> for moving the partitions that were on the failed disk, but not
> >> necessarily
> >>> all the others on that broker?
> >>>
> >>> Jason
> >>>
> >>>
> >>> On Thu, Jun 20, 2013 at 3:15 PM, Jason Rosenberg <jbr@squareup.com>
> >> wrote:
> >>>
> >>>> yeah, that would work!
> >>>>
> >>>>
> >>>> On Thu, Jun 20, 2013 at 1:20 PM, Jay Kreps <jay.kreps@gmail.com>
> wrote:
> >>>>
> >>>>> Yeah we didn't go as far as adding weighting or anything like that--I
> >>>>> think we'd be open to a patch that did that as long as it was
> >>>>> optional. In the short term you can obviously add multiple
> directories
> >>>>> on the same disk to increase its share.
> >>>>>
> >>>>> -Jay
> >>>>>
> >>>>> On Thu, Jun 20, 2013 at 12:59 PM, Jason Rosenberg <jbr@squareup.com>
> >>>>> wrote:
> >>>>>> This sounds like a great idea, to just disks as "just a bunch
of
> >> disks"
> >>>>> or
> >>>>>> JBOD.....hdfs works well this way.
> >>>>>>
> >>>>>> Do all the disks need to be the same size, to use them evenly?
>  Since
> >> it
> >>>>>> will allocate partitions randomly?
> >>>>>>
> >>>>>> It would be nice if you had 2 disks, with one twice as large
as the
> >>>>> other,
> >>>>>> if the larger would be twice as likely to receive partitions
as the
> >>>>> smaller
> >>>>>> one, etc.
> >>>>>>
> >>>>>> I suppose this goes into my earlier question to the list, vis-a-vis
> >>>>>> heterogeneous brokers (e.g. utilize brokers with different sized
> >>>>> storage,
> >>>>>> using some sort of weighting scheme, etc.).
> >>>>>>
> >>>>>> Jason
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Jun 20, 2013 at 11:07 AM, Jay Kreps <jay.kreps@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> The intention is to allow the use of multiple disks without
RAID or
> >>>>>>> logical volume management. We have found that there are
a lot of
> >>>>>>> downsides to RAID--in particular a huge throughput hit.
Since we
> >>>>>>> already have a parallelism model due to partitioning and
a fault
> >>>>>>> tolerance model with replication RAID doesn't actually buy
much.
> With
> >>>>>>> this feature you can directly mount multiple disks as their
own
> >>>>>>> directory and the server will randomly assign partitions
to them.
> >>>>>>>
> >>>>>>> Obviously this will only work well if there are enough
> >> high-throughput
> >>>>>>> partitions to make load balance evenly (e.g. if you have
only one
> big
> >>>>>>> partition per server then this isn't going to work).
> >>>>>>>
> >>>>>>> -Jay
> >>>>>>>
> >>>>>>> On Wed, Jun 19, 2013 at 11:01 PM, Jason Rosenberg <
> jbr@squareup.com>
> >>>>>>> wrote:
> >>>>>>>> is it possible for a partition to have multiple replicas
on
> >> different
> >>>>>>>> directories on the same broker?  (hopefully no!)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Jun 19, 2013 at 10:47 PM, Jun Rao <junrao@gmail.com>
> wrote:
> >>>>>>>>
> >>>>>>>>> It takes a comma separated list and partition replicas
are
> randomly
> >>>>>>>>> distributed to the list.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>
> >>>>>>>>> Jun
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 19, 2013 at 10:25 PM, Jason Rosenberg
<
> >> jbr@squareup.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> In the 0.8 config, log.dir is now log.dirs.
 It looks like the
> >>>>>>> singular
> >>>>>>>>>> log.dir is still supported, but under the covers
the property is
> >>>>>>>>> log.dirs.
> >>>>>>>>>>
> >>>>>>>>>> I'm curious, does this take a comma separated
list of
> directories?
> >>>>>>> The
> >>>>>>>>> new
> >>>>>>>>>> config page just says:
> >>>>>>>>>> "The directories in which the log data is kept"
> >>>>>>>>>>
> >>>>>>>>>> Also, how does kafka handle multiple directories?
 Does it treat
> >>>>> each
> >>>>>>>>>> directory as a separate replica partition, or
what?
> >>>>>>>>>>
> >>>>>>>>>> Jason
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message