kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Drew Goya <d...@gradientx.com>
Subject Re: upgrade from beta1 to 0.81
Date Sat, 21 Dec 2013 00:05:34 GMT
This is the exception I ran into, I was able to fix it by deleting the
/data/kafka/logs/Events2-124/ directory.  That directory contained a non
zero size index file and a zero size log file.  I had a bunch of these
directories scattered around the cluster.

[2013-12-18 02:40:37,163] FATAL Fatal error during KafkaServerStable
startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.IllegalArgumentException: requirement failed: Corrupt index
found, index file (/data/kafka/logs/Events2-124/00000000000000000000.index)
has non-zero size but the last offset is 0 and the base offset is 0
at scala.Predef$.require(Predef.scala:145)
at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:160)
at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:159)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at
scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
at
scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
at kafka.log.Log.loadSegments(Log.scala:159)
at kafka.log.Log.<init>(Log.scala:64)
at
kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:120)
at
kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:115)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:115)
at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:107)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
at kafka.log.LogManager.loadLogs(LogManager.scala:107)
at kafka.log.LogManager.<init>(LogManager.scala:59)


On Fri, Dec 20, 2013 at 9:06 AM, Jun Rao <junrao@gmail.com> wrote:

> Drew,
>
> Even without kafka-1074, brokers shouldn't hit exceptions during startup.
> What exceptions do you see?
>
> Thanks,
>
> Jun
>
>
> On Thu, Dec 19, 2013 at 9:36 PM, Drew Goya <drew@gradientx.com> wrote:
>
> > We migrated from 0.8.0 to 0.8.1 last week.  We have a 15 broker cluster
> so
> > it took a while to roll through them one by one.  Once I finished I was
> > finally able to complete a partition reassignment.  I also had to do some
> > manual cleanup, but Neha says it will be fixed soon:
> >
> > https://issues.apache.org/jira/browse/KAFKA-1074
> >
> > Until then, if you have done any partition reassignment you will have to
> > watch your brokers as they come up.  They may fail and you will have to
> go
> > delete the empty partition directories.
> >
> >
> > On Thu, Dec 19, 2013 at 11:07 AM, Guozhang Wang <wangguoz@gmail.com>
> > wrote:
> >
> > > 0.8.1 is working in stable at LinkedIn now.
> > >
> > > Guozhang
> > >
> > >
> > > On Thu, Dec 19, 2013 at 10:52 AM, Yu, Libo <libo.yu@citi.com> wrote:
> > >
> > > > I also want to know how stable the 0.81 will be, compared with 0.8 or
> > > > 0.8-beta1.
> > > >
> > > > Regards,
> > > >
> > > > Libo
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Jason Rosenberg [mailto:jbr@squareup.com]
> > > > Sent: Thursday, December 19, 2013 12:54 PM
> > > > To: users@kafka.apache.org
> > > > Subject: Re: upgrade from beta1 to 0.81
> > > >
> > > > How stable is 0.8.1, will there be a 'release' of this soon, or are
> > there
> > > > still significant open issues?
> > > >
> > > > Thanks,
> > > >
> > > > Jason
> > > >
> > > >
> > > > On Thu, Dec 19, 2013 at 12:17 PM, Guozhang Wang <wangguoz@gmail.com>
> > > > wrote:
> > > >
> > > > > Libo, yes the upgrade from 0.8 to 0.8.1 can be done in place.
> > > > >
> > > > > Guozhang
> > > > >
> > > > >
> > > > > On Thu, Dec 19, 2013 at 8:57 AM, Yu, Libo <libo.yu@citi.com>
> wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > As the tools in 0.8 are not stable and we don't want to take
the
> > > > > > risk. we want to skip 0.8 and upgrade from beta1 to 0.81
> directly.
> > > > > > So my question is whether we can do an in place upgrade and
let
> > 0.81
> > > > > > use beta1's zk and kf data.
> > > > > > Assume
> > > > > > that we will disable log compaction. Thanks.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Libo
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > -- Guozhang
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message