chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <>
Subject Re: log rotation
Date Wed, 27 Jan 2010 21:08:29 GMT
This thread reminded me of another related question I've been meaning to
ask. My implementation is similar to the one you describe Ari, where we tail
a single file indefinitely, but periodically it gets rotated by the Linux
logrotate script, which handles cleanup of old files (I don't think Chukwa
should handle this necessarily).

My question is how does the agent handle the character offset marker when a
file it's tailing gets truncated during rotation? Does it just reset it back
to 0 when it sees that the file suddenly has no content?

On Wed, Jan 27, 2010 at 10:48 AM, Ariel Rabkin <> wrote:

> Suppose you have a file, foo, with a FileTailingAdaptor watching it.
> After rotation, you'll have a foo.1 and then another foo.  The
> FileTailingAdaptor will look at the latter.
> After those files have been collected, it's safe to delete them.
> Right now, Chukwa doesn't do this itself. Should it?
> --Ari
> On Wed, Jan 27, 2010 at 10:18 AM, Corbin Hoenes <> wrote:
> > I've noticed some chatter about log rotation and FileTailingAdaptor:
> >
> > Is there some kind of documentation on how it deals with say a directory
> of files that get rotated.  At first glance it looks like it keeps an
> adaptor open forever and there doesn't appear to be a way to know that a
> file has been processed by the adaptor so we can't really move files out,
> etc...  Right now we are piping apache access logs through rotatelogs and
> then using chukwa to tail these files.  Is there a better way?
> >
> >
> >
> >
> --
> Ari Rabkin
> UC Berkeley Computer Science Department

View raw message