lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-2787) disable atime for DirectIOLinuxDirectory
Date Wed, 01 Dec 2010 16:51:11 GMT


Michael McCandless commented on LUCENE-2787:

+1, this is a no brainer.  I had no idea linux lets you turn off atime per file desriptor!

It's ridiculous that the OS maintains an atime on our index files.

Uwe, I agree about the intention of the man page (so eg back when contrib/benchmark used to
write 10,000 files to run its tests, and then index them, we could've seen a big speedup :)
), but still it can't hurt to also turn it off when opening the index files for reading.

I think atime is updated per-read not just at open ( though
I'm not sure.  Even so, it's presumably cached in the OS's write buffer and then only flushed
periodically, so I don't think we'll see sizable gains here.  But every bit counts so I think
we should do it.

> disable atime for DirectIOLinuxDirectory
> ----------------------------------------
>                 Key: LUCENE-2787
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: contrib/*
>            Reporter: Robert Muir
>             Fix For: 3.1, 4.0
>         Attachments: LUCENE-2787.patch
> In Linux's open():
>     (Since Linux 2.6.8) Do not update the file last access time (st_atime in the inode)
when the file is read(2). This flag is intended for use by indexing or backup programs, where
its use can significantly reduce the amount of disk activity. This flag may not be effective
on all filesystems. One example is NFS, where the server maintains the access time.
> So we should do this in our linux-specific DirectIOLinuxDirectory.
> Separately (offtopic), it would be better if this was a LinuxDirectory that only uses
O_DIRECT when it should :)
> It would be nice to think about an optional modules/native for common platforms similar
to what tomcat provides
> Its easier to test directories like this now (

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message