lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-753) Use NIO positional read to avoid synchronization in FSIndexInput
Date Thu, 17 Jul 2008 18:19:31 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-753:
--------------------------------------

    Attachment: FSDirectoryPool.patch


I attached FSDirectoryPool.patch, which adds
oal.store.FSDirectoryPool, a Directory that will open a new file for
every unique thread.

This is intended only as a test (to see if shows consistent
improvement in concurrency) -- eg it does not close all these files,
nor make any effort to budget itself if there are too many threads,
it's not really a pool, etc.  But it should give us an upper bound on
the gains we could hope for.

I also added a "pool=true|false" config option to contrib/benchmark so
you can run tests with and without separate files.

I ran some quick initial tests but didn't see obvious gains.  I'll go
back & re-run more carefully to confirm, and post back.


> Use NIO positional read to avoid synchronization in FSIndexInput
> ----------------------------------------------------------------
>
>                 Key: LUCENE-753
>                 URL: https://issues.apache.org/jira/browse/LUCENE-753
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>            Reporter: Yonik Seeley
>         Attachments: FileReadTest.java, FileReadTest.java, FileReadTest.java, FileReadTest.java,
FileReadTest.java, FileReadTest.java, FileReadTest.java, FSDirectoryPool.patch, FSIndexInput.patch,
FSIndexInput.patch, lucene-753.patch, lucene-753.patch
>
>
> As suggested by Doug, we could use NIO pread to avoid synchronization on the underlying
file.
> This could mitigate any MT performance drop caused by reducing the number of files in
the index format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message