lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-6684) TestSimpleFSLockFactory.testStressLock assert trip only on Windows?
Date Sun, 30 Aug 2015 09:24:45 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14721459#comment-14721459
] 

Michael McCandless commented on LUCENE-6684:
--------------------------------------------

Copying my email to the dev list about this:

OK I dug a bit, and this behavior is maybe "by design" in Windows:

Here's a java issue about it:

    https://bugs.openjdk.java.net/browse/JDK-8029608

Here's the "by design" explanation (though it's very old, I think it's
likely the semantics of NTFS have not changed in this regard):

    https://support.microsoft.com/en-us/kb/159199

I didn't realize a file on NTFS could be in this "pending delete"
state: I think this means the Files.delete call against the file
succeeded (maybe?) in one thread, but then other threads still have
the file open (in "shared access" mode).  This puts the file into
"pending delete" state such that 1) it is still included in the dir
listing, but 2) you cannot open it for read or write, nor delete it
again ( ? ).

If this interpretation is correct, it easily explains this failure,
and it might explain LUCENE-6684 if (maybe?) the original Files.delete
call fails w/ an IOException.

I'm going to disable these asserts for Windows: they are just causing
noisy false test failures.


> TestSimpleFSLockFactory.testStressLock assert trip only on Windows?
> -------------------------------------------------------------------
>
>                 Key: LUCENE-6684
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6684
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Michael McCandless
>
> It's happened at least twice before; here's the second occurrence:
> http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/5042/
> {noformat}
>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSimpleFSLockFactory
-Dtests.method=testStressLocks -Dtests.seed=86DA7F883D64B036 -Dtests.slow=true -Dtests.locale=th
-Dtests.timezone=America/Metlakatla -Dtests.asserts=true -Dtests.file.encoding=Cp1252
>    [junit4] ERROR   1.58s J1 | TestSimpleFSLockFactory.testStressLocks <<<
>    [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError:
Captured an uncaught exception in thread: Thread[id=62, name=Thread-35, state=RUNNABLE, group=TGRP-TestSimpleFSLockFactory]
>    [junit4]    > 	at __randomizedtesting.SeedInfo.seed([86DA7F883D64B036:D8EB317521C87850]:0)
>    [junit4]    > Caused by: java.lang.AssertionError: hit unexpected NoSuchFileException:
file=_3.cfs
>    [junit4]    > 	at __randomizedtesting.SeedInfo.seed([86DA7F883D64B036]:0)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:750)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deletePendingFiles(IndexFileDeleter.java:528)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:731)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4686)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriter$DeleteNewFilesEvent.process(DocumentsWriter.java:732)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4743)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4734)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3115)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3082)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1064)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1109)
>    [junit4]    > 	at org.apache.lucene.store.BaseLockFactoryTestCase$WriterThread.run(BaseLockFactoryTestCase.java:221)
>    [junit4] IGNOR/A 0.01s J1 | TestSimpleFSLockFactory.testDeleteLockFile
>    [junit4]    > Assumption #1: test requires the ability to delete a locked file(throwable:
java.io.IOException: cannot delete file: test.lock, a virus scanner has it open)
>    [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity,
locale=th, timezone=America/Metlakatla
>    [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=1,free=44713016,total=64880640
>    [junit4]   2> NOTE: All tests run in this JVM: [TestRAMDirectory, TestFastDecompressionMode,
TestSetOnce, TestRegexpQuery, TestDeterminism, TestCachingTokenFilter, TestIOUtils, TestSimpleFSLockFactory]
>    [junit4] Completed [16/398] on J1 in 5.05s, 7 tests, 1 error, 1 skipped <<<
FAILURES!
> {noformat}
> It means that somehow on flush IW was asked to remove a new file _3.cfs yet that file
does not exist.  I don't think this test generates fake (for testing) exceptions so it's especially
curious...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message