lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Woodward (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-8714) BaseNormsFormatTestCase should not use NoMergePolicy
Date Fri, 01 Mar 2019 10:24:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alan Woodward updated LUCENE-8714:
----------------------------------
    Description: 
Jenkins found a test failure that stems from a combination of LimitHandleFS and NoMergePolicy. 
There's no particular reason for BaseNormsFormatTestCase to prevent merges, so we should just
remove this IndexWriteConfig setting in the test.

[~hossman] writes on LUCENE-8585:

This is from branch_8_0...

{code}
  [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene70NormsFormat
-Dtests.method=testFewValues -Dtests.seed=C3613DC62817C401 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL -Dtests.timezone=Asia/Anadyr
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
  [junit4] ERROR   7.04s | TestLucene70NormsFormat.testFewValues <<<
  [junit4]    > Throwable #1: java.nio.file.FileSystemException: /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
Too many open files
  [junit4]    > at __randomizedtesting.SeedInfo.seed([C3613DC62817C401:E1CC12DA3AA6A4FA]:0)
  [junit4]    > at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
  [junit4]    > at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
  [junit4]    > at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:197)
  [junit4]    > at org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202)
  [junit4]    > at java.nio.channels.FileChannel.open(FileChannel.java:287)
  [junit4]    > at java.nio.channels.FileChannel.open(FileChannel.java:335)
  [junit4]    > at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)
  [junit4]    > at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2801)
  [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:747)
  [junit4]    > at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
  [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1069)
  [junit4]    > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:128)
  [junit4]    > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
  [junit4]    > at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
  [junit4]    > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
  [junit4]    > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
  [junit4]    > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
  [junit4]    > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
  [junit4]    > at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
  [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
  [junit4]    > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
  [junit4]    > at org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:494)
  [junit4]    > at org.apache.lucene.index.BaseNormsFormatTestCase.testFewValues(BaseNormsFormatTestCase.java:181)
  [junit4]    > at java.lang.Thread.run(Thread.java:748)
  [junit4]   2> NOTE: leaving temporary files on disk at: /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001
  [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{},
maxPointsInLeafNode=322, maxMBSortInHeap=7.12719968491226, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4d8a3cb),
locale=es-CL, timezone=Asia/Anadyr
  [junit4]   2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=292853816,total=321912832
  [junit4]   2> NOTE: All tests run in this JVM: [TestLucene70NormsFormat]
  [junit4] Completed [1/1 (1!)] in 7.48s, 1 test, 1 error <<< FAILURES!
{code}

(when running all methods of that test w/that seed, many of them fail _after_ this method,
with identical exceptions, but those same methods pass in isolation -- suggesting perhaps
leaked open files? or maybe only leaked on failure?)

Also: this test is called \{{TestLucene70NormsFormat}} but it has \{{new Lucene80Codec()}}
hardcoded in it ... which seems like a pretty big WTF?

  was:Jenkins found a test failure that stems from a combination of LimitHandleFS and NoMergePolicy. 
There's no particular reason for BaseNormsFormatTestCase to prevent merges, so we should just
remove this IndexWriteConfig setting in the test.


> BaseNormsFormatTestCase should not use NoMergePolicy
> ----------------------------------------------------
>
>                 Key: LUCENE-8714
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8714
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Alan Woodward
>            Assignee: Alan Woodward
>            Priority: Major
>         Attachments: LUCENE-8714.patch
>
>
> Jenkins found a test failure that stems from a combination of LimitHandleFS and NoMergePolicy. 
There's no particular reason for BaseNormsFormatTestCase to prevent merges, so we should just
remove this IndexWriteConfig setting in the test.
> [~hossman] writes on LUCENE-8585:
> This is from branch_8_0...
> {code}
>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLucene70NormsFormat
-Dtests.method=testFewValues -Dtests.seed=C3613DC62817C401 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL -Dtests.timezone=Asia/Anadyr
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>   [junit4] ERROR   7.04s | TestLucene70NormsFormat.testFewValues <<<
>   [junit4]    > Throwable #1: java.nio.file.FileSystemException: /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
Too many open files
>   [junit4]    > at __randomizedtesting.SeedInfo.seed([C3613DC62817C401:E1CC12DA3AA6A4FA]:0)
>   [junit4]    > at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
>   [junit4]    > at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
>   [junit4]    > at org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:197)
>   [junit4]    > at org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202)
>   [junit4]    > at java.nio.channels.FileChannel.open(FileChannel.java:287)
>   [junit4]    > at java.nio.channels.FileChannel.open(FileChannel.java:335)
>   [junit4]    > at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)
>   [junit4]    > at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2801)
>   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:747)
>   [junit4]    > at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
>   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1069)
>   [junit4]    > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:128)
>   [junit4]    > at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
>   [junit4]    > at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
>   [junit4]    > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
>   [junit4]    > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
>   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
>   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
>   [junit4]    > at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
>   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
>   [junit4]    > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
>   [junit4]    > at org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:494)
>   [junit4]    > at org.apache.lucene.index.BaseNormsFormatTestCase.testFewValues(BaseNormsFormatTestCase.java:181)
>   [junit4]    > at java.lang.Thread.run(Thread.java:748)
>   [junit4]   2> NOTE: leaving temporary files on disk at: /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001
>   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{},
maxPointsInLeafNode=322, maxMBSortInHeap=7.12719968491226, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4d8a3cb),
locale=es-CL, timezone=Asia/Anadyr
>   [junit4]   2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation 1.8.0_144
(64-bit)/cpus=4,threads=1,free=292853816,total=321912832
>   [junit4]   2> NOTE: All tests run in this JVM: [TestLucene70NormsFormat]
>   [junit4] Completed [1/1 (1!)] in 7.48s, 1 test, 1 error <<< FAILURES!
> {code}
> (when running all methods of that test w/that seed, many of them fail _after_ this method,
with identical exceptions, but those same methods pass in isolation -- suggesting perhaps
leaked open files? or maybe only leaked on failure?)
> Also: this test is called \{{TestLucene70NormsFormat}} but it has \{{new Lucene80Codec()}}
hardcoded in it ... which seems like a pretty big WTF?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message