lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+140) - Build # 18103 - Unstable!
Date Fri, 21 Oct 2016 10:19:33 GMT
We had similar tests before though, with BS1 vs BS2. Because BS1
scored out-of-order, it performed additions in different order, too.

But we did the math in these boolean scorers with double precision
internally and this kept the test happy (I am sure, if you had a
massive massive BQ it would have eventually failed).

What changed?

Sorry, I think as soon as you start adding leniency here, the test is
instantly worthless. You might as well delete it, because you can no
longer tell if a difference in score "matters" or not.

On Fri, Oct 21, 2016 at 6:15 AM, Adrien Grand <jpountz@gmail.com> wrote:
> I don't think this test failure proves that the optimization is broken. The
> other failure is a bit easier to understand so I'll use it as an example,
> but this one is the same. We have an initial query which is ((((#body:d
> (+body:b (body:d body:a))^4.0 +body:c body:d)^4.0)^8.0)^3.0)^9.0 and we are
> trying to check that we get the same scores if we rewrite it or if we don't.
>
> For that particular query, the rewritten form is:
> ((+body:b (body:d body:a))^4.0 +body:c +body:d)^864.0
>
> The only boolean simplification that happened is that body:d was made a MUST
> clause since it was both a FILTER and a SHOULD clause. The rest is only
> about simplifications of nested BoostQueries. The score is different due to
> the fact that additions are performed in a different order, which is
> expected with floats. And then the difference is amplified by the fact that
> this particular query has a high boost.
>
> Le ven. 21 oct. 2016 à 11:30, Robert Muir <rcmuir@gmail.com> a écrit :
>>
>> Personally I think that is not correct to do. Some optimization or
>> another is broken if scores differ in this way...
>>
>> Just because differences are "small" does not make them insignificant.
>> Think about the tie-break case to users and so on, especially with
>> smaller documents / less terms / etc.
>>
>> Even 1 ulp is gonna mess stuff up, i do not believe leniency in the
>> tests is the solution.
>>
>>
>>
>> On Fri, Oct 21, 2016 at 3:05 AM, Adrien Grand <jpountz@gmail.com> wrote:
>> > It is the same one as the other day, we need to relax a bit the check on
>> > scores somehow.
>> >
>> > Le ven. 21 oct. 2016 à 08:40, Policeman Jenkins Server
>> > <jenkins@thetaphi.de>
>> > a écrit :
>> >>
>> >> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18103/
>> >> Java: 64bit/jdk-9-ea+140 -XX:+UseCompressedOops -XX:+UseParallelGC
>> >>
>> >> 1 tests failed.
>> >> FAILED:  org.apache.lucene.search.TestBooleanRewrites.testRandom
>> >>
>> >> Error Message:
>> >> expected:<2048.99267578125> but was:<2048.992919921875>
>> >>
>> >> Stack Trace:
>> >> java.lang.AssertionError: expected:<2048.99267578125> but
>> >> was:<2048.992919921875>
>> >>         at
>> >>
>> >> __randomizedtesting.SeedInfo.seed([DDF676110AA095BA:AFBA531EBBC023C9]:0)
>> >>         at org.junit.Assert.fail(Assert.java:93)
>> >>         at org.junit.Assert.failNotEquals(Assert.java:647)
>> >>         at org.junit.Assert.assertEquals(Assert.java:443)
>> >>         at org.junit.Assert.assertEquals(Assert.java:512)
>> >>         at
>> >>
>> >> org.apache.lucene.search.TestBooleanRewrites.assertEquals(TestBooleanRewrites.java:427)
>> >>         at
>> >>
>> >> org.apache.lucene.search.TestBooleanRewrites.testRandom(TestBooleanRewrites.java:367)
>> >>         at
>> >>
>> >> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native
>> >> Method)
>> >>         at
>> >>
>> >> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
>> >>         at
>> >>
>> >> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
>> >>         at
>> >> java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>> >>         at
>> >>
>> >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>> >>         at
>> >>
>> >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> >>         at
>> >>
>> >> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> >>         at
>> >>
>> >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>> >>         at java.lang.Thread.run(java.base@9-ea/Thread.java:843)
>> >>
>> >>
>> >>
>> >>
>> >> Build Log:
>> >> [...truncated 1309 lines...]
>> >>    [junit4] Suite: org.apache.lucene.search.TestBooleanRewrites
>> >>    [junit4]   2> NOTE: reproduce with: ant test
>> >> -Dtestcase=TestBooleanRewrites -Dtests.method=testRandom
>> >> -Dtests.seed=DDF676110AA095BA -Dtests.multiplier=3 -Dtests.slow=true
>> >> -Dtests.locale=qu -Dtests.timezone=JST -Dtests.asserts=true
>> >> -Dtests.file.encoding=UTF-8
>> >>    [junit4] FAILURE 0.05s J1 | TestBooleanRewrites.testRandom <<<
>> >>    [junit4]    > Throwable #1: java.lang.AssertionError:
>> >> expected:<2048.99267578125> but was:<2048.992919921875>
>> >>    [junit4]    >        at
>> >>
>> >> __randomizedtesting.SeedInfo.seed([DDF676110AA095BA:AFBA531EBBC023C9]:0)
>> >>    [junit4]    >        at
>> >>
>> >> org.apache.lucene.search.TestBooleanRewrites.assertEquals(TestBooleanRewrites.java:427)
>> >>    [junit4]    >        at
>> >>
>> >> org.apache.lucene.search.TestBooleanRewrites.testRandom(TestBooleanRewrites.java:367)
>> >>    [junit4]    >        at
>> >>
>> >> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native
>> >> Method)
>> >>    [junit4]    >        at
>> >>
>> >> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
>> >>    [junit4]    >        at
>> >>
>> >> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
>> >>    [junit4]    >        at
>> >> java.lang.Thread.run(java.base@9-ea/Thread.java:843)
>> >>    [junit4]   2> NOTE: test params are:
>> >>
>> >> codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION,
>> >> chunkSize=16269, maxDocsPerChunk=9, blockSize=4),
>> >>
>> >> termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION,
>> >> chunkSize=16269, blockSize=4)), sim=ClassicSimilarity, locale=qu,
>> >> timezone=JST
>> >>    [junit4]   2> NOTE: Linux 4.4.0-42-generic amd64/Oracle Corporation
>> >> 9-ea (64-bit)/cpus=12,threads=1,free=153179392,total=333971456
>> >>    [junit4]   2> NOTE: All tests run in this JVM:
>> >> [TestStandardAnalyzer,
>> >> TestByteBlockPool, TestNumericUtils, Test2BPostingsBytes,
>> >> TestSumDocFreq,
>> >> TestCharTermAttributeImpl, TestByteArrayDataInput,
>> >> TestLucene50StoredFieldsFormat, TestPhraseQuery, MultiCollectorTest,
>> >> TestShardSearching, TestQueryRescorer, Test2BBKDPoints,
>> >> TestIndexReaderClose, TestSpanContainQuery, TestSloppyMath,
>> >> TestLongPostings, TestSmallFloat, TestTimSorterWorstCase,
>> >> TestStressNRT,
>> >> TestMultiDocValues, TestFieldCacheRewriteMethod,
>> >> TestSameTokenSamePosition,
>> >> TestCheckIndex, TestHugeRamFile, TestPagedBytes, TestEarlyTermination,
>> >> TestTermVectorsWriter, TestDemoParallelLeafReader,
>> >> TestDocumentsWriterStallControl, TestAutomatonQueryUnicode,
>> >> TestOmitNorms,
>> >> TestIntBlockPool, TestMultiset, TestSearcherManager, TestIndexSearcher,
>> >> TestIndexWriterDelete, TestLiveFieldValues,
>> >> TestScoreCachingWrappingScorer,
>> >> TestDocument, TestDirectPacked, TestFieldsReader,
>> >> TestSimpleFSDirectory,
>> >> TestPayloads, TestAutomaton, FuzzyTermOnShortTermsTest,
>> >> TestSortedSetSortField, TestSortRandom, TestBooleanQuery,
>> >> TestIntArrayDocIdSet, TestCharArraySet, TestRadixSelector,
>> >> TestSleepingLockWrapper, TestTermdocPerf, TestDocsAndPositions,
>> >> TestNamedSPILoader, TestRecyclingIntBlockAllocator,
>> >> TestIndexWriterUnicode,
>> >> TestStressAdvance, TestFixedLengthBytesRefArray,
>> >> TestConsistentFieldNumbers,
>> >> Test2BDocs, TestMultiFields, TestDocumentsWriterDeleteQueue,
>> >> TestMutablePointsReaderUtils, TestIndexableField,
>> >> TestExitableDirectoryReader, TestSortRescorer, TestDuelingCodecs,
>> >> TestNRTReaderCleanup, TestCharsRef, TestIndexFileDeleter,
>> >> TestIndexWriterMerging, TestFSTs, TestIndexWriterWithThreads,
>> >> TestMultiMMap,
>> >> TestBasics, TestIndexWriterMergePolicy, TestAtomicUpdate,
>> >> TestHighCompressionMode, TestTopDocsMerge, TestTieredMergePolicy,
>> >> TestLevenshteinAutomata, TestByteSlices, TestNRTCachingDirectory,
>> >> TestParallelCompositeReader, TestIndexWriterNRTIsCurrent, TestCrash,
>> >> TestSegmentTermDocs, TestPerFieldPostingsFormat2, TestTermsEnum2,
>> >> TestPostingsOffsets, TestTransactionRollback, TestFlex,
>> >> TestStressIndexing2,
>> >> TestBagOfPostings, TestWildcard, TestSimilarity2,
>> >> TestFieldMaskingSpanQuery,
>> >> TestExternalCodecs, TestNoDeletionPolicy, TestRegexpQuery,
>> >> TestPerSegmentDeletes, TestSegmentTermEnum,
>> >> TestComplexExplanationsOfNonMatches, TestPrefixCodedTerms,
>> >> TestAllFilesHaveCodecHeader, TestTermScorer, TestMultiLevelSkipList,
>> >> TestSearchForDuplicates, TestCompiledAutomaton,
>> >> TestElevationComparator,
>> >> TestSubScorerFreqs, TestConjunctions, TestPositionIncrement,
>> >> TestSpanExplanationsOfNonMatches, TestSetOnce, TestFilterIterator,
>> >> TestPriorityQueue, TestAttributeSource, TestBooleanScorer,
>> >> TestAssertions,
>> >> TestTwoPhaseCommitTool, TestIndexWriterOnJRECrash, Test4GBStoredFields,
>> >> TestNGramPhraseQuery, TestSimpleAttributeImpl, TestIndexCommit,
>> >> TestIOUtils,
>> >> TestNewestSegment, Test2BPagedBytes, TestStopFilter,
>> >> TestPackedTokenAttributeImpl, TestCodecUtil,
>> >> TestGrowableByteArrayDataOutput, TestLucene50TermVectorsFormat,
>> >> TestAllFilesHaveChecksumFooter, TestDuelingCodecsAtNight,
>> >> TestFilterDirectoryReader, TestIndexWriterMaxDocs,
>> >> TestIndexWriterOnVMError,
>> >> TestIndexWriterOutOfFileDescriptors, TestInfoStream, TestManyFields,
>> >> TestMaxPosition, TestMergePolicyWrapper, TestMergeRateLimiter,
>> >> TestMultiTermsEnum, TestNumericDocValuesUpdates, TestOrdinalMap,
>> >> TestPointValues, TestReadOnlyIndex, TestReaderWrapperDVTypeCheck,
>> >> TestSegmentInfos, TestSwappedIndexFiles, TestTermVectors, TestTerms,
>> >> TestTragicIndexWriterDeadlock, TestTryDelete, TestBlendedTermQuery,
>> >> TestBooleanRewrites]
>> >>    [junit4] Completed [340/434 (1!)] on J1 in 0.06s, 10 tests, 1
>> >> failure
>> >> <<< FAILURES!
>> >>
>> >> [...truncated 61242 lines...]
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message