lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-6386) TestIndexWriterForceMerge still unreliable in NIGHTLY
Date Thu, 02 Apr 2015 08:34:52 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392380#comment-14392380
] 

Shai Erera commented on LUCENE-6386:
------------------------------------

bq. s14 really should have been s13.cfs

You're right, my mistake. The example should have listed {{s13.cfs}}.

bq. We do NOT delete level 1 segments after merging to the level 2 non-CFS segment before
creating the result s14

I see. That explains the 4X. But then what happens if there are multi-level merges? Do we
delete any of these files? Let me give a concrete example:

{noformat}
source: [s1,s2] [s3,s4] [s5,s6] [s7,s8]
level1: [s9,s10], [s11,s12]
level2: [s13,s14]
level3: [s15]
final:  [s15.cfs]
{noformat}

Would we take 5X disk space in that case??

> TestIndexWriterForceMerge still unreliable in NIGHTLY
> -----------------------------------------------------
>
>                 Key: LUCENE-6386
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6386
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>            Assignee: Michael McCandless
>             Fix For: Trunk, 5.1
>
>         Attachments: LUCENE-6386.patch
>
>
> Discovered by ryan beasting (trunk):
> ant test -Dtestcase=TestIndexWriterForceMerge -Dtests.method=testForceMergeTempSpaceUsage
-Dtests.seed=DC9ADB74850A581B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true
-Dtests.locale=sr__#Latn -Dtests.timezone=Indian/Chagos -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
> {noformat}
>   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterForceMerge
-Dtests.method=testForceMergeTempSpaceUsage -Dtests.seed=DC9ADB74850A581B -Dtests.multiplier=2
-Dtests.nightly=true -Dtests.slow=true -Dtests.locale=sr__#Latn -Dtests.timezone=Indian/Chagos
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>    [junit4] FAILURE 1.20s | TestIndexWriterForceMerge.testForceMergeTempSpaceUsage <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError: forceMerge used too much
temporary space: starting usage was 291570 bytes; final usage was 262469 bytes; max temp usage
was 1079501 but should have been 874710 (= 3X starting usage), BEFORE=
>    [junit4]    > _u.scf              146329
>    [junit4]    > _u.si               635
>    [junit4]    >  |- (inside compound file) _u.fld              2214
>    [junit4]    >  |- (inside compound file) _u.inf              392
>    [junit4]    >  |- (inside compound file) _u.len              2381
>    [junit4]    >  |- (inside compound file) _u.pst              36758
>    [junit4]    >  |- (inside compound file) _u.vec              104144
>    [junit4]    > _s.pst              1338
>    [junit4]    > _s.inf              392
>    [junit4]    > _s.fld              94
>    [junit4]    > _s.len              221
>    [junit4]    > _s.vec              3744
>    [junit4]    > _s.si               624
>    [junit4]    > _t.fld              94
>    [junit4]    > _t.len              221
>    [junit4]    > _t.pst              1338
>    [junit4]    > _t.inf              392
>    [junit4]    > _t.vec              3744
>    [junit4]    > _t.si               624
>    [junit4]    > _v.fld              94
>    [junit4]    > _v.pst              1338
>    [junit4]    > _v.inf              392
>    [junit4]    > _v.vec              3744
>    [junit4]    > _v.si               624
>    [junit4]    > _v.len              221
>    [junit4]    > _w.len              221
>    [junit4]    > _w.pst              1338
>    [junit4]    > _w.inf              392
>    [junit4]    > _w.fld              94
>    [junit4]    > _w.si               624
>    [junit4]    > _w.vec              3744
>    [junit4]    > _x.vec              3744
>    [junit4]    > _x.inf              392
>    [junit4]    > _x.pst              1338
>    [junit4]    > _x.fld              94
>    [junit4]    > _x.si               624
>    [junit4]    > _x.len              221
>    [junit4]    > _y.fld              94
>    [junit4]    > _y.pst              1338
>    [junit4]    > _y.inf              392
>    [junit4]    > _y.si               624
>    [junit4]    > _y.vec              3744
>    [junit4]    > _y.len              221
>    [junit4]    > _z.fld              94
>    [junit4]    > _z.pst              1338
>    [junit4]    > _z.inf              392
>    [junit4]    > _z.len              221
>    [junit4]    > _z.vec              3744
>    [junit4]    > _z.si               624
>    [junit4]    > _10.si              630
>    [junit4]    > _10.fld             94
>    [junit4]    > _10.pst             1338
>    [junit4]    > _10.inf             392
>    [junit4]    > _10.vec             3744
>    [junit4]    > _10.len             221
>    [junit4]    > _11.len             221
>    [junit4]    > _11.si              630
>    [junit4]    > _11.vec             3744
>    [junit4]    > _11.pst             1338
>    [junit4]    > _11.inf             392
>    [junit4]    > _11.fld             94
>    [junit4]    > _12.vec             3744
>    [junit4]    > _12.si              630
>    [junit4]    > _12.len             221
>    [junit4]    > _12.fld             94
>    [junit4]    > _12.pst             1338
>    [junit4]    > _12.inf             392
>    [junit4]    > _13.fld             94
>    [junit4]    > _13.vec             3744
>    [junit4]    > _13.si              630
>    [junit4]    > _13.pst             1338
>    [junit4]    > _13.inf             392
>    [junit4]    > _13.len             221
>    [junit4]    > _14.fld             94
>    [junit4]    > _14.pst             1338
>    [junit4]    > _14.inf             392
>    [junit4]    > _14.si              630
>    [junit4]    > _14.vec             3744
>    [junit4]    > _14.len             221
>    [junit4]    > _15.len             221
>    [junit4]    > _15.vec             3744
>    [junit4]    > _15.si              630
>    [junit4]    > _15.pst             1338
>    [junit4]    > _15.inf             392
>    [junit4]    > _15.fld             94
>    [junit4]    > _16.vec             3744
>    [junit4]    > _16.len             221
>    [junit4]    > _16.fld             94
>    [junit4]    > _16.si              630
>    [junit4]    > _16.pst             1338
>    [junit4]    > _16.inf             392
>    [junit4]    > _17.vec             3744
>    [junit4]    > _17.pst             1338
>    [junit4]    > _17.inf             392
>    [junit4]    > _17.len             221
>    [junit4]    > _17.si              630
>    [junit4]    > _17.fld             94
>    [junit4]    > _18.pst             1338
>    [junit4]    > _18.inf             392
>    [junit4]    > _18.len             221
>    [junit4]    > _18.vec             3744
>    [junit4]    > _18.si              630
>    [junit4]    > _18.fld             94
>    [junit4]    > _19.fld             94
>    [junit4]    > _19.si              630
>    [junit4]    > _19.len             221
>    [junit4]    > _19.vec             3744
>    [junit4]    > _19.pst             1338
>    [junit4]    > _19.inf             392
>    [junit4]    > _1a.fld             94
>    [junit4]    > _1a.pst             1338
>    [junit4]    > _1a.inf             392
>    [junit4]    > _1a.len             221
>    [junit4]    > _1a.vec             3744
>    [junit4]    > _1a.si              630
>    [junit4]    > _1b.fld             94
>    [junit4]    > _1b.si              630
>    [junit4]    > _1b.pst             1338
>    [junit4]    > _1b.inf             392
>    [junit4]    > _1b.vec             3744
>    [junit4]    > _1b.len             221
>    [junit4]    > _1c.vec             3744
>    [junit4]    > _1c.pst             1338
>    [junit4]    > _1c.inf             392
>    [junit4]    > _1c.fld             94
>    [junit4]    > _1c.len             221
>    [junit4]    > _1c.si              630
>    [junit4]    > _1d.pst             1338
>    [junit4]    > _1d.inf             392
>    [junit4]    > _1d.fld             94
>    [junit4]    > _1d.len             221
>    [junit4]    > _1d.vec             3744
>    [junit4]    > _1d.si              630
>    [junit4]    > _1e.fld             94
>    [junit4]    > _1e.pst             1338
>    [junit4]    > _1e.inf             392
>    [junit4]    > _1e.len             221
>    [junit4]    > _1e.vec             3744
>    [junit4]    > _1e.si              630
>    [junit4]    > _1f.fld             40
>    [junit4]    > _1f.pst             195
>    [junit4]    > _1f.inf             392
>    [junit4]    > _1f.len             149
>    [junit4]    > _1f.vec             405
>    [junit4]    > _1f.si              629
>    [junit4]    > AFTER=
>    [junit4]    > _1j.si              635
>    [junit4]    > _1j.scf             261701
>    [junit4]    >  |- (inside compound file) _1j.fld             3982
>    [junit4]    >  |- (inside compound file) _1j.inf             392
>    [junit4]    >  |- (inside compound file) _1j.len             4149
>    [junit4]    >  |- (inside compound file) _1j.pst             66151
>    [junit4]    >  |- (inside compound file) _1j.vec             186577
>    [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DC9ADB74850A581B:C6581887EB1ABBDC]:0)
>    [junit4]    > 	at org.apache.lucene.index.TestIndexWriterForceMerge.testForceMergeTempSpaceUsage(TestIndexWriterForceMerge.java:199)
>    [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
>    [junit4]   2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=no):
{id=IB SPL-D2, content=DFR I(F)B1}, locale=sr__#Latn, timezone=Indian/Chagos
>    [junit4]   2> NOTE: Linux 3.13.0-45-generic amd64/Oracle Corporation 1.8.0_40-ea
(64-bit)/cpus=8,threads=1,free=208092000,total=253231104
>    [junit4]   2> NOTE: All tests run in this JVM: [TestIndexWriterForceMerge]
>    [junit4] Completed in 1.54s, 1 test, 1 failure <<< FAILURES!
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message