lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doron Cohen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1115) Some small fixes to contrib/benchmark
Date Wed, 02 Jan 2008 23:26:34 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555441#action_12555441
] 

Doron Cohen commented on LUCENE-1115:
-------------------------------------

Definitely a bug.
Patch looks good, and I like the optimization, thanks for fixing this Mike.

Perhaps rename in TaskSequence from *anyExhaustedTasks* to *anyExhaustableTasks*?

Also, this new test (belongs in TestPerfTaskLogic) passes with the fix but fails without it:
{code}
  /**
   * Test that exhaust in loop works as expected (LUCENE-1115).
   */
  public void testExhaustedLooped() throws Exception {
    // 1. alg definition (required in every "logic" test)
    String algLines[] = {
        "# ----- properties ",
        "doc.maker="+Reuters20DocMaker.class.getName(),
        "doc.add.log.step=3",
        "doc.term.vector=false",
        "doc.maker.forever=false",
        "directory=RAMDirectory",
        "doc.stored=false",
        "doc.tokenized=false",
        "debug.level=1",
        "# ----- alg ",
        "{ \"Rounds\"",
        "  ResetSystemErase",
        "  CreateIndex",
        "  { \"AddDocs\"  AddDoc > : * ",
        "  CloseIndex",
        "} : 2",
    };
    
    // 2. execute the algorithm  (required in every "logic" test)
    Benchmark benchmark = execBenchmark(algLines);

    // 3. test number of docs in the index
    IndexReader ir = IndexReader.open(benchmark.getRunData().getDirectory());
    int ndocsExpected = 20; // Reuters20DocMaker exhausts after 20 docs.
    assertEquals("wrong number of docs in the index!", ndocsExpected, ir.numDocs());
    ir.close();
  }
{code}

Cheers,
Doron

> Some small fixes to contrib/benchmark
> -------------------------------------
>
>                 Key: LUCENE-1115
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1115
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.3
>
>         Attachments: LUCENE-1115.patch
>
>
> I've fixed a few small issues I've hit in contrib/benchmark.
> First, this alg was only doing work on the first round.  All
> subsequent rounds immediately finished:
> {code}
> analyzer=org.apache.lucene.analysis.standard.StandardAnalyzer
> doc.maker=org.apache.lucene.benchmark.byTask.feeds.LineDocMaker
> work.dir = /lucene/work
> docs.file=work/reuters.lines.txt
> doc.maker.forever=false
> directory=FSDirectory
> doc.add.log.step=3000
> { "Rounds"
>   ResetSystemErase
>   CreateIndex
>   { "AddDocs" AddDoc > : *
>   CloseIndex
>   NewRound
> } : 3
> {code}
> I think this is because we are failing to reset "exhausted" to false
> in PerfTask.doLogic(), so I added that.  Plus I had to re-open the
> file in LineDocMaker.
> Second, I made a small optimization to not call updateExhausted unless
> any of the child tasks are TaskSequence or ResetInputsTask (which I
> compute up-front).
> Finally, we were not allowing flushing by RAM and doc count, so I
> fixed the logic in Create/OpenIndexTask to set both RAMBufferSizeMB
> and MaxBufferedDocs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message