lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2548) Multithreaded faceting
Date Mon, 09 Sep 2013 22:56:53 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762425#comment-13762425
] 

Erick Erickson commented on SOLR-2548:
--------------------------------------

[~dsmiley]

See below...

I'm not seeing points 1-3. I think you might be missing the distinction between adding fields
to the pending queue and actually doing the faceting:

(1) I don't think so. If facet.threads == 2, the third time around the counter is -1 so the
field gets added to the pending structure, it's not executed on at all until one of the other
threads completes.

(2) I'm not seeing it. Every time a task completes, another is started from the pending list.
The main thread is just sitting around waiting for the child threads to complete. Mostly this
is for my edification, I have no objection to the semaphore approach. In fact it's a little
cleaner, the second "for (String f : facetFs) {" loop is somewhat loosely coupled.

(3) Not quite sure about this either. I don't see where the main thread is used to compute
any facets. Well, except in the intentionally serial case when the directExecutor is used
and the old behavior is desired. Items are just added to the pending queue once you exceed
facet.threads. That queue is consumed to submit other tasks to new threads via "completionService.submit(pending.removeFirst());"
in the second loop. The main thread never computes facets. Or I'm just blind to it.

(4) That makes sense, although I'll defer to Robert.

(5) OK. I did have some trouble in the tests though, some of them were expecting 400 response
code and the SERVER_ERROR is 500 as I remember so don't be surprised if there's an issue there
when you run the full test suite if you haven't already. I made some effort to give back the
same errors as the tests expected which may account for some of the weirdness you saw in the
exception handling.

You'll notice I punted on Adrien's comment "Is there any reason why you didn't make facet
queries and facet ranges multi-threaded"... feel free ;).
                
> Multithreaded faceting
> ----------------------
>
>                 Key: SOLR-2548
>                 URL: https://issues.apache.org/jira/browse/SOLR-2548
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Janne Majaranta
>            Assignee: Erick Erickson
>            Priority: Minor
>              Labels: facet
>             Fix For: 4.5, 5.0
>
>         Attachments: SOLR-2548_4.2.1.patch, SOLR-2548_for_31x.patch, SOLR-2548_multithreaded_faceting,_dsmiley.patch,
SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch, SOLR-2548.patch
>
>
> Add multithreading support for faceting.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message