lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gregory Chanan (JIRA)" <>
Subject [jira] [Commented] (SOLR-7789) Introduce a ConfigSet management API
Date Sat, 22 Aug 2015 20:55:46 GMT


Gregory Chanan commented on SOLR-7789:

Thanks for the comments, Shalin.

{quote}Why should there be a separate queue for such operations? i.e. Why should ZkController
have method called getOverseerConfigSetQueue() at all? It should just use the collection queue.
Similarily why does this API need a new ConfigSetsHandler and not use CollectionsHandler?{quote}

Let's start with the second question.  There is a ConfigSetsHandler because it operates on
a separate end point -- it doesn't make sense to send ConfigSet-related commands to /admin/collections,
it makes more sense from the end user perspective to send ConfigSet-related commands to /admin/configs.
 Given separate end points, separate handlers also make sense.

On the second question, the concern is a little more subtle.  The point I am trying to make
is that how the Overseer processes ConfigSet operations is an implementation detail of the
Overseer.  The ConfigSetHandler (which is not part of the Overseer) just needs an interface
in order to tell the Overseer that it wants a ConfigSet operation processed, it shouldn't
be concerned with the implementation details.  Right now we happen to use the same queue under
the covers, but maybe we find in the future that's a bad idea (e.g. users have different QoS
expectations between ConfigSet and Collection operations and we add ConfigSet operations that
are long lived and block important Collection operations).  If the interface between the Overseer
and the ConfigSet handler doesn't refer to collections, we don't need to change anything outside
of the Overseer if we change the processing in the future.

bq. The only reason why it is called a OverseerCollectionQueue is to indicate that it is the
queue for OverseerCollectionProcessor.

That can be indicated with a variable/method name, not the type name.

{quote}TBH, all this refactoring of OverseerCollectionProcessor confuses me very much. It
looks like you want the APIs and tasks run in the overseer to be pluggable but I haven't noticed
you saying that anywhere. In the absence of them being truly pluggable, the current API has
become more complicated than it was before. We do not need complex dispatch mechanisms in
the collection processor.{quote}

I don't wish to make the API/tasks pluggable so I understand your concern.  That being said,
there is a middle ground between API/tasks being pluggable and putting everything in the collection
processor.  All I'm arguing for is clean interfaces.  Take SOLR-7855 as an example; because
we had Overseer/role related commands in the collection processor it made refactoring much
more difficult.  Doing what you suggest in my opinion would have the same effect.

{quote}We do not need complex dispatch mechanisms in the collection processor. If you think
that it is wrong to perform tasks that do not involve a Collection then it could simply be
renamed to OverseerTaskProcessor and we can be done.{quote}

I don't think the dispatching here is complex and it is completely contained in the Overseer.
 I'm not sure what you mean by OverseerTaskProcessor, this seems like a separate issue.  After
SOLR-7855 we have an OverseerCollectionMessageHandler to handle overseer collection messages.
 If you are suggesting throwing the ConfigSet related commands in there (from OverseerConfigSetMessageHandler),
you've just moved the dispatch code somewhere else.

bq. Btw we have had APIs such as clusterprops and request_status in the overseer collection
processor and I don't think it is confusing anyone.

I gave the example above the overseer/role related commands making code hard to refactor.
 I agree with you that clusterprops and request_status aren't particularly confusing -- that
doesn't mean we can't do better.

> Introduce a ConfigSet management API
> ------------------------------------
>                 Key: SOLR-7789
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Gregory Chanan
>            Assignee: Gregory Chanan
>         Attachments: SOLR-7789.patch, SOLR-7789.patch, SOLR-7789.patch, SOLR-7789.patch
> SOLR-5955 describes a feature to automatically create a ConfigSet, based on another one,
from a collection API call (i.e. one step collection creation).  Discussion there yielded
SOLR-7742, Immutable ConfigSet support.  To close the loop, we need support for a ConfigSet
management API.
> The simplest ConfigSet API could have one operation:
> create a new config set, based on an existing one, possible modifying the ConfigSet properties.
 Note you need to be able to modify the ConfigSet properties at creation time because otherwise
Immutable could not be changed.
> Another logical operation to support is ConfigSet deletion; that may be more complicated
to implement than creation because you need to handle the case where a collection is already
using the configuration.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message