lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christine Poerschke (JIRA)" <>
Subject [jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource
Date Fri, 25 Aug 2017 15:05:00 GMT


Christine Poerschke commented on SOLR-11250:

Thanks [~yuyano] for attaching the new patch! I haven't had an opportunity to really look
at it but wanted to share three non-code related things that crossed my mind, in no particular

* Naming of the new class to perhaps be something other than _ExternalModel_ for clarity.
To illustrate, given the configuration
  "class" : "org.apache.solr.ltr.model.URIExternalModel",
  "name" : "modelZ",
  "features" : [],
  "params" : {
    "uri" : "file:///var/models/modelF.json"
we have _modelZ_ which is stored _inside_ ZooKeeper but is an instance of class URI _External_
Model and we have _modelF_ which is stored _outside_ ZooKeeper i.e. _externally_ in a file
(but it is _not_ an instance of _External_ Model). In the code base there's a Foo/FilterFoo
naming pattern e.g. but (say) {{FilterLTRScoringModel}} may
not be particularly clear from a Solr user's perspective. {{WrapperModel}} or {{WrappingModel}}
or something like that might be alternatives. What do you think?

* The Solr Reference Guide documentation is now under version control e.g.
i.e. changes to it can be included in the same patch as the actual code changes. When we add
the documentation let's mention somehow in the [Applying Changes|]
section about changes to the modelF.json physical file not taking effect immediately.

* Custom model format flexibility: one motivation for storing the model definition externally
is the difficulty of storing large models in ZooKeeper. Additionally, perhaps as a side effect,
given the abstract ExternalModel class and its
public abstract Map<String, Object> fetchModelMap() throws ModelException;
method it is now then also possible for custom (external) models to use a non-JSON format
potentially e.g.
public class MyURIExternalModel extends URIExternalModel {
  public Map<String, Object> fetchModelMap() throws ModelException {
could use a
class: org.apache.solr.ltr.model.LinearModel
name: modelC
  documentRecency: 0.1
  isBook: 1.0
  originalScore: 0.5
non-json input file. Custom models are custom models of course but still I think this is a
nice side effect of the abstract ExternalModel.fetchModelMap signature :-)

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>                 Key: SOLR-11250
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250_master_v2.patch, SOLR-11250.patch
> We add new model which contains only the location of the external model and loads it
during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external
model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json",
the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will
be used for scoring documents.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message