lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christine Poerschke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource
Date Fri, 25 Aug 2017 15:05:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141738#comment-16141738
] 

Christine Poerschke commented on SOLR-11250:
--------------------------------------------

Thanks [~yuyano] for attaching the new patch! I haven't had an opportunity to really look
at it but wanted to share three non-code related things that crossed my mind, in no particular
order:

* Naming of the new class to perhaps be something other than _ExternalModel_ for clarity.
To illustrate, given the configuration
{code}
{
  "class" : "org.apache.solr.ltr.model.URIExternalModel",
  "name" : "modelZ",
  "features" : [],
  "params" : {
    "uri" : "file:///var/models/modelF.json"
  }
}
{code}
we have _modelZ_ which is stored _inside_ ZooKeeper but is an instance of class URI _External_
Model and we have _modelF_ which is stored _outside_ ZooKeeper i.e. _externally_ in a file
(but it is _not_ an instance of _External_ Model). In the code base there's a Foo/FilterFoo
naming pattern e.g. Weight.java/FilterWeight.java but (say) {{FilterLTRScoringModel}} may
not be particularly clear from a Solr user's perspective. {{WrapperModel}} or {{WrappingModel}}
or something like that might be alternatives. What do you think?

* The Solr Reference Guide documentation is now under version control e.g. https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/solr-ref-guide/src/learning-to-rank.adoc
i.e. changes to it can be included in the same patch as the actual code changes. When we add
the documentation let's mention somehow in the [Applying Changes|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/solr-ref-guide/src/learning-to-rank.adoc#applying-changes]
section about changes to the modelF.json physical file not taking effect immediately.

* Custom model format flexibility: one motivation for storing the model definition externally
is the difficulty of storing large models in ZooKeeper. Additionally, perhaps as a side effect,
given the abstract ExternalModel class and its
{code}
public abstract Map<String, Object> fetchModelMap() throws ModelException;
{code}
method it is now then also possible for custom (external) models to use a non-JSON format
potentially e.g.
{code}
public class MyURIExternalModel extends URIExternalModel {
  ...
  @Override
  public Map<String, Object> fetchModelMap() throws ModelException {
    ...
  }
  ...
}
{code}
could use a
{code}
class: org.apache.solr.ltr.model.LinearModel
name: modelC
multipliers:
  documentRecency: 0.1
  isBook: 1.0
  originalScore: 0.5
{code}
non-json input file. Custom models are custom models of course but still I think this is a
nice side effect of the abstract ExternalModel.fetchModelMap signature :-)

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>
>                 Key: SOLR-11250
>                 URL: https://issues.apache.org/jira/browse/SOLR-11250
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250_master_v2.patch, SOLR-11250.patch
>
>
> We add new model which contains only the location of the external model and loads it
during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external
model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json",
the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will
be used for scoring documents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message