lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christine Poerschke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource
Date Fri, 18 Aug 2017 19:13:03 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133498#comment-16133498
] 

Christine Poerschke commented on SOLR-11250:
--------------------------------------------

bq. ... whether we need to revise {{getFeatures}} and {{getNorms}} which affect {{toLTRScoringModelMap}}.
... On the other hand, because it delegates {{getFeatures}} and {{getNorms}} to wrapped model,
we still upload features and norms of the  wrapped model to ZooKeeper. ...

Great question. I think it's important that what is uploaded to ZooKeeper matches what is
in the {{myExternalModel.json}} file that was passed to the http://localhost:8983/solr/mycollection/schema/model-store
end-point i.e. for the upload to ZooKeeper to include the features and norms of the wrapped
model but the params of the {{myExternalModel.json}} would be confusing.

In the {{ExternalModel.validate()}} we can ensure that {{myExternalModel.json}} contains no
features (since there being any would indicate a misconfiguration essentially). And then how
about in {{ManagedModelStore.toLTRScoringModelMap}} something like this?

{code}
  final List<Map<String,Object>> features = new ArrayList<>();
+ if (!model instanceof ExternalModel) {
    final List<Feature> featuresList = model.getFeatures();
    final List<Normalizer> normsList = model.getNorms();
    for (int ii=0; ii<featuresList.size(); ++ii) {
      features.add(toFeatureMap(featuresList.get(ii), normsList.get(ii)));
    }
+ }
  modelMap.put(FEATURES_KEY, features);
{code}

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>
>                 Key: SOLR-11250
>                 URL: https://issues.apache.org/jira/browse/SOLR-11250
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250.patch
>
>
> We add new model which contains only the location of the external model and loads it
during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external
model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json",
the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will
be used for scoring documents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message