lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Yano (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource
Date Fri, 18 Aug 2017 01:42:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131583#comment-16131583
] 

Yuki Yano commented on SOLR-11250:
----------------------------------

[~cpoerschke]
Thank you for revising the patch! I agree with your proposal and try refactoring codes based
on new patch.

By the way, I'm wandering whether we need to revise {{getFeatures}} and {{getNorms}} which
affect {{toLTRScoringModelMap}}. 

Current implementation only overwrites {{getParams}} which tends to grow with large model
(ex. large ensemble trees). On the other hand, because it delegates {{getFeatures}} and {{getNorms}}
to wrapped model, we still upload features and norms of the  wrapped model to ZooKeeper. This
can be problem when uploading huge linear model which may contain many features.

As for search engines, I think this is very rare case because extracting many features is
too costly. Do you think we need to consider the above case with this patch?

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>
>                 Key: SOLR-11250
>                 URL: https://issues.apache.org/jira/browse/SOLR-11250
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250.patch
>
>
> We add new model which contains only the location of the external model and loads it
during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external
model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json",
the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will
be used for scoring documents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message