lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Yano (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-11250) Add new LTR model which loads the model definition from the external resource
Date Thu, 31 Aug 2017 06:43:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148527#comment-16148527
] 

Yuki Yano commented on SOLR-11250:
----------------------------------

[~shalinmangar]
Hmm, I need to clear my head. Summarizing the exchange about this issue and also SOLR-11049,
my understanding is as belows:

1. Accessing external resources like {{http}} or {{ftp}} is prohibitted due to security issues.
2. We can access local resources by {{SolrResourceLoader}} if resources are located in {{instanceDir}}
i.e., {{ ${solr.solr.home} }} except {{solr.allow.unsafe.resourceloading}} is true.
3. If we need to share resources in SolrCloud, using blob store is recommended.

By these reasons, if I want to add the model with external resources, I should implement following
two models:

1. The model which loads external resources by using {{SolrResourceLoader}} (for standalone
mode).
2. The model which loads external resources from {{BlobRepository}} (for SolrCloud mode).

Is my understanding correct?

> Add new LTR model which loads the model definition from the external resource
> -----------------------------------------------------------------------------
>
>                 Key: SOLR-11250
>                 URL: https://issues.apache.org/jira/browse/SOLR-11250
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>            Reporter: Yuki Yano
>            Priority: Minor
>         Attachments: SOLR-11250_master.patch, SOLR-11250_master_v2.patch, SOLR-11250_master_v3.patch,
SOLR-11250.patch
>
>
> We add new model which contains only the location of the external model and loads it
during the initialization.
> By this procedure, large models which are difficult to upload to ZooKeeper can be available.
> The new model works as the wrapper of existing models, and deligates APIs to them.
> We add two classes by this patch:
> * {{ExternalModel}} : a base class for models with external resources.
> * {{URIExternalModel}} : an implementation of {{ExternalModel}} which loads the external
model from specified URI (ex. file:, http:, etc.).
> For example, if you have a model on the local disk "file:///var/models/myModel.json",
the definition of {{URIExternalModel}} will be like the following.
> {code}
> {
>   "class" : "org.apache.solr.ltr.model.URIExternalModel",
>   "name" : "myURIExternalModel",
>   "features" : [],
>   "params" : {
>     "uri" : "file:///var/models/myModel.json"
>   }
> }
> {code}
> If you use LTR with {{model=myURIExternalModel}}, the model of {{myModel.json}} will
be used for scoring documents.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message