lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Smiley (JIRA)" <>
Subject [jira] [Commented] (SOLR-7576) Implement RequestHandler in Javascript
Date Fri, 07 Aug 2015 05:07:46 GMT


David Smiley commented on SOLR-7576:

bq. we can't keep large amounts of data in ZK

Yes I am aware.  I see now why you mention this: it's because the "blob store" feature is
itself a Solr collection and it keeps all versions.  I didn't know that until now (or lapse
of memory ;-P); I had guessed (wrongly) it was just a REST API into ZooKeeper.   I think I
better understand where you're coming from now.  There are other features like sources for
Suggesters that I look forward to seeing getting upgraded to use the blob store, since it's
bad to put large files in ZooKeeper in your conf dir.

For this specific feature (request handlers in scripting language) why should the executable
be put in the Blob store at all?  Generally these scripts are going to be very small.   Instead,
why not simply access it via the SolrResourceLoader abstraction, thus it'll work for stand-alone
file-system & ZooKeeper if SolrCloud (i.e. exactly how the Script URP works)?  A REST
API on top could add another convenient access mechanism to "conf" dir stuff -- obviously
with access control required by default though disable'able at the command-line for developers
working on scripts locally.  AFAIK ZooKeeper clients (e.g. Solr) can monitor ZK paths for
updates; and likewise this is possible of the file system in modern Java.  In summary, what
I'm advocating is that the script be treated like the rest of the configuration for a core,
even though technically it's code not configuration.  I don't think that distinction is pertinent.

> Implement RequestHandler in Javascript
> --------------------------------------
>                 Key: SOLR-7576
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Noble Paul
>         Attachments: SOLR-7576.patch, SOLR-7576.patch
> Solr now support dynamic loading (SOLR-7073) of components and it is secured in SOLR-7126
> We can extend the same functionality with JS as well
> the handler {{/js}} is implicitly registered
> To make this work
> * Solr should be started with {{-Denable.js.loading=true}}
> * The javascript must be loaded to the {{.system}} collection using the blob store API
> * Sign the javascript and pass the signature in a param called {{_sig}}
> The {{JSRequestHandler}} is implicitly defined and it can be accessed by hitting {{/js/<jsname>/<version>}}

> Steps for developing scripts
> # start the cluster with the {{enable.js.loading}} . If you are starting using our script
it would be {{bin/solr start -e cloud -a "-Denable.js.loading=true"}} . You would not need
security during development , so don't add the private keys to Solr
> # create {{.system}} collection {{bin/solr create -c .system}}
> # Write your javascript code . (say {{test.js}} )
> # post it to {{.system}} collection . {{curl -X POST -H 'Content-Type: application/octet-stream'
--data-binary @test.js http://localhost:8983/solr/.system/blob/test}}
> # run your script {{http://host:8983/solr/gettingstarted/js/test/1}}
> # Edit your script and repeat from step #4 . Keep in mind that the version would be bumped
up every time you post a new script . So, the second time the url would be {{http://host:8983/solr/gettingstarted/js/test/2}}
. So on and so forth
> sample programs
> 1) writes a val to output
> {code:javascript}
> //empty line
> $.response().add('testkey','Test Val');
> {code}
> 2)  manipulate the output to add an extra field to each doc 
> {code}
> //empty line
> var l = [];
> $.query({
>               q: '*:*',
>               qt: '/select',
>               start:0,
>           }).forEach('response', function(doc) {
>                                          doc.put('script', 'Added this value');
>                                          l.push(doc);
>           });
>  $.response().add('alldocs', l);
> {code}
> 3)  stream through all the docs
> {code:Javascript}
> //empty line
> $.query({
>               q: '*:*',
>               qt: '/select',
>               start:0,
>               distrib:'false'
>           }).pipe('response', 'docs', function(doc) { // the pipe function is executed
right before the response writer and right after the transformers   
>                                          if('IT'== doc.get('genre_s')) return null;
>                                          doc.put('script', 'Added this value');
>                                          return doc;
>           });
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message