lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <>
Subject [jira] [Commented] (SOLR-9684) Add schedule Streaming Expression
Date Mon, 02 Jan 2017 02:34:58 GMT


Joel Bernstein commented on SOLR-9684:

Ok, then let's go with *priority* as the name for this function.

About the *merge* function. The merge function is shorthand for "mergeSort". It's designed
to merge two streams sorted on the same keys and maintain the sort order. Originally the idea
was that the /export handler was a giant sorting engine, and merge was a way to efficiently
merge the sorted streams.

The priority function behaves more like the SQL UNIONALL with priority. But it's different
in that priority is only picks one stream to iterate on each open/close. This design allows
it to iterate the high priority topic until it's empty, and only then iterate through the
lower priority topic.

Eventually we'll need to implement both a UnionStream and UnionAllStream as well.

> Add schedule Streaming Expression
> ---------------------------------
>                 Key: SOLR-9684
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Joel Bernstein
>            Assignee: Joel Bernstein
>             Fix For: master (7.0), 6.4
>         Attachments: SOLR-9684.patch, SOLR-9684.patch, SOLR-9684.patch
> SOLR-9559 adds a general purpose *parallel task executor* for streaming expressions.
The executor() function executes a stream of tasks and doesn't have any concept of task priority.
> The scheduler() function wraps two streams, a high priority stream and a low priority
stream. The scheduler function emits tuples from the high priority stream first, and then
the low priority stream.
> The executor() function can then wrap the scheduler function to see tasks in priority
> Pseudo syntax:
> {code}
> daemon(executor(schedule(topic(tasks, q="priority:high"), topic(tasks, q="priority:low"))))
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message