lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-8588) Add TopicStream to the streaming API to support publish/subscribe messaging
Date Sat, 20 Feb 2016 01:04:18 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15155249#comment-15155249
] 

ASF subversion and git services commented on SOLR-8588:
-------------------------------------------------------

Commit f9127a919ac212c4a5c36e66fb0d0c15a7867c0e in lucene-solr's branch refs/heads/master
from jbernste
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f9127a9 ]

SOLR-8588: Update CHANGES.txt


> Add TopicStream to the streaming API to support publish/subscribe messaging
> ---------------------------------------------------------------------------
>
>                 Key: SOLR-8588
>                 URL: https://issues.apache.org/jira/browse/SOLR-8588
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Joel Bernstein
>            Assignee: Joel Bernstein
>             Fix For: 6.0
>
>         Attachments: SOLR-8588.patch, SOLR-8588.patch, SOLR-8588.patch, SOLR-8588.patch,
SOLR-8588.patch
>
>
> The TopicStream is a *publish/subscribe messaging service* built on top of SolrCloud.
 The TopicStream returns all *new* documents for a specific query. Version numbers will be
used as checkpoints for Topics to ensure single delivery of each document. When combined with
the DaemonStream (SOLR-8550), Topics can provide continuous streaming. Sample syntax:
> {code}
> topic(checkpointCollection, dataCollection, id="topicA",  q="awesome stuff" checkpointEvery="1000")
> {code}
> The checkpoint collection will be used to persist the topic checkpoints.
> Example combined with the DaemonStream:
> {code}
> daemon(topic(...)...)
> {code}
> When combined with SOLR-7739 this allows for messaging based on *machine learned* classifications.
> The TopicStream supports 3 models of publish/subscribe messaging:
> 1) *Request & response*: In this model a topic(...) expression can be saved and executed
at any time. In this scenario the TopicStream will always retrieve it's checkpoints and start
from where it left off.
> 2) *Continuous pull streaming*: In this model you would wrap the TopicStream in a DaemonStream
and call read() in a loop inside a java program.  This would provide a continuous stream of
new content as it arrives in the index.
> 3) *Continuous push streaming*: In this model you would send an expression like this
to the /stream handler: *daemon(update(topic(...)...)...)*. This daemon process would run
inside Solr and continuously stream new documents from the topic and push them to another
SolrCloud collection. Other pushing expressions can be created to push documents in different
ways or take other types of actions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message