lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Høydahl <>
Subject Re: [jira] [Updated] (SOLR-9659) Add zookeeper DataWatch API
Date Tue, 18 Oct 2016 13:01:39 GMT
This JIRA got me thinking of a more general topic. It may be controversial, but are there any
thoughts around supporting alternatives to Zookeeper in the future? Would be cool if we could
have a *functional* oriented internal APIs for Collections, live-nodes, config-sets, cluster-state
etc, hiding the underlying technology. The first implementation would be a ZK, then we could
further down the road provide other implementations for each, based upon e.g. Hazelcast or
other suitable technologies.

I mean, we started down this abstraction path with ZkSolrResourceLoader as one of many implementations
to resolve config/resources. But it kind of stopped there. Nowadays there are tons of “isZkAware”
tests across our codebase, assuming there will ever only be two options: ZK or non-zk. You
probably guessed already that I'd love a quicker way of launching a CloudCluster without setting
up ZK, then people could move to ZK to get top notch robustness… That would perhaps be what
we need to deprecate standalone mode forever, i.e. we would always run “cloud” even if
powered by an in-memory config grid or similar.

Jan Høydahl, search solution architect
Cominvent AS -

> 18. okt. 2016 kl. 14.26 skrev Alan Woodward (JIRA) <>:
>     [
> Alan Woodward updated SOLR-9659:
> --------------------------------
>    Attachment: SOLR-9659.patch
> Patch, outlining the API, but not actually cutting over anything to use it yet.
> There are two types of watch:
> * DataWatch, called any time a znode is updated
> * ChildWatch, called any time a znode's children are changed
> Both are registered with SolrZkClient, which maintains ZK watches and lists of interested
parties.  When a znode is changed, or its children are changed, the SolrZkClient will read
the new data/new children, and then send this new information to all registered watchers for
that znode.  ZK connection exceptions are all handled internally.  If a connection is dropped
and re-established, SolrZkClient will reset all ZK watches, read the latest data, and notify
all registered watchers.
>> Add zookeeper DataWatch API
>> ---------------------------
>>                Key: SOLR-9659
>>                URL:
>>            Project: Solr
>>         Issue Type: Improvement
>>     Security Level: Public(Default Security Level. Issues are Public) 
>>           Reporter: Alan Woodward
>>           Assignee: Alan Woodward
>>        Attachments: SOLR-9659.patch
>> We have several components which need to set up watches on ZooKeeper nodes for various
aspects of cluster management.  At the moment, all of these components do this themselves,
leading to large amounts of duplicated code, and complicated logic for dealing with reconnections,
etc, scattered across the codebase.  We should replace this with a simple API controlled by
SolrZkClient, which should make the code more robust, and testing considerably easier.
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message