lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Walter Underwood <wun...@wunderwood.org>
Subject Re: Solr TCP layer
Date Mon, 09 Mar 2015 03:24:45 GMT
I agree. I have some servers that use a TCP socket protocol and they are beastly hard to monitor,
load-balance, all that stuff. HTTP rules. I need a really big advantage to recommend a non-HTTP
server.

Understand that I helped design at least one socket protocol, HP JetDirect. This was designed
before HTTP, so I have an excuse.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)


On Mar 8, 2015, at 8:15 PM, Shawn Heisey <apache@elyograg.org> wrote:

> On 3/8/2015 2:05 PM, Saumitra Srivastav wrote:
>> I want to start working on adding a TCP layer for client to node and
>> inter-node communication.
>> 
>> I am not up to date on recent changes happening to Solr. So before I start
>> looking into code, I would like to know if there is already some work done
>> in this direction, which I can reuse. Are there any know
>> challenges/complexities?
>> 
>> I would appreciate any help to kick start this effort. Also, what would be
>> the best way to discuss and get feedback on design from contributors? Open a
>> JIRA??
> 
> 
> What follows is my mostly my opinion, interspersed with a few things
> that might be loosely called facts:
> 
> I personally do not mind at all that Solr uses HTTP.
> 
> Some people view it as an inefficient protocol, but the overhead is low
> on all but the slowest connections.  On a LAN, it is probably not even
> worth discussing.  Most of the time, there will be a LAN connection
> between the Solr client and the Solr server.
> 
> I think there are two primary advantages to HTTP:
> 
> *) Testing can be carried out in a browser with hand-typed URLs.
> *) We don't have to worry about writing the code to handle the
> underlying protocol.
> 
> The first point makes testing throughout the Solr deployment process
> VERY easy.
> 
> Expanding on the second point:  There are well-tested and mostly
> bug-free HTTP client and server libraries available for us to use.  We
> don't have to figure out how to write them, troubleshoot them, or
> optimize them.  There are hundreds or thousands of other people using
> those libraries that can find and report bugs and inefficiencies, often
> including the fix in the bug report.
> 
> We might be able to make Solr a little more efficient if we use our own
> TCP protocol instead of HTTP, but there are drawbacks.  It takes a lot
> of experience to write a network protocol from scratch, and we are bound
> to make mistakes, mistakes that will cause users a LOT of problems.  The
> authors of those libraries that I mentioned have already gone through
> that pain, often many times.
> 
> Solr must be a standalone application to realistically use a custom TCP
> protocol directly.  Although there are already loose plans in place to
> pull the HTTP and network layers into Solr and make it a standalone
> application, we are not there yet.
> 
> If you want to work on a new protocol, feel free.  If there is anything
> I can do to help in between my other duties, I will.  I do not know of
> any existing work that you can re-use, although it's possible there
> might be something.  You might look at the Zookeeper project for an
> existing implementation of a custom network protocol in Java.
> 
> I suspect that it will take a very long time before any such work is as
> stable as HttpClient and the Servlet API (implemented by Jetty).  Unless
> you can demonstrate that stability, it will not become the default protocol.
> 
> Thanks,
> Shawn
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message