lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jed Glazner <jglaz...@beyondoblivion.com>
Subject Re: Replicaiton Fails with Unreachable error when master host is responding.
Date Wed, 04 May 2011 05:14:08 GMT
So it turns out that it's the host names.  According the DNS RFC 
underscores are not valid in host names. Most DNS servers now support 
them, but it's not in the rfc strictly speaking.  So there must be 
something in the underlying java classes that bork when using 
underscores in host names, though  I didn't see anything in the stack 
trace that indicated an invalid host name exception. That was most the 
issue though.  Once I changed the host name to the master's IP address  
replication worked great.  So I'm working with our IT to remove 
underscores from our host names.

Just thought I would post my answer here in case anyone else had that 
issue.

Thanks.

Jed.

On 04/28/2011 02:03 PM, Mike Sokolov wrote:
> No clue. Try wireshark to gather more data?
>
> On 04/28/2011 02:53 PM, Jed Glazner wrote:
>> Anybody?
>>
>> On 04/27/2011 01:51 PM, Jed Glazner wrote:
>>> Hello All,
>>>
>>> I'm having a very strange problem that I just can't figure out. The
>>> slave is not able to replicate from the master, even though the master
>>> is reachable from the slave machine.  I can telnet to the port it's
>>> running on, I can use text based browsers to navigate the master from
>>> the slave. I just don't understand why it won't replicate.  The admin
>>> screen gives me an Unreachable in the status, and in the log there is an
>>> exception thrown.  Details below:
>>>
>>> BACKGROUND:
>>>
>>> OS: Arch Linux
>>> Solr Version: svn revision 1096983 from
>>> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/
>>> No custom plugins, just whatever came with the version above.
>>> Java Setup:
>>>
>>> java version "1.6.0_22"
>>> OpenJDK Runtime Environment (IcedTea6 1.10) (ArchLinux-6.b22_1.10-1-x86_64)
>>> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>>>
>>> We have 3 cores running, all 3 cores are not able to replicate.
>>>
>>> The admin on the slave shows  the Master as
>>> http://solr-master-01_dev.la.bo:8983/solr/music/replication  - *Unreachable*
>>> Replicaiton def on the slave
>>>
>>>     529<requestHandler name="/replication" class="solr.ReplicationHandler">
>>>     530<lst name="${slave:slave}">
>>>     531<str
>>> name="masterUrl">http://solr-master-01_dev.la.bo:8983/solr/music/replication</str>
>>>     532<str name="pollInterval">00:15:00</str>
>>>     533</lst>
>>>     534</requestHandler>
>>>
>>> Replication def on the master:
>>>
>>>     529<requestHandler name="/replication" class="solr.ReplicationHandler">
>>>     530<lst name="${master:master}">
>>>     531<str name="replicateAfter">commit</str>
>>>     532<str name="replicateAfter">startup</str>
>>>     533<str name="confFiles">schema.xml,stopwords.txt</str>
>>>     534</lst>
>>>     535</requestHandler>
>>>
>>> Below is the log start to finish for replication attempts, note that it
>>> says connection refused, however, I can telnet to 8983 from the slave to
>>> the master, so I know it's up and reachable from the slave:
>>>
>>> telnet solr-master-01_dev.la.bo 8983
>>> Trying 172.12.65.58...
>>> Connected to solr-master-01_dev.la.bo.
>>> Escape character is '^]'.
>>>
>>> I double checked the master to make sure that it didn't have replication
>>> turned off, and it's not.  So I should be able to replicate but it
>>> can't.  I just dont' know what else to check.  The log from the slave is
>>> below.
>>>
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.request.SolrQueryResponse<init>
>>> WARNING: org.apache.solr.request.SolrQueryResponse is deprecated. Please
>>> use the corresponding class in org.apache.solr.response
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.handler.ReplicationHandler
>>> getReplicationDetails
>>> WARNING: Exception while invoking 'details' method for replication on
>>> master
>>> java.net.ConnectException: Connection refused
>>>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>        at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
>>>        at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
>>>        at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
>>>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
>>>        at java.net.Socket.connect(Socket.java:546)
>>>        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>        at java.lang.reflect.Method.invoke(Method.java:616)
>>>        at
>>> org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:140)
>>>        at
>>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
>>>        at
>>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>>>        at
>>> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
>>>        at
>>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>>>        at
>>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>>>        at
>>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>>>        at
>>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
>>>        at
>>> org.apache.solr.handler.SnapPuller.getNamedListResponse(SnapPuller.java:193)
>>>        at
>>> org.apache.solr.handler.SnapPuller.getCommandResponse(SnapPuller.java:188)
>>>        at
>>> org.apache.solr.handler.ReplicationHandler.getReplicationDetails(ReplicationHandler.java:588)
>>>        at
>>> org.apache.solr.handler.ReplicationHandler.handleRequestBody(ReplicationHandler.java:183)
>>>        at
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
>>>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1360)
>>>        at
>>> org.apache.jsp.admin.replication.index_jsp.executeCommand(org.apache.jsp.admin.replication.index_jsp:50)
>>>        at
>>> org.apache.jsp.admin.replication.index_jsp._jspService(org.apache.jsp.admin.replication.index_jsp:232)
>>>        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
>>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>        at
>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:389)
>>>        at
>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
>>>        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
>>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>        at
>>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>>>        at
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
>>>        at
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>        at
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>        at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>>>        at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>        at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327)
>>>        at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)
>>>        at
>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
>>>        at
>>> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>>>        at
>>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>>>        at
>>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>>>        at
>>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>>>        at
>>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>>>        at
>>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>>>        at
>>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>>>        at
>>> org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
>>>        at
>>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>>>        at org.mortbay.jetty.Server.handle(Server.java:326)
>>>        at
>>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>>>        at
>>> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
>>>        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
>>>        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>>>        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>>>        at
>>> org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
>>>        at
>>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.core.SolrCore execute
>>> INFO: [music] webapp=null path=null params={command=details} status=0
>>> QTime=66
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.request.SolrQueryResponse<init>
>>> WARNING: org.apache.solr.request.SolrQueryResponse is deprecated. Please
>>> use the corresponding class in org.apache.solr.response
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.core.SolrCore execute
>>> INFO: [music] webapp=null path=null params={command=fetchindex} status=0
>>> QTime=1
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:39:45 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:39:45 PM org.apache.solr.handler.SnapPuller fetchLatestIndex
>>> SEVERE: Master at:
>>> http://solr-master-01_dev.la.bo:8983/solr/music/replication  is not
>>> available. Index fetch failed. Exception: Connection refused
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: I/O exception (java.net.ConnectException) caught when processing
>>> request: Connection refused
>>> Apr 27, 2011 7:40:00 PM org.apache.commons.httpclient.HttpMethodDirector
>>> executeWithRetry
>>> INFO: Retrying request
>>> Apr 27, 2011 7:40:00 PM org.apache.solr.handler.SnapPuller fetchLatestIndex
>>> SEVERE: Master at:http://solr-master-01_dev:8983/solr/rovi/replication
>>> is not available. Index fetch failed. Exception: Connection refused
>>>
>>>
>>
>> --
>> Jed Glazner (801-360-0181)
>>
>> This email and its attachments (if any) are for the sole use of the
>> intended recipient, and may contain private, confidential, and
>> privileged material. Any review, copying, or distribution of this
>> email, its attachments or the information contained herein is strictly
>> prohibited. If you are not the intended recipient, please contact the
>> sender immediately and permanently delete the original and any copies
>> of this email and any attachments.
>>


Mime
View raw message