qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DISPATCH-966) Qpid dispatch unstable inter-router connections
Date Thu, 17 May 2018 13:11:00 GMT

    [ https://issues.apache.org/jira/browse/DISPATCH-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479026#comment-16479026
] 

Ganesh Murthy commented on DISPATCH-966:
----------------------------------------

Thanks Marcel for finding the problem. I was looking at the last part of your comment - "What
I did notice it that proton trace logging is showing inconsistent settlement flag for messages
that are split over multiple transfer frames " and was wondering if what you are observing
would be fixed by https://issues.apache.org/jira/browse/PROTON-1514

Please take a look and let me know. Thank you

> Qpid dispatch unstable inter-router connections
> -----------------------------------------------
>
>                 Key: DISPATCH-966
>                 URL: https://issues.apache.org/jira/browse/DISPATCH-966
>             Project: Qpid Dispatch
>          Issue Type: Bug
>          Components: Routing Engine
>    Affects Versions: 1.0.1
>            Reporter: Marcel Meulemans
>            Assignee: Ted Ross
>            Priority: Blocker
>             Fix For: 1.1.0
>
>         Attachments: inconsistent-settlement.log, qdrouterd-unsettled-true.log, qdrouterd.conf,
qdrouterd.log, router-unsettled-true.dump, router.dump
>
>
> I am running a three node fully connected mesh of dispatch routers with 10000 attached
clients and I am seeing some unstable inter-router connections (I am sending around 1000
small, less than 1K, messages per second through the network). The inter-router connections
fail every so many seconds with the message:
> {{Connection to router-2:55672 failed: amqp:session:invalid-field sequencing error,
expected delivery-id 7, got 6}}
> (the numbers 7 and 6 differ per connection loss)
> In wireshark, using the attached tcpdump capture, I can see that every time before the
inter router connection is dropped, therw is a rejected disposition with the message:
> {{Condition: qd:forbidden}}
> {{Description: Deliveries to a multicast address must be pre-settled}}
> The routers are connected as follows:
>  * router-0 -> router-1
>  * router-0 -> router-2
>  * router-1 -> router-2
> The routers are running as a docker container (debian stretch) on google compute engine
machines (every router on a separate node).
> Attached are:
>  * my qdrouter.conf (from one of the routers)
>  * a log snippet from router-0 at debug level from connection drop to connection re-established
to connection drop again.
>  * a tcpdump capture of the inter-router connection between router-0 and router-1 during
which several of the failures occur
> Versions:
>  * qpid-dispatch@1.0.1-rc1
>  * qpid-proton@0.20.0
>  
> [^qdrouterd.log]
> [^qdrouterd.conf]
> [^router.dump]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message