qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [qpid-dispatch] kgiusti opened a new pull request #716: DISPATCH-1545: Prevent Head-of-line blocking on shared inter-router l…
Date Tue, 07 Apr 2020 20:26:41 GMT
kgiusti opened a new pull request #716: DISPATCH-1545: Prevent Head-of-line blocking on shared
inter-router l…
URL: https://github.com/apache/qpid-dispatch/pull/716
 
 
   …inks
   
   This patch prevents head-of-line blocking on shared links by
   dynamically assigning a dedicated inter-router link for the transfer
   of a streaming message.
   
   **Patch Notes**
   
   There are 3 primary changes made by this patch:
   
   - Classification of inbound messages as streaming
   - Modifications to the forwarding path to handle streaming messages
   - The creation of a per-connection pool of links available for streaming a message
   
   
   **Classification**
   
   There is no practical way to determine the length of an inbound message prior to its complete
arrival.  This means the router has to monitor the size of an incoming message and classify
it as streaming once it gets "too big" (and has not yet finished arriving).  This requires
defining a size threshold over which the message is classified as streaming.
   
   Note well that this classification MUST happen BEFORE the message is forwarded by the core:
the core uses the classification to determine the proper outgoing link.
   
   This means the I/O thread must buffer enough of an incoming message to make the classification
before the message can be handed to the core for forwarding.  In other words an incoming message
will be buffered until it completely arrives OR hits the "too big" threshold.
   
   Rather than introducing yet another user-facing "streaming message threshold" configuration
knob (and the additional code complexity, test matrix dimension and user documentation it
would require), this patch uses the existing Q2 threshold to classify the message. The Q2
threshold makes sense: it already sets a topmost limit for incoming message buffering.  Once
Q2 is hit the router MUST forward the message in order to unblock the link, so Q2 essentially
forces us to make that classification in any case.
   
   The patch does this buffering/threshold check in the rx handler in router_node.c at the
point in the code where the message has yet to be forwarded.  Once forwarded further changes
to the Q2 state are irrelevant to the classification process.
   
   **Forwarding Path Modifications**
   
   Once the classification is complete all streaming messages are flagged.  I've done this
rather than constantly checking the "qd_message_receive_complete()" throughout the message's
lifecycle because the receive status can change at any moment via the I/O thread.  This opens
to possibility of race conditions.
   
   Each forwarder has been updated to check the message's streaming flag during the outbound
link selection process.  If the candidate link's connection supports outgoing streaming links
the forwarder will obtain a dedicated outgoing anonymous link for forwarding the message.
 This link is either created on demand or obtained from the connection's pool of free streaming
links.
   
   **Connection Link Pool**
   
   Every connection that supports outgoing streaming links maintains a free streaming link
pool.  This pool has been added to the qdr_connection_t structure. This pool is initialized
empty.  When a forwarder needs a dedicated streaming link it will check the pool for an available
link.  If none are available a new qdr_link_t will be created for the outgoing message.  This
link is marked as "pooled" so when the message is finally delivered the qdr_link_t will be
returned to the pool, ready for the next streaming message.
   
   The pool is designed to keep at least 128 standby links.  More links can be allocated,
but links in excess of the 128 will eventually be released.  This is done by the streaming_link_scrubber
core module added by this patch.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


Mime
View raw message