flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Jiang (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-22698) RabbitMQ source does not stop unless message arrives in queue
Date Wed, 26 May 2021 01:52:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-22698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17351424#comment-17351424
] 

Nicholas Jiang edited comment on FLINK-22698 at 5/26/21, 1:51 AM:
------------------------------------------------------------------

Thanks for [~cmick]'s investigation. The solution which adds a deliveryTimeout parameter to
the configuration makes sense to me. IMO, this deliveryTimeout could be regarded as configuration
option and users could configure this timeout. But in unit test, how do you configure the
delivery timeout which could cause the failure of the unit test?
[~austince], should the FLIP-27 upgrade of RabbiteMQ Source concern this problem?


was (Author: nicholasjiang):
Thanks for [~cmick]'s investigation. The solution which adds a deliveryTimeout parameter to
the configuration makes sense to me. IMO, this deliveryTimeout could be regarded as configuration
option and users could configure this timeout. And [~austince], should the FLIP-27 upgrade
of RabbiteMQ Source concern this problem?

> RabbitMQ source does not stop unless message arrives in queue
> -------------------------------------------------------------
>
>                 Key: FLINK-22698
>                 URL: https://issues.apache.org/jira/browse/FLINK-22698
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors/ RabbitMQ
>    Affects Versions: 1.12.0
>            Reporter: Austin Cawley-Edwards
>            Assignee: Michał Ciesielczyk
>            Priority: Major
>         Attachments: taskmanager_thread_dump.json
>
>
> In a streaming job with multiple RMQSources, a stop-with-savepoint request has unexpected
behavior. Regular checkpoints and savepoints complete successfully, it is only the stop-with-savepoint
request where this behavior is seen.
>  
> *Expected Behavior:*
> The stop-with-savepoint request stops the job with a FINISHED state.
>  
> *Actual Behavior:*
> The stop-with-savepoint request either times out or hangs indefinitely unless a message
arrives in all the queues that the job consumes from after the stop-with-savepoint request
is made.
>  
> *Current workaround:*
> Send a sentinel value to each of the queues consumed by the job that the deserialization
schema checks in its isEndOfStream method. This is cumbersome and makes it difficult to do
stateful upgrades, as coordination with another system is now necessary. 
>  
>  
> The TaskManager thread dump is attached.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message