flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Jiang (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-22698) RabbitMQ source does not stop unless message arrives in queue
Date Thu, 20 May 2021 02:09:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-22698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348011#comment-17348011

Nicholas Jiang commented on FLINK-22698:

[~austince], thanks for tracking this issue. From this ML thread, the discussion is in the
obstruct progress. IMO, we could fix the IsEndStream method to solve the above problem possiblely.

> RabbitMQ source does not stop unless message arrives in queue
> -------------------------------------------------------------
>                 Key: FLINK-22698
>                 URL: https://issues.apache.org/jira/browse/FLINK-22698
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors/ RabbitMQ
>    Affects Versions: 1.12.0
>            Reporter: Austin Cawley-Edwards
>            Priority: Major
>         Attachments: taskmanager_thread_dump.json
> In a streaming job with multiple RMQSources, a stop-with-savepoint request has unexpected
behavior. Regular checkpoints and savepoints complete successfully, it is only the stop-with-savepoint
request where this behavior is seen.
> *Expected Behavior:*
> The stop-with-savepoint request stops the job with a FINISHED state.
> *Actual Behavior:*
> The stop-with-savepoint request either times out or hangs indefinitely unless a message
arrives in all the queues that the job consumes from after the stop-with-savepoint request
is made.
> *Current workaround:*
> Send a sentinel value to each of the queues consumed by the job that the deserialization
schema checks in its isEndOfStream method. This is cumbersome and makes it difficult to do
stateful upgrades, as coordination with another system is now necessary. 
> The TaskManager thread dump is attached.

This message was sent by Atlassian Jira

View raw message