flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juho Autio <juho.au...@rovio.com>
Subject Re: Kafka topic partition skewness causes watermark not being emitted
Date Tue, 17 Apr 2018 06:41:07 GMT
A possible workaround while waiting for FLINK-5479, if someone is hitting
the same problem: we chose to send "heartbeat" messages periodically to all
topics & partitions found on our Kafka. We do that through the service that
normally writes to our Kafka. This way every partition always has some
~recent timestamps.

On Wed, Dec 13, 2017 at 1:06 PM, Gerard Garcia <gerard@talaia.io> wrote:

> Thanks Gordon.
>
> Don't worry, I'll be careful to not have empty partitions until the next
> release.
> Also, I'll keep an eye to FLINK-5479 and if at some point I see that
> there is a fix and the issue bothers us too much I'll try to apply the
> patch myself to the latest stable release.
>
> Gerard
>
> On Wed, Dec 13, 2017 at 10:31 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org
> > wrote:
>
>> Hi,
>>
>> I've just elevated FLINK-5479 to BLOCKER for 1.5.
>>
>> Unfortunately, AFAIK there is no easy workaround solution for this issue
>> yet in the releases so far.
>> The min watermark logic that controls per-partition watermark emission is
>> hidden inside the consumer, making it hard to work around it.
>>
>> One possible solution I can imagine, but perhaps not that trivial, is to
>> inject some special marker event into all partitions periodically.
>> The watermark assigner should be able to recognize this special marker
>> and try to provide some watermark for it.
>> Another option is that I can provide some patch you can apply for a
>> custom build of the Kafka connector that handles partition idleness
>> properly.
>> However, given that we're aiming for a faster release cycle for Flink 1.5
>> (proposed release date is Feb. 2018), it might not be worth the extra
>> maintenance effort on your side of a custom build.
>>
>> Best,
>> Gordon
>>
>>
>> On Tue, Dec 12, 2017 at 9:28 PM, gerardg <gerard@talaia.io> wrote:
>>
>>> I'm also affected by this behavior. There are no updates in FLINK-5479
>>> but
>>> did you manage to find a way to workaround this?
>>>
>>> Thanks,
>>>
>>> Gerard
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nab
>>> ble.com/
>>>
>>
>>
>

Mime
View raw message