ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Mashenkov <andrey.mashen...@gmail.com>
Subject Re: Backup queue for local continuous query
Date Fri, 10 Aug 2018 10:41:20 GMT
Hi Denis,

As my understanding right, to prevent potential OOM Local CQ either
shouldn't collect anything in backup queue or other nodes should send akcs.

Second way look strange as Local CQ runs on local node only and doesn't
care whats going on other nodes.
It is not clear what events and when other nodes should ack as they doesn't
participate in query?
When a primary node for a partition goes away and node with local CQ
running become a new primary
then seems, it make no sense to notify CQ with stale events from backup
queue and only new events make sense (as from new primary).

Instead of using backup queue we can notify CQ on backup events instantly
and let user filter out backup events if needed in listener.

So, I'd expect local query shouldn't involve remote nodes at all.
Also, I'm not sure we should filter out backup events. Can we let users to
do this in their listener code as it shouldn't costs for local CQ?





On Thu, Aug 2, 2018 at 3:35 PM Denis Mekhanikov <dmekhanikov@gmail.com>
wrote:

> Igniters,
>
> I noticed, that local continuous queries store queues with backup entries.
> And since the queries are local, other nodes never send acknowledges.
>
> It has two negative effects:
> 1) Backup queue may grow indefinitely and cause OOME.
> 2) When topology changes, the backup queue is flushed, and old
> notifications are delivered to the listener, even if they happened long
> time ago.
>
> I prepared a fix for it. You can find it in the ticket: IGNITE-9009
> <https://issues.apache.org/jira/browse/IGNITE-9009>
> It makes local continuous queries ignore backup entries and not put them
> into the backup queues.
>
> Nikolay, Semyon,
> You are the people, who implemented this piece of functionality.
> Could you review these changes and share your opinion?
>
> I can think of another way to fix it: we could register
> *CacheContinuousQueryHandlers* on all nodes even for a local CQ, and make
> them send backup notifications to the subscriber node.
> It will help in cases, when you create local listeners on all nodes and
> want all nodes to process changes, that happen on their primary partitions.
> In this case backup entries may help not to lose updates. But actually it
> will leave a time window between sending a backup acknowledge and notifying
> the local listener. And such use-case will generate quadratic number of
> handlers and pollute the network communication.
>
> So, I prefer the first option.
>
> What do you think?
>
> Denis
>


-- 
Best regards,
Andrey V. Mashenkov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message