ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@gridgain.com>
Subject Re: IgniteCache.invoke on ALL keys
Date Tue, 31 May 2016 13:18:07 GMT
Ok then, looks like there are no obstacles or objections, I've created a
JIRA ticket:
https://issues.apache.org/jira/browse/IGNITE-3222

Thanks,
Pavel.

On Tue, May 31, 2016 at 3:44 PM, Denis Magda <dmagda@gridgain.com> wrote:

> Pavel,
>
> > This seems to be a common task, why don't we implement invokeAll as I
> > suggested above?
>
> I would implement such a method using the approach shown in the example.
> In my understanding it would be the most efficient way. Data locality will
> longer be not an issue when IGNITE-2310 is implemented.
>
> —
> Denis
>
> > On May 31, 2016, at 1:16 PM, Pavel Tupitsyn <ptupitsyn@gridgain.com>
> wrote:
> >
> > Denis, thank you, this may work, but:
> > * it is too complicated
> > * it does not guarantee locality
> >
> > This seems to be a common task, why don't we implement invokeAll as I
> > suggested above?
> >
> > On Tue, May 31, 2016 at 1:05 PM, Denis Magda <dmagda@gridgain.com>
> wrote:
> >
> >> Pavel,
> >>
> >> Here is an example on how to execute scan queries on partitions’ owners
> >> and perform some operation for every entry
> >>
> >>
> https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java
> >> <
> >>
> https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java
> >>>
> >>
> >> When this feature is implemented [1] it will be possible to postpone
> >> partitions movement until such an operation is in progress. Presently if
> >> the rebalancing happens the data will be retrieved from a remote node
> (new
> >> partition owner), however the result will be consistent in any case.
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-2310 <
> >> https://issues.apache.org/jira/browse/IGNITE-2310>
> >>
> >> —
> >> Denis
> >>
> >>> On May 31, 2016, at 7:42 AM, Yakov Zhdanov <yzhdanov@gridgain.com>
> >> wrote:
> >>>
> >>> Vova, even though it operates on single key we will need to "pin"
> >> partition
> >>> so key does not go to another node.
> >>>
> >>> Pavel, you can also send closures to all primary nodes to do local scan
> >>> query for each partition. This way you will go over each entry.
> >>>
> >>> Thanks!
> >>> --
> >>> Yakov Zhdanov, Director R&D
> >>> *GridGain Systems*
> >>> www.gridgain.com
> >>>
> >>> 2016-05-30 16:05 GMT-04:00 Vladimir Ozerov <vozerov@gridgain.com>:
> >>>
> >>>> Affinity run/call operate on a single key AFAIK.
> >>>>
> >>>> On Mon, May 30, 2016 at 10:55 PM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Actually I have seen a ticket to block moving partitions if
> >> affinityCall
> >>>> or
> >>>>> affinityRun are called. I think once these tickets are implemented,
> the
> >>>>> process will become reliable, no?
> >>>>>
> >>>>> D.
> >>>>>
> >>>>> On Mon, May 30, 2016 at 9:13 AM, Pavel Tupitsyn <
> >> ptupitsyn@gridgain.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Dmitriy, as I understand, there is no reliable way to do that
if
> >>>>>> rebalancing happens.
> >>>>>>
> >>>>>> On Mon, May 30, 2016 at 6:50 PM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I think we do support this use case. Why not send a computation
to
> a
> >>>>>> server
> >>>>>>> and then perform the iteration through the cache entries
locally on
> >>>>> that
> >>>>>>> server?
> >>>>>>>
> >>>>>>> On Mon, May 30, 2016 at 4:44 AM, Pavel Tupitsyn <
> >>>>> ptupitsyn@gridgain.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Igniters,
> >>>>>>>>
> >>>>>>>> Looks like we do not have an efficient way to perform
an action on
> >>>>>> EVERY
> >>>>>>>> cache entry.
> >>>>>>>>
> >>>>>>>> Let's say I want to remove all entries that match a
predicate.
> >>>>>>>> My only option is to retrieve these entries via Scan
or SQL query,
> >>>>> and
> >>>>>>> then
> >>>>>>>> call removeAll.
> >>>>>>>> This involves a lot of unnecessary network trips (send
keys to
> >>>> caller
> >>>>>>> node,
> >>>>>>>> send them back to primary nodes).
> >>>>>>>>
> >>>>>>>> Would it be possible to implement a method like
> >>>>>>>> void IgniteCache.invokeAll(entryProcessor)
> >>>>>>>> that invokes the processor on all entries and does not
return
> >>>>> anything?
> >>>>>>>> There could be more overloads that return results or
only return
> >>>>>> results
> >>>>>>>> for changed entries.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> Pavel.
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message