ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Missing ways to get access to Ignite CacheEntry
Date Mon, 21 Dec 2015 07:59:46 GMT
Again, my issue is with unwrap method. Are we all in agreement that
unwrap(…) will not work on the client side?

D.

On Sun, Dec 20, 2015 at 11:30 PM, Denis Magda <dmagda@gridgain.com> wrote:

> Actually I think that Andrey's proposal on addition of
> IgniteCache.getEntry(k) sounds natural.
>
> Is there any platform limitation that will prevent us from loading an
> entry with its version from a remote node when such a particular method is
> used?
> I think that it's feasible to support it and in addition I would add on
> more method to the API - IgniteCache.getAllEntries(keysSet).
>
> --
> Denis
>
>
> On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote:
>
>> On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <andrewkornev@hotmail.com>
>> wrote:
>>
>> How would I know how to interpret the returned null?
>>>
>>> Can we just have a simple getEntry(K) method that has semantics of the
>>> regular get(K)? Why invent something?
>>>
>>> If we do this, how would you unwrap such entry on the client side?
>>
>>
>> Cheers
>>> Andrey
>>>
>>> From: dsetrakyan@apache.org
>>>> Date: Fri, 18 Dec 2015 12:55:47 -0800
>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>> To: dev@ignite.apache.org
>>>>
>>>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev <
>>>>
>>> andrewkornev@hotmail.com>
>>>
>>>> wrote:
>>>>
>>>> Dmitriy,
>>>>>
>>>>> I understand that we're talking about Ignite CacheEntry and not
>>>>>
>>>> JCache's
>>>
>>>> CacheEntry that has no version property to speak of.
>>>>>
>>>>> But I don't understand why the method is named localEntry()? What does
>>>>> "local" mean in this context? Does it mean that I can only use it on
>>>>>
>>>> the
>>>
>>>> primary node for a given key? What would happen if I call this method
>>>>>
>>>> on a
>>>
>>>> non-primary node? And what would I do if I needed the entry (along
>>>>>
>>>> with the
>>>
>>>> valid version it contains) on a non-primary node?
>>>>>
>>>>> If the node is either primary or backup, then you will get the entry
>>>>
>>> back.
>>>
>>>> Otherwise, you will get null.
>>>>
>>>>
>>>> From: dsetrakyan@apache.org
>>>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800
>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>>> To: dev@ignite.apache.org
>>>>>>
>>>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <
>>>>>>
>>>>> andrewkornev@hotmail.com
>>>
>>>> wrote:
>>>>>>
>>>>>> Dmitriy,
>>>>>>>
>>>>>>> Based on what criteria does one determine which information is
>>>>>>>
>>>>>> redundant
>>>>>
>>>>>> and which is not?
>>>>>>>
>>>>>>> To me, if an API declares a method getSomeRedundantInfo(), then
the
>>>>>>> implementer has no choice, but to comply with the contract and
>>>>>>>
>>>>>> provide
>>>
>>>> such
>>>>>
>>>>>> info. One can't just return a null or ignore it some other way,
>>>>>>>
>>>>>> simply
>>>
>>>> because it appears to have some performance impact.
>>>>>>>
>>>>>>
>>>>>> Andrey, the CacheEntry is not the JCache entry, but rather the
>>>>>>
>>>>> internal
>>>
>>>> Ignite entry that you can get when unwrapping the JCache entry. The
>>>>>> unwrapping, obviously, can only happen on the server side.
>>>>>>
>>>>> Therefore, I
>>>
>>>> am
>>>>>
>>>>>> suggesting that we also add a localEntry(K) method which will
>>>>>>
>>>>> provide the
>>>
>>>> entry API on the server side directly (not from iterator or EP).
>>>>>>
>>>>>> Now I understand that the version() method currently does not work
>>>>>>
>>>>> properly
>>>>>
>>>>>> on the server side, but this is something we can easily fix.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>>
>>>>>> Besides, the overhead of the version number may turn out to be
>>>>>>>
>>>>>> negligible
>>>>>
>>>>>> when benchmarked. Most likely there will be near zero overhead
>>>>>>>
>>>>>> (due to
>>>
>>>> the
>>>>>
>>>>>> way the network stack packs the bytes into the ethernet frames).
>>>>>>>
>>>>>> The
>>>
>>>> workaround for the broken API however may be much more expensive in
>>>>>>>
>>>>>> terms
>>>>>
>>>>>> of performance and user experience.
>>>>>>>
>>>>>>> Please let's keep the API consistent and predictable.
>>>>>>>
>>>>>>> Regards
>>>>>>> Andrey
>>>>>>>
>>>>>>> From: dsetrakyan@apache.org
>>>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800
>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>>>>> To: dev@ignite.apache.org
>>>>>>>>
>>>>>>>> Andrey,
>>>>>>>>
>>>>>>>> The main issue with returning version is sending redundant
>>>>>>>>
>>>>>>> information
>>>>>
>>>>>> over
>>>>>>>
>>>>>>>> the wire. What if we added a method IgniteCache.localEntry(K)
>>>>>>>>
>>>>>>> which
>>>
>>>> would
>>>>>
>>>>>> return a local entry instance with a version. You would only be
>>>>>>>>
>>>>>>> able
>>>
>>>> to
>>>>>
>>>>>> invoke this method on the server node (not client).
>>>>>>>>
>>>>>>>> Would this work for you?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <
>>>>>>>>
>>>>>>> andrewkornev@hotmail.com
>>>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis,
>>>>>>>>>
>>>>>>>>> I'm not suggesting to have every cache read operation
return a
>>>>>>>>>
>>>>>>>> CacheEntry.
>>>>>>>
>>>>>>>> The "classical" GridGain used to have a getEntry(key) method
>>>>>>>>>
>>>>>>>> that
>>>
>>>> was
>>>>>
>>>>>> very
>>>>>>>
>>>>>>>> useful. The current version of Ignite provides virtually
no
>>>>>>>>>
>>>>>>>> way of
>>>
>>>> obtaining a cache entry except thru an iterator, or using an
>>>>>>>>>
>>>>>>>> EntryProcessor
>>>>>>>
>>>>>>>> invocation which seems more like a hack.
>>>>>>>>>
>>>>>>>>> As for the question of what's stored in Ignite cache
and the
>>>>>>>>>
>>>>>>>> pre-compute,
>>>>>>>
>>>>>>>> it is really an off-topic. It just came about as we were
>>>>>>>>>
>>>>>>>> looking
>>>
>>>> for a
>>>>>
>>>>>> way
>>>>>>>
>>>>>>>> to work around the issue with Ignite query not returning
cache
>>>>>>>>>
>>>>>>>> entries'
>>>>>
>>>>>> version numbers in the result iterator. (There is now a ticket
>>>>>>>>>
>>>>>>>> that
>>>
>>>> addresses this issue:
>>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2202
>>>>>>>
>>>>>>>> )This is when we realized that there is no longer any way
in
>>>>>>>>>
>>>>>>>> Ignite to
>>>>>
>>>>>> retrieve the CacheEntry by its key.
>>>>>>>>>
>>>>>>>>> But for the sake of completeness I quickly go over the
entire
>>>>>>>>>
>>>>>>>> use
>>>
>>>> case,
>>>>>
>>>>>> because it seems I wasn't clear enough in the previous post.
>>>>>>>>>
>>>>>>>>> Ignite cache holds the data entries that are used as
the input
>>>>>>>>>
>>>>>>>> into
>>>
>>>> some
>>>>>>>
>>>>>>>> expensive computation. The data in the cache gets updated
from
>>>>>>>>>
>>>>>>>> time to
>>>>>
>>>>>> time, the computation only needs to be re-run
>>>>>>>>> - on-demand -- when a user asks for the computed value
and
>>>>>>>>> - the pre-computed value is obsolete -- the source cache
data
>>>>>>>>>
>>>>>>>> has
>>>
>>>> been
>>>>>
>>>>>> updated since the last time the value was computed.
>>>>>>>>>
>>>>>>>>> Otherwise, we return the pre-computed value. Note that
we don't
>>>>>>>>>
>>>>>>>> want to
>>>>>
>>>>>> rerun the computation for each cache update! Only when the
>>>>>>>>>
>>>>>>>> users
>>>
>>>> asks
>>>>>
>>>>>> for
>>>>>>>
>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> I hope it make it more clear now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Andrey
>>>>>>>>>
>>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>>>>>>> To: dev@ignite.apache.org
>>>>>>>>>> From: dmagda@gridgain.com
>>>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300
>>>>>>>>>>
>>>>>>>>>> Hi Andrey,
>>>>>>>>>>
>>>>>>>>>> I got you, thanks for the clarification.
>>>>>>>>>>
>>>>>>>>>> So since you're storing a computed value in some
local data
>>>>>>>>>>
>>>>>>>>> structure
>>>>>
>>>>>> what is stored in the Ignite cache as a value in such a case?
>>>>>>>>>>
>>>>>>>>> There
>>>>>
>>>>>> should be something.
>>>>>>>>>> Why don't you (or can't you) store a version identifier
in
>>>>>>>>>>
>>>>>>>>> that
>>>
>>>> value
>>>>>
>>>>>> that is located in the Ignite cache? This should work
>>>>>>>>>>
>>>>>>>>> perfectly
>>>
>>>> well
>>>>>
>>>>>> for
>>>>>>>
>>>>>>>> you use-case.
>>>>>>>>>>
>>>>>>>>>> Personally I'm against adding CacheEntry to a response
of
>>>>>>>>>>
>>>>>>>>> every
>>>
>>>> get
>>>>>
>>>>>> like
>>>>>>>
>>>>>>>> operation that happens over a cache (get, getAll, SQL &
Scan
>>>>>>>>>>
>>>>>>>>> queries).
>>>>>>>
>>>>>>>> This will add extra size to every response and only in rare
>>>>>>>>>>
>>>>>>>>> cases
>>>
>>>> someone will benefit from it.
>>>>>>>>>> However, what if we add a special configuration parameter
to
>>>>>>>>>> CacheConfiguration that will manage whether CacheEntry
has
>>>>>>>>>>
>>>>>>>>> to be
>>>
>>>> passed
>>>>>>>
>>>>>>>> as a part of a response or not? Igniters, what do you think
>>>>>>>>>>
>>>>>>>>> on
>>>
>>>> this?
>>>>>
>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote:
>>>>>>>>>>
>>>>>>>>>>> Denis,In this case I used the term "caching"
in general
>>>>>>>>>>>
>>>>>>>>>> sense,
>>>
>>>> meaning
>>>>>>>
>>>>>>>> saving the computed value for later. I didn't mean the Ignite
>>>>>>>>>
>>>>>>>> cache.Sorry
>>>>>>>
>>>>>>>> about the confusion.Andrey
>>>>>>>>>
>>>>>>>>>>       _____________________________
>>>>>>>>>>> From: Denis Magda <dmagda@gridgain.com>
>>>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM
>>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite
>>>>>>>>>>>
>>>>>>>>>> CacheEntry
>>>
>>>> To:  <dev@ignite.apache.org>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                      Hi Romain,
>>>>>>>>>>>
>>>>>>>>>>>    I'm a bit confused cause initially you saying
that
>>>>>>>>>>>
>>>>>>>>>>>    /The value is computed on first access and
it is then
>>>>>>>>>>>
>>>>>>>>>> cached
>>>
>>>>    /and then you add that
>>>>>>>>>>>
>>>>>>>>>>>    /One constraint is that the computed value
is not
>>>>>>>>>>>
>>>>>>>>>> serializable.
>>>>>
>>>>>> //
>>>>>>>
>>>>>>>>    /
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>    Ignite won't be able to store a value of cache
if
>>>>>>>>>>>
>>>>>>>>>> serialization
>>>>>
>>>>>> is
>>>>>>>
>>>>>>>> not
>>>>>>>>>
>>>>>>>>>>    supported for it by some reason.
>>>>>>>>>>>
>>>>>>>>>>>    Please clarify here. Probably I'm missing
something from
>>>>>>>>>>>
>>>>>>>>>> your
>>>
>>>> description.
>>>>>>>>>
>>>>>>>>>>    --
>>>>>>>>>>>    Denis
>>>>>>>>>>>
>>>>>>>>>>>    On 12/16/2015 6:21 PM, Andrey Kornev wrote:
>>>>>>>>>>>    > Romain
>>>>>>>>>>>    >
>>>>>>>>>>>    > I suggest you be very careful using the
invoke()
>>>>>>>>>>>
>>>>>>>>>> functionality. As
>>>>>>>
>>>>>>>> explained in this posting (and the follow-ups)
>>>>>>>>>
>>>>>>>>>>    >
>>>>>>>>>>>
>>>>>>>>>>
>>> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E
>>>
>>>>    > the correct usage of the invoke() may be difficult, to
>>>>>>>>>>>
>>>>>>>>>> put
>>>
>>>> it
>>>>>
>>>>>> politely.
>>>>>>>>>
>>>>>>>>>>    >
>>>>>>>>>>>    > @Denis Very briefly, the use case is
"conditional
>>>>>>>>>>>
>>>>>>>>>> re-compute".
>>>>>
>>>>>> There is a pretty expensive computation that needs to be
>>>>>>>>>
>>>>>>>> applied to
>>>
>>>> each
>>>>>>>
>>>>>>>> entry on demand. The value is computed on first access and
it
>>>>>>>>>
>>>>>>>> is
>>>
>>>> then
>>>>>
>>>>>> cached along with the current version of the cache entry.  The
>>>>>>>>>
>>>>>>>> value
>>>>>
>>>>>> must
>>>>>>>
>>>>>>>> be recomputed on next access only if the entry has been
>>>>>>>>>
>>>>>>>> modified.
>>>
>>>>    >
>>>>>>>>>>>    > If anyone has a better idea how to achieve
something
>>>>>>>>>>>
>>>>>>>>>> like
>>>
>>>> that,
>>>>>
>>>>>> please do share! One constraint is that the computed value is
>>>>>>>>>
>>>>>>>> not
>>>
>>>> serializable.
>>>>>>>>>
>>>>>>>>>>    >
>>>>>>>>>>>    > Cheers
>>>>>>>>>>>    > Andrey
>>>>>>>>>>>    >
>>>>>>>>>>>    >> Subject: Re: Missing ways to get
access to Ignite
>>>>>>>>>>>
>>>>>>>>>> CacheEntry
>>>>>
>>>>>>    >> To: dev@ignite.apache.org
>>>>>>>>>>>    >> From: dmagda@gridgain.com
>>>>>>>>>>>    >> Date: Wed, 16 Dec 2015 16:11:48 +0300
>>>>>>>>>>>    >>
>>>>>>>>>>>    >> Romain,
>>>>>>>>>>>    >>
>>>>>>>>>>>    >> If it's implemented this way for
now it doesn't mean
>>>>>>>>>>>
>>>>>>>>>> that
>>>
>>>> we
>>>>>
>>>>>> can't
>>>>>>>
>>>>>>>>    >> enhance the API for other use cases ;)
>>>>>>>>>>>    >> As I said presently it's supported
for limited number
>>>>>>>>>>>
>>>>>>>>>> of
>>>
>>>> methods
>>>>>>>
>>>>>>>> mostly
>>>>>>>>>
>>>>>>>>>>    >> by performance reasons.
>>>>>>>>>>>    >>
>>>>>>>>>>>    >> Please go ahead and try invoke/invokeAll
and if it
>>>>>>>>>>>
>>>>>>>>>> doesn't
>>>
>>>> work
>>>>>>>
>>>>>>>> fine for
>>>>>>>>>
>>>>>>>>>>    >> you we can keep discussing what to do
next.
>>>>>>>>>>>    >>
>>>>>>>>>>>    >> Regards,
>>>>>>>>>>>    >> Denis
>>>>>>>>>>>    >>
>>>>>>>>>>>    >> On 12/16/2015 1:53 PM, Romain Gilles
wrote:
>>>>>>>>>>>    >>> Hi Denis,
>>>>>>>>>>>    >>> Thanks for you replay. And sorry
to not double check
>>>>>>>>>>>
>>>>>>>>>> it
>>>
>>>> before.
>>>>>>>
>>>>>>>> I see that
>>>>>>>>>
>>>>>>>>>>    >>> if I want to work with CacheEntry,
I need to use the
>>>>>>>>>>>
>>>>>>>>>> invoke
>>>>>
>>>>>> method in order
>>>>>>>>>
>>>>>>>>>>    >>> to return the CacheEntry. This is
the way I should do
>>>>>>>>>>>
>>>>>>>>>> it. It
>>>>>
>>>>>> sounds like
>>>>>>>>>
>>>>>>>>>>    >>> complicated for "just" getting an
entry. But if you
>>>>>>>>>>>
>>>>>>>>>> say
>>>
>>>> this
>>>>>
>>>>>> is
>>>>>>>
>>>>>>>> the way I
>>>>>>>>>
>>>>>>>>>>    >>> will do it like that. I was just
think that it could
>>>>>>>>>>>
>>>>>>>>>> be a
>>>
>>>> common
>>>>>>>
>>>>>>>> use case
>>>>>>>>>
>>>>>>>>>>    >>> and therefore provide it as a shortcut.
>>>>>>>>>>>    >>>
>>>>>>>>>>>    >>> Thanks,
>>>>>>>>>>>    >>>
>>>>>>>>>>>    >>> Romain.
>>>>>>>>>>>    >>>
>>>>>>>>>>>    >>> Le mer. 16 déc. 2015 à 11:34,
Denis Magda <
>>>>>>>>>>>
>>>>>>>>>> dmagda@gridgain.com>
>>>>>>>
>>>>>>>> a écrit :
>>>>>>>>>
>>>>>>>>>>    >>>
>>>>>>>>>>>    >>>> Hi Romain,
>>>>>>>>>>>    >>>>
>>>>>>>>>>>    >>>> As the current documentation
of
>>>>>>>>>>>
>>>>>>>>>> org.apache.ignite.cache.CacheEntry
>>>>>>>>>
>>>>>>>>>>    >>>> states it's possible to get a
reference to
>>>>>>>>>>>
>>>>>>>>>> CacheEntry
>>>
>>>> and
>>>>>
>>>>>> its
>>>>>>>
>>>>>>>> version
>>>>>>>>>
>>>>>>>>>>    >>>> only for methods that are executed
over local data
>>>>>>>>>>>
>>>>>>>>>> set
>>>
>>>> stored
>>>>>>>
>>>>>>>> on a node.
>>>>>>>>>
>>>>>>>>>>    >>>> Among such methods are invoke
& invokeAll and
>>>>>>>>>>>
>>>>>>>>>> randomEntry.
>>>>>
>>>>>>    >>>>
>>>>>>>>>>>    >>>> You probably can get a CacheEntry
and its non null
>>>>>>>>>>>
>>>>>>>>>> version
>>>>>
>>>>>> when
>>>>>>>
>>>>>>>> a cache
>>>>>>>>>
>>>>>>>>>>    >>>> iterator is in use but the version
will be 'null',
>>>>>>>>>>>
>>>>>>>>>> as
>>>
>>>> far
>>>>>
>>>>>> as I
>>>>>>>
>>>>>>>> remember,
>>>>>>>>>
>>>>>>>>>>    >>>> for those entries that are loaded
from remote nodes.
>>>>>>>>>>>    >>>> Presently Ignite doesn't
transfer the version from
>>>>>>>>>>>
>>>>>>>>>> remote
>>>>>
>>>>>> nodes
>>>>>>>
>>>>>>>> as a
>>>>>>>>>
>>>>>>>>>>    >>>> part of response by perform reasons.
>>>>>>>>>>>    >>>>
>>>>>>>>>>>    >>>> Please elaborate more on
your particular use case
>>>>>>>>>>>
>>>>>>>>>> and
>>>
>>>> what
>>>>>
>>>>>> API
>>>>>>>
>>>>>>>> you would
>>>>>>>>>
>>>>>>>>>>    >>>> like to add in order to support
it.
>>>>>>>>>>>    >>>>
>>>>>>>>>>>    >>>> --
>>>>>>>>>>>    >>>> Denis
>>>>>>>>>>>    >>>>
>>>>>>>>>>>    >>>> On 12/16/2015 12:58 PM, Romain
Gilles wrote:
>>>>>>>>>>>    >>>>> Hi Igniters,
>>>>>>>>>>>    >>>>> I'm looking for a way
to get access to the Ignite
>>>>>>>>>>>
>>>>>>>>>> CacheEntry.
>>>>>>>
>>>>>>>> For now
>>>>>>>>>
>>>>>>>>>>    >>>> this
>>>>>>>>>>>    >>>>> is the ways I found:
>>>>>>>>>>>    >>>>>
>>>>>>>>>>>    >>>>>       - Through the queris
>>>>>>>>>>>    >>>>>       - Through jsr 107
Cache Iterable
>>>>>>>>>>>    >>>>>       - Through jsr 107
Cache itterator
>>>>>>>>>>>    >>>>>       - Through IgniteCache::randomEntry()
>>>>>>>>>>>    >>>>>
>>>>>>>>>>>    >>>>> If I remember correctly
it was possible to get the
>>>>>>>>>>>
>>>>>>>>>> CacheEntry
>>>>>>>
>>>>>>>> from a
>>>>>>>>>
>>>>>>>>>>    >>>> given
>>>>>>>>>>>    >>>>> key in old version of
gridgain community
>>>>>>>>>>>    >>>>> version: GridCacheProjection::entry(K
key)
>>>>>>>>>>>
>>>>>>>>>> GridCacheEntry<K,V>
>>>>>>>
>>>>>>>>    >>>>> I think it could be a good to introduce
this
>>>>>>>>>>>
>>>>>>>>>> feature at
>>>
>>>> IgniteCache
>>>>>>>>>
>>>>>>>>>>    >>>> level.
>>>>>>>>>>>    >>>>> Or maybe there is an
other way to do it.
>>>>>>>>>>>    >>>>>
>>>>>>>>>>>    >>>>> Thanks,
>>>>>>>>>>>    >>>>>
>>>>>>>>>>>    >>>>> Romain.
>>>>>>>>>>>    >>>>>
>>>>>>>>>>>    >
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message