ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Kornev <andrewkor...@hotmail.com>
Subject RE: Missing ways to get access to Ignite CacheEntry
Date Fri, 18 Dec 2015 17:47:45 GMT
Dmitriy,

Based on what criteria does one determine which information is redundant and which is not?


To me, if an API declares a method getSomeRedundantInfo(), then the implementer has no choice,
but to comply with the contract and provide such info. One can't just return a null or ignore
it some other way, simply because it appears to have some performance impact. Besides, the
overhead of the version number may turn out to be negligible when benchmarked. Most likely
there will be near zero overhead (due to the way the network stack packs the bytes into the
ethernet frames). The workaround for the broken API however may be much more expensive in
terms of performance and user experience.

Please let's keep the API consistent and predictable.

Regards
Andrey

> From: dsetrakyan@apache.org
> Date: Fri, 18 Dec 2015 08:41:00 -0800
> Subject: Re: Missing ways to get access to Ignite CacheEntry
> To: dev@ignite.apache.org
> 
> Andrey,
> 
> The main issue with returning version is sending redundant information over
> the wire. What if we added a method IgniteCache.localEntry(K) which would
> return a local entry instance with a version. You would only be able to
> invoke this method on the server node (not client).
> 
> Would this work for you?
> 
> D.
> 
> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <andrewkornev@hotmail.com>
> wrote:
> 
> > Denis,
> >
> > I'm not suggesting to have every cache read operation return a CacheEntry.
> > The "classical" GridGain used to have a getEntry(key) method that was very
> > useful. The current version of Ignite provides virtually no way of
> > obtaining a cache entry except thru an iterator, or using an EntryProcessor
> > invocation which seems more like a hack.
> >
> > As for the question of what's stored in Ignite cache and the pre-compute,
> > it is really an off-topic. It just came about as we were looking for a way
> > to work around the issue with Ignite query not returning cache entries'
> > version numbers in the result iterator. (There is now a ticket that
> > addresses this issue: https://issues.apache.org/jira/browse/IGNITE-2202
> > )This is when we realized that there is no longer any way in Ignite to
> > retrieve the CacheEntry by its key.
> >
> > But for the sake of completeness I quickly go over the entire use case,
> > because it seems I wasn't clear enough in the previous post.
> >
> > Ignite cache holds the data entries that are used as the input into some
> > expensive computation. The data in the cache gets updated from time to
> > time, the computation only needs to be re-run
> > - on-demand -- when a user asks for the computed value and
> > - the pre-computed value is obsolete -- the source cache data has been
> > updated since the last time the value was computed.
> >
> > Otherwise, we return the pre-computed value. Note that we don't want to
> > rerun the computation for each cache update! Only when the users asks for
> > it.
> >
> > I hope it make it more clear now.
> >
> > Thanks
> > Andrey
> >
> > > Subject: Re: Missing ways to get access to Ignite CacheEntry
> > > To: dev@ignite.apache.org
> > > From: dmagda@gridgain.com
> > > Date: Fri, 18 Dec 2015 13:44:16 +0300
> > >
> > > Hi Andrey,
> > >
> > > I got you, thanks for the clarification.
> > >
> > > So since you're storing a computed value in some local data structure
> > > what is stored in the Ignite cache as a value in such a case? There
> > > should be something.
> > > Why don't you (or can't you) store a version identifier in that value
> > > that is located in the Ignite cache? This should work perfectly well for
> > > you use-case.
> > >
> > > Personally I'm against adding CacheEntry to a response of every get like
> > > operation that happens over a cache (get, getAll, SQL & Scan queries).
> > > This will add extra size to every response and only in rare cases
> > > someone will benefit from it.
> > > However, what if we add a special configuration parameter to
> > > CacheConfiguration that will manage whether CacheEntry has to be passed
> > > as a part of a response or not? Igniters, what do you think on this?
> > >
> > >
> > > Regards,
> > > Denis
> > >
> > >
> > > On 12/17/2015 9:09 AM, Andrey Kornev wrote:
> > > > Denis,In this case I used the term "caching" in general sense, meaning
> > saving the computed value for later. I didn't mean the Ignite cache.Sorry
> > about the confusion.Andrey
> > > >      _____________________________
> > > > From: Denis Magda <dmagda@gridgain.com>
> > > > Sent: Wednesday, December 16, 2015 9:48 PM
> > > > Subject: Re: Missing ways to get access to Ignite CacheEntry
> > > > To:  <dev@ignite.apache.org>
> > > >
> > > >
> > > >                     Hi Romain,
> > > >
> > > >   I'm a bit confused cause initially you saying that
> > > >
> > > >   /The value is computed on first access and it is then cached
> > > >
> > > >   /and then you add that
> > > >
> > > >   /One constraint is that the computed value is not serializable. //
> > > >   /
> > > >
> > > >
> > > >   Ignite won't be able to store a value of cache if serialization is
> > not
> > > >   supported for it by some reason.
> > > >
> > > >   Please clarify here. Probably I'm missing something from your
> > description.
> > > >
> > > >   --
> > > >   Denis
> > > >
> > > >   On 12/16/2015 6:21 PM, Andrey Kornev wrote:
> > > >   > Romain
> > > >   >
> > > >   > I suggest you be very careful using the invoke() functionality.
As
> > explained in this posting (and the follow-ups)
> > > >   >
> > http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E
> > > >   > the correct usage of the invoke() may be difficult, to put it
> > politely.
> > > >   >
> > > >   > @Denis Very briefly, the use case is "conditional re-compute".
> > There is a pretty expensive computation that needs to be applied to each
> > entry on demand. The value is computed on first access and it is then
> > cached along with the current version of the cache entry.  The  value must
> > be recomputed on next access only if the entry has been modified.
> > > >   >
> > > >   > If anyone has a better idea how to achieve something like that,
> > please do share! One constraint is that the computed value is not
> > serializable.
> > > >   >
> > > >   > Cheers
> > > >   > Andrey
> > > >   >
> > > >   >> Subject: Re: Missing ways to get access to Ignite CacheEntry
> > > >   >> To: dev@ignite.apache.org
> > > >   >> From: dmagda@gridgain.com
> > > >   >> Date: Wed, 16 Dec 2015 16:11:48 +0300
> > > >   >>
> > > >   >> Romain,
> > > >   >>
> > > >   >> If it's implemented this way for now it doesn't mean that we
can't
> > > >   >> enhance the API for other use cases ;)
> > > >   >> As I said presently it's supported for limited number of methods
> > mostly
> > > >   >> by performance reasons.
> > > >   >>
> > > >   >> Please go ahead and try invoke/invokeAll and if it doesn't
work
> > fine for
> > > >   >> you we can keep discussing what to do next.
> > > >   >>
> > > >   >> Regards,
> > > >   >> Denis
> > > >   >>
> > > >   >> On 12/16/2015 1:53 PM, Romain Gilles wrote:
> > > >   >>> Hi Denis,
> > > >   >>> Thanks for you replay. And sorry to not double check it
before.
> > I see that
> > > >   >>> if I want to work with CacheEntry, I need to use the invoke
> > method in order
> > > >   >>> to return the CacheEntry. This is the way I should do it.
It
> > sounds like
> > > >   >>> complicated for "just" getting an entry. But if you say
this is
> > the way I
> > > >   >>> will do it like that. I was just think that it could be
a common
> > use case
> > > >   >>> and therefore provide it as a shortcut.
> > > >   >>>
> > > >   >>> Thanks,
> > > >   >>>
> > > >   >>> Romain.
> > > >   >>>
> > > >   >>> Le mer. 16 déc. 2015 à 11:34, Denis Magda <dmagda@gridgain.com>
> > a écrit :
> > > >   >>>
> > > >   >>>> Hi Romain,
> > > >   >>>>
> > > >   >>>> As the current documentation of
> > org.apache.ignite.cache.CacheEntry
> > > >   >>>> states it's possible to get a reference to CacheEntry
and its
> > version
> > > >   >>>> only for methods that are executed over local data
set stored
> > on a node.
> > > >   >>>> Among such methods are invoke & invokeAll and randomEntry.
> > > >   >>>>
> > > >   >>>> You probably can get a CacheEntry and its non null
version when
> > a cache
> > > >   >>>> iterator is in use but the version will be 'null',
as far as I
> > remember,
> > > >   >>>> for those entries that are loaded from remote nodes.
> > > >   >>>> Presently Ignite doesn't transfer the version from
remote nodes
> > as a
> > > >   >>>> part of response by perform reasons.
> > > >   >>>>
> > > >   >>>> Please elaborate more on your particular use case and
what API
> > you would
> > > >   >>>> like to add in order to support it.
> > > >   >>>>
> > > >   >>>> --
> > > >   >>>> Denis
> > > >   >>>>
> > > >   >>>> On 12/16/2015 12:58 PM, Romain Gilles wrote:
> > > >   >>>>> Hi Igniters,
> > > >   >>>>> I'm looking for a way to get access to the Ignite
CacheEntry.
> > For now
> > > >   >>>> this
> > > >   >>>>> is the ways I found:
> > > >   >>>>>
> > > >   >>>>>       - Through the queris
> > > >   >>>>>       - Through jsr 107 Cache Iterable
> > > >   >>>>>       - Through jsr 107 Cache itterator
> > > >   >>>>>       - Through IgniteCache::randomEntry()
> > > >   >>>>>
> > > >   >>>>> If I remember correctly it was possible to get
the CacheEntry
> > from a
> > > >   >>>> given
> > > >   >>>>> key in old version of gridgain community
> > > >   >>>>> version: GridCacheProjection::entry(K key) GridCacheEntry<K,V>
> > > >   >>>>> I think it could be a good to introduce this feature
at
> > IgniteCache
> > > >   >>>> level.
> > > >   >>>>> Or maybe there is an other way to do it.
> > > >   >>>>>
> > > >   >>>>> Thanks,
> > > >   >>>>>
> > > >   >>>>> Romain.
> > > >   >>>>>
> > > >   >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message