ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: Stable binary key representation
Date Sun, 09 Apr 2017 14:00:40 GMT
Sergi, Vovan,

Sorry for being annoying but I still didn't get an answer on whether the
resolvers are the must for DML. The main reason why we made them up some
time ago is to support specific DML use cases. However I can't recall the
use cases.

--
Denis

On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <sergi.vladykin@gmail.com>
wrote:

> Ok, we need to do 2 things here:
>
> 1. Drop the resolvers from the source code.
> 2. Write a good page in docs on "What makes a correct cache key".
>
> Who can do that?
>
> Sergi
>
> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:
>
> > It is possible to try adding support of comparison to Resolvers, but the
> > whole approach looks wrong and for now it is better to get rid of it
> while
> > we have a chance to break compatibility.
> >
> > Sergi
> >
> > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> >
> >> The discussion should've been started with that :) If supporting
> resolvers
> >> in new architecture is not possible or means too big effort, then it's
> >> definitely not worth it.
> >>
> >> -Val
> >>
> >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vozerov@gridgain.com>
> >> wrote:
> >>
> >> > Dima,
> >> >
> >> > Yes, they may explode some internals of our indexes.
> >> >
> >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan"
<
> >> > dsetrakyan@apache.org> написал:
> >> >
> >> > > Guys,
> >> > >
> >> > > Isn't the main issue here that we cannot use the Identity Resolvers
> in
> >> > > BTrees in the 2.0 version? If yes, then we have to remove them no
> >> matter
> >> > > what.
> >> > >
> >> > > D.
> >> > >
> >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >> sergi.vladykin@gmail.com
> >> > >
> >> > > wrote:
> >> > >
> >> > > > Binary key representation is stable when we always have equal
> >> > serialized
> >> > > > bytes when the original keys are equal.
> >> > > >
> >> > > > Resolver allows you to have some extra info in the Key and equal
> >> Keys
> >> > > will
> >> > > > be serialized into different bytes, which is wrong.
> >> > > >
> >> > > > Look at the example what you can do with resolvers:
> >> > > >
> >> > > > We may have some data entry with fields a, b, c. Let's say the
> >> unique
> >> > > part
> >> > > > here is `a` and it the only fields used in Key equals() and
> >> hashCode().
> >> > > > Still we may have the following layouts:
> >> > > >
> >> > > > 1. Ka -> Vbc
> >> > > > 2. Kab -> Vc
> >> > > > 3. Kabc -> Boolean.TRUE
> >> > > >
> >> > > > The only 1 is a correct layout, others are plain wrong variants
> (but
> >> > they
> >> > > > are still possible with Resolvers) because everything that does
> not
> >> > make
> >> > > > Key unique must be in Value.
> >> > > >
> >> > > > We want to clearly state that if you have something in Key, that
> is
> >> not
> >> > > > part of equals(), then the Key is invalid and that stuff must
be
> in
> >> > > Value.
> >> > > > This allows us to rely on binary representation of a Key to be
> >> stable
> >> > and
> >> > > > have some more optimizations and code simplifications with respect
> >> to
> >> > > these
> >> > > > assumptions.
> >> > > >
> >> > > > Sergi
> >> > > >
> >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >> > > > valentin.kulichenko@gmail.com>:
> >> > > >
> >> > > > > Even with my vast expirience I would never claim that I've
seen
> >> > > > > "everything" :)
> >> > > > >
> >> > > > > What do you mean by stable binary key representation and
how
> >> > resolvers
> >> > > > make
> >> > > > > it unstable?
> >> > > > >
> >> > > > > -Val
> >> > > > >
> >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> >> > > sergi.vladykin@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Val,
> >> > > > > >
> >> > > > > > I know that you have really vast experience in Ignite
> >> deployments
> >> > and
> >> > > > > > probably saw everything that can happen. Did you ever
see
> >> identity
> >> > > > > > resolvers use in real life? I guess no.
> >> > > > > >
> >> > > > > > Hibernate example is bad here, because if their key
is
> unstable
> >> > > across
> >> > > > > > multiple JVMs, it means that it was not designed for
> distributed
> >> > > > caches a
> >> > > > > > priori.
> >> > > > > >
> >> > > > > > Also knowing in advance about stable binary key representation
> >> > allows
> >> > > > us
> >> > > > > to
> >> > > > > > apply additional optimizations, like comparing keys
without
> >> > detaching
> >> > > > > them
> >> > > > > > from offheap memory.
> >> > > > > >
> >> > > > > > We always will be able to add this stuff back if we
see users
> >> > really
> >> > > > need
> >> > > > > > it. Let's remove it for 2.0.
> >> > > > > >
> >> > > > > > Sergi
> >> > > > > >
> >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
> >> > > > > > valentin.kulichenko@gmail.com>:
> >> > > > > >
> >> > > > > > > Alex,
> >> > > > > > >
> >> > > > > > > To be honest, I don't understand the reasoning
behind the
> >> > removal.
> >> > > I
> >> > > > > > think
> >> > > > > > > resolvers provide good flexibility for different
corner
> cases
> >> and
> >> > > > it's
> >> > > > > a
> >> > > > > > > good thing to have them. Note that they can be
applied not
> >> only
> >> > to
> >> > > > > cache
> >> > > > > > > keys, but to any binary objects.
> >> > > > > > >
> >> > > > > > > Hibernate issue is actually a good example of
such use case.
> >> The
> >> > > fact
> >> > > > > > that
> >> > > > > > > we found an alternative solution doesn't actually
mean
> >> anything,
> >> > > > > because
> >> > > > > > > what if this happened not in our module, but in
user's
> >> > application?
> >> > > > > > > Unfortunately, we can't predict everything.
> >> > > > > > >
> >> > > > > > > Error proneness is not a very strong argument
either,
> because
> >> in
> >> > my
> >> > > > > view
> >> > > > > > > these resolvers are as much error prone as BinaryIdMapper,
> for
> >> > > > example.
> >> > > > > > >
> >> > > > > > > -Val
> >> > > > > > >
> >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk
<
> >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> >> > > > > > >
> >> > > > > > > > Denis,
> >> > > > > > > >
> >> > > > > > > > Can you suggest a use-case where identity
resolver is
> needed
> >> > > (given
> >> > > > > > that
> >> > > > > > > we
> >> > > > > > > > agree that a key must contain only valuable
fields)?
> >> > > > > > > >
> >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dmagda@apache.org
> >:
> >> > > > > > > >
> >> > > > > > > > > Where do you want to remove the identity
resolvers from?
> >> If
> >> > > it’s
> >> > > > > > > related
> >> > > > > > > > > to the internals of Hibernate module
then it’s fine but
> if
> >> > you
> >> > > > > > suggest
> >> > > > > > > > > removing identity resolvers public interfaces
then it
> >> might
> >> > be
> >> > > a
> >> > > > > > haste
> >> > > > > > > > > decision.
> >> > > > > > > > >
> >> > > > > > > > > —
> >> > > > > > > > > Denis
> >> > > > > > > > >
> >> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey
Goncharuk <
> >> > > > > > > > alexey.goncharuk@gmail.com>
> >> > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > +1, I see no other reasons to keep
it.
> >> > > > > > > > > >
> >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi
Vladykin <
> >> > > > > > sergi.vladykin@gmail.com
> >> > > > > > > >:
> >> > > > > > > > > >
> >> > > > > > > > > >> +1
> >> > > > > > > > > >>
> >> > > > > > > > > >> Lets drop them.
> >> > > > > > > > > >>
> >> > > > > > > > > >> Sergi
> >> > > > > > > > > >>
> >> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00
Dmitriy Govorukhin <
> >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> >> > > > > > > > > >> :
> >> > > > > > > > > >>
> >> > > > > > > > > >>> Hi guys, i implemented
proxy for IgniteCache in
> >> hibernate
> >> > > > > > > > integration,
> >> > > > > > > > > >> this
> >> > > > > > > > > >>> proxy transformate cacheKey
to our key wrapper,
> leaves
> >> > only
> >> > > > > > > required
> >> > > > > > > > > >>> field. I think we can remove
identity resolve, it
> >> should
> >> > > not
> >> > > > > > broke
> >> > > > > > > > > >>> integration with hibernate.
Any objections?
> >> > > > > > > > > >>>
> >> > > > > > > > > >>> On Wed, Mar 29, 2017 at
11:07 PM, Valentin
> Kulichenko
> >> <
> >> > > > > > > > > >>> valentin.kulichenko@gmail.com>
wrote:
> >> > > > > > > > > >>>
> >> > > > > > > > > >>>> I'm not saying there
is no alternative solution.
> But
> >> > let's
> >> > > > > > > implement
> >> > > > > > > > > it
> >> > > > > > > > > >>> and
> >> > > > > > > > > >>>> prove that it works
first, and remove resolvers
> only
> >> > after
> >> > > > > that.
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>> -Val
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>> On Wed, Mar 29, 2017
at 12:18 PM, Sergi Vladykin <
> >> > > > > > > > > >>> sergi.vladykin@gmail.com
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>> wrote:
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>>> Guys, nothing is
impossible if you know a bit
> about
> >> > > > > reflection
> >> > > > > > in
> >> > > > > > > > > >> Java
> >> > > > > > > > > >>> :)
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> We had a look at
the CacheKey class and it is
> easily
> >> > > > > > replaceable.
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> Sergi
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>> 2017-03-29 21:49
GMT+03:00 Dmitriy Setrakyan <
> >> > > > > > > > dsetrakyan@apache.org
> >> > > > > > > > > >>> :
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>>> On Wed, Mar
29, 2017 at 11:43 AM, Valentin
> >> Kulichenko
> >> > <
> >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com>
wrote:
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>>> "Hibernate
key" is the CacheKey class I was
> >> referring
> >> > > to.
> >> > > > > > It's
> >> > > > > > > > > >>>> provided
> >> > > > > > > > > >>>>>> by
> >> > > > > > > > > >>>>>>> Hibernate,
not by user and not by us. So I'm not
> >> sure
> >> > > > it's
> >> > > > > > > > > >> possible
> >> > > > > > > > > >>>> to
> >> > > > > > > > > >>>>>>> replace
it.
> >> > > > > > > > > >>>>>>>
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>> If it is impossible
to replace or get rid of the
> >> > > Hibernate
> >> > > > > > key,
> >> > > > > > > is
> >> > > > > > > > > >>> this
> >> > > > > > > > > >>>>>> discussion
valid at all?
> >> > > > > > > > > >>>>>>
> >> > > > > > > > > >>>>>
> >> > > > > > > > > >>>>
> >> > > > > > > > > >>>
> >> > > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message