ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Stable binary key representation
Date Sun, 09 Apr 2017 14:39:13 GMT
Resolvers were essential for DML because we had broken comparison semantics
of binary objects. This is not the case now.

Resolver as a whole is normal practice. E.g. it is implemented in .NET on
core language level and widely used in many cases. Hazelcast has it as well
AFAIK. So it is wrong to think that the whole idea is useless. Think of it
as a comparator's brother.

The only reason why we need to remove it is missing hash index in new
architecture. It makes sense, as it is better to have AI 2.0 without them,
than no AI 2.0 :-)

09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
sergi.vladykin@gmail.com> написал:

> I guess Resolvers were added to DML just because they already existed since
> 1.9 and we were forced to support them in all the parts of our product.
>
> We have to stop this practice to add features without clear real life use
> cases for them.
>
> Sergi
>
> 2017-04-09 17:00 GMT+03:00 Denis Magda <dmagda@gridgain.com>:
>
> > Sergi, Vovan,
> >
> > Sorry for being annoying but I still didn't get an answer on whether the
> > resolvers are the must for DML. The main reason why we made them up some
> > time ago is to support specific DML use cases. However I can't recall the
> > use cases.
> >
> > --
> > Denis
> >
> > On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <sergi.vladykin@gmail.com
> >
> > wrote:
> >
> > > Ok, we need to do 2 things here:
> > >
> > > 1. Drop the resolvers from the source code.
> > > 2. Write a good page in docs on "What makes a correct cache key".
> > >
> > > Who can do that?
> > >
> > > Sergi
> > >
> > > 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:
> > >
> > > > It is possible to try adding support of comparison to Resolvers, but
> > the
> > > > whole approach looks wrong and for now it is better to get rid of it
> > > while
> > > > we have a chance to break compatibility.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com>:
> > > >
> > > >> The discussion should've been started with that :) If supporting
> > > resolvers
> > > >> in new architecture is not possible or means too big effort, then
> it's
> > > >> definitely not worth it.
> > > >>
> > > >> -Val
> > > >>
> > > >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > >> wrote:
> > > >>
> > > >> > Dima,
> > > >> >
> > > >> > Yes, they may explode some internals of our indexes.
> > > >> >
> > > >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan"
<
> > > >> > dsetrakyan@apache.org> написал:
> > > >> >
> > > >> > > Guys,
> > > >> > >
> > > >> > > Isn't the main issue here that we cannot use the Identity
> > Resolvers
> > > in
> > > >> > > BTrees in the 2.0 version? If yes, then we have to remove
them
> no
> > > >> matter
> > > >> > > what.
> > > >> > >
> > > >> > > D.
> > > >> > >
> > > >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > > >> sergi.vladykin@gmail.com
> > > >> > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Binary key representation is stable when we always
have equal
> > > >> > serialized
> > > >> > > > bytes when the original keys are equal.
> > > >> > > >
> > > >> > > > Resolver allows you to have some extra info in the
Key and
> equal
> > > >> Keys
> > > >> > > will
> > > >> > > > be serialized into different bytes, which is wrong.
> > > >> > > >
> > > >> > > > Look at the example what you can do with resolvers:
> > > >> > > >
> > > >> > > > We may have some data entry with fields a, b, c. Let's
say the
> > > >> unique
> > > >> > > part
> > > >> > > > here is `a` and it the only fields used in Key equals()
and
> > > >> hashCode().
> > > >> > > > Still we may have the following layouts:
> > > >> > > >
> > > >> > > > 1. Ka -> Vbc
> > > >> > > > 2. Kab -> Vc
> > > >> > > > 3. Kabc -> Boolean.TRUE
> > > >> > > >
> > > >> > > > The only 1 is a correct layout, others are plain wrong
> variants
> > > (but
> > > >> > they
> > > >> > > > are still possible with Resolvers) because everything
that
> does
> > > not
> > > >> > make
> > > >> > > > Key unique must be in Value.
> > > >> > > >
> > > >> > > > We want to clearly state that if you have something
in Key,
> that
> > > is
> > > >> not
> > > >> > > > part of equals(), then the Key is invalid and that
stuff must
> be
> > > in
> > > >> > > Value.
> > > >> > > > This allows us to rely on binary representation of
a Key to be
> > > >> stable
> > > >> > and
> > > >> > > > have some more optimizations and code simplifications
with
> > respect
> > > >> to
> > > >> > > these
> > > >> > > > assumptions.
> > > >> > > >
> > > >> > > > Sergi
> > > >> > > >
> > > >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > > >> > > > valentin.kulichenko@gmail.com>:
> > > >> > > >
> > > >> > > > > Even with my vast expirience I would never claim
that I've
> > seen
> > > >> > > > > "everything" :)
> > > >> > > > >
> > > >> > > > > What do you mean by stable binary key representation
and how
> > > >> > resolvers
> > > >> > > > make
> > > >> > > > > it unstable?
> > > >> > > > >
> > > >> > > > > -Val
> > > >> > > > >
> > > >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin
<
> > > >> > > sergi.vladykin@gmail.com
> > > >> > > > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Val,
> > > >> > > > > >
> > > >> > > > > > I know that you have really vast experience
in Ignite
> > > >> deployments
> > > >> > and
> > > >> > > > > > probably saw everything that can happen.
Did you ever see
> > > >> identity
> > > >> > > > > > resolvers use in real life? I guess no.
> > > >> > > > > >
> > > >> > > > > > Hibernate example is bad here, because if
their key is
> > > unstable
> > > >> > > across
> > > >> > > > > > multiple JVMs, it means that it was not designed
for
> > > distributed
> > > >> > > > caches a
> > > >> > > > > > priori.
> > > >> > > > > >
> > > >> > > > > > Also knowing in advance about stable binary
key
> > representation
> > > >> > allows
> > > >> > > > us
> > > >> > > > > to
> > > >> > > > > > apply additional optimizations, like comparing
keys
> without
> > > >> > detaching
> > > >> > > > > them
> > > >> > > > > > from offheap memory.
> > > >> > > > > >
> > > >> > > > > > We always will be able to add this stuff
back if we see
> > users
> > > >> > really
> > > >> > > > need
> > > >> > > > > > it. Let's remove it for 2.0.
> > > >> > > > > >
> > > >> > > > > > Sergi
> > > >> > > > > >
> > > >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko
<
> > > >> > > > > > valentin.kulichenko@gmail.com>:
> > > >> > > > > >
> > > >> > > > > > > Alex,
> > > >> > > > > > >
> > > >> > > > > > > To be honest, I don't understand the
reasoning behind
> the
> > > >> > removal.
> > > >> > > I
> > > >> > > > > > think
> > > >> > > > > > > resolvers provide good flexibility for
different corner
> > > cases
> > > >> and
> > > >> > > > it's
> > > >> > > > > a
> > > >> > > > > > > good thing to have them. Note that they
can be applied
> not
> > > >> only
> > > >> > to
> > > >> > > > > cache
> > > >> > > > > > > keys, but to any binary objects.
> > > >> > > > > > >
> > > >> > > > > > > Hibernate issue is actually a good example
of such use
> > case.
> > > >> The
> > > >> > > fact
> > > >> > > > > > that
> > > >> > > > > > > we found an alternative solution doesn't
actually mean
> > > >> anything,
> > > >> > > > > because
> > > >> > > > > > > what if this happened not in our module,
but in user's
> > > >> > application?
> > > >> > > > > > > Unfortunately, we can't predict everything.
> > > >> > > > > > >
> > > >> > > > > > > Error proneness is not a very strong
argument either,
> > > because
> > > >> in
> > > >> > my
> > > >> > > > > view
> > > >> > > > > > > these resolvers are as much error prone
as
> BinaryIdMapper,
> > > for
> > > >> > > > example.
> > > >> > > > > > >
> > > >> > > > > > > -Val
> > > >> > > > > > >
> > > >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey
Goncharuk <
> > > >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Denis,
> > > >> > > > > > > >
> > > >> > > > > > > > Can you suggest a use-case where
identity resolver is
> > > needed
> > > >> > > (given
> > > >> > > > > > that
> > > >> > > > > > > we
> > > >> > > > > > > > agree that a key must contain only
valuable fields)?
> > > >> > > > > > > >
> > > >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis
Magda <
> > dmagda@apache.org
> > > >:
> > > >> > > > > > > >
> > > >> > > > > > > > > Where do you want to remove
the identity resolvers
> > from?
> > > >> If
> > > >> > > it’s
> > > >> > > > > > > related
> > > >> > > > > > > > > to the internals of Hibernate
module then it’s fine
> > but
> > > if
> > > >> > you
> > > >> > > > > > suggest
> > > >> > > > > > > > > removing identity resolvers
public interfaces then
> it
> > > >> might
> > > >> > be
> > > >> > > a
> > > >> > > > > > haste
> > > >> > > > > > > > > decision.
> > > >> > > > > > > > >
> > > >> > > > > > > > > —
> > > >> > > > > > > > > Denis
> > > >> > > > > > > > >
> > > >> > > > > > > > > > On Apr 5, 2017, at 7:42
AM, Alexey Goncharuk <
> > > >> > > > > > > > alexey.goncharuk@gmail.com>
> > > >> > > > > > > > > wrote:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > +1, I see no other reasons
to keep it.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00
Sergi Vladykin <
> > > >> > > > > > sergi.vladykin@gmail.com
> > > >> > > > > > > >:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >> +1
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> Lets drop them.
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> Sergi
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >> 2017-04-05 13:50
GMT+03:00 Dmitriy Govorukhin <
> > > >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > > >> > > > > > > > > >> :
> > > >> > > > > > > > > >>
> > > >> > > > > > > > > >>> Hi guys, i implemented
proxy for IgniteCache in
> > > >> hibernate
> > > >> > > > > > > > integration,
> > > >> > > > > > > > > >> this
> > > >> > > > > > > > > >>> proxy transformate
cacheKey to our key wrapper,
> > > leaves
> > > >> > only
> > > >> > > > > > > required
> > > >> > > > > > > > > >>> field. I think
we can remove identity resolve,
> it
> > > >> should
> > > >> > > not
> > > >> > > > > > broke
> > > >> > > > > > > > > >>> integration with
hibernate. Any objections?
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>> On Wed, Mar 29,
2017 at 11:07 PM, Valentin
> > > Kulichenko
> > > >> <
> > > >> > > > > > > > > >>> valentin.kulichenko@gmail.com>
wrote:
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>>> I'm not saying
there is no alternative
> solution.
> > > But
> > > >> > let's
> > > >> > > > > > > implement
> > > >> > > > > > > > > it
> > > >> > > > > > > > > >>> and
> > > >> > > > > > > > > >>>> prove that
it works first, and remove resolvers
> > > only
> > > >> > after
> > > >> > > > > that.
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>> -Val
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>> On Wed, Mar
29, 2017 at 12:18 PM, Sergi
> Vladykin
> > <
> > > >> > > > > > > > > >>> sergi.vladykin@gmail.com
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>> wrote:
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>>> Guys,
nothing is impossible if you know a bit
> > > about
> > > >> > > > > reflection
> > > >> > > > > > in
> > > >> > > > > > > > > >> Java
> > > >> > > > > > > > > >>> :)
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> We had
a look at the CacheKey class and it is
> > > easily
> > > >> > > > > > replaceable.
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> Sergi
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>> 2017-03-29
21:49 GMT+03:00 Dmitriy Setrakyan <
> > > >> > > > > > > > dsetrakyan@apache.org
> > > >> > > > > > > > > >>> :
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>>> On
Wed, Mar 29, 2017 at 11:43 AM, Valentin
> > > >> Kulichenko
> > > >> > <
> > > >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com>
wrote:
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>>>
"Hibernate key" is the CacheKey class I was
> > > >> referring
> > > >> > > to.
> > > >> > > > > > It's
> > > >> > > > > > > > > >>>> provided
> > > >> > > > > > > > > >>>>>> by
> > > >> > > > > > > > > >>>>>>>
Hibernate, not by user and not by us. So I'm
> > not
> > > >> sure
> > > >> > > > it's
> > > >> > > > > > > > > >> possible
> > > >> > > > > > > > > >>>> to
> > > >> > > > > > > > > >>>>>>>
replace it.
> > > >> > > > > > > > > >>>>>>>
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>> If
it is impossible to replace or get rid of
> > the
> > > >> > > Hibernate
> > > >> > > > > > key,
> > > >> > > > > > > is
> > > >> > > > > > > > > >>> this
> > > >> > > > > > > > > >>>>>> discussion
valid at all?
> > > >> > > > > > > > > >>>>>>
> > > >> > > > > > > > > >>>>>
> > > >> > > > > > > > > >>>>
> > > >> > > > > > > > > >>>
> > > >> > > > > > > > > >>
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message