ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Stable binary key representation
Date Mon, 10 Apr 2017 18:14:30 GMT
There is no more such resolver. It was removed.

On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dmagda@apache.org> wrote:

> Vovan,
>
> Before I fix the documentation, what’t the replacement for
> BinaryFieldIdentiyResolver we used to define field for hash code
> calculation and equality comparison when DML statements are used?
> https://apacheignite.readme.io/docs/binary-marshaller#
> section-binary-field-identity-resolver <https://apacheignite.readme.
> io/docs/binary-marshaller#section-binary-field-identity-resolver>
>
> —
> Denis
>
> > On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
> >
> > Resolvers were essential for DML because we had broken comparison
> semantics
> > of binary objects. This is not the case now.
> >
> > Resolver as a whole is normal practice. E.g. it is implemented in .NET on
> > core language level and widely used in many cases. Hazelcast has it as
> well
> > AFAIK. So it is wrong to think that the whole idea is useless. Think of
> it
> > as a comparator's brother.
> >
> > The only reason why we need to remove it is missing hash index in new
> > architecture. It makes sense, as it is better to have AI 2.0 without
> them,
> > than no AI 2.0 :-)
> >
> > 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> > sergi.vladykin@gmail.com> написал:
> >
> >> I guess Resolvers were added to DML just because they already existed
> since
> >> 1.9 and we were forced to support them in all the parts of our product.
> >>
> >> We have to stop this practice to add features without clear real life
> use
> >> cases for them.
> >>
> >> Sergi
> >>
> >> 2017-04-09 17:00 GMT+03:00 Denis Magda <dmagda@gridgain.com>:
> >>
> >>> Sergi, Vovan,
> >>>
> >>> Sorry for being annoying but I still didn't get an answer on whether
> the
> >>> resolvers are the must for DML. The main reason why we made them up
> some
> >>> time ago is to support specific DML use cases. However I can't recall
> the
> >>> use cases.
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> sergi.vladykin@gmail.com
> >>>
> >>> wrote:
> >>>
> >>>> Ok, we need to do 2 things here:
> >>>>
> >>>> 1. Drop the resolvers from the source code.
> >>>> 2. Write a good page in docs on "What makes a correct cache key".
> >>>>
> >>>> Who can do that?
> >>>>
> >>>> Sergi
> >>>>
> >>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:
> >>>>
> >>>>> It is possible to try adding support of comparison to Resolvers,
but
> >>> the
> >>>>> whole approach looks wrong and for now it is better to get rid of
it
> >>>> while
> >>>>> we have a chance to break compatibility.
> >>>>>
> >>>>> Sergi
> >>>>>
> >>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> >>>>> valentin.kulichenko@gmail.com>:
> >>>>>
> >>>>>> The discussion should've been started with that :) If supporting
> >>>> resolvers
> >>>>>> in new architecture is not possible or means too big effort,
then
> >> it's
> >>>>>> definitely not worth it.
> >>>>>>
> >>>>>> -Val
> >>>>>>
> >>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> >> vozerov@gridgain.com
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Dima,
> >>>>>>>
> >>>>>>> Yes, they may explode some internals of our indexes.
> >>>>>>>
> >>>>>>> 06 апр. 2017 г. 23:32 пользователь "Dmitriy
Setrakyan" <
> >>>>>>> dsetrakyan@apache.org> написал:
> >>>>>>>
> >>>>>>>> Guys,
> >>>>>>>>
> >>>>>>>> Isn't the main issue here that we cannot use the Identity
> >>> Resolvers
> >>>> in
> >>>>>>>> BTrees in the 2.0 version? If yes, then we have to remove
them
> >> no
> >>>>>> matter
> >>>>>>>> what.
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> >>>>>> sergi.vladykin@gmail.com
> >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Binary key representation is stable when we always
have equal
> >>>>>>> serialized
> >>>>>>>>> bytes when the original keys are equal.
> >>>>>>>>>
> >>>>>>>>> Resolver allows you to have some extra info in the
Key and
> >> equal
> >>>>>> Keys
> >>>>>>>> will
> >>>>>>>>> be serialized into different bytes, which is wrong.
> >>>>>>>>>
> >>>>>>>>> Look at the example what you can do with resolvers:
> >>>>>>>>>
> >>>>>>>>> We may have some data entry with fields a, b, c.
Let's say the
> >>>>>> unique
> >>>>>>>> part
> >>>>>>>>> here is `a` and it the only fields used in Key equals()
and
> >>>>>> hashCode().
> >>>>>>>>> Still we may have the following layouts:
> >>>>>>>>>
> >>>>>>>>> 1. Ka -> Vbc
> >>>>>>>>> 2. Kab -> Vc
> >>>>>>>>> 3. Kabc -> Boolean.TRUE
> >>>>>>>>>
> >>>>>>>>> The only 1 is a correct layout, others are plain
wrong
> >> variants
> >>>> (but
> >>>>>>> they
> >>>>>>>>> are still possible with Resolvers) because everything
that
> >> does
> >>>> not
> >>>>>>> make
> >>>>>>>>> Key unique must be in Value.
> >>>>>>>>>
> >>>>>>>>> We want to clearly state that if you have something
in Key,
> >> that
> >>>> is
> >>>>>> not
> >>>>>>>>> part of equals(), then the Key is invalid and that
stuff must
> >> be
> >>>> in
> >>>>>>>> Value.
> >>>>>>>>> This allows us to rely on binary representation
of a Key to be
> >>>>>> stable
> >>>>>>> and
> >>>>>>>>> have some more optimizations and code simplifications
with
> >>> respect
> >>>>>> to
> >>>>>>>> these
> >>>>>>>>> assumptions.
> >>>>>>>>>
> >>>>>>>>> Sergi
> >>>>>>>>>
> >>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> >>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Even with my vast expirience I would never claim
that I've
> >>> seen
> >>>>>>>>>> "everything" :)
> >>>>>>>>>>
> >>>>>>>>>> What do you mean by stable binary key representation
and how
> >>>>>>> resolvers
> >>>>>>>>> make
> >>>>>>>>>> it unstable?
> >>>>>>>>>>
> >>>>>>>>>> -Val
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin
<
> >>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Val,
> >>>>>>>>>>>
> >>>>>>>>>>> I know that you have really vast experience
in Ignite
> >>>>>> deployments
> >>>>>>> and
> >>>>>>>>>>> probably saw everything that can happen.
Did you ever see
> >>>>>> identity
> >>>>>>>>>>> resolvers use in real life? I guess no.
> >>>>>>>>>>>
> >>>>>>>>>>> Hibernate example is bad here, because if
their key is
> >>>> unstable
> >>>>>>>> across
> >>>>>>>>>>> multiple JVMs, it means that it was not
designed for
> >>>> distributed
> >>>>>>>>> caches a
> >>>>>>>>>>> priori.
> >>>>>>>>>>>
> >>>>>>>>>>> Also knowing in advance about stable binary
key
> >>> representation
> >>>>>>> allows
> >>>>>>>>> us
> >>>>>>>>>> to
> >>>>>>>>>>> apply additional optimizations, like comparing
keys
> >> without
> >>>>>>> detaching
> >>>>>>>>>> them
> >>>>>>>>>>> from offheap memory.
> >>>>>>>>>>>
> >>>>>>>>>>> We always will be able to add this stuff
back if we see
> >>> users
> >>>>>>> really
> >>>>>>>>> need
> >>>>>>>>>>> it. Let's remove it for 2.0.
> >>>>>>>>>>>
> >>>>>>>>>>> Sergi
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko
<
> >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Alex,
> >>>>>>>>>>>>
> >>>>>>>>>>>> To be honest, I don't understand the
reasoning behind
> >> the
> >>>>>>> removal.
> >>>>>>>> I
> >>>>>>>>>>> think
> >>>>>>>>>>>> resolvers provide good flexibility for
different corner
> >>>> cases
> >>>>>> and
> >>>>>>>>> it's
> >>>>>>>>>> a
> >>>>>>>>>>>> good thing to have them. Note that they
can be applied
> >> not
> >>>>>> only
> >>>>>>> to
> >>>>>>>>>> cache
> >>>>>>>>>>>> keys, but to any binary objects.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hibernate issue is actually a good example
of such use
> >>> case.
> >>>>>> The
> >>>>>>>> fact
> >>>>>>>>>>> that
> >>>>>>>>>>>> we found an alternative solution doesn't
actually mean
> >>>>>> anything,
> >>>>>>>>>> because
> >>>>>>>>>>>> what if this happened not in our module,
but in user's
> >>>>>>> application?
> >>>>>>>>>>>> Unfortunately, we can't predict everything.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Error proneness is not a very strong
argument either,
> >>>> because
> >>>>>> in
> >>>>>>> my
> >>>>>>>>>> view
> >>>>>>>>>>>> these resolvers are as much error prone
as
> >> BinaryIdMapper,
> >>>> for
> >>>>>>>>> example.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Val
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44 PM, Alexey
Goncharuk <
> >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Can you suggest a use-case where
identity resolver is
> >>>> needed
> >>>>>>>> (given
> >>>>>>>>>>> that
> >>>>>>>>>>>> we
> >>>>>>>>>>>>> agree that a key must contain only
valuable fields)?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00 Denis
Magda <
> >>> dmagda@apache.org
> >>>>> :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Where do you want to remove
the identity resolvers
> >>> from?
> >>>>>> If
> >>>>>>>> it’s
> >>>>>>>>>>>> related
> >>>>>>>>>>>>>> to the internals of Hibernate
module then it’s fine
> >>> but
> >>>> if
> >>>>>>> you
> >>>>>>>>>>> suggest
> >>>>>>>>>>>>>> removing identity resolvers
public interfaces then
> >> it
> >>>>>> might
> >>>>>>> be
> >>>>>>>> a
> >>>>>>>>>>> haste
> >>>>>>>>>>>>>> decision.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> —
> >>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Apr 5, 2017, at 7:42
AM, Alexey Goncharuk <
> >>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> +1, I see no other reasons
to keep it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2017-04-05 13:59 GMT+03:00
Sergi Vladykin <
> >>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> +1
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Lets drop them.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-04-05 13:50 GMT+03:00
Dmitriy Govorukhin <
> >>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> >>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi guys, i implemented
proxy for IgniteCache in
> >>>>>> hibernate
> >>>>>>>>>>>>> integration,
> >>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>> proxy transformate
cacheKey to our key wrapper,
> >>>> leaves
> >>>>>>> only
> >>>>>>>>>>>> required
> >>>>>>>>>>>>>>>>> field. I think we
can remove identity resolve,
> >> it
> >>>>>> should
> >>>>>>>> not
> >>>>>>>>>>> broke
> >>>>>>>>>>>>>>>>> integration with
hibernate. Any objections?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Mar 29,
2017 at 11:07 PM, Valentin
> >>>> Kulichenko
> >>>>>> <
> >>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>
wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'm not saying
there is no alternative
> >> solution.
> >>>> But
> >>>>>>> let's
> >>>>>>>>>>>> implement
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> prove that it
works first, and remove resolvers
> >>>> only
> >>>>>>> after
> >>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar
29, 2017 at 12:18 PM, Sergi
> >> Vladykin
> >>> <
> >>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Guys, nothing
is impossible if you know a bit
> >>>> about
> >>>>>>>>>> reflection
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We had a
look at the CacheKey class and it is
> >>>> easily
> >>>>>>>>>>> replaceable.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> 2017-03-29
21:49 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed,
Mar 29, 2017 at 11:43 AM, Valentin
> >>>>>> Kulichenko
> >>>>>>> <
> >>>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>
wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
"Hibernate key" is the CacheKey class I was
> >>>>>> referring
> >>>>>>>> to.
> >>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>> provided
> >>>>>>>>>>>>>>>>>>>> by
> >>>>>>>>>>>>>>>>>>>>>
Hibernate, not by user and not by us. So I'm
> >>> not
> >>>>>> sure
> >>>>>>>>> it's
> >>>>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>
replace it.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If it
is impossible to replace or get rid of
> >>> the
> >>>>>>>> Hibernate
> >>>>>>>>>>> key,
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>> discussion
valid at all?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message