ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: All BinaryObjects created by BinaryObjectBuilder stored at the same partition by default
Date Sat, 06 Aug 2016 08:25:31 GMT
Keep in mind that we need to support affinity keys in BinaryObjects. It
means that it will have to consist from at least two fields: one for exact
equality check and another one for hashCode calculation.

It looks to me that configuration of cache key is a part of cache
configuration. Thus cache key builder must be bound to cache.

Sergi

2016-08-06 6:18 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:

> How about we add a property - auto-generate hashCode() in binary
> configuration. If set, then we auto-generate the hashCode() every time a
> binary object is built.
>
> Thoughts?
>
> On Fri, Aug 5, 2016 at 5:29 AM, Alexander Paschenko <
> alexander.a.paschenko@gmail.com> wrote:
>
> > Denis,
> >
> > Thank you very much for your proposed solution, I will reflect it in
> > issue's comments and implement this check in code. Most likely this
> > has to be an issue by itself.
> >
> > However, it all does not answer the main question of this thread - how
> > do we automatically supply hash codes for newly built binary objects?
> > This is very important for convenient use of SQL inserts. Please, all,
> > share your thoughts.
> >
> > - Alex
> >
> > 2016-08-03 3:23 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> > > On Tue, Aug 2, 2016 at 7:36 AM, Denis Magda <dmagda@gridgain.com>
> wrote:
> > >
> > >> Vova,
> > >>
> > >> By default hasCode field will be initialized this way in
> > >> BinaryObjectBuilderImpl
> > >>
> > >> /** */
> > >> private static Integer DFLT_HASH_CODE_MAGIC = 0;
> > >>
> > >> /** */
> > >> private Integer hashCode = DFLT_HASH_CODE_MAGIC;
> > >>
> > >> Also we will introduce the following flag in BinaryUtils
> > >>
> > >> /** Flag indicating whether as hashCode was explicitly set or not. **/
> > >> public static final short EMPTY_HASH_CODE_FLAG = 0x0032;
> > >>
> > >> At the BinaryObjectBuilder.build() time we will perform the check
> below
> > >> and write hashCodeFlag.
> > >>
> > >>
> > >> short hashCodeFlag = hashCode == DFLT_HASH_CODE_MAGIC ?
> > >> BinaryUtils.EMPTY_HASH_CODE_FLAG : 0;
> > >>
> > >> // Write hashCode flag as well.
> > >> writer.postWrite(true, registeredType, hashCode, hashCodeFlag);
> > >>
> > >> Later when a BinaryObject is used as a key we can check the value of
> > >> BinaryUtils.EMPTY_HASH_CODE_FLAG
> > >> and throw an exception if it’s not empty.
> > >>
> > >> Makes sense?
> > >>
> > >
> > > It does to me. If there are no objections, then we should list this in
> > the
> > > ticket and implement the proposed suggestion of throwing exception if a
> > > binary object without hashcode is used as a key.
> > >
> > >
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On Aug 2, 2016, at 7:09 AM, Vladimir Ozerov <vozerov@gridgain.com>
> > >> wrote:
> > >> >
> > >> > Denis,
> > >> >
> > >> > I hardly can imagine how it could work in our binary protocol. Can
> you
> > >> > please elaborate?
> > >> >
> > >> > On Tue, Aug 2, 2016 at 5:06 PM, Denis Magda <dmagda@gridgain.com>
> > wrote:
> > >> >
> > >> >> There is a technique we already use to see if a field is
> initialized
> > by
> > >> >> application code. By default a field has to be a reference to
a
> > >> predefined
> > >> >> object and the reference comparison (not equals) is used to check
> if
> > the
> > >> >> field is initialized by the user.
> > >> >>
> > >> >> Refer to IgniteConfiguration.failureDetectionTimeout and
> > >> >> IgniteSpiAdapter.initializeFailureDetectionTimeout for more
> details.
> > >> >>
> > >> >> —
> > >> >> Denis
> > >> >>
> > >> >>> On Aug 2, 2016, at 12:14 AM, Alexander Paschenko <
> > >> >> alexander.a.paschenko@gmail.com> wrote:
> > >> >>>
> > >> >>> Dmitriy,
> > >> >>>
> > >> >>> Good point, however, currently there's no way to distinguish
hash
> > code
> > >> >>> of zero which is a valid case from missing hash code. We probably
> > >> >>> should enhance binary builder for it to handle this case.
> > >> >>>
> > >> >>> - Alex
> > >> >>>
> > >> >>> 2016-08-02 9:47 GMT+03:00 Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >:
> > >> >>>> On Mon, Aug 1, 2016 at 11:38 PM, Vladimir Ozerov <
> > >> vozerov@gridgain.com>
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> Andrey,
> > >> >>>>>
> > >> >>>>> The question is when to print this warning. I doubt
we can
> print a
> > >> >> warning
> > >> >>>>> when calling *BinaryObjectBuilder.build() *method,
because an
> > object
> > >> >>>>> without a hash code is normal situation.
> > >> >>>>>
> > >> >>>>>
> > >> >>>> I would not only print warning, but throw exception, if
an object
> > >> >> without a
> > >> >>>> hashCode ends up on a put or read operation in cache.
> > >> >>>>
> > >> >>>>
> > >> >>>>> On Tue, Aug 2, 2016 at 9:00 AM, Andrey Gura <agura@gridgain.com
> >
> > >> >> wrote:
> > >> >>>>>
> > >> >>>>>> I think we also should print some warning in case
when
> hashCode()
> > >> >> wasn't
> > >> >>>>>> called on BinaryObject explicitly.
> > >> >>>>>>
> > >> >>>>>> On Tue, Aug 2, 2016 at 2:20 AM, Dmitriy Setrakyan
<
> > >> >> dsetrakyan@apache.org
> > >> >>>>>>
> > >> >>>>>> wrote:
> > >> >>>>>>
> > >> >>>>>>> On Mon, Aug 1, 2016 at 10:01 AM, Alexey Goncharuk
<
> > >> >>>>>>> alexey.goncharuk@gmail.com> wrote:
> > >> >>>>>>>
> > >> >>>>>>>> Dmitriy,
> > >> >>>>>>>>
> > >> >>>>>>>> The question is how do you calculate the
value of the
> > hashCode? Do
> > >> >>>>> you
> > >> >>>>>>> want
> > >> >>>>>>>> it to be specified explicitly in INSERT
statement?
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> I think optionally we should allow to specify
hashCode as part
> > of
> > >> the
> > >> >>>>>>> INSERT statement. However, if it is not specified,
we should
> > >> >> calculate
> > >> >>>>> it
> > >> >>>>>>> automatically based in the key fields defined
in the
> > schema/type.
> > >> >>>>> Agree?
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>> 2016-08-01 19:47 GMT+03:00 Dmitriy Setrakyan
<
> > >> dsetrakyan@apache.org
> > >> >>>>>> :
> > >> >>>>>>>>
> > >> >>>>>>>>> Alex,
> > >> >>>>>>>>>
> > >> >>>>>>>>> In your case, why not just explicitly
set hashcode every
> time
> > you
> > >> >>>>>>> create
> > >> >>>>>>>> an
> > >> >>>>>>>>> object? There is BinaryObjectBuilder.hashCode(...)
method.
> > >> >>>>>>>>>
> > >> >>>>>>>>> D.
> > >> >>>>>>>>>
> > >> >>>>>>>>> On Mon, Aug 1, 2016 at 7:42 AM, al.psc
<
> > >> >>>>>>> alexander.a.paschenko@gmail.com>
> > >> >>>>>>>>> wrote:
> > >> >>>>>>>>>
> > >> >>>>>>>>>> Guys,
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> It seems like this problem has
become an important one once
> > >> >>>>> again.
> > >> >>>>>>>>>> In the course of working on
> > >> >>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2294
(DML
> > support)
> > >> >>>>>>>> there's
> > >> >>>>>>>>>> need
> > >> >>>>>>>>>> to support binary marshaller.
And, although we can build
> just
> > >> >>>>>>>>> BinaryObject
> > >> >>>>>>>>>> and put it to cache, without adequate
hash code it won't be
> > >> >>>>> stored
> > >> >>>>>>>>>> properly.
> > >> >>>>>>>>>> Currently SQL MERGE works simply
by deserializing newly
> built
> > >> >>>>>> object,
> > >> >>>>>>>> but
> > >> >>>>>>>>>> it's obviously wrong and is just
a workaround rather a
> > solution.
> > >> >>>>>>>>>> Has anyone come with possible
design proposals for this
> > >> problem's
> > >> >>>>>>>>> solution?
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Thanks.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> - Alex
> > >> >>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> --
> > >> >>>>>>>>>> View this message in context:
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>
> > >> >>
> > >> http://apache-ignite-developers.2346864.n4.nabble.
> > com/All-BinaryObjects-created-by-BinaryObjectBuilder-stored-
> > at-the-same-partition-by-default-tp8042p10304.html
> > >> >>>>>>>>>> Sent from the Apache Ignite Developers
mailing list archive
> > at
> > >> >>>>>>>>> Nabble.com.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> --
> > >> >>>>>> Andrey Gura
> > >> >>>>>> GridGain Systems, Inc.
> > >> >>>>>> www.gridgain.com
> > >> >>>>>>
> > >> >>>>>
> > >> >>
> > >> >>
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message