ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Json support in Ignite
Date Sat, 16 Jan 2016 07:47:22 GMT
One other thing to keep in mind, that JSON data is often not-flattened ie
coming with hierarchical structures. In which case, querying it from Ignite
would be a non-trivial if at all possible.

Cos

On Thu, Jan 14, 2016 at 09:09AM, Dmitriy Setrakyan wrote:
> I would consider a case for generating hash-code by iterating through all
> the fields by default. We need to make sure that the iteration order is the
> same on both ends.
> 
> User can always override it, no?
> 
> D.
> 
> On Thu, Jan 14, 2016 at 8:10 AM, Andrey Gura <agura@gridgain.com> wrote:
> 
> > Hi,
> >
> >
> > > > Also we need following functionality for json objects
> > > >
> > > > 1. Possibility to map json cache key to affinity key. This can be
> > > achieved
> > > > with custom affinity mapper:
> > > >
> > > >     cache.setAffinityMapper(new AffinityKeyMapper() {
> > > >         @Override public Object affinityKey(Object key) {
> > > >             return
> > ((JsonObject)key).getJsonNumber("orgId").longValue();
> > > >         }
> > > >     });
> > > >
> > >
> > > We can also have a special field within JSON Object, called
> > > ignite_affinity_field=bla
> >
> >
> > +1
> >
> > It looks like we are over architecting here.
> > >
> > > Keys should not have many fields, and most likely will have one or two.
> > > Iterating through fields and generating hash-code or equals the standard
> > > way seems reasonable (as opposed to creating a whole architecture around
> > > how to get hash and equals fields).
> > >
> > > If it makes sense, we should cache the hash code value inside the key
> > > object after it has been generated once.
> > >
> >
> > I don't think that we should have some constraints here. If user wants to
> > use specific set of fields for hashCode/equals he should have such
> > possibility.
> >
> > There is one more problem in hashCode/equals: how to handle cases when
> > specified field isn't mandatory?
> >
> >
> > On Wed, Jul 15, 2015 at 11:32 AM, Dmitriy Setrakyan <dsetrakyan@apache.org
> > >
> > wrote:
> >
> > > On Wed, Jul 15, 2015 at 12:00 AM, Semyon Boikov <sboikov@gridgain.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > As part of integration with Node.Js (IGNITE-961) we age going to add in
> > > > Ignite support for JSON (possibility for IgniteCache to put, get, index
> > > and
> > > > query JSON data).
> > > >
> > > > We can implement standard 'javax.json' API (JSR 353), with this API
> > work
> > > > with cache will look like in this example:
> > > >
> > > >
> > > Are we going to implement JSON specification ourselves? Sounds a bit
> > fishy.
> > > Can we just take some available implementation (assuming that the license
> > > fits)?
> > >
> > >
> > >
> > > > Ignite should somehow provide access to 'javax.json' API, unfortunately
> > > > 'javax.json' is not part of JDK so we have several options how it can
> > be
> > > > added in Ignite:
> > > >
> > > > 1. Add dependency for 'javax.json-api' in core module, in this case we
> > > can
> > > > have method to get access for json API on 'org.apache.ignite.Ignite':
> > > >
> > > > * interface Ignite {*
> > > > *    ...*
> > > >
> > > > *    javax.json.JsonProvider json();*
> > > > * }*
> > > >
> > > > * JsonProvider json = ignite.json();*
> > > >
> > > > Not sure it is ok, since now we don't have 3rd party dependencies in
> > > ignite
> > > > 'core' module except 'javax.cache' API.
> > > >
> > > > 2. Added another optional module 'json', this module will have
> > dependency
> > > > for 'javax.json' API, and in this module we can have factory providing
> > > > access to 'javax.json' API:
> > > >
> > >
> > > I think an optional module makes sense.
> > >
> > >
> > >
> > > >
> > > > 3. Create plugin for json, in this case plugin will provide access to
> > > > 'javax.json' API:
> > > >
> > > > * IgniteJsonPlugin jsonPlugin = ignite.plugin(IgniteJsonPlugin.NAME);*
> > > >
> > > > * javax.json.JsonProvider provider = jsonPlugin.json();*
> > > >
> > > >
> > > > Not sure this aproach with plugin is ok, since now we don't have 'core'
> > > > ignite functionality implemented as plugins.
> > > >
> > >
> > > Agree, this probably won't work.
> > >
> > >
> > >
> > > > Also we need following functionality for json objects
> > > >
> > > > 1. Possibility to map json cache key to affinity key. This can be
> > > achieved
> > > > with custom affinity mapper:
> > > >
> > > >     cache.setAffinityMapper(new AffinityKeyMapper() {
> > > >         @Override public Object affinityKey(Object key) {
> > > >             return
> > ((JsonObject)key).getJsonNumber("orgId").longValue();
> > > >         }
> > > >     });
> > > >
> > >
> > > We can also have a special field within JSON Object, called
> > > ignite_affinity_field=bla
> > >
> > >
> > > >
> > > > 2. Efficient implementation for equals/hashCode for json cache keys.
> > > >
> > > > One option is just have equals/hashCode like in HashMap.
> > > >
> > > > Another option can be add special JsonConfiguration, it should be set
> > in
> > > > CacheConfiguration and can contain following settings:
> > > > - list of fields to use in 'equals' and 'hashCode' calculation
> > > > - alternatively for equals/hashCode we can provide possibility to
> > > implement
> > > > custom comparator/hash code calculator
> > > > - JsonConfiguration can have setting for affinity key
> > > >
> > > > So JsonConfiguration can look like this:
> > > >
> > > > *class JsonConfiguration {*
> > > > *    /***
> > > > *     * Field names for hash code calculation.*
> > > > *     */*
> > > > *    public Collection<String> getKeyHashCodeFields();*
> > > >
> > > > *    /***
> > > > *     * Field names to use .*
> > > > *     */*
> > > > *    public Collection<String> getKeyEqualsFields();*
> > > >
> > > > *    /***
> > > > *     * Field name to get key which will be used for node affinity
> > > > calculation.*
> > > > *     */*
> > > > *    public String getAffinityKeyField();*
> > > > *}*
> > > >
> > >
> > > It looks like we are over architecting here.
> > >
> > > Keys should not have many fields, and most likely will have one or two.
> > > Iterating through fields and generating hash-code or equals the standard
> > > way seems reasonable (as opposed to creating a whole architecture around
> > > how to get hash and equals fields).
> > >
> > > If it makes sense, we should cache the hash code value inside the key
> > > object after it has been generated once.
> > >
> > >
> > > >
> > > >
> > > > I need your feedback on these questions:
> > > > - is it ok to implement 'javax.json' API to support json for Ignite
> > > caches?
> > > >
> > >
> > > Can we get some existing implementation instead?
> > >
> > >
> > > > - if we implement 'javax.json' what is best way to add it in Ignite
> > (see
> > > my
> > > > suggestion above: just add dependency for 'javax.json' API in 'core'
> > > > module, add special module or plugin)
> > > >
> > >
> > > Optional module?
> > >
> > >
> > > > - do we need to adde JsonConfiguration I described above?
> > > >
> > >
> > > Most likely not
> > >
> > >
> > > >
> > > > Thanks!
> > > >
> > >
> >
> >
> >
> > --
> > Andrey Gura
> > GridGain Systems, Inc.
> > www.gridgain.com
> >

Mime
View raw message