ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: IEP-54: Schema-first approach for 3.0
Date Mon, 07 Sep 2020 19:26:13 GMT
Ivan,

I see your point. I agree that with the automatic updates we step into the
schema-last territory.

Actually, if we support automatic evolution, we can as well support
creating a cache without schema and inferring it from the first insert. In
other words, we can have both "schema-first" and "schema-last" modes.

Alexey, what do you think?

-Val

On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk <alexey.goncharuk@gmail.com>
wrote:

> Ivan,
>
> Thank you, I got your concern now. As it is mostly regarding the
> terminology, I am absolutely fine with changing the name to whatever fits
> the approach best. Dynamic or evolving schema sounds great. I will make
> corresponding changes to the IEP once we settle on the name.
>
> пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo100@gmail.com>:
>
> > Hi Val,
> >
> > Thank you for your answer!
> >
> > My understanding is a little bit different. Yes, schema evolution
> > definitely should be possible. But I see a main difference in "how
> > schema is updated". I treat a common SQL approach schema-first. Schema
> > and data manipulation operations are clearly separated and it enables
> > interesting capabilities, e.g. preventing untended schema changes by
> > mistaken data operations, restricting user permissions to change
> > schema.
> >
> > > Schema-first means that schema exists in advance and all the stored
> data
> > is compliant with it - that's exactly what is proposed.
> >
> > A schema-last approach mentioned in [1] also assumes that schema
> > exists, but it is inferred from data. Is not it more similar to the
> > proposing approach?
> >
> > And I would like to say, that my main concern so far is mostly about
> > terminology. And I suppose if it confuses me then others might be
> > confused as well. My feeling is closer to "dynamic or liquid or may be
> > evolving schema".
> >
> > [1]
> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> >
> > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
> > valentin.kulichenko@gmail.com>:
> > > Hi Ivan,
> > >
> > > I don't see an issue with that. Schema-first means that schema exists
> in
> > > advance and all the stored data is compliant with it - that's exactly
> > what
> > > is proposed. There are no restrictions prohibiting changes to the
> schema.
> > >
> > > -Val
> > >
> > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo100@gmail.com>
> > wrote:
> > >
> > >> Alexey,
> > >>
> > >> I am a little bit confused with terminology. My understanding conforms
> > >> to a survey [1] (see part X Semi Structured Data). Can we really treat
> > >> a "dynamic schema" approach as a kind of "schema-first"?
> > >>
> > >> [1]
> > >>
> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> > >>
> > >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dmagda@apache.org>:
> > >> >>
> > >> >> However, could you please elaborate on the relation between Ignite
> > and
> > >> >> ORM?
> > >> >> Is there a use case for Hibernate running on top of Ignite (I
> haven't
> > >> >> seen
> > >> >> one so far)? If so, what is missing exactly on the Ignite side
to
> > >> support
> > >> >> this? In my understanding, all you need is SQL API which we already
> > >> have.
> > >> >> Am I missing something?
> > >> >
> > >> >
> > >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
> > >> > internally, then they can easily translate an Entity object into an
> > >> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
> > >> > our
> > >> > Spring Data integration is already based on the Ignite SQL APIs and
> > >> > needs
> > >> > to be improved once the schema-first approach is supported. That
> would
> > >> > solve a ton of usability issues.
> > >> >
> > >> > I would revise the Hibernate integration as well during the Ignite
> 3.0
> > >> dev
> > >> > phase. Can't say if it's used a lot but Spring Data is getting
> > traction
> > >> for
> > >> > sure.
> > >> >
> > >> > @Michael Pollind, I'll loop you in as long as you've started working
> > on
> > >> the
> > >> > Ignite support for Micornaut Data
> > >> > <https://micronaut-projects.github.io/micronaut-data/latest/guide/>
> > and
> > >> > came across some challenges. Just watch this discussion. That's what
> > is
> > >> > coming in Ignite 3.0.
> > >> >
> > >> >
> > >> > -
> > >> > Denis
> > >> >
> > >> >
> > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
> > >> > valentin.kulichenko@gmail.com> wrote:
> > >> >
> > >> >> Hi Denis,
> > >> >>
> > >> >> Generally speaking, I believe that the schema-first approach
> natively
> > >> >> addresses the issue if duplicate fields in key and value objects,
> > >> because
> > >> >> schema will be created for a cache, not for an object, as it
> happens
> > >> now.
> > >> >> Basically, the schema will define whether there is a primary key
or
> > >> >> not,
> > >> >> and which fields are included in case there is one. Any API that
we
> > >> would
> > >> >> have must be compliant with this, so it becomes fairly easy to
work
> > >> >> with
> > >> >> data as with a set of records, rather than key-value pairs.
> > >> >>
> > >> >> However, could you please elaborate on the relation between Ignite
> > and
> > >> >> ORM?
> > >> >> Is there a use case for Hibernate running on top of Ignite (I
> haven't
> > >> >> seen
> > >> >> one so far)? If so, what is missing exactly on the Ignite side
to
> > >> support
> > >> >> this? In my understanding, all you need is SQL API which we already
> > >> have.
> > >> >> Am I missing something?
> > >> >>
> > >> >> -Val
> > >> >>
> > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dmagda@apache.org>
> > wrote:
> > >> >>
> > >> >> > Val,
> > >> >> >
> > >> >> > I would propose adding another point to the motivations list
> which
> > >> >> > is
> > >> >> > related to the ORM frameworks such as Spring Data, Hibernate,
> > >> Micronaut
> > >> >> and
> > >> >> > many others.
> > >> >> >
> > >> >> > Presently, the storage engine requires to distinguish key
objects
> > >> >> > from
> > >> >> the
> > >> >> > value ones that complicate the usage of Ignite with those
ORM
> > >> >> > frameworks
> > >> >> > (especially if a key object comprises several fields). More
on
> this
> > >> can
> > >> >> be
> > >> >> > found here:
> > >> >> >
> > >> >> >
> > >> >>
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html
> > >> >> >
> > >> >> > It will be nice if the new schema-first approach allows us
to
> work
> > >> with
> > >> >> > a
> > >> >> > single entity object when it comes to the ORMs. With no need
to
> > >> >> > split
> > >> >> > the
> > >> >> > entity into a key and value. Just want to be sure that the
Ignite
> > >> >> > 3.0
> > >> >> > has
> > >> >> > all the essential public APIs that would support the
> single-entity
> > >> >> > based
> > >> >> > approach.
> > >> >> >
> > >> >> > What do you think?
> > >> >> >
> > >> >> > -
> > >> >> > Denis
> > >> >> >
> > >> >> >
> > >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko <
> > >> >> > valentin.kulichenko@gmail.com> wrote:
> > >> >> >
> > >> >> > > Igniters,
> > >> >> > >
> > >> >> > > One of the big changes proposed for Ignite 3.0 is the
so-called
> > >> >> > > "schema-first approach". To add more clarity, I've started
> > writing
> > >> >> > > the
> > >> >> > IEP
> > >> >> > > for this change:
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > >> >> > >
> > >> >> > > Please take a look and let me know if there are any
immediate
> > >> >> > > thoughts,
> > >> >> > > suggestions, or objections.
> > >> >> > >
> > >> >> > > -Val
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >>
> > >>
> > >> --
> > >>
> > >> Best regards,
> > >> Ivan Pavlukhin
> > >>
> > >
> >
> >
> > --
> >
> > Best regards,
> > Ivan Pavlukhin
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message