ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: IEP-54: Schema-first approach for 3.0
Date Mon, 07 Sep 2020 12:58:59 GMT
Ivan,

Thank you, I got your concern now. As it is mostly regarding the
terminology, I am absolutely fine with changing the name to whatever fits
the approach best. Dynamic or evolving schema sounds great. I will make
corresponding changes to the IEP once we settle on the name.

пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo100@gmail.com>:

> Hi Val,
>
> Thank you for your answer!
>
> My understanding is a little bit different. Yes, schema evolution
> definitely should be possible. But I see a main difference in "how
> schema is updated". I treat a common SQL approach schema-first. Schema
> and data manipulation operations are clearly separated and it enables
> interesting capabilities, e.g. preventing untended schema changes by
> mistaken data operations, restricting user permissions to change
> schema.
>
> > Schema-first means that schema exists in advance and all the stored data
> is compliant with it - that's exactly what is proposed.
>
> A schema-last approach mentioned in [1] also assumes that schema
> exists, but it is inferred from data. Is not it more similar to the
> proposing approach?
>
> And I would like to say, that my main concern so far is mostly about
> terminology. And I suppose if it confuses me then others might be
> confused as well. My feeling is closer to "dynamic or liquid or may be
> evolving schema".
>
> [1]
> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>
> 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
> > Hi Ivan,
> >
> > I don't see an issue with that. Schema-first means that schema exists in
> > advance and all the stored data is compliant with it - that's exactly
> what
> > is proposed. There are no restrictions prohibiting changes to the schema.
> >
> > -Val
> >
> > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo100@gmail.com>
> wrote:
> >
> >> Alexey,
> >>
> >> I am a little bit confused with terminology. My understanding conforms
> >> to a survey [1] (see part X Semi Structured Data). Can we really treat
> >> a "dynamic schema" approach as a kind of "schema-first"?
> >>
> >> [1]
> >> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> >>
> >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dmagda@apache.org>:
> >> >>
> >> >> However, could you please elaborate on the relation between Ignite
> and
> >> >> ORM?
> >> >> Is there a use case for Hibernate running on top of Ignite (I haven't
> >> >> seen
> >> >> one so far)? If so, what is missing exactly on the Ignite side to
> >> support
> >> >> this? In my understanding, all you need is SQL API which we already
> >> have.
> >> >> Am I missing something?
> >> >
> >> >
> >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
> >> > internally, then they can easily translate an Entity object into an
> >> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
> >> > our
> >> > Spring Data integration is already based on the Ignite SQL APIs and
> >> > needs
> >> > to be improved once the schema-first approach is supported. That would
> >> > solve a ton of usability issues.
> >> >
> >> > I would revise the Hibernate integration as well during the Ignite 3.0
> >> dev
> >> > phase. Can't say if it's used a lot but Spring Data is getting
> traction
> >> for
> >> > sure.
> >> >
> >> > @Michael Pollind, I'll loop you in as long as you've started working
> on
> >> the
> >> > Ignite support for Micornaut Data
> >> > <https://micronaut-projects.github.io/micronaut-data/latest/guide/>
> and
> >> > came across some challenges. Just watch this discussion. That's what
> is
> >> > coming in Ignite 3.0.
> >> >
> >> >
> >> > -
> >> > Denis
> >> >
> >> >
> >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
> >> > valentin.kulichenko@gmail.com> wrote:
> >> >
> >> >> Hi Denis,
> >> >>
> >> >> Generally speaking, I believe that the schema-first approach natively
> >> >> addresses the issue if duplicate fields in key and value objects,
> >> because
> >> >> schema will be created for a cache, not for an object, as it happens
> >> now.
> >> >> Basically, the schema will define whether there is a primary key or
> >> >> not,
> >> >> and which fields are included in case there is one. Any API that we
> >> would
> >> >> have must be compliant with this, so it becomes fairly easy to work
> >> >> with
> >> >> data as with a set of records, rather than key-value pairs.
> >> >>
> >> >> However, could you please elaborate on the relation between Ignite
> and
> >> >> ORM?
> >> >> Is there a use case for Hibernate running on top of Ignite (I haven't
> >> >> seen
> >> >> one so far)? If so, what is missing exactly on the Ignite side to
> >> support
> >> >> this? In my understanding, all you need is SQL API which we already
> >> have.
> >> >> Am I missing something?
> >> >>
> >> >> -Val
> >> >>
> >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dmagda@apache.org>
> wrote:
> >> >>
> >> >> > Val,
> >> >> >
> >> >> > I would propose adding another point to the motivations list which
> >> >> > is
> >> >> > related to the ORM frameworks such as Spring Data, Hibernate,
> >> Micronaut
> >> >> and
> >> >> > many others.
> >> >> >
> >> >> > Presently, the storage engine requires to distinguish key objects
> >> >> > from
> >> >> the
> >> >> > value ones that complicate the usage of Ignite with those ORM
> >> >> > frameworks
> >> >> > (especially if a key object comprises several fields). More on
this
> >> can
> >> >> be
> >> >> > found here:
> >> >> >
> >> >> >
> >> >>
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html
> >> >> >
> >> >> > It will be nice if the new schema-first approach allows us to
work
> >> with
> >> >> > a
> >> >> > single entity object when it comes to the ORMs. With no need to
> >> >> > split
> >> >> > the
> >> >> > entity into a key and value. Just want to be sure that the Ignite
> >> >> > 3.0
> >> >> > has
> >> >> > all the essential public APIs that would support the single-entity
> >> >> > based
> >> >> > approach.
> >> >> >
> >> >> > What do you think?
> >> >> >
> >> >> > -
> >> >> > Denis
> >> >> >
> >> >> >
> >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko <
> >> >> > valentin.kulichenko@gmail.com> wrote:
> >> >> >
> >> >> > > Igniters,
> >> >> > >
> >> >> > > One of the big changes proposed for Ignite 3.0 is the so-called
> >> >> > > "schema-first approach". To add more clarity, I've started
> writing
> >> >> > > the
> >> >> > IEP
> >> >> > > for this change:
> >> >> > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> >> >> > >
> >> >> > > Please take a look and let me know if there are any immediate
> >> >> > > thoughts,
> >> >> > > suggestions, or objections.
> >> >> > >
> >> >> > > -Val
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >>
> >> --
> >>
> >> Best regards,
> >> Ivan Pavlukhin
> >>
> >
>
>
> --
>
> Best regards,
> Ivan Pavlukhin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message