ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Pavlukhin <vololo...@gmail.com>
Subject Re: IEP-54: Schema-first approach for 3.0
Date Mon, 07 Sep 2020 08:33:34 GMT
Hi Val,

Thank you for your answer!

My understanding is a little bit different. Yes, schema evolution
definitely should be possible. But I see a main difference in "how
schema is updated". I treat a common SQL approach schema-first. Schema
and data manipulation operations are clearly separated and it enables
interesting capabilities, e.g. preventing untended schema changes by
mistaken data operations, restricting user permissions to change
schema.

> Schema-first means that schema exists in advance and all the stored data is compliant
with it - that's exactly what is proposed.

A schema-last approach mentioned in [1] also assumes that schema
exists, but it is inferred from data. Is not it more similar to the
proposing approach?

And I would like to say, that my main concern so far is mostly about
terminology. And I suppose if it confuses me then others might be
confused as well. My feeling is closer to "dynamic or liquid or may be
evolving schema".

[1] https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf

2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <valentin.kulichenko@gmail.com>:
> Hi Ivan,
>
> I don't see an issue with that. Schema-first means that schema exists in
> advance and all the stored data is compliant with it - that's exactly what
> is proposed. There are no restrictions prohibiting changes to the schema.
>
> -Val
>
> On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo100@gmail.com> wrote:
>
>> Alexey,
>>
>> I am a little bit confused with terminology. My understanding conforms
>> to a survey [1] (see part X Semi Structured Data). Can we really treat
>> a "dynamic schema" approach as a kind of "schema-first"?
>>
>> [1]
>> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
>>
>> 2020-09-02 1:53 GMT+03:00, Denis Magda <dmagda@apache.org>:
>> >>
>> >> However, could you please elaborate on the relation between Ignite and
>> >> ORM?
>> >> Is there a use case for Hibernate running on top of Ignite (I haven't
>> >> seen
>> >> one so far)? If so, what is missing exactly on the Ignite side to
>> support
>> >> this? In my understanding, all you need is SQL API which we already
>> have.
>> >> Am I missing something?
>> >
>> >
>> > Good point, yes, if all the ORM integrations use Ignite SQL APIs
>> > internally, then they can easily translate an Entity object into an
>> > INSERT/UPDATE statement that lists all the object's fields. Luckily,
>> > our
>> > Spring Data integration is already based on the Ignite SQL APIs and
>> > needs
>> > to be improved once the schema-first approach is supported. That would
>> > solve a ton of usability issues.
>> >
>> > I would revise the Hibernate integration as well during the Ignite 3.0
>> dev
>> > phase. Can't say if it's used a lot but Spring Data is getting traction
>> for
>> > sure.
>> >
>> > @Michael Pollind, I'll loop you in as long as you've started working on
>> the
>> > Ignite support for Micornaut Data
>> > <https://micronaut-projects.github.io/micronaut-data/latest/guide/> and
>> > came across some challenges. Just watch this discussion. That's what is
>> > coming in Ignite 3.0.
>> >
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
>> > valentin.kulichenko@gmail.com> wrote:
>> >
>> >> Hi Denis,
>> >>
>> >> Generally speaking, I believe that the schema-first approach natively
>> >> addresses the issue if duplicate fields in key and value objects,
>> because
>> >> schema will be created for a cache, not for an object, as it happens
>> now.
>> >> Basically, the schema will define whether there is a primary key or
>> >> not,
>> >> and which fields are included in case there is one. Any API that we
>> would
>> >> have must be compliant with this, so it becomes fairly easy to work
>> >> with
>> >> data as with a set of records, rather than key-value pairs.
>> >>
>> >> However, could you please elaborate on the relation between Ignite and
>> >> ORM?
>> >> Is there a use case for Hibernate running on top of Ignite (I haven't
>> >> seen
>> >> one so far)? If so, what is missing exactly on the Ignite side to
>> support
>> >> this? In my understanding, all you need is SQL API which we already
>> have.
>> >> Am I missing something?
>> >>
>> >> -Val
>> >>
>> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dmagda@apache.org> wrote:
>> >>
>> >> > Val,
>> >> >
>> >> > I would propose adding another point to the motivations list which
>> >> > is
>> >> > related to the ORM frameworks such as Spring Data, Hibernate,
>> Micronaut
>> >> and
>> >> > many others.
>> >> >
>> >> > Presently, the storage engine requires to distinguish key objects
>> >> > from
>> >> the
>> >> > value ones that complicate the usage of Ignite with those ORM
>> >> > frameworks
>> >> > (especially if a key object comprises several fields). More on this
>> can
>> >> be
>> >> > found here:
>> >> >
>> >> >
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html
>> >> >
>> >> > It will be nice if the new schema-first approach allows us to work
>> with
>> >> > a
>> >> > single entity object when it comes to the ORMs. With no need to
>> >> > split
>> >> > the
>> >> > entity into a key and value. Just want to be sure that the Ignite
>> >> > 3.0
>> >> > has
>> >> > all the essential public APIs that would support the single-entity
>> >> > based
>> >> > approach.
>> >> >
>> >> > What do you think?
>> >> >
>> >> > -
>> >> > Denis
>> >> >
>> >> >
>> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko <
>> >> > valentin.kulichenko@gmail.com> wrote:
>> >> >
>> >> > > Igniters,
>> >> > >
>> >> > > One of the big changes proposed for Ignite 3.0 is the so-called
>> >> > > "schema-first approach". To add more clarity, I've started writing
>> >> > > the
>> >> > IEP
>> >> > > for this change:
>> >> > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
>> >> > >
>> >> > > Please take a look and let me know if there are any immediate
>> >> > > thoughts,
>> >> > > suggestions, or objections.
>> >> > >
>> >> > > -Val
>> >> > >
>> >> >
>> >>
>> >
>>
>>
>> --
>>
>> Best regards,
>> Ivan Pavlukhin
>>
>


-- 

Best regards,
Ivan Pavlukhin

Mime
View raw message