ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: IEP-54: Schema-first approach for 3.0
Date Wed, 09 Sep 2020 01:01:14 GMT
Hi Denis,

I guess the wording in the IEP is a little bit confusing. All it means is
that you should not create nested POJOs, but rather inline fields into a
single POJO that is mapped to a particular schema. In other words, nested
POJOs are not supported.

Alex, is this correct? Please let me know if I'm missing something.

As for the "cache" term, I agree that it is outdated, but I'm not sure what
we can replace it with. "Table" is tightly associated with SQL, but SQL is
optional in our case. Do you want to create a separate discussion about
this?

-Val

On Tue, Sep 8, 2020 at 4:37 PM Denis Magda <dmagda@apache.org> wrote:

> Val,
>
> I've checked the IEP again and have a few questions.
>
> Arbitrary nested objects and collections are not allowed as column values.
> > Nested POJOs should either be inlined into schema, or stored as BLOBs
>
>
> Could you provide a DDL code snippet showing how the inlining of POJOs is
> supposed to work?
>
> Also, we keep using the terms "cache" and "table" throughout the IEP. Is it
> the right time to discuss an alternate name that would replace those too?
> Personally, the "table" should stay and the "cache" should go considering
> that SQL is one of the primary APIs in Ignite and that DDL is supported
> out-of-the-box.
>
>
> -
> Denis
>
>
> On Mon, Sep 7, 2020 at 12:26 PM Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
> > Ivan,
> >
> > I see your point. I agree that with the automatic updates we step into
> the
> > schema-last territory.
> >
> > Actually, if we support automatic evolution, we can as well support
> > creating a cache without schema and inferring it from the first insert.
> In
> > other words, we can have both "schema-first" and "schema-last" modes.
> >
> > Alexey, what do you think?
> >
> > -Val
> >
> > On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk <
> > alexey.goncharuk@gmail.com>
> > wrote:
> >
> > > Ivan,
> > >
> > > Thank you, I got your concern now. As it is mostly regarding the
> > > terminology, I am absolutely fine with changing the name to whatever
> fits
> > > the approach best. Dynamic or evolving schema sounds great. I will make
> > > corresponding changes to the IEP once we settle on the name.
> > >
> > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo100@gmail.com>:
> > >
> > > > Hi Val,
> > > >
> > > > Thank you for your answer!
> > > >
> > > > My understanding is a little bit different. Yes, schema evolution
> > > > definitely should be possible. But I see a main difference in "how
> > > > schema is updated". I treat a common SQL approach schema-first.
> Schema
> > > > and data manipulation operations are clearly separated and it enables
> > > > interesting capabilities, e.g. preventing untended schema changes by
> > > > mistaken data operations, restricting user permissions to change
> > > > schema.
> > > >
> > > > > Schema-first means that schema exists in advance and all the stored
> > > data
> > > > is compliant with it - that's exactly what is proposed.
> > > >
> > > > A schema-last approach mentioned in [1] also assumes that schema
> > > > exists, but it is inferred from data. Is not it more similar to the
> > > > proposing approach?
> > > >
> > > > And I would like to say, that my main concern so far is mostly about
> > > > terminology. And I suppose if it confuses me then others might be
> > > > confused as well. My feeling is closer to "dynamic or liquid or may
> be
> > > > evolving schema".
> > > >
> > > > [1]
> > > >
> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> > > >
> > > > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko <
> > > > valentin.kulichenko@gmail.com>:
> > > > > Hi Ivan,
> > > > >
> > > > > I don't see an issue with that. Schema-first means that schema
> exists
> > > in
> > > > > advance and all the stored data is compliant with it - that's
> exactly
> > > > what
> > > > > is proposed. There are no restrictions prohibiting changes to the
> > > schema.
> > > > >
> > > > > -Val
> > > > >
> > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo100@gmail.com
> >
> > > > wrote:
> > > > >
> > > > >> Alexey,
> > > > >>
> > > > >> I am a little bit confused with terminology. My understanding
> > conforms
> > > > >> to a survey [1] (see part X Semi Structured Data). Can we really
> > treat
> > > > >> a "dynamic schema" approach as a kind of "schema-first"?
> > > > >>
> > > > >> [1]
> > > > >>
> > >
> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
> > > > >>
> > > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dmagda@apache.org>:
> > > > >> >>
> > > > >> >> However, could you please elaborate on the relation
between
> > Ignite
> > > > and
> > > > >> >> ORM?
> > > > >> >> Is there a use case for Hibernate running on top of
Ignite (I
> > > haven't
> > > > >> >> seen
> > > > >> >> one so far)? If so, what is missing exactly on the Ignite
side
> to
> > > > >> support
> > > > >> >> this? In my understanding, all you need is SQL API which
we
> > already
> > > > >> have.
> > > > >> >> Am I missing something?
> > > > >> >
> > > > >> >
> > > > >> > Good point, yes, if all the ORM integrations use Ignite
SQL APIs
> > > > >> > internally, then they can easily translate an Entity object
into
> > an
> > > > >> > INSERT/UPDATE statement that lists all the object's fields.
> > Luckily,
> > > > >> > our
> > > > >> > Spring Data integration is already based on the Ignite SQL
APIs
> > and
> > > > >> > needs
> > > > >> > to be improved once the schema-first approach is supported.
That
> > > would
> > > > >> > solve a ton of usability issues.
> > > > >> >
> > > > >> > I would revise the Hibernate integration as well during
the
> Ignite
> > > 3.0
> > > > >> dev
> > > > >> > phase. Can't say if it's used a lot but Spring Data is getting
> > > > traction
> > > > >> for
> > > > >> > sure.
> > > > >> >
> > > > >> > @Michael Pollind, I'll loop you in as long as you've started
> > working
> > > > on
> > > > >> the
> > > > >> > Ignite support for Micornaut Data
> > > > >> > <
> > https://micronaut-projects.github.io/micronaut-data/latest/guide/>
> > > > and
> > > > >> > came across some challenges. Just watch this discussion.
That's
> > what
> > > > is
> > > > >> > coming in Ignite 3.0.
> > > > >> >
> > > > >> >
> > > > >> > -
> > > > >> > Denis
> > > > >> >
> > > > >> >
> > > > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko <
> > > > >> > valentin.kulichenko@gmail.com> wrote:
> > > > >> >
> > > > >> >> Hi Denis,
> > > > >> >>
> > > > >> >> Generally speaking, I believe that the schema-first
approach
> > > natively
> > > > >> >> addresses the issue if duplicate fields in key and value
> objects,
> > > > >> because
> > > > >> >> schema will be created for a cache, not for an object,
as it
> > > happens
> > > > >> now.
> > > > >> >> Basically, the schema will define whether there is a
primary
> key
> > or
> > > > >> >> not,
> > > > >> >> and which fields are included in case there is one.
Any API
> that
> > we
> > > > >> would
> > > > >> >> have must be compliant with this, so it becomes fairly
easy to
> > work
> > > > >> >> with
> > > > >> >> data as with a set of records, rather than key-value
pairs.
> > > > >> >>
> > > > >> >> However, could you please elaborate on the relation
between
> > Ignite
> > > > and
> > > > >> >> ORM?
> > > > >> >> Is there a use case for Hibernate running on top of
Ignite (I
> > > haven't
> > > > >> >> seen
> > > > >> >> one so far)? If so, what is missing exactly on the Ignite
side
> to
> > > > >> support
> > > > >> >> this? In my understanding, all you need is SQL API which
we
> > already
> > > > >> have.
> > > > >> >> Am I missing something?
> > > > >> >>
> > > > >> >> -Val
> > > > >> >>
> > > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dmagda@apache.org
> >
> > > > wrote:
> > > > >> >>
> > > > >> >> > Val,
> > > > >> >> >
> > > > >> >> > I would propose adding another point to the motivations
list
> > > which
> > > > >> >> > is
> > > > >> >> > related to the ORM frameworks such as Spring Data,
Hibernate,
> > > > >> Micronaut
> > > > >> >> and
> > > > >> >> > many others.
> > > > >> >> >
> > > > >> >> > Presently, the storage engine requires to distinguish
key
> > objects
> > > > >> >> > from
> > > > >> >> the
> > > > >> >> > value ones that complicate the usage of Ignite
with those ORM
> > > > >> >> > frameworks
> > > > >> >> > (especially if a key object comprises several fields).
More
> on
> > > this
> > > > >> can
> > > > >> >> be
> > > > >> >> > found here:
> > > > >> >> >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html
> > > > >> >> >
> > > > >> >> > It will be nice if the new schema-first approach
allows us to
> > > work
> > > > >> with
> > > > >> >> > a
> > > > >> >> > single entity object when it comes to the ORMs.
With no need
> to
> > > > >> >> > split
> > > > >> >> > the
> > > > >> >> > entity into a key and value. Just want to be sure
that the
> > Ignite
> > > > >> >> > 3.0
> > > > >> >> > has
> > > > >> >> > all the essential public APIs that would support
the
> > > single-entity
> > > > >> >> > based
> > > > >> >> > approach.
> > > > >> >> >
> > > > >> >> > What do you think?
> > > > >> >> >
> > > > >> >> > -
> > > > >> >> > Denis
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko
<
> > > > >> >> > valentin.kulichenko@gmail.com> wrote:
> > > > >> >> >
> > > > >> >> > > Igniters,
> > > > >> >> > >
> > > > >> >> > > One of the big changes proposed for Ignite
3.0 is the
> > so-called
> > > > >> >> > > "schema-first approach". To add more clarity,
I've started
> > > > writing
> > > > >> >> > > the
> > > > >> >> > IEP
> > > > >> >> > > for this change:
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > >> >> > >
> > > > >> >> > > Please take a look and let me know if there
are any
> immediate
> > > > >> >> > > thoughts,
> > > > >> >> > > suggestions, or objections.
> > > > >> >> > >
> > > > >> >> > > -Val
> > > > >> >> > >
> > > > >> >> >
> > > > >> >>
> > > > >> >
> > > > >>
> > > > >>
> > > > >> --
> > > > >>
> > > > >> Best regards,
> > > > >> Ivan Pavlukhin
> > > > >>
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message