ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: Interoperable Ignite.NET Dates
Date Tue, 03 Nov 2020 12:48:17 GMT
Alexey,

The proposal looks good to me.

Usually I would try to avoid extra dependencies in the core assembly,
but NodaTime seems to be worth it.

Would you like to prepare an IEP?

Thanks,
Pavel



On Tue, Nov 3, 2020 at 3:15 PM Alexey Kukushkin <kukushkinalexey@gmail.com>
wrote:

> We already created tickets and tested the proposed solution with our
> applications (already in PROD) and an internal fork of Ignite. This
> discussion is a proposal to donate this change to open source Apache Ignite
> is the community accepts it:
>
>    - IGNITE-12824 .NET: Write DateTime in interoperable format by default
>    <https://issues.apache.org/jira/browse/IGNITE-12824>
>    - IGNITE-12825 Serialize Java and .NET dates using same calendars
>    <https://issues.apache.org/jira/browse/IGNITE-12825>
>
>
> вт, 3 нояб. 2020 г. в 15:08, Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
> > Cool, makes sense to me then. Please feel free to create a ticket and
> mark
> > it with the "ignite-3" label.
> >
> > -Val
> >
> > On Tue, Nov 3, 2020 at 4:03 AM Alexey Kukushkin <
> kukushkinalexey@gmail.com
> > >
> > wrote:
> >
> > > Val, absolutely - our proposal affects .NET API only.
> > >
> > > вт, 3 нояб. 2020 г. в 15:00, Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > > > Got it. So it only affects the .NET side, correct?
> > > >
> > > > -Val
> > > >
> > > > On Tue, Nov 3, 2020 at 3:52 AM Alexey Kukushkin <
> > > kukushkinalexey@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Val,
> > > > >
> > > > > Our proposal does not overlap with IEP-54
> > > > > <
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >,
> > > > > which proposes changing Ignite date storage format. Our proposal
is
> > > > > enhancing Ignite to handle .NET dates in a truly portable way
> instead
> > > of
> > > > > requiring the application developers to repeat this task every
> time:
> > > > >
> > > > >    - Write .NET dates as Ignite dates (today .NET dates are written
> > as
> > > > >    generic Ignite objects)
> > > > >    - Convert Local .NET dates to UTC inside Ignite and to it using
> > Java
> > > > >    calendars.
> > > > >
> > > > >
> > > > > вт, 3 нояб. 2020 г. в 11:10, Valentin Kulichenko <
> > > > > valentin.kulichenko@gmail.com>:
> > > > >
> > > > > > Hi Alexey,
> > > > > >
> > > > > > The IEP-54 [1] describes the data layout proposed for Ignite
3.0,
> > it
> > > > > > includes various date/time types. Can you please take a look
and
> > > check
> > > > if
> > > > > > this addresses your concerns? We can then discuss further if
> > needed.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Tue, Nov 3, 2020 at 12:05 AM Alexey Kukushkin <
> > > > > > kukushkinalexey@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > > We propose changing public API so this is for Ignite 3.0.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > вт, 3 нояб. 2020 г. в 02:17, Pavel Tupitsyn <
> > ptupitsyn@apache.org
> > > >:
> > > > > > >
> > > > > > > > Alexey,
> > > > > > > >
> > > > > > > > Just to clarify before we start the discussion:
> > > > > > > > this proposal seems to introduce some breaking changes,
so we
> > are
> > > > > > talking
> > > > > > > > about Ignite 3.0, correct?
> > > > > > > >
> > > > > > > > Pavel
> > > > > > > >
> > > > > > > > On Tue, Nov 3, 2020 at 12:13 AM Alexey Kukushkin <
> > > > > > > > kukushkinalexey@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > What do you think about changing .NET API to
read/write
> > > portable
> > > > > > dates
> > > > > > > by
> > > > > > > > > default and making that really portable?
> > > > > > > > >
> > > > > > > > > *The Problem*
> > > > > > > > > Presently .NET API writes dates as composite
Ignite
> objects.
> > > Only
> > > > > > .NET
> > > > > > > > > clients can read such dates: any other client
(JDBC, Java,
> > etc)
> > > > > does
> > > > > > > not
> > > > > > > > > understand it without custom deserialization.
> > > > > > > > >
> > > > > > > > > It is still possible to configure .NET serialization
to
> write
> > > > dates
> > > > > > as
> > > > > > > > > Ignite dates - see DateTime Serialization note
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility
> > > > > > > > > >.
> > > > > > > > > But then Ignite accepts only UTC dates, requiring
the
> > > application
> > > > > > > > > developers to convert local dates to UTC dates
and back.
> This
> > > > task
> > > > > is
> > > > > > > not
> > > > > > > > > trivial: DateTime.ToUniversalTime
> > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1
> > > > > > > > > >
> > > > > > > > > uses
> > > > > > > > > calendars different from Java (and the .NET calendars
are
> the
> > > > > invalid
> > > > > > > > ones
> > > > > > > > > - especially for pre-1990 dates).
> > > > > > > > >
> > > > > > > > > The motivation for the current default behavior
was
> probably
> > > the
> > > > > > desire
> > > > > > > > to
> > > > > > > > > keep the Time Zone information: Ignite dates
do not store
> > time
> > > > > zones.
> > > > > > > > >
> > > > > > > > > In our experience interoperability is more important
than
> > > storing
> > > > > > time
> > > > > > > > zone
> > > > > > > > > info.
> > > > > > > > >
> > > > > > > > > *The Proposal*
> > > > > > > > >
> > > > > > > > >    1. Always write .NET dates as portable Ignite
dates: get
> > rid
> > > > of
> > > > > > the
> > > > > > > > >    BinaryReflectiveSerializer.ForceTimestamp
flag that
> > > currently
> > > > > > > triggers
> > > > > > > > > this
> > > > > > > > >    behavior.
> > > > > > > > >       - We could still keep the ForceTimestamp
flag if
> saving
> > > > .NET
> > > > > > > dates
> > > > > > > > as
> > > > > > > > >       transparent objects seems a useful case.
We do not
> > think
> > > it
> > > > > is
> > > > > > > > > useful.
> > > > > > > > >    2. Automatically convert Local dates to UTC
and back
> > > *inside*
> > > > > > > > >    Ignite.NET.
> > > > > > > > >       - In this case we lose the DateTime.Kind
of UTC
> dates:
> > we
> > > > > > write a
> > > > > > > > UTC
> > > > > > > > >       date but we would read a Local date since
Ignite
> would
> > > > always
> > > > > > > > > convert UTC
> > > > > > > > >       to Local when reading.
> > > > > > > > >       We could add a UtcDate date flag to
> > > QuerySqlFieldAttribute
> > > > > > > > >       and BinaryReflectiveSerializer to control
the
> > > > deserialization
> > > > > > > > > behavior if
> > > > > > > > >       keeping dates in UTC format use case seems
important.
> > > > > > > > >    3. Use NodaTime <https://nodatime.org/>
for UTC<->Local
> > > > > > > conversions.
> > > > > > > > >    Noda time uses Java calendars making the conversion
> truely
> > > > > > portable.
> > > > > > > > >
> > > > > > > > > *The Benefits*
> > > > > > > > >
> > > > > > > > >    1. We think portable dates are much more important
than
> > > > storing
> > > > > > time
> > > > > > > > >    zone info. Why do we store time zones for
every date on
> > the
> > > > > server
> > > > > > > > > anyway?
> > > > > > > > >    Time zone is client-side info.
> > > > > > > > >    2. Simpler application code: no more manual
> configuration
> > to
> > > > > > trigger
> > > > > > > > the
> > > > > > > > >    portable behavior.
> > > > > > > > >    3. Non-trivial code to make the truly portable
> UTC<->Local
> > > > > > > conversion
> > > > > > > > is
> > > > > > > > >    implemented once inside Ignite instead of
having every
> > > > > Ignite.NET
> > > > > > > > >    application implementing it.
> > > > > > > > >
> > > > > > > > > Thank you!
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Alexey
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Alexey
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Alexey
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Alexey
> > >
> >
>
>
> --
> Best regards,
> Alexey
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message