ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Sapego <isap...@apache.org>
Subject Re: IEP-76 Thin Client Protocol for Ignite 3.0
Date Thu, 01 Jul 2021 15:42:22 GMT
Ivan, what are extra serde steps you are talking about?

Best Regards,
Igor


On Thu, Jul 1, 2021 at 5:52 PM Ivan Daschinsky <ivandasch@gmail.com> wrote:

> > I agree. But this was decided before in IEP-54, and is out of scope for
> current IEP.
> Would you like to start a separate thread to discuss this? Or I can do this
> a bit later.
>
> Great idea, let's discuss it. I suppose this will simplify many aspects of
> realization and improve performance a lot
>
> чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky <ivandasch@gmail.com>:
>
> > > Here is the description of TUPLE_GET_ALL:
> > - UUID: table ID
> > - int: schema ID
> > - arr of arr: array of rows with values for all columns in given schema
> >
> > I suppose that we should describe this more verbose and explicit. I
> > nevertheless suggest to also consider writing values this way:
> > - arr of fields names (if name is missed, corresponding field is nil)
> > - arr of rows (row as array, length equal to fields array)
> >
> > It is quite simple and if we use str8 (it is more than enough for any
> > utf-8 reasonable field name), overhead will be negligible, but
> realization
> > of a client will be way simpler
> >
> > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn <ptupitsyn@apache.org>:
> >
> >> > No it isn't, I have carefully read code and IEP, in your code you
> write
> >> > schema id in each tuple.
> >>
> >> There is no code for batch operations yet.
> >>
> >> Here is the description of TUPLE_GET_ALL:
> >> - UUID: table ID
> >> - int: schema ID
> >> - arr of arr: array of rows with values for all columns in given schema
> >> (nil when value is missing for a column)
> >>
> >> As you can see, schema ID is written once for all rows.
> >> A row is just a set of values according to the schema.
> >>
> >>
> >> > Also, my biggest concern -- extra serde step. I suppose we should pass
> >> > bytearray to internal api, and use msgpack throughout all wire
> >> protocols,
> >> > as tarantool does.
> >>
> >> I agree. But this was decided before in IEP-54, and is out of scope for
> >> current IEP.
> >> Would you like to start a separate thread to discuss this? Or I can do
> >> this
> >> a bit later.
> >>
> >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky <ivandasch@gmail.com>
> >> wrote:
> >>
> >> > > This is described in all operations that include multiple tuples.
> >> > No it isn't, I have carefully read code and IEP, in your code you
> write
> >> > schema id in each tuple.
> >> >
> >> > Also, my biggest concern -- extra serde step. I suppose we should pass
> >> > bytearray to internal api, and use msgpack throughout all wire
> >> protocols,
> >> > as tarantool does.
> >> >
> >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn <ptupitsyn@apache.org>:
> >> >
> >> > > Ivan,
> >> > >
> >> > > >  that there is not neccesary to write schema versions in each
row
> >> > > > in collectionof tuples
> >> > >
> >> > > This is described in all operations that include multiple tuples.
> >> > >
> >> > >
> >> > > > it is not clear from your code (probably
> >> > > > mistake?) how differ key tuples and value tuples from each other
> >> > >
> >> > > Key tuples include only key columns. Key columns come first in the
> >> > schema.
> >> > > Value tuples include all columns, key and value. Added "Key tuples"
> >> > > section.
> >> > >
> >> > >
> >> > > > As for me, these excercises with schema's doesn't worth a lot
> >> > >
> >> > > I'll add a benchmark and we'll see.
> >> > >
> >> > >
> >> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky <ivandasch@gmail.com
> >
> >> > > wrote:
> >> > >
> >> > > > I suppose, that there is not neccesary to write schema versions
in
> >> each
> >> > > row
> >> > > > in collectionof tuples. Also it is not clear from your code
> >> (probably
> >> > > > mistake?) how differ key tuples and value tuples from each other.
> In
> >> > > > readTuple you always read full schema and check for full length.
> As
> >> for
> >> > > me,
> >> > > > these excercises with schema's doesn't worth a lot. I.e. postgres
> >> just
> >> > > > writes field names and then simpy rows with data. Saving few
bytes
> >> > > doesn't
> >> > > > make much deal. Btw, msgpack has special types for short strings
> >> (i.e.
> >> > > > str8). It is much easier use it and write field name as is.
> >> > > >
> >> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn <ptupitsyn@apache.org>:
> >> > > >
> >> > > > > Ivan, tuple serialization section added to the IEP, let
me know
> >> if it
> >> > > is
> >> > > > > clear enough.
> >> > > > >
> >> > > > > Thanks!
> >> > > > >
> >> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky <
> >> ivandasch@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > I can't find any description of tuple serialization
in IEP,
> >> only in
> >> > > > code
> >> > > > > >
> >> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn <
> ptupitsyn@apache.org
> >> >:
> >> > > > > >
> >> > > > > > > Ivan,
> >> > > > > > >
> >> > > > > > > 0. The IEP is not in progress, it is ready for
review and
> >> > > discussion.
> >> > > > > > > 1. Tuple serialization is described in the IEP
and
> >> demonstrated
> >> > in
> >> > > > the
> >> > > > > > PoC
> >> > > > > > > (see ClientMessageHandler#readTuple), let me know
if more
> >> details
> >> > > are
> >> > > > > > > required
> >> > > > > > > 2. Tuple schema serialization is described in
SCHEMAS_GET
> >> > section.
> >> > > > > Table
> >> > > > > > > schema (configuration) needs more details, you
are right -
> >> I'll
> >> > add
> >> > > > > them.
> >> > > > > > > 3. This IEP is about tables (tuple-based) API
only, since it
> >> is
> >> > the
> >> > > > > only
> >> > > > > > > API that we have right now, as noted in Risks
and
> Assumptions.
> >> > > > > > >
> >> > > > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky
<
> >> > > ivandasch@gmail.com>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Also, is there any clear information about
KV api? Is
> there
> >> any
> >> > > > plan
> >> > > > > to
> >> > > > > > > > implement it? Or is there any proposal about
it?
> >> > > > > > > >
> >> > > > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky
<
> >> ivandasch@gmail.com
> >> > >:
> >> > > > > > > >
> >> > > > > > > > > Pavel, but IEP is in progress, isn't
it?
> >> > > > > > > > >
> >> > > > > > > > > 1. There is not any information about
tuple
> serialization.
> >> > And
> >> > > > > there
> >> > > > > > > > isn't
> >> > > > > > > > > a clear consensus about it.
> >> > > > > > > > > 2. There is not any information about
schrma
> serialization
> >> > > > format.
> >> > > > > > And
> >> > > > > > > > > AFAIK, there isn't a clear consensus
also.
> >> > > > > > > > >
> >> > > > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel
Tupitsyn <
> >> > ptupitsyn@apache.org
> >> > > >:
> >> > > > > > > > >
> >> > > > > > > > >> Igniters,
> >> > > > > > > > >>
> >> > > > > > > > >> Please review the IEP for thin client
protocol in 3.0
> >> [1].
> >> > > > > > > > >> PoC is in progress [2]
> >> > > > > > > > >>
> >> > > > > > > > >> [1]
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
> >> > > > > > > > >>
> >> > > > > > > > >> [2] https://github.com/apache/ignite-3/pull/191
> >> > > > > > > > >>
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message