ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Daschinsky <ivanda...@gmail.com>
Subject Re: IEP-76 Thin Client Protocol for Ignite 3.0
Date Thu, 01 Jul 2021 14:51:40 GMT
> I agree. But this was decided before in IEP-54, and is out of scope for
current IEP.
Would you like to start a separate thread to discuss this? Or I can do this
a bit later.

Great idea, let's discuss it. I suppose this will simplify many aspects of
realization and improve performance a lot

чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky <ivandasch@gmail.com>:

> > Here is the description of TUPLE_GET_ALL:
> - UUID: table ID
> - int: schema ID
> - arr of arr: array of rows with values for all columns in given schema
>
> I suppose that we should describe this more verbose and explicit. I
> nevertheless suggest to also consider writing values this way:
> - arr of fields names (if name is missed, corresponding field is nil)
> - arr of rows (row as array, length equal to fields array)
>
> It is quite simple and if we use str8 (it is more than enough for any
> utf-8 reasonable field name), overhead will be negligible, but realization
> of a client will be way simpler
>
> чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn <ptupitsyn@apache.org>:
>
>> > No it isn't, I have carefully read code and IEP, in your code you write
>> > schema id in each tuple.
>>
>> There is no code for batch operations yet.
>>
>> Here is the description of TUPLE_GET_ALL:
>> - UUID: table ID
>> - int: schema ID
>> - arr of arr: array of rows with values for all columns in given schema
>> (nil when value is missing for a column)
>>
>> As you can see, schema ID is written once for all rows.
>> A row is just a set of values according to the schema.
>>
>>
>> > Also, my biggest concern -- extra serde step. I suppose we should pass
>> > bytearray to internal api, and use msgpack throughout all wire
>> protocols,
>> > as tarantool does.
>>
>> I agree. But this was decided before in IEP-54, and is out of scope for
>> current IEP.
>> Would you like to start a separate thread to discuss this? Or I can do
>> this
>> a bit later.
>>
>> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky <ivandasch@gmail.com>
>> wrote:
>>
>> > > This is described in all operations that include multiple tuples.
>> > No it isn't, I have carefully read code and IEP, in your code you write
>> > schema id in each tuple.
>> >
>> > Also, my biggest concern -- extra serde step. I suppose we should pass
>> > bytearray to internal api, and use msgpack throughout all wire
>> protocols,
>> > as tarantool does.
>> >
>> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn <ptupitsyn@apache.org>:
>> >
>> > > Ivan,
>> > >
>> > > >  that there is not neccesary to write schema versions in each row
>> > > > in collectionof tuples
>> > >
>> > > This is described in all operations that include multiple tuples.
>> > >
>> > >
>> > > > it is not clear from your code (probably
>> > > > mistake?) how differ key tuples and value tuples from each other
>> > >
>> > > Key tuples include only key columns. Key columns come first in the
>> > schema.
>> > > Value tuples include all columns, key and value. Added "Key tuples"
>> > > section.
>> > >
>> > >
>> > > > As for me, these excercises with schema's doesn't worth a lot
>> > >
>> > > I'll add a benchmark and we'll see.
>> > >
>> > >
>> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky <ivandasch@gmail.com>
>> > > wrote:
>> > >
>> > > > I suppose, that there is not neccesary to write schema versions in
>> each
>> > > row
>> > > > in collectionof tuples. Also it is not clear from your code
>> (probably
>> > > > mistake?) how differ key tuples and value tuples from each other.
In
>> > > > readTuple you always read full schema and check for full length. As
>> for
>> > > me,
>> > > > these excercises with schema's doesn't worth a lot. I.e. postgres
>> just
>> > > > writes field names and then simpy rows with data. Saving few bytes
>> > > doesn't
>> > > > make much deal. Btw, msgpack has special types for short strings
>> (i.e.
>> > > > str8). It is much easier use it and write field name as is.
>> > > >
>> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn <ptupitsyn@apache.org>:
>> > > >
>> > > > > Ivan, tuple serialization section added to the IEP, let me know
>> if it
>> > > is
>> > > > > clear enough.
>> > > > >
>> > > > > Thanks!
>> > > > >
>> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky <
>> ivandasch@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > I can't find any description of tuple serialization in IEP,
>> only in
>> > > > code
>> > > > > >
>> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn <ptupitsyn@apache.org
>> >:
>> > > > > >
>> > > > > > > Ivan,
>> > > > > > >
>> > > > > > > 0. The IEP is not in progress, it is ready for review
and
>> > > discussion.
>> > > > > > > 1. Tuple serialization is described in the IEP and
>> demonstrated
>> > in
>> > > > the
>> > > > > > PoC
>> > > > > > > (see ClientMessageHandler#readTuple), let me know if
more
>> details
>> > > are
>> > > > > > > required
>> > > > > > > 2. Tuple schema serialization is described in SCHEMAS_GET
>> > section.
>> > > > > Table
>> > > > > > > schema (configuration) needs more details, you are
right -
>> I'll
>> > add
>> > > > > them.
>> > > > > > > 3. This IEP is about tables (tuple-based) API only,
since it
>> is
>> > the
>> > > > > only
>> > > > > > > API that we have right now, as noted in Risks and Assumptions.
>> > > > > > >
>> > > > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky <
>> > > ivandasch@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Also, is there any clear information about KV
api? Is there
>> any
>> > > > plan
>> > > > > to
>> > > > > > > > implement it? Or is there any proposal about it?
>> > > > > > > >
>> > > > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky
<
>> ivandasch@gmail.com
>> > >:
>> > > > > > > >
>> > > > > > > > > Pavel, but IEP is in progress, isn't it?
>> > > > > > > > >
>> > > > > > > > > 1. There is not any information about tuple
serialization.
>> > And
>> > > > > there
>> > > > > > > > isn't
>> > > > > > > > > a clear consensus about it.
>> > > > > > > > > 2. There is not any information about schrma
serialization
>> > > > format.
>> > > > > > And
>> > > > > > > > > AFAIK, there isn't a clear consensus also.
>> > > > > > > > >
>> > > > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn
<
>> > ptupitsyn@apache.org
>> > > >:
>> > > > > > > > >
>> > > > > > > > >> Igniters,
>> > > > > > > > >>
>> > > > > > > > >> Please review the IEP for thin client
protocol in 3.0
>> [1].
>> > > > > > > > >> PoC is in progress [2]
>> > > > > > > > >>
>> > > > > > > > >> [1]
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
>> > > > > > > > >>
>> > > > > > > > >> [2] https://github.com/apache/ignite-3/pull/191
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message