ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Plehanov <plehanov.a...@gmail.com>
Subject Re: Thin client: transactions support
Date Mon, 13 May 2019 10:58:05 GMT
Hi, Ivan.

Thanks for your comments.

1. Transaction id in thin client protocol it's just a tx counter for the
current connection. It's not related to GridCacheVersion. If we want to
know GridCacheVersion on the client side, I think we should introduce a new
type of operation (for example OP_TX_INFO).
2. Error handling is already provided by thin client protocol, even in case
of empty response. Of course, the client will know if there is a failure
occurred during OP_TX_END operation.
3. AFAIK some of thin client implementations already send requests and
process responses in an async way (.NET for example). As for java thin
client, in the current implementation channel is locked exclusively before
request send and until the response is processed. I have some ideas about
how to fix this (split send/receive process into two different parts and
acquire locks for this parts separately or create futures on request sent
and complete it after processing the response in a dedicated thread), I've
created ticket [1] for this issue and will try to implement prototype in a
couple of days.

About suspend/resume, yes, on server-side we should resume tx before each
transactional cache operation and suspend the tx after the operation. In my
opinion, suspend/resume approach have several advantages over approach with
explicit tx id argument:
- Introducing explicit tx id argument for cache operations leads to a
significant API change
- It's not clear how to use it together with current (tx-per-thread)
approach (for example, what if a thread is already held transaction and
someone call cache operation with explicit tx id?)
- Suspend/resume feature will also be useful for thick clients
- Suspend/resume functionality is already partially implemented (for
optimistic transactions only)

[1] https://issues.apache.org/jira/browse/IGNITE-11685

пт, 3 мая 2019 г. в 08:10, Павлухин Иван <vololo100@gmail.com>:

> Hi Alex,
>
> I went through IEP [1] and I have a couple of questions:
> 1. What is going to be used as transaction id? In a described protocol
> I see an int field for it. Should not it be GridCacheVersion
> corresponding to IgniteInternalTx#xidVersion?
> 2. OP_TX_END message assumes an empty response, but I think that
> errors during tx finish are possible and should be returned in a
> response.
> 3. In IEP it is stated that async processing of lock operations should
> be introduced on a client side to enable concurrent operations from
> different client threads. Do you have an idea how to achieve it?
>
> Also, a bit about a suspend/resume trait. I tried to think about it
> leaving away an existing transactions implementation in Ignite. As I
> understood we are going to resume a tx before each cache operation in
> the tx and resume the tx after the operation. All this to make an
> executing thread available for other operations (e.g. in other txs).
> From the first glance it seems like an inversed logic. A
> straightforward way is to execute a cache operation within a
> particular transaction defined as an explicit tx id argument (e.g.
> cache.put(key, value, txid)). Can we do so?
>
> And leaving for now thin client API. I cannot say that one proposed in
> IEP is good or bad. I can only say that it ressembles current thick
> client API. And perhaps it should not. I think that we should consider
> similar APIs provided by other vendors and keep in mind that we have a
> bunch of client implementations for different languages. I suppose
> that we can return to it a little bit later. And I hope that we will
> do it.
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
>
> вт, 30 апр. 2019 г. в 13:24, Alex Plehanov <plehanov.alex@gmail.com>:
> >
> > Hello, Igniters!
> >
> > I've update IEP [1] and implement PoC according to new approach (multiple
> > concurrent transactions per connection).
> > But to move forward another feature need to be implemented:
> suspend/resume
> > for pessimistic transactions (IGNITE-5714 [2]). Implementation of
> > suspend/resume is ready now and ticket in 'Patch available' status. Can
> any
> > transactions expert help with review of IGNITE-5714?
> >
> > [1]:
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > [2]: https://issues.apache.org/jira/browse/IGNITE-5714
> >
> > чт, 4 апр. 2019 г. в 11:32, Alex Plehanov <plehanov.alex@gmail.com>:
> >
> > > Vladimir,
> > >
> > > Ok, then I will rewrite IEP in the near future.
> > >
> > > чт, 4 апр. 2019 г. в 11:14, Vladimir Ozerov <vozerov@gridgain.com>:
> > >
> > >> Hi Alex,
> > >>
> > >> I think we should be able to handle many transactions through a single
> > >> connection. This will make our protocol and client implementations
> much
> > >> more efficient, and simplicity from developer's perspective is not our
> > >> goal. Consider normal nodes. We have server nodes and client nodes.
> You
> > >> may
> > >> span whatever number of transactions you need, but all of them are
> > >> coordinated through a single connection. The same should be
> applicable to
> > >> thin clients. Protocol is already designed to handle this, as we pass
> > >> unique operation ID in order to distinguish one operation from
> another. It
> > >> is true, though, that we will have to introduce a kind of "session"
> > >> concept, and pass additional identifier along with cache operations,
> but
> > >> this doesn't sound like a problem to me.
> > >>
> > >> And provided that currently server-side transactions are bound to
> threads
> > >> artificially, I would say that the first step in implementation of
> > >> transactions on thin clients should be decoupling server-side
> transactions
> > >> from threads. Without this we will have very inefficient
> implementation,
> > >> when every new client transaction have to spawn a new thread. This is
> slow
> > >> and introduces high memory pressure on a cluster node. We already work
> > >> this
> > >> way for MVCC transactions which are spawned from JDBC driver, and
> believe
> > >> me, we do not want to replicated this bad practice to other clients
> :-)
> > >>
> > >> Vladimir.
> > >>
> > >> On Thu, Apr 4, 2019 at 10:08 AM Alex Plehanov <
> plehanov.alex@gmail.com>
> > >> wrote:
> > >>
> > >> > Guys, so, do we need multiple concurrent transactions per
> connection?
> > >> >
> > >> > There are pros and cons for each approach. Difference between
> > >> approaches:
> > >> >
> > >> > One transaction at a time per connection:
> > >> >  - This approach is used in RDBMS world and users got used to it
> > >> >  - To use transactions concurrently users need to use different
> > >> connections
> > >> > and get these connections via something like a connection pool
> > >> >  - Easy to implement (in fact, PoC is already done)
> > >> >
> > >> > Multiple concurrent transactions per connection:
> > >> >  - At least for java thin client, we can implement transaction per
> > >> thread
> > >> > approach as implemented now for the thick client (perhaps other thin
> > >> > clients can implement the same abstraction)
> > >> >  - There is also protocol change for all cache operations needed (to
> > >> bind
> > >> > cache operation to the transaction)
> > >> >  - Significant changes to all implemented clients are needed
> > >> >  - Implementation on the server side is more complex
> > >> >
> > >> > What do you think?
> > >> >
> > >> >
> > >> > вт, 2 апр. 2019 г. в 16:29, Alex Plehanov <plehanov.alex@gmail.com
> >:
> > >> >
> > >> > > Ilya,
> > >> > >
> > >> > > > We should be able to multiplex several transactions using a
> single
> > >> > > Client connection.
> > >> > > In this case, we should significantly change cache operations
> syntax
> > >> (for
> > >> > > each implemented client), to bind each operation to the
> transaction.
> > >> > >
> > >> > > > I want to also ask if "Number of entries participating in
> > >> transaction
> > >> > > (may be approximate). 0 - default value." is needed.
> > >> > > I've tried to minimize API changes between thick and thin client
> to
> > >> > > simplify move from one to another. It's the only reason. But I
> agree
> > >> with
> > >> > > you, the parameter is not very useful.
> > >> > >
> > >> > >
> > >> > > вт, 2 апр. 2019 г. в 14:48, Ilya Kasnacheev <
> > >> ilya.kasnacheev@gmail.com>:
> > >> > >
> > >> > >> Hello!
> > >> > >>
> > >> > >> Pavel, I agree with you thorougly. We should be able to multiplex
> > >> > several
> > >> > >> transactions using a single Client connection. This means adding
> > >> > >> Transaction id parameter to every affected cache operation / SQL
> > >> > statement
> > >> > >> (if applicable) to make sure we do cache operations on relevant
> > >> > >> transaction.
> > >> > >>
> > >> > >> This is how other things work in Ignite, such as communication.
> We do
> > >> > not
> > >> > >> open dozens of connections, we multiplex operations
> asynchronously
> > >> > through
> > >> > >> a single connection.
> > >> > >>
> > >> > >> I think that trying to pool Ignite connections will be highly
> > >> > >> inconvenient,
> > >> > >> since there is no existing infrastructure for such pooling (like
> > >> there
> > >> > >> exists for JDBC).
> > >> > >>
> > >> > >> I want to also ask if "Number of entries participating in
> transaction
> > >> > (may
> > >> > >> be approximate). 0 - default value." is needed. Does it actually
> do
> > >> > >> anything in our tx protocol? Users of existing APIs are already
> > >> confused
> > >> > >> by
> > >> > >> this parameter, if we could get rid of it in thin client
> protocol it
> > >> > would
> > >> > >> be nice clean-up.
> > >> > >>
> > >> > >> Regards,
> > >> > >> --
> > >> > >> Ilya Kasnacheev
> > >> > >>
> > >> > >>
> > >> > >> вт, 2 апр. 2019 г. в 09:55, Pavel Tupitsyn <ptupitsyn@apache.org
> >:
> > >> > >>
> > >> > >> > Alex,
> > >> > >> >
> > >> > >> > > now we can only support one active transaction per connection
> > >> > >> >
> > >> > >> > I totally understand server-side and protocol limitations that
> are
> > >> > >> causing
> > >> > >> > this.
> > >> > >> > But I have no idea how to support this in .NET Thin Client, for
> > >> > example.
> > >> > >> >
> > >> > >> > It is thread-safe and can handle multiple async operations in
> > >> > parallel.
> > >> > >> > But with TX support we have to somehow switch to
> single-threaded
> > >> mode
> > >> > to
> > >> > >> > avoid unexpected effects.
> > >> > >> >
> > >> > >> > Any ideas?
> > >> > >> >
> > >> > >> >
> > >> > >> > On Mon, Apr 1, 2019 at 6:38 PM Alex Plehanov <
> > >> plehanov.alex@gmail.com
> > >> > >
> > >> > >> > wrote:
> > >> > >> >
> > >> > >> > > Dmitriy, thank you!
> > >> > >> > >
> > >> > >> > > Guys, I've created the IEP [1] on wiki, please have a look.
> > >> > >> > >
> > >> > >> > > [1]
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > чт, 28 мар. 2019 г. в 14:33, Dmitriy Pavlov <
> dpavlov@apache.org
> > >> >:
> > >> > >> > >
> > >> > >> > > > Hi,
> > >> > >> > > >
> > >> > >> > > > I've added permissions to account plehanov.alex
> > >> > >> > > >
> > >> > >> > > > Recently Infra integrated Apache LDAP with confluence, so
> it is
> > >> > >> > possible
> > >> > >> > > to
> > >> > >> > > > login using Apache credentials. Probably we can ask infra
> if
> > >> extra
> > >> > >> > > > permissions to edit pages should be added for committers.
> > >> > >> > > >
> > >> > >> > > > Sincerely,
> > >> > >> > > > Dmitriy Pavlov
> > >> > >> > > >
> > >> > >> > > > ср, 27 мар. 2019 г. в 13:37, Alex Plehanov <
> > >> > plehanov.alex@gmail.com
> > >> > >> >:
> > >> > >> > > >
> > >> > >> > > > > Vladimir,
> > >> > >> > > > >
> > >> > >> > > > > About current tx: ok, then we don't need tx() method in
> the
> > >> > >> interface
> > >> > >> > > at
> > >> > >> > > > > all (the same cached transaction info user can store by
> > >> > himself).
> > >> > >> > > > >
> > >> > >> > > > > About decoupling transactions from threads on the server
> > >> side:
> > >> > for
> > >> > >> > now,
> > >> > >> > > > we
> > >> > >> > > > > can start with thread-per-connection approach (we only
> can
> > >> > support
> > >> > >> > one
> > >> > >> > > > > active transaction per connection, see below, so we need
> one
> > >> > >> > additional
> > >> > >> > > > > dedicated thread for each connection with active
> > >> transaction),
> > >> > and
> > >> > >> > > later
> > >> > >> > > > > change server-side internals to process client
> transactions
> > >> in
> > >> > any
> > >> > >> > > server
> > >> > >> > > > > thread (not dedicated to this connection). This change
> will
> > >> not
> > >> > >> > affect
> > >> > >> > > > the
> > >> > >> > > > > thin client protocol, it only affects the server side.
> > >> > >> > > > > In any case, we can't support concurrent transactions per
> > >> > >> connection
> > >> > >> > on
> > >> > >> > > > > the client side without fundamental changes to the
> current
> > >> > >> protocol
> > >> > >> > > > (cache
> > >> > >> > > > > operation doesn't bound to transaction or thread and the
> > >> server
> > >> > >> > doesn't
> > >> > >> > > > > know which thread on the client side do this cache
> > >> operation).
> > >> > In
> > >> > >> my
> > >> > >> > > > > opinion, if a user wants to use concurrent transactions,
> he
> > >> must
> > >> > >> use
> > >> > >> > > > > different connections from a connection pool.
> > >> > >> > > > >
> > >> > >> > > > > About semantics of suspend/resume on the client-side:
> it's
> > >> > >> absolutely
> > >> > >> > > > > different than server-side semantics (we don't need to do
> > >> > >> > > suspend/resume
> > >> > >> > > > to
> > >> > >> > > > > pass transaction between threads on the client-side), but
> > >> can't
> > >> > be
> > >> > >> > > > > implemented efficiently without implemented
> suspend/resume on
> > >> > >> > > > server-side.
> > >> > >> > > > >
> > >> > >> > > > > Can anyone give me permissions to create IEP on Apache
> wiki?
> > >> > >> > > > >
> > >> > >> > > > > ср, 27 мар. 2019 г. в 11:59, Vladimir Ozerov <
> > >> > >> vozerov@gridgain.com>:
> > >> > >> > > > >
> > >> > >> > > > > > Hi Alex,
> > >> > >> > > > > >
> > >> > >> > > > > > My comments was only about the protocol. Getting
> current
> > >> info
> > >> > >> about
> > >> > >> > > > > > transaction should be handled by the client itself. It
> is
> > >> not
> > >> > >> > > protocl's
> > >> > >> > > > > > concern. Same about other APIs and behavior in case
> another
> > >> > >> > > transaction
> > >> > >> > > > > is
> > >> > >> > > > > > attempted from the same thread.
> > >> > >> > > > > >
> > >> > >> > > > > > Putting protocol aside, transaction support is
> complicated
> > >> > >> matter.
> > >> > >> > I
> > >> > >> > > > > would
> > >> > >> > > > > > propose to route through IEP and wide community
> > >> discussion. We
> > >> > >> need
> > >> > >> > > to
> > >> > >> > > > > > review API and semantics very carefully, taking
> > >> SUSPEND/RESUME
> > >> > >> in
> > >> > >> > > > count.
> > >> > >> > > > > > Also I do not see how we support client transactions
> > >> > efficiently
> > >> > >> > > > without
> > >> > >> > > > > > decoupling transactions from threads on the server side
> > >> first.
> > >> > >> > > Because
> > >> > >> > > > > > without it you will need a dedicated server thread for
> > >> every
> > >> > >> > client's
> > >> > >> > > > > > transaction which is slow and may even crash the
> server.
> > >> > >> > > > > >
> > >> > >> > > > > > Vladimir.
> > >> > >> > > > > >
> > >> > >> > > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov <
> > >> > >> > > > plehanov.alex@gmail.com>
> > >> > >> > > > > > wrote:
> > >> > >> > > > > >
> > >> > >> > > > > > > Vladimir, what if we want to get current transaction
> info
> > >> > >> (tx()
> > >> > >> > > > > method)?
> > >> > >> > > > > > >
> > >> > >> > > > > > > Does close() method mapped to TX_END(rollback)?
> > >> > >> > > > > > >
> > >> > >> > > > > > > For example, this code:
> > >> > >> > > > > > >
> > >> > >> > > > > > > try(tx = txStart()) {
> > >> > >> > > > > > >     tx.commit();
> > >> > >> > > > > > > }
> > >> > >> > > > > > >
> > >> > >> > > > > > > Will produce:
> > >> > >> > > > > > > TX_START
> > >> > >> > > > > > > TX_END(commit)
> > >> > >> > > > > > > TX_END(rollback)
> > >> > >> > > > > > >
> > >> > >> > > > > > > Am I understand you right?
> > >> > >> > > > > > >
> > >> > >> > > > > > > About xid. There is yet another proposal. Use some
> unique
> > >> > per
> > >> > >> > > > > connection
> > >> > >> > > > > > id
> > >> > >> > > > > > > (integer, simple counter) for identifying the
> > >> transaction on
> > >> > >> > > > > > > commit/rollback message. The client gets this id
> from the
> > >> > >> server
> > >> > >> > > with
> > >> > >> > > > > > > transaction info and sends it back to the server when
> > >> trying
> > >> > >> to
> > >> > >> > > > > > > commit/rollback transaction. This id is not shown to
> > >> users.
> > >> > >> But
> > >> > >> > > also
> > >> > >> > > > we
> > >> > >> > > > > > can
> > >> > >> > > > > > > pass from server to client real transaction id (xid)
> with
> > >> > >> > > transaction
> > >> > >> > > > > > info
> > >> > >> > > > > > > for diagnostic purposes.
> > >> > >> > > > > > >
> > >> > >> > > > > > > And one more question: what should we do if the
> client
> > >> > starts
> > >> > >> a
> > >> > >> > new
> > >> > >> > > > > > > transaction without ending the old one? Should we
> end the
> > >> > old
> > >> > >> > > > > transaction
> > >> > >> > > > > > > implicitly (rollback) or throw an exception to the
> > >> client?
> > >> > In
> > >> > >> my
> > >> > >> > > > > opinion,
> > >> > >> > > > > > > the first option is better. For example, if we got a
> > >> > >> previously
> > >> > >> > > used
> > >> > >> > > > > > > connection from the connection pool, we should not
> worry
> > >> > about
> > >> > >> > any
> > >> > >> > > > > > > uncompleted transaction started by the previous user
> of
> > >> this
> > >> > >> > > > > connection.
> > >> > >> > > > > > >
> > >> > >> > > > > > > ср, 27 мар. 2019 г. в 11:02, Vladimir Ozerov <
> > >> > >> > vozerov@gridgain.com
> > >> > >> > > >:
> > >> > >> > > > > > >
> > >> > >> > > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not
> support
> > >> > them
> > >> > >> > yet,
> > >> > >> > > > and
> > >> > >> > > > > > > adding
> > >> > >> > > > > > > > them in future should not conflict with simple
> > >> START/END
> > >> > >> > > > > > infrastructure.
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov <
> > >> > >> > > > > vozerov@gridgain.com
> > >> > >> > > > > > >
> > >> > >> > > > > > > > wrote:
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > > Hi Alex,
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > I am not sure we need 5 commands. Wouldn't it be
> > >> enough
> > >> > to
> > >> > >> > have
> > >> > >> > > > > only
> > >> > >> > > > > > > two?
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > START - accepts optional parameters, returns
> > >> transaction
> > >> > >> info
> > >> > >> > > > > > > > > END - provides commit flag, returns void
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > Vladimir.
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov <
> > >> > >> > > > > > plehanov.alex@gmail.com
> > >> > >> > > > > > > >
> > >> > >> > > > > > > > > wrote:
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > > >> Sergey, yes, the close is something like silent
> > >> > rollback.
> > >> > >> > But
> > >> > >> > > we
> > >> > >> > > > > can
> > >> > >> > > > > > > > >> also implement this on the client side, just
> using
> > >> > >> rollback
> > >> > >> > > and
> > >> > >> > > > > > > ignoring
> > >> > >> > > > > > > > >> errors in the response.
> > >> > >> > > > > > > > >>
> > >> > >> > > > > > > > >> ср, 27 мар. 2019 г. в 00:04, Sergey Kozlov <
> > >> > >> > > > skozlov@gridgain.com
> > >> > >> > > > > >:
> > >> > >> > > > > > > > >>
> > >> > >> > > > > > > > >> > Nikolay
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > Am I correctly understand you points:
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> >    - close: rollback
> > >> > >> > > > > > > > >> >    - commit, close: do nothing
> > >> > >> > > > > > > > >> >    - rollback, close: do what? (I suppose
> nothing)
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > Also you assume that after commit/rollback we
> may
> > >> > need
> > >> > >> to
> > >> > >> > > free
> > >> > >> > > > > > some
> > >> > >> > > > > > > > >> > resources on server node(s)or just do on
> client
> > >> > started
> > >> > >> > TX?
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex
> Plehanov <
> > >> > >> > > > > > > > plehanov.alex@gmail.com
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > wrote:
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > > Sergey, we have the close() method in the
> thick
> > >> > >> client,
> > >> > >> > > it's
> > >> > >> > > > > > > > behavior
> > >> > >> > > > > > > > >> is
> > >> > >> > > > > > > > >> > > slightly different than rollback() method
> (it
> > >> > should
> > >> > >> > > > rollback
> > >> > >> > > > > if
> > >> > >> > > > > > > the
> > >> > >> > > > > > > > >> > > transaction is not committed and do nothing
> if
> > >> the
> > >> > >> > > > transaction
> > >> > >> > > > > > is
> > >> > >> > > > > > > > >> already
> > >> > >> > > > > > > > >> > > committed). I think we should support
> > >> > >> try-with-resource
> > >> > >> > > > > > semantics
> > >> > >> > > > > > > in
> > >> > >> > > > > > > > >> the
> > >> > >> > > > > > > > >> > > thin client and OP_TX_CLOSE will be useful
> here.
> > >> > >> > > > > > > > >> > >
> > >> > >> > > > > > > > >> > > Nikolay, suspend/resume didn't work yet for
> > >> > >> pessimistic
> > >> > >> > > > > > > > transactions.
> > >> > >> > > > > > > > >> > Also,
> > >> > >> > > > > > > > >> > > the main goal of suspend/resume operations
> is to
> > >> > >> support
> > >> > >> > > > > > > transaction
> > >> > >> > > > > > > > >> > > passing between threads. In the thin
> client, the
> > >> > >> > > transaction
> > >> > >> > > > > is
> > >> > >> > > > > > > > bound
> > >> > >> > > > > > > > >> to
> > >> > >> > > > > > > > >> > > the client connection, not client thread. I
> > >> think
> > >> > >> > passing
> > >> > >> > > a
> > >> > >> > > > > > > > >> transaction
> > >> > >> > > > > > > > >> > > between different client connections is not
> a
> > >> very
> > >> > >> > useful
> > >> > >> > > > > case.
> > >> > >> > > > > > > > >> > >
> > >> > >> > > > > > > > >> > > вт, 26 мар. 2019 г. в 22:17, Nikolay
> Izhikov <
> > >> > >> > > > > > nizhikov@apache.org
> > >> > >> > > > > > > >:
> > >> > >> > > > > > > > >> > >
> > >> > >> > > > > > > > >> > > > Hello, Alex.
> > >> > >> > > > > > > > >> > > >
> > >> > >> > > > > > > > >> > > > We also have suspend and resume
> operations.
> > >> > >> > > > > > > > >> > > > I think we should support them
> > >> > >> > > > > > > > >> > > >
> > >> > >> > > > > > > > >> > > > вт, 26 марта 2019 г., 22:07 Sergey Kozlov
> <
> > >> > >> > > > > > skozlov@gridgain.com
> > >> > >> > > > > > > >:
> > >> > >> > > > > > > > >> > > >
> > >> > >> > > > > > > > >> > > > > Hi
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > > Looks like I missed something but why we
> > >> need
> > >> > >> > > > OP_TX_CLOSE
> > >> > >> > > > > > > > >> operation?
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > > Also I suggest to reserve a code for
> > >> SAVEPOINT
> > >> > >> > > operation
> > >> > >> > > > > > which
> > >> > >> > > > > > > > >> very
> > >> > >> > > > > > > > >> > > > useful
> > >> > >> > > > > > > > >> > > > > to understand where transaction has been
> > >> rolled
> > >> > >> back
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex
> > >> Plehanov <
> > >> > >> > > > > > > > >> > plehanov.alex@gmail.com
> > >> > >> > > > > > > > >> > > >
> > >> > >> > > > > > > > >> > > > > wrote:
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > > > Hello Igniters!
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > I want to pick up the ticket
> IGNITE-7369
> > >> and
> > >> > >> add
> > >> > >> > > > > > > transactions
> > >> > >> > > > > > > > >> > support
> > >> > >> > > > > > > > >> > > > to
> > >> > >> > > > > > > > >> > > > > > our thin client implementation.
> > >> > >> > > > > > > > >> > > > > > I've looked at our current
> implementation
> > >> and
> > >> > >> have
> > >> > >> > > > some
> > >> > >> > > > > > > > >> proposals
> > >> > >> > > > > > > > >> > to
> > >> > >> > > > > > > > >> > > > > > support transactions:
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > Add new operations to thin client
> > >> protocol:
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > >     OP_TX_GET, 4000, Get current
> > >> transaction
> > >> > >> for
> > >> > >> > > > client
> > >> > >> > > > > > > > >> connection
> > >> > >> > > > > > > > >> > > > > >     OP_TX_START, 4001, Start a new
> > >> > transaction
> > >> > >> > > > > > > > >> > > > > >     OP_TX_COMMIT, 4002, Commit
> transaction
> > >> > >> > > > > > > > >> > > > > >     OP_TX_ROLLBACK, 4003, Rollback
> > >> > transaction
> > >> > >> > > > > > > > >> > > > > >     OP_TX_CLOSE, 4004, Close
> transaction
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > From the client side (java) new
> interfaces
> > >> > >> will be
> > >> > >> > > > > added:
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > public interface ClientTransactions {
> > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> txStart();
> > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> > >> > >> > > > > > txStart(TransactionConcurrency
> > >> > >> > > > > > > > >> > > > concurrency,
> > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation);
> > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> > >> > >> > > > > > txStart(TransactionConcurrency
> > >> > >> > > > > > > > >> > > > concurrency,
> > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation, long
> > >> timeout,
> > >> > >> int
> > >> > >> > > > > txSize);
> > >> > >> > > > > > > > >> > > > > >     public ClientTransaction tx(); //
> Get
> > >> > >> current
> > >> > >> > > > > > connection
> > >> > >> > > > > > > > >> > > > transaction
> > >> > >> > > > > > > > >> > > > > >     public ClientTransactions
> > >> > withLabel(String
> > >> > >> > lb);
> > >> > >> > > > > > > > >> > > > > > }
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > public interface ClientTransaction
> extends
> > >> > >> > > > > AutoCloseable {
> > >> > >> > > > > > > > >> > > > > >     public IgniteUuid xid(); // Do we
> need
> > >> > it?
> > >> > >> > > > > > > > >> > > > > >     public TransactionIsolation
> > >> isolation();
> > >> > >> > > > > > > > >> > > > > >     public TransactionConcurrency
> > >> > >> concurrency();
> > >> > >> > > > > > > > >> > > > > >     public long timeout();
> > >> > >> > > > > > > > >> > > > > >     public String label();
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > >     public void commit();
> > >> > >> > > > > > > > >> > > > > >     public void rollback();
> > >> > >> > > > > > > > >> > > > > >     public void close();
> > >> > >> > > > > > > > >> > > > > > }
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > From the server side, I think as a
> first
> > >> step
> > >> > >> > (while
> > >> > >> > > > > > > > >> transactions
> > >> > >> > > > > > > > >> > > > > > suspend/resume is not fully
> implemented)
> > >> we
> > >> > can
> > >> > >> > use
> > >> > >> > > > the
> > >> > >> > > > > > same
> > >> > >> > > > > > > > >> > approach
> > >> > >> > > > > > > > >> > > > as
> > >> > >> > > > > > > > >> > > > > > for JDBC: add a new worker to each
> > >> > >> > > > ClientRequestHandler
> > >> > >> > > > > > and
> > >> > >> > > > > > > > >> process
> > >> > >> > > > > > > > >> > > > > > requests by this worker if the
> > >> transaction is
> > >> > >> > > started
> > >> > >> > > > > > > > >> explicitly.
> > >> > >> > > > > > > > >> > > > > > ClientRequestHandler is bound to
> client
> > >> > >> > connection,
> > >> > >> > > so
> > >> > >> > > > > > there
> > >> > >> > > > > > > > >> will
> > >> > >> > > > > > > > >> > be
> > >> > >> > > > > > > > >> > > > 1:1
> > >> > >> > > > > > > > >> > > > > > relation between client connection and
> > >> > thread,
> > >> > >> > which
> > >> > >> > > > > > process
> > >> > >> > > > > > > > >> > > operations
> > >> > >> > > > > > > > >> > > > > in
> > >> > >> > > > > > > > >> > > > > > a transaction.
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > Also, there is a couple of issues I
> want
> > >> to
> > >> > >> > discuss:
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > We have overloaded method txStart
> with a
> > >> > >> different
> > >> > >> > > set
> > >> > >> > > > > of
> > >> > >> > > > > > > > >> > arguments.
> > >> > >> > > > > > > > >> > > > Some
> > >> > >> > > > > > > > >> > > > > > of the arguments may be missing. To
> pass
> > >> > >> arguments
> > >> > >> > > > with
> > >> > >> > > > > > > > >> OP_TX_START
> > >> > >> > > > > > > > >> > > > > > operation we have the next options:
> > >> > >> > > > > > > > >> > > > > >  * Serialize full set of arguments
> and use
> > >> > some
> > >> > >> > > value
> > >> > >> > > > > for
> > >> > >> > > > > > > > >> missing
> > >> > >> > > > > > > > >> > > > > > arguments. For example -1 for int/long
> > >> types
> > >> > >> and
> > >> > >> > > null
> > >> > >> > > > > for
> > >> > >> > > > > > > > string
> > >> > >> > > > > > > > >> > > type.
> > >> > >> > > > > > > > >> > > > We
> > >> > >> > > > > > > > >> > > > > > can't use 0 for int/long types since 0
> > >> it's a
> > >> > >> > valid
> > >> > >> > > > > value
> > >> > >> > > > > > > for
> > >> > >> > > > > > > > >> > > > > concurrency,
> > >> > >> > > > > > > > >> > > > > > isolation and timeout arguments.
> > >> > >> > > > > > > > >> > > > > >  * Serialize arguments as a
> collection of
> > >> > >> > > > property-value
> > >> > >> > > > > > > pairs
> > >> > >> > > > > > > > >> > (like
> > >> > >> > > > > > > > >> > > > it's
> > >> > >> > > > > > > > >> > > > > > implemented now for
> CacheConfiguration).
> > >> In
> > >> > >> this
> > >> > >> > > case
> > >> > >> > > > > only
> > >> > >> > > > > > > > >> > explicitly
> > >> > >> > > > > > > > >> > > > > > provided arguments will be serialized.
> > >> > >> > > > > > > > >> > > > > > Which way is better? The simplest
> > >> solution is
> > >> > >> to
> > >> > >> > use
> > >> > >> > > > the
> > >> > >> > > > > > > first
> > >> > >> > > > > > > > >> > option
> > >> > >> > > > > > > > >> > > > > and I
> > >> > >> > > > > > > > >> > > > > > want to use it if there were no
> > >> objections.
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > Do we need transaction id (xid) on the
> > >> client
> > >> > >> > side?
> > >> > >> > > > > > > > >> > > > > > If yes, we can pass xid along with
> > >> > >> OP_TX_COMMIT,
> > >> > >> > > > > > > > OP_TX_ROLLBACK,
> > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE operations back to the
> server
> > >> and
> > >> > >> do
> > >> > >> > > > > > additional
> > >> > >> > > > > > > > >> check
> > >> > >> > > > > > > > >> > on
> > >> > >> > > > > > > > >> > > > the
> > >> > >> > > > > > > > >> > > > > > server side (current transaction id
> for
> > >> > >> connection
> > >> > >> > > ==
> > >> > >> > > > > > > > >> transaction
> > >> > >> > > > > > > > >> > id
> > >> > >> > > > > > > > >> > > > > passed
> > >> > >> > > > > > > > >> > > > > > from client side). This, perhaps, will
> > >> > protect
> > >> > >> > > clients
> > >> > >> > > > > > > against
> > >> > >> > > > > > > > >> some
> > >> > >> > > > > > > > >> > > > > errors
> > >> > >> > > > > > > > >> > > > > > (for example when client try to commit
> > >> > outdated
> > >> > >> > > > > > > transaction).
> > >> > >> > > > > > > > >> But
> > >> > >> > > > > > > > >> > > > > > currently, we don't have data type
> > >> IgniteUuid
> > >> > >> in
> > >> > >> > > thin
> > >> > >> > > > > > client
> > >> > >> > > > > > > > >> > > protocol.
> > >> > >> > > > > > > > >> > > > Do
> > >> > >> > > > > > > > >> > > > > > we need to add it too?
> > >> > >> > > > > > > > >> > > > > > Also, we can pass xid as a string
> just to
> > >> > >> inform
> > >> > >> > the
> > >> > >> > > > > > client
> > >> > >> > > > > > > > and
> > >> > >> > > > > > > > >> do
> > >> > >> > > > > > > > >> > > not
> > >> > >> > > > > > > > >> > > > > pass
> > >> > >> > > > > > > > >> > > > > > it back to the server with
> commit/rollback
> > >> > >> > > operation.
> > >> > >> > > > > > > > >> > > > > > Or not to pass xid at all (.NET thick
> > >> client
> > >> > >> works
> > >> > >> > > > this
> > >> > >> > > > > > way
> > >> > >> > > > > > > as
> > >> > >> > > > > > > > >> far
> > >> > >> > > > > > > > >> > > as I
> > >> > >> > > > > > > > >> > > > > > know).
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > What do you think?
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > ср, 7 мар. 2018 г. в 16:22, Vladimir
> > >> Ozerov <
> > >> > >> > > > > > > > >> vozerov@gridgain.com
> > >> > >> > > > > > > > >> > >:
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > > > > We already have transactions
> support in
> > >> > JDBC
> > >> > >> > > driver
> > >> > >> > > > in
> > >> > >> > > > > > TX
> > >> > >> > > > > > > > SQL
> > >> > >> > > > > > > > >> > > branch
> > >> > >> > > > > > > > >> > > > > > > (ignite-4191). Currently it is
> > >> implemented
> > >> > >> > through
> > >> > >> > > > > > > separate
> > >> > >> > > > > > > > >> > thread,
> > >> > >> > > > > > > > >> > > > > which
> > >> > >> > > > > > > > >> > > > > > > is not that efficient. Ideally we
> need
> > >> to
> > >> > >> finish
> > >> > >> > > > > > > decoupling
> > >> > >> > > > > > > > >> > > > > transactions
> > >> > >> > > > > > > > >> > > > > > > from threads. But alternatively we
> can
> > >> > change
> > >> > >> > the
> > >> > >> > > > > logic
> > >> > >> > > > > > on
> > >> > >> > > > > > > > >> how we
> > >> > >> > > > > > > > >> > > > > assign
> > >> > >> > > > > > > > >> > > > > > > thread ID to specific transaction
> and
> > >> > >> > > "impersonate"
> > >> > >> > > > > thin
> > >> > >> > > > > > > > >> client
> > >> > >> > > > > > > > >> > > > worker
> > >> > >> > > > > > > > >> > > > > > > threads when serving requests from
> > >> multiple
> > >> > >> > users.
> > >> > >> > > > > > > > >> > > > > > >
> > >> > >> > > > > > > > >> > > > > > >
> > >> > >> > > > > > > > >> > > > > > >
> > >> > >> > > > > > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM,
> Denis
> > >> > Magda
> > >> > >> <
> > >> > >> > > > > > > > >> dmagda@apache.org>
> > >> > >> > > > > > > > >> > > > > wrote:
> > >> > >> > > > > > > > >> > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > Here is an original discussion
> with a
> > >> > >> > reference
> > >> > >> > > to
> > >> > >> > > > > the
> > >> > >> > > > > > > > JIRA
> > >> > >> > > > > > > > >> > > ticket:
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > http://apache-ignite-developers.2346864.n4.nabble
> > >> > >> > > > .
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > >
> > >> > com/Re-Transaction-operations-using-the-Ignite-Thin-Client-
> > >> > >> > > > > > > > >> > > > > > > > Protocol-td25914.html
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > --
> > >> > >> > > > > > > > >> > > > > > > > Denis
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18 AM,
> > >> Dmitriy
> > >> > >> > > Setrakyan
> > >> > >> > > > <
> > >> > >> > > > > > > > >> > > > > > dsetrakyan@apache.org
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > wrote:
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > Hi Dmitriy. I don't think we
> have a
> > >> > >> design
> > >> > >> > > > > proposal
> > >> > >> > > > > > > for
> > >> > >> > > > > > > > >> > > > transaction
> > >> > >> > > > > > > > >> > > > > > > > support
> > >> > >> > > > > > > > >> > > > > > > > > in thin clients. Do you mind
> taking
> > >> > this
> > >> > >> > > > > initiative
> > >> > >> > > > > > > and
> > >> > >> > > > > > > > >> > > creating
> > >> > >> > > > > > > > >> > > > an
> > >> > >> > > > > > > > >> > > > > > IEP
> > >> > >> > > > > > > > >> > > > > > > > on
> > >> > >> > > > > > > > >> > > > > > > > > Wiki?
> > >> > >> > > > > > > > >> > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > D.
> > >> > >> > > > > > > > >> > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > On Tue, Mar 6, 2018 at 8:46 AM,
> > >> Dmitriy
> > >> > >> > > > > Govorukhin <
> > >> > >> > > > > > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com>
> > >> wrote:
> > >> > >> > > > > > > > >> > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > > Hi, Igniters.
> > >> > >> > > > > > > > >> > > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > > I've seen a lot of discussions
> > >> about
> > >> > >> thin
> > >> > >> > > > client
> > >> > >> > > > > > and
> > >> > >> > > > > > > > >> binary
> > >> > >> > > > > > > > >> > > > > > protocol,
> > >> > >> > > > > > > > >> > > > > > > > > but I
> > >> > >> > > > > > > > >> > > > > > > > > > did not hear anything about
> > >> > >> transactions
> > >> > >> > > > > support.
> > >> > >> > > > > > Do
> > >> > >> > > > > > > > we
> > >> > >> > > > > > > > >> > have
> > >> > >> > > > > > > > >> > > > some
> > >> > >> > > > > > > > >> > > > > > > draft
> > >> > >> > > > > > > > >> > > > > > > > > for
> > >> > >> > > > > > > > >> > > > > > > > > > this purpose?
> > >> > >> > > > > > > > >> > > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > > As I understand we have
> several
> > >> > >> problems:
> > >> > >> > > > > > > > >> > > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > >    - thread and transaction
> have
> > >> hard
> > >> > >> > > related
> > >> > >> > > > > (we
> > >> > >> > > > > > > use
> > >> > >> > > > > > > > >> > > > > thread-local
> > >> > >> > > > > > > > >> > > > > > > > > variable
> > >> > >> > > > > > > > >> > > > > > > > > >    and thread name)
> > >> > >> > > > > > > > >> > > > > > > > > >    - we can process only one
> > >> > >> transaction
> > >> > >> > at
> > >> > >> > > > the
> > >> > >> > > > > > same
> > >> > >> > > > > > > > >> time
> > >> > >> > > > > > > > >> > in
> > >> > >> > > > > > > > >> > > > one
> > >> > >> > > > > > > > >> > > > > > > thread
> > >> > >> > > > > > > > >> > > > > > > > > (it
> > >> > >> > > > > > > > >> > > > > > > > > >    mean we need hold thread
> per
> > >> > >> client. If
> > >> > >> > > > > connect
> > >> > >> > > > > > > 100
> > >> > >> > > > > > > > >> thin
> > >> > >> > > > > > > > >> > > > > clients
> > >> > >> > > > > > > > >> > > > > > > to
> > >> > >> > > > > > > > >> > > > > > > > 1
> > >> > >> > > > > > > > >> > > > > > > > > >    server node, then need to
> hold
> > >> 100
> > >> > >> > thread
> > >> > >> > > > on
> > >> > >> > > > > > the
> > >> > >> > > > > > > > >> server
> > >> > >> > > > > > > > >> > > > side)
> > >> > >> > > > > > > > >> > > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > > > Let's discuss how we can
> implement
> > >> > >> > > > transactions
> > >> > >> > > > > > for
> > >> > >> > > > > > > > the
> > >> > >> > > > > > > > >> > thin
> > >> > >> > > > > > > > >> > > > > > client.
> > >> > >> > > > > > > > >> > > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > > >
> > >> > >> > > > > > > > >> > > > > > > >
> > >> > >> > > > > > > > >> > > > > > >
> > >> > >> > > > > > > > >> > > > > >
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > > > --
> > >> > >> > > > > > > > >> > > > > Sergey Kozlov
> > >> > >> > > > > > > > >> > > > > GridGain Systems
> > >> > >> > > > > > > > >> > > > > www.gridgain.com
> > >> > >> > > > > > > > >> > > > >
> > >> > >> > > > > > > > >> > > >
> > >> > >> > > > > > > > >> > >
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >> > --
> > >> > >> > > > > > > > >> > Sergey Kozlov
> > >> > >> > > > > > > > >> > GridGain Systems
> > >> > >> > > > > > > > >> > www.gridgain.com
> > >> > >> > > > > > > > >> >
> > >> > >> > > > > > > > >>
> > >> > >> > > > > > > > >
> > >> > >> > > > > > > >
> > >> > >> > > > > > >
> > >> > >> > > > > >
> > >> > >> > > > >
> > >> > >> > > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message