ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Sapego <isap...@apache.org>
Subject Re: Thin client: transactions support
Date Fri, 17 May 2019 15:49:07 GMT
Hi,

I had a look at IEP and have several comments:

1. Why would one want to use to use server's default values for Concurrency
or Isolation?
I believe, client should have its own defaults which should be explicitly
documented, so that
behaviour of transactions will not depend on the server node it was routed
to. The same goes
for timeout.

2. Not sure about transaction ID represented by int. Why not to use
IgniteUuid? It should simplify
server code. Also it may help with monitoring/debugging if thin clients and
server nodes use the
same identifier for transactions. It does not seem as a big overhead to me
either.

3. Maybe it makes sense to add "In transaction" boolean flag to cache
operation request header
to avoid bloating message size in non-affected scenarios.

Best Regards,
Igor


On Mon, May 13, 2019 at 1:58 PM Alex Plehanov <plehanov.alex@gmail.com>
wrote:

> Hi, Ivan.
>
> Thanks for your comments.
>
> 1. Transaction id in thin client protocol it's just a tx counter for the
> current connection. It's not related to GridCacheVersion. If we want to
> know GridCacheVersion on the client side, I think we should introduce a new
> type of operation (for example OP_TX_INFO).
> 2. Error handling is already provided by thin client protocol, even in case
> of empty response. Of course, the client will know if there is a failure
> occurred during OP_TX_END operation.
> 3. AFAIK some of thin client implementations already send requests and
> process responses in an async way (.NET for example). As for java thin
> client, in the current implementation channel is locked exclusively before
> request send and until the response is processed. I have some ideas about
> how to fix this (split send/receive process into two different parts and
> acquire locks for this parts separately or create futures on request sent
> and complete it after processing the response in a dedicated thread), I've
> created ticket [1] for this issue and will try to implement prototype in a
> couple of days.
>
> About suspend/resume, yes, on server-side we should resume tx before each
> transactional cache operation and suspend the tx after the operation. In my
> opinion, suspend/resume approach have several advantages over approach with
> explicit tx id argument:
> - Introducing explicit tx id argument for cache operations leads to a
> significant API change
> - It's not clear how to use it together with current (tx-per-thread)
> approach (for example, what if a thread is already held transaction and
> someone call cache operation with explicit tx id?)
> - Suspend/resume feature will also be useful for thick clients
> - Suspend/resume functionality is already partially implemented (for
> optimistic transactions only)
>
> [1] https://issues.apache.org/jira/browse/IGNITE-11685
>
> пт, 3 мая 2019 г. в 08:10, Павлухин Иван <vololo100@gmail.com>:
>
> > Hi Alex,
> >
> > I went through IEP [1] and I have a couple of questions:
> > 1. What is going to be used as transaction id? In a described protocol
> > I see an int field for it. Should not it be GridCacheVersion
> > corresponding to IgniteInternalTx#xidVersion?
> > 2. OP_TX_END message assumes an empty response, but I think that
> > errors during tx finish are possible and should be returned in a
> > response.
> > 3. In IEP it is stated that async processing of lock operations should
> > be introduced on a client side to enable concurrent operations from
> > different client threads. Do you have an idea how to achieve it?
> >
> > Also, a bit about a suspend/resume trait. I tried to think about it
> > leaving away an existing transactions implementation in Ignite. As I
> > understood we are going to resume a tx before each cache operation in
> > the tx and resume the tx after the operation. All this to make an
> > executing thread available for other operations (e.g. in other txs).
> > From the first glance it seems like an inversed logic. A
> > straightforward way is to execute a cache operation within a
> > particular transaction defined as an explicit tx id argument (e.g.
> > cache.put(key, value, txid)). Can we do so?
> >
> > And leaving for now thin client API. I cannot say that one proposed in
> > IEP is good or bad. I can only say that it ressembles current thick
> > client API. And perhaps it should not. I think that we should consider
> > similar APIs provided by other vendors and keep in mind that we have a
> > bunch of client implementations for different languages. I suppose
> > that we can return to it a little bit later. And I hope that we will
> > do it.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> >
> > вт, 30 апр. 2019 г. в 13:24, Alex Plehanov <plehanov.alex@gmail.com>:
> > >
> > > Hello, Igniters!
> > >
> > > I've update IEP [1] and implement PoC according to new approach
> (multiple
> > > concurrent transactions per connection).
> > > But to move forward another feature need to be implemented:
> > suspend/resume
> > > for pessimistic transactions (IGNITE-5714 [2]). Implementation of
> > > suspend/resume is ready now and ticket in 'Patch available' status. Can
> > any
> > > transactions expert help with review of IGNITE-5714?
> > >
> > > [1]:
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > > [2]: https://issues.apache.org/jira/browse/IGNITE-5714
> > >
> > > чт, 4 апр. 2019 г. в 11:32, Alex Plehanov <plehanov.alex@gmail.com>:
> > >
> > > > Vladimir,
> > > >
> > > > Ok, then I will rewrite IEP in the near future.
> > > >
> > > > чт, 4 апр. 2019 г. в 11:14, Vladimir Ozerov <vozerov@gridgain.com>:
> > > >
> > > >> Hi Alex,
> > > >>
> > > >> I think we should be able to handle many transactions through a
> single
> > > >> connection. This will make our protocol and client implementations
> > much
> > > >> more efficient, and simplicity from developer's perspective is not
> our
> > > >> goal. Consider normal nodes. We have server nodes and client nodes.
> > You
> > > >> may
> > > >> span whatever number of transactions you need, but all of them are
> > > >> coordinated through a single connection. The same should be
> > applicable to
> > > >> thin clients. Protocol is already designed to handle this, as we
> pass
> > > >> unique operation ID in order to distinguish one operation from
> > another. It
> > > >> is true, though, that we will have to introduce a kind of "session"
> > > >> concept, and pass additional identifier along with cache operations,
> > but
> > > >> this doesn't sound like a problem to me.
> > > >>
> > > >> And provided that currently server-side transactions are bound to
> > threads
> > > >> artificially, I would say that the first step in implementation of
> > > >> transactions on thin clients should be decoupling server-side
> > transactions
> > > >> from threads. Without this we will have very inefficient
> > implementation,
> > > >> when every new client transaction have to spawn a new thread. This
> is
> > slow
> > > >> and introduces high memory pressure on a cluster node. We already
> work
> > > >> this
> > > >> way for MVCC transactions which are spawned from JDBC driver, and
> > believe
> > > >> me, we do not want to replicated this bad practice to other clients
> > :-)
> > > >>
> > > >> Vladimir.
> > > >>
> > > >> On Thu, Apr 4, 2019 at 10:08 AM Alex Plehanov <
> > plehanov.alex@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Guys, so, do we need multiple concurrent transactions per
> > connection?
> > > >> >
> > > >> > There are pros and cons for each approach. Difference between
> > > >> approaches:
> > > >> >
> > > >> > One transaction at a time per connection:
> > > >> >  - This approach is used in RDBMS world and users got used to it
> > > >> >  - To use transactions concurrently users need to use different
> > > >> connections
> > > >> > and get these connections via something like a connection pool
> > > >> >  - Easy to implement (in fact, PoC is already done)
> > > >> >
> > > >> > Multiple concurrent transactions per connection:
> > > >> >  - At least for java thin client, we can implement transaction per
> > > >> thread
> > > >> > approach as implemented now for the thick client (perhaps other
> thin
> > > >> > clients can implement the same abstraction)
> > > >> >  - There is also protocol change for all cache operations needed
> (to
> > > >> bind
> > > >> > cache operation to the transaction)
> > > >> >  - Significant changes to all implemented clients are needed
> > > >> >  - Implementation on the server side is more complex
> > > >> >
> > > >> > What do you think?
> > > >> >
> > > >> >
> > > >> > вт, 2 апр. 2019 г. в 16:29, Alex Plehanov <
> plehanov.alex@gmail.com
> > >:
> > > >> >
> > > >> > > Ilya,
> > > >> > >
> > > >> > > > We should be able to multiplex several transactions using a
> > single
> > > >> > > Client connection.
> > > >> > > In this case, we should significantly change cache operations
> > syntax
> > > >> (for
> > > >> > > each implemented client), to bind each operation to the
> > transaction.
> > > >> > >
> > > >> > > > I want to also ask if "Number of entries participating in
> > > >> transaction
> > > >> > > (may be approximate). 0 - default value." is needed.
> > > >> > > I've tried to minimize API changes between thick and thin client
> > to
> > > >> > > simplify move from one to another. It's the only reason. But I
> > agree
> > > >> with
> > > >> > > you, the parameter is not very useful.
> > > >> > >
> > > >> > >
> > > >> > > вт, 2 апр. 2019 г. в 14:48, Ilya Kasnacheev <
> > > >> ilya.kasnacheev@gmail.com>:
> > > >> > >
> > > >> > >> Hello!
> > > >> > >>
> > > >> > >> Pavel, I agree with you thorougly. We should be able to
> multiplex
> > > >> > several
> > > >> > >> transactions using a single Client connection. This means
> adding
> > > >> > >> Transaction id parameter to every affected cache operation /
> SQL
> > > >> > statement
> > > >> > >> (if applicable) to make sure we do cache operations on relevant
> > > >> > >> transaction.
> > > >> > >>
> > > >> > >> This is how other things work in Ignite, such as communication.
> > We do
> > > >> > not
> > > >> > >> open dozens of connections, we multiplex operations
> > asynchronously
> > > >> > through
> > > >> > >> a single connection.
> > > >> > >>
> > > >> > >> I think that trying to pool Ignite connections will be highly
> > > >> > >> inconvenient,
> > > >> > >> since there is no existing infrastructure for such pooling
> (like
> > > >> there
> > > >> > >> exists for JDBC).
> > > >> > >>
> > > >> > >> I want to also ask if "Number of entries participating in
> > transaction
> > > >> > (may
> > > >> > >> be approximate). 0 - default value." is needed. Does it
> actually
> > do
> > > >> > >> anything in our tx protocol? Users of existing APIs are already
> > > >> confused
> > > >> > >> by
> > > >> > >> this parameter, if we could get rid of it in thin client
> > protocol it
> > > >> > would
> > > >> > >> be nice clean-up.
> > > >> > >>
> > > >> > >> Regards,
> > > >> > >> --
> > > >> > >> Ilya Kasnacheev
> > > >> > >>
> > > >> > >>
> > > >> > >> вт, 2 апр. 2019 г. в 09:55, Pavel Tupitsyn <
> ptupitsyn@apache.org
> > >:
> > > >> > >>
> > > >> > >> > Alex,
> > > >> > >> >
> > > >> > >> > > now we can only support one active transaction per
> connection
> > > >> > >> >
> > > >> > >> > I totally understand server-side and protocol limitations
> that
> > are
> > > >> > >> causing
> > > >> > >> > this.
> > > >> > >> > But I have no idea how to support this in .NET Thin Client,
> for
> > > >> > example.
> > > >> > >> >
> > > >> > >> > It is thread-safe and can handle multiple async operations in
> > > >> > parallel.
> > > >> > >> > But with TX support we have to somehow switch to
> > single-threaded
> > > >> mode
> > > >> > to
> > > >> > >> > avoid unexpected effects.
> > > >> > >> >
> > > >> > >> > Any ideas?
> > > >> > >> >
> > > >> > >> >
> > > >> > >> > On Mon, Apr 1, 2019 at 6:38 PM Alex Plehanov <
> > > >> plehanov.alex@gmail.com
> > > >> > >
> > > >> > >> > wrote:
> > > >> > >> >
> > > >> > >> > > Dmitriy, thank you!
> > > >> > >> > >
> > > >> > >> > > Guys, I've created the IEP [1] on wiki, please have a look.
> > > >> > >> > >
> > > >> > >> > > [1]
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > чт, 28 мар. 2019 г. в 14:33, Dmitriy Pavlov <
> > dpavlov@apache.org
> > > >> >:
> > > >> > >> > >
> > > >> > >> > > > Hi,
> > > >> > >> > > >
> > > >> > >> > > > I've added permissions to account plehanov.alex
> > > >> > >> > > >
> > > >> > >> > > > Recently Infra integrated Apache LDAP with confluence, so
> > it is
> > > >> > >> > possible
> > > >> > >> > > to
> > > >> > >> > > > login using Apache credentials. Probably we can ask infra
> > if
> > > >> extra
> > > >> > >> > > > permissions to edit pages should be added for committers.
> > > >> > >> > > >
> > > >> > >> > > > Sincerely,
> > > >> > >> > > > Dmitriy Pavlov
> > > >> > >> > > >
> > > >> > >> > > > ср, 27 мар. 2019 г. в 13:37, Alex Plehanov <
> > > >> > plehanov.alex@gmail.com
> > > >> > >> >:
> > > >> > >> > > >
> > > >> > >> > > > > Vladimir,
> > > >> > >> > > > >
> > > >> > >> > > > > About current tx: ok, then we don't need tx() method in
> > the
> > > >> > >> interface
> > > >> > >> > > at
> > > >> > >> > > > > all (the same cached transaction info user can store by
> > > >> > himself).
> > > >> > >> > > > >
> > > >> > >> > > > > About decoupling transactions from threads on the
> server
> > > >> side:
> > > >> > for
> > > >> > >> > now,
> > > >> > >> > > > we
> > > >> > >> > > > > can start with thread-per-connection approach (we only
> > can
> > > >> > support
> > > >> > >> > one
> > > >> > >> > > > > active transaction per connection, see below, so we
> need
> > one
> > > >> > >> > additional
> > > >> > >> > > > > dedicated thread for each connection with active
> > > >> transaction),
> > > >> > and
> > > >> > >> > > later
> > > >> > >> > > > > change server-side internals to process client
> > transactions
> > > >> in
> > > >> > any
> > > >> > >> > > server
> > > >> > >> > > > > thread (not dedicated to this connection). This change
> > will
> > > >> not
> > > >> > >> > affect
> > > >> > >> > > > the
> > > >> > >> > > > > thin client protocol, it only affects the server side.
> > > >> > >> > > > > In any case, we can't support concurrent transactions
> per
> > > >> > >> connection
> > > >> > >> > on
> > > >> > >> > > > > the client side without fundamental changes to the
> > current
> > > >> > >> protocol
> > > >> > >> > > > (cache
> > > >> > >> > > > > operation doesn't bound to transaction or thread and
> the
> > > >> server
> > > >> > >> > doesn't
> > > >> > >> > > > > know which thread on the client side do this cache
> > > >> operation).
> > > >> > In
> > > >> > >> my
> > > >> > >> > > > > opinion, if a user wants to use concurrent
> transactions,
> > he
> > > >> must
> > > >> > >> use
> > > >> > >> > > > > different connections from a connection pool.
> > > >> > >> > > > >
> > > >> > >> > > > > About semantics of suspend/resume on the client-side:
> > it's
> > > >> > >> absolutely
> > > >> > >> > > > > different than server-side semantics (we don't need to
> do
> > > >> > >> > > suspend/resume
> > > >> > >> > > > to
> > > >> > >> > > > > pass transaction between threads on the client-side),
> but
> > > >> can't
> > > >> > be
> > > >> > >> > > > > implemented efficiently without implemented
> > suspend/resume on
> > > >> > >> > > > server-side.
> > > >> > >> > > > >
> > > >> > >> > > > > Can anyone give me permissions to create IEP on Apache
> > wiki?
> > > >> > >> > > > >
> > > >> > >> > > > > ср, 27 мар. 2019 г. в 11:59, Vladimir Ozerov <
> > > >> > >> vozerov@gridgain.com>:
> > > >> > >> > > > >
> > > >> > >> > > > > > Hi Alex,
> > > >> > >> > > > > >
> > > >> > >> > > > > > My comments was only about the protocol. Getting
> > current
> > > >> info
> > > >> > >> about
> > > >> > >> > > > > > transaction should be handled by the client itself.
> It
> > is
> > > >> not
> > > >> > >> > > protocl's
> > > >> > >> > > > > > concern. Same about other APIs and behavior in case
> > another
> > > >> > >> > > transaction
> > > >> > >> > > > > is
> > > >> > >> > > > > > attempted from the same thread.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Putting protocol aside, transaction support is
> > complicated
> > > >> > >> matter.
> > > >> > >> > I
> > > >> > >> > > > > would
> > > >> > >> > > > > > propose to route through IEP and wide community
> > > >> discussion. We
> > > >> > >> need
> > > >> > >> > > to
> > > >> > >> > > > > > review API and semantics very carefully, taking
> > > >> SUSPEND/RESUME
> > > >> > >> in
> > > >> > >> > > > count.
> > > >> > >> > > > > > Also I do not see how we support client transactions
> > > >> > efficiently
> > > >> > >> > > > without
> > > >> > >> > > > > > decoupling transactions from threads on the server
> side
> > > >> first.
> > > >> > >> > > Because
> > > >> > >> > > > > > without it you will need a dedicated server thread
> for
> > > >> every
> > > >> > >> > client's
> > > >> > >> > > > > > transaction which is slow and may even crash the
> > server.
> > > >> > >> > > > > >
> > > >> > >> > > > > > Vladimir.
> > > >> > >> > > > > >
> > > >> > >> > > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov <
> > > >> > >> > > > plehanov.alex@gmail.com>
> > > >> > >> > > > > > wrote:
> > > >> > >> > > > > >
> > > >> > >> > > > > > > Vladimir, what if we want to get current
> transaction
> > info
> > > >> > >> (tx()
> > > >> > >> > > > > method)?
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > Does close() method mapped to TX_END(rollback)?
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > For example, this code:
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > try(tx = txStart()) {
> > > >> > >> > > > > > >     tx.commit();
> > > >> > >> > > > > > > }
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > Will produce:
> > > >> > >> > > > > > > TX_START
> > > >> > >> > > > > > > TX_END(commit)
> > > >> > >> > > > > > > TX_END(rollback)
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > Am I understand you right?
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > About xid. There is yet another proposal. Use some
> > unique
> > > >> > per
> > > >> > >> > > > > connection
> > > >> > >> > > > > > id
> > > >> > >> > > > > > > (integer, simple counter) for identifying the
> > > >> transaction on
> > > >> > >> > > > > > > commit/rollback message. The client gets this id
> > from the
> > > >> > >> server
> > > >> > >> > > with
> > > >> > >> > > > > > > transaction info and sends it back to the server
> when
> > > >> trying
> > > >> > >> to
> > > >> > >> > > > > > > commit/rollback transaction. This id is not shown
> to
> > > >> users.
> > > >> > >> But
> > > >> > >> > > also
> > > >> > >> > > > we
> > > >> > >> > > > > > can
> > > >> > >> > > > > > > pass from server to client real transaction id
> (xid)
> > with
> > > >> > >> > > transaction
> > > >> > >> > > > > > info
> > > >> > >> > > > > > > for diagnostic purposes.
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > And one more question: what should we do if the
> > client
> > > >> > starts
> > > >> > >> a
> > > >> > >> > new
> > > >> > >> > > > > > > transaction without ending the old one? Should we
> > end the
> > > >> > old
> > > >> > >> > > > > transaction
> > > >> > >> > > > > > > implicitly (rollback) or throw an exception to the
> > > >> client?
> > > >> > In
> > > >> > >> my
> > > >> > >> > > > > opinion,
> > > >> > >> > > > > > > the first option is better. For example, if we got
> a
> > > >> > >> previously
> > > >> > >> > > used
> > > >> > >> > > > > > > connection from the connection pool, we should not
> > worry
> > > >> > about
> > > >> > >> > any
> > > >> > >> > > > > > > uncompleted transaction started by the previous
> user
> > of
> > > >> this
> > > >> > >> > > > > connection.
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > ср, 27 мар. 2019 г. в 11:02, Vladimir Ozerov <
> > > >> > >> > vozerov@gridgain.com
> > > >> > >> > > >:
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not
> > support
> > > >> > them
> > > >> > >> > yet,
> > > >> > >> > > > and
> > > >> > >> > > > > > > adding
> > > >> > >> > > > > > > > them in future should not conflict with simple
> > > >> START/END
> > > >> > >> > > > > > infrastructure.
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov
> <
> > > >> > >> > > > > vozerov@gridgain.com
> > > >> > >> > > > > > >
> > > >> > >> > > > > > > > wrote:
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > > Hi Alex,
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > I am not sure we need 5 commands. Wouldn't it
> be
> > > >> enough
> > > >> > to
> > > >> > >> > have
> > > >> > >> > > > > only
> > > >> > >> > > > > > > two?
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > START - accepts optional parameters, returns
> > > >> transaction
> > > >> > >> info
> > > >> > >> > > > > > > > > END - provides commit flag, returns void
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > Vladimir.
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov <
> > > >> > >> > > > > > plehanov.alex@gmail.com
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > > > > wrote:
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > > >> Sergey, yes, the close is something like
> silent
> > > >> > rollback.
> > > >> > >> > But
> > > >> > >> > > we
> > > >> > >> > > > > can
> > > >> > >> > > > > > > > >> also implement this on the client side, just
> > using
> > > >> > >> rollback
> > > >> > >> > > and
> > > >> > >> > > > > > > ignoring
> > > >> > >> > > > > > > > >> errors in the response.
> > > >> > >> > > > > > > > >>
> > > >> > >> > > > > > > > >> ср, 27 мар. 2019 г. в 00:04, Sergey Kozlov <
> > > >> > >> > > > skozlov@gridgain.com
> > > >> > >> > > > > >:
> > > >> > >> > > > > > > > >>
> > > >> > >> > > > > > > > >> > Nikolay
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > Am I correctly understand you points:
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> >    - close: rollback
> > > >> > >> > > > > > > > >> >    - commit, close: do nothing
> > > >> > >> > > > > > > > >> >    - rollback, close: do what? (I suppose
> > nothing)
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > Also you assume that after commit/rollback
> we
> > may
> > > >> > need
> > > >> > >> to
> > > >> > >> > > free
> > > >> > >> > > > > > some
> > > >> > >> > > > > > > > >> > resources on server node(s)or just do on
> > client
> > > >> > started
> > > >> > >> > TX?
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex
> > Plehanov <
> > > >> > >> > > > > > > > plehanov.alex@gmail.com
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > wrote:
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > > Sergey, we have the close() method in the
> > thick
> > > >> > >> client,
> > > >> > >> > > it's
> > > >> > >> > > > > > > > behavior
> > > >> > >> > > > > > > > >> is
> > > >> > >> > > > > > > > >> > > slightly different than rollback() method
> > (it
> > > >> > should
> > > >> > >> > > > rollback
> > > >> > >> > > > > if
> > > >> > >> > > > > > > the
> > > >> > >> > > > > > > > >> > > transaction is not committed and do
> nothing
> > if
> > > >> the
> > > >> > >> > > > transaction
> > > >> > >> > > > > > is
> > > >> > >> > > > > > > > >> already
> > > >> > >> > > > > > > > >> > > committed). I think we should support
> > > >> > >> try-with-resource
> > > >> > >> > > > > > semantics
> > > >> > >> > > > > > > in
> > > >> > >> > > > > > > > >> the
> > > >> > >> > > > > > > > >> > > thin client and OP_TX_CLOSE will be useful
> > here.
> > > >> > >> > > > > > > > >> > >
> > > >> > >> > > > > > > > >> > > Nikolay, suspend/resume didn't work yet
> for
> > > >> > >> pessimistic
> > > >> > >> > > > > > > > transactions.
> > > >> > >> > > > > > > > >> > Also,
> > > >> > >> > > > > > > > >> > > the main goal of suspend/resume operations
> > is to
> > > >> > >> support
> > > >> > >> > > > > > > transaction
> > > >> > >> > > > > > > > >> > > passing between threads. In the thin
> > client, the
> > > >> > >> > > transaction
> > > >> > >> > > > > is
> > > >> > >> > > > > > > > bound
> > > >> > >> > > > > > > > >> to
> > > >> > >> > > > > > > > >> > > the client connection, not client thread.
> I
> > > >> think
> > > >> > >> > passing
> > > >> > >> > > a
> > > >> > >> > > > > > > > >> transaction
> > > >> > >> > > > > > > > >> > > between different client connections is
> not
> > a
> > > >> very
> > > >> > >> > useful
> > > >> > >> > > > > case.
> > > >> > >> > > > > > > > >> > >
> > > >> > >> > > > > > > > >> > > вт, 26 мар. 2019 г. в 22:17, Nikolay
> > Izhikov <
> > > >> > >> > > > > > nizhikov@apache.org
> > > >> > >> > > > > > > >:
> > > >> > >> > > > > > > > >> > >
> > > >> > >> > > > > > > > >> > > > Hello, Alex.
> > > >> > >> > > > > > > > >> > > >
> > > >> > >> > > > > > > > >> > > > We also have suspend and resume
> > operations.
> > > >> > >> > > > > > > > >> > > > I think we should support them
> > > >> > >> > > > > > > > >> > > >
> > > >> > >> > > > > > > > >> > > > вт, 26 марта 2019 г., 22:07 Sergey
> Kozlov
> > <
> > > >> > >> > > > > > skozlov@gridgain.com
> > > >> > >> > > > > > > >:
> > > >> > >> > > > > > > > >> > > >
> > > >> > >> > > > > > > > >> > > > > Hi
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > > Looks like I missed something but why
> we
> > > >> need
> > > >> > >> > > > OP_TX_CLOSE
> > > >> > >> > > > > > > > >> operation?
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > > Also I suggest to reserve a code for
> > > >> SAVEPOINT
> > > >> > >> > > operation
> > > >> > >> > > > > > which
> > > >> > >> > > > > > > > >> very
> > > >> > >> > > > > > > > >> > > > useful
> > > >> > >> > > > > > > > >> > > > > to understand where transaction has
> been
> > > >> rolled
> > > >> > >> back
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex
> > > >> Plehanov <
> > > >> > >> > > > > > > > >> > plehanov.alex@gmail.com
> > > >> > >> > > > > > > > >> > > >
> > > >> > >> > > > > > > > >> > > > > wrote:
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > > > Hello Igniters!
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > I want to pick up the ticket
> > IGNITE-7369
> > > >> and
> > > >> > >> add
> > > >> > >> > > > > > > transactions
> > > >> > >> > > > > > > > >> > support
> > > >> > >> > > > > > > > >> > > > to
> > > >> > >> > > > > > > > >> > > > > > our thin client implementation.
> > > >> > >> > > > > > > > >> > > > > > I've looked at our current
> > implementation
> > > >> and
> > > >> > >> have
> > > >> > >> > > > some
> > > >> > >> > > > > > > > >> proposals
> > > >> > >> > > > > > > > >> > to
> > > >> > >> > > > > > > > >> > > > > > support transactions:
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > Add new operations to thin client
> > > >> protocol:
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > >     OP_TX_GET, 4000, Get current
> > > >> transaction
> > > >> > >> for
> > > >> > >> > > > client
> > > >> > >> > > > > > > > >> connection
> > > >> > >> > > > > > > > >> > > > > >     OP_TX_START, 4001, Start a new
> > > >> > transaction
> > > >> > >> > > > > > > > >> > > > > >     OP_TX_COMMIT, 4002, Commit
> > transaction
> > > >> > >> > > > > > > > >> > > > > >     OP_TX_ROLLBACK, 4003, Rollback
> > > >> > transaction
> > > >> > >> > > > > > > > >> > > > > >     OP_TX_CLOSE, 4004, Close
> > transaction
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > From the client side (java) new
> > interfaces
> > > >> > >> will be
> > > >> > >> > > > > added:
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > public interface ClientTransactions
> {
> > > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> > txStart();
> > > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> > > >> > >> > > > > > txStart(TransactionConcurrency
> > > >> > >> > > > > > > > >> > > > concurrency,
> > > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation);
> > > >> > >> > > > > > > > >> > > > > >     public ClientTransaction
> > > >> > >> > > > > > txStart(TransactionConcurrency
> > > >> > >> > > > > > > > >> > > > concurrency,
> > > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation, long
> > > >> timeout,
> > > >> > >> int
> > > >> > >> > > > > txSize);
> > > >> > >> > > > > > > > >> > > > > >     public ClientTransaction tx();
> //
> > Get
> > > >> > >> current
> > > >> > >> > > > > > connection
> > > >> > >> > > > > > > > >> > > > transaction
> > > >> > >> > > > > > > > >> > > > > >     public ClientTransactions
> > > >> > withLabel(String
> > > >> > >> > lb);
> > > >> > >> > > > > > > > >> > > > > > }
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > public interface ClientTransaction
> > extends
> > > >> > >> > > > > AutoCloseable {
> > > >> > >> > > > > > > > >> > > > > >     public IgniteUuid xid(); // Do
> we
> > need
> > > >> > it?
> > > >> > >> > > > > > > > >> > > > > >     public TransactionIsolation
> > > >> isolation();
> > > >> > >> > > > > > > > >> > > > > >     public TransactionConcurrency
> > > >> > >> concurrency();
> > > >> > >> > > > > > > > >> > > > > >     public long timeout();
> > > >> > >> > > > > > > > >> > > > > >     public String label();
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > >     public void commit();
> > > >> > >> > > > > > > > >> > > > > >     public void rollback();
> > > >> > >> > > > > > > > >> > > > > >     public void close();
> > > >> > >> > > > > > > > >> > > > > > }
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > From the server side, I think as a
> > first
> > > >> step
> > > >> > >> > (while
> > > >> > >> > > > > > > > >> transactions
> > > >> > >> > > > > > > > >> > > > > > suspend/resume is not fully
> > implemented)
> > > >> we
> > > >> > can
> > > >> > >> > use
> > > >> > >> > > > the
> > > >> > >> > > > > > same
> > > >> > >> > > > > > > > >> > approach
> > > >> > >> > > > > > > > >> > > > as
> > > >> > >> > > > > > > > >> > > > > > for JDBC: add a new worker to each
> > > >> > >> > > > ClientRequestHandler
> > > >> > >> > > > > > and
> > > >> > >> > > > > > > > >> process
> > > >> > >> > > > > > > > >> > > > > > requests by this worker if the
> > > >> transaction is
> > > >> > >> > > started
> > > >> > >> > > > > > > > >> explicitly.
> > > >> > >> > > > > > > > >> > > > > > ClientRequestHandler is bound to
> > client
> > > >> > >> > connection,
> > > >> > >> > > so
> > > >> > >> > > > > > there
> > > >> > >> > > > > > > > >> will
> > > >> > >> > > > > > > > >> > be
> > > >> > >> > > > > > > > >> > > > 1:1
> > > >> > >> > > > > > > > >> > > > > > relation between client connection
> and
> > > >> > thread,
> > > >> > >> > which
> > > >> > >> > > > > > process
> > > >> > >> > > > > > > > >> > > operations
> > > >> > >> > > > > > > > >> > > > > in
> > > >> > >> > > > > > > > >> > > > > > a transaction.
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > Also, there is a couple of issues I
> > want
> > > >> to
> > > >> > >> > discuss:
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > We have overloaded method txStart
> > with a
> > > >> > >> different
> > > >> > >> > > set
> > > >> > >> > > > > of
> > > >> > >> > > > > > > > >> > arguments.
> > > >> > >> > > > > > > > >> > > > Some
> > > >> > >> > > > > > > > >> > > > > > of the arguments may be missing. To
> > pass
> > > >> > >> arguments
> > > >> > >> > > > with
> > > >> > >> > > > > > > > >> OP_TX_START
> > > >> > >> > > > > > > > >> > > > > > operation we have the next options:
> > > >> > >> > > > > > > > >> > > > > >  * Serialize full set of arguments
> > and use
> > > >> > some
> > > >> > >> > > value
> > > >> > >> > > > > for
> > > >> > >> > > > > > > > >> missing
> > > >> > >> > > > > > > > >> > > > > > arguments. For example -1 for
> int/long
> > > >> types
> > > >> > >> and
> > > >> > >> > > null
> > > >> > >> > > > > for
> > > >> > >> > > > > > > > string
> > > >> > >> > > > > > > > >> > > type.
> > > >> > >> > > > > > > > >> > > > We
> > > >> > >> > > > > > > > >> > > > > > can't use 0 for int/long types
> since 0
> > > >> it's a
> > > >> > >> > valid
> > > >> > >> > > > > value
> > > >> > >> > > > > > > for
> > > >> > >> > > > > > > > >> > > > > concurrency,
> > > >> > >> > > > > > > > >> > > > > > isolation and timeout arguments.
> > > >> > >> > > > > > > > >> > > > > >  * Serialize arguments as a
> > collection of
> > > >> > >> > > > property-value
> > > >> > >> > > > > > > pairs
> > > >> > >> > > > > > > > >> > (like
> > > >> > >> > > > > > > > >> > > > it's
> > > >> > >> > > > > > > > >> > > > > > implemented now for
> > CacheConfiguration).
> > > >> In
> > > >> > >> this
> > > >> > >> > > case
> > > >> > >> > > > > only
> > > >> > >> > > > > > > > >> > explicitly
> > > >> > >> > > > > > > > >> > > > > > provided arguments will be
> serialized.
> > > >> > >> > > > > > > > >> > > > > > Which way is better? The simplest
> > > >> solution is
> > > >> > >> to
> > > >> > >> > use
> > > >> > >> > > > the
> > > >> > >> > > > > > > first
> > > >> > >> > > > > > > > >> > option
> > > >> > >> > > > > > > > >> > > > > and I
> > > >> > >> > > > > > > > >> > > > > > want to use it if there were no
> > > >> objections.
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > Do we need transaction id (xid) on
> the
> > > >> client
> > > >> > >> > side?
> > > >> > >> > > > > > > > >> > > > > > If yes, we can pass xid along with
> > > >> > >> OP_TX_COMMIT,
> > > >> > >> > > > > > > > OP_TX_ROLLBACK,
> > > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE operations back to the
> > server
> > > >> and
> > > >> > >> do
> > > >> > >> > > > > > additional
> > > >> > >> > > > > > > > >> check
> > > >> > >> > > > > > > > >> > on
> > > >> > >> > > > > > > > >> > > > the
> > > >> > >> > > > > > > > >> > > > > > server side (current transaction id
> > for
> > > >> > >> connection
> > > >> > >> > > ==
> > > >> > >> > > > > > > > >> transaction
> > > >> > >> > > > > > > > >> > id
> > > >> > >> > > > > > > > >> > > > > passed
> > > >> > >> > > > > > > > >> > > > > > from client side). This, perhaps,
> will
> > > >> > protect
> > > >> > >> > > clients
> > > >> > >> > > > > > > against
> > > >> > >> > > > > > > > >> some
> > > >> > >> > > > > > > > >> > > > > errors
> > > >> > >> > > > > > > > >> > > > > > (for example when client try to
> commit
> > > >> > outdated
> > > >> > >> > > > > > > transaction).
> > > >> > >> > > > > > > > >> But
> > > >> > >> > > > > > > > >> > > > > > currently, we don't have data type
> > > >> IgniteUuid
> > > >> > >> in
> > > >> > >> > > thin
> > > >> > >> > > > > > client
> > > >> > >> > > > > > > > >> > > protocol.
> > > >> > >> > > > > > > > >> > > > Do
> > > >> > >> > > > > > > > >> > > > > > we need to add it too?
> > > >> > >> > > > > > > > >> > > > > > Also, we can pass xid as a string
> > just to
> > > >> > >> inform
> > > >> > >> > the
> > > >> > >> > > > > > client
> > > >> > >> > > > > > > > and
> > > >> > >> > > > > > > > >> do
> > > >> > >> > > > > > > > >> > > not
> > > >> > >> > > > > > > > >> > > > > pass
> > > >> > >> > > > > > > > >> > > > > > it back to the server with
> > commit/rollback
> > > >> > >> > > operation.
> > > >> > >> > > > > > > > >> > > > > > Or not to pass xid at all (.NET
> thick
> > > >> client
> > > >> > >> works
> > > >> > >> > > > this
> > > >> > >> > > > > > way
> > > >> > >> > > > > > > as
> > > >> > >> > > > > > > > >> far
> > > >> > >> > > > > > > > >> > > as I
> > > >> > >> > > > > > > > >> > > > > > know).
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > What do you think?
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > ср, 7 мар. 2018 г. в 16:22, Vladimir
> > > >> Ozerov <
> > > >> > >> > > > > > > > >> vozerov@gridgain.com
> > > >> > >> > > > > > > > >> > >:
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > > > > We already have transactions
> > support in
> > > >> > JDBC
> > > >> > >> > > driver
> > > >> > >> > > > in
> > > >> > >> > > > > > TX
> > > >> > >> > > > > > > > SQL
> > > >> > >> > > > > > > > >> > > branch
> > > >> > >> > > > > > > > >> > > > > > > (ignite-4191). Currently it is
> > > >> implemented
> > > >> > >> > through
> > > >> > >> > > > > > > separate
> > > >> > >> > > > > > > > >> > thread,
> > > >> > >> > > > > > > > >> > > > > which
> > > >> > >> > > > > > > > >> > > > > > > is not that efficient. Ideally we
> > need
> > > >> to
> > > >> > >> finish
> > > >> > >> > > > > > > decoupling
> > > >> > >> > > > > > > > >> > > > > transactions
> > > >> > >> > > > > > > > >> > > > > > > from threads. But alternatively we
> > can
> > > >> > change
> > > >> > >> > the
> > > >> > >> > > > > logic
> > > >> > >> > > > > > on
> > > >> > >> > > > > > > > >> how we
> > > >> > >> > > > > > > > >> > > > > assign
> > > >> > >> > > > > > > > >> > > > > > > thread ID to specific transaction
> > and
> > > >> > >> > > "impersonate"
> > > >> > >> > > > > thin
> > > >> > >> > > > > > > > >> client
> > > >> > >> > > > > > > > >> > > > worker
> > > >> > >> > > > > > > > >> > > > > > > threads when serving requests from
> > > >> multiple
> > > >> > >> > users.
> > > >> > >> > > > > > > > >> > > > > > >
> > > >> > >> > > > > > > > >> > > > > > >
> > > >> > >> > > > > > > > >> > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM,
> > Denis
> > > >> > Magda
> > > >> > >> <
> > > >> > >> > > > > > > > >> dmagda@apache.org>
> > > >> > >> > > > > > > > >> > > > > wrote:
> > > >> > >> > > > > > > > >> > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > Here is an original discussion
> > with a
> > > >> > >> > reference
> > > >> > >> > > to
> > > >> > >> > > > > the
> > > >> > >> > > > > > > > JIRA
> > > >> > >> > > > > > > > >> > > ticket:
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > http://apache-ignite-developers.2346864.n4.nabble
> > > >> > >> > > > .
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > com/Re-Transaction-operations-using-the-Ignite-Thin-Client-
> > > >> > >> > > > > > > > >> > > > > > > > Protocol-td25914.html
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > --
> > > >> > >> > > > > > > > >> > > > > > > > Denis
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18 AM,
> > > >> Dmitriy
> > > >> > >> > > Setrakyan
> > > >> > >> > > > <
> > > >> > >> > > > > > > > >> > > > > > dsetrakyan@apache.org
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > wrote:
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > Hi Dmitriy. I don't think we
> > have a
> > > >> > >> design
> > > >> > >> > > > > proposal
> > > >> > >> > > > > > > for
> > > >> > >> > > > > > > > >> > > > transaction
> > > >> > >> > > > > > > > >> > > > > > > > support
> > > >> > >> > > > > > > > >> > > > > > > > > in thin clients. Do you mind
> > taking
> > > >> > this
> > > >> > >> > > > > initiative
> > > >> > >> > > > > > > and
> > > >> > >> > > > > > > > >> > > creating
> > > >> > >> > > > > > > > >> > > > an
> > > >> > >> > > > > > > > >> > > > > > IEP
> > > >> > >> > > > > > > > >> > > > > > > > on
> > > >> > >> > > > > > > > >> > > > > > > > > Wiki?
> > > >> > >> > > > > > > > >> > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > D.
> > > >> > >> > > > > > > > >> > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > On Tue, Mar 6, 2018 at 8:46
> AM,
> > > >> Dmitriy
> > > >> > >> > > > > Govorukhin <
> > > >> > >> > > > > > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com>
> > > >> wrote:
> > > >> > >> > > > > > > > >> > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > > Hi, Igniters.
> > > >> > >> > > > > > > > >> > > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > > I've seen a lot of
> discussions
> > > >> about
> > > >> > >> thin
> > > >> > >> > > > client
> > > >> > >> > > > > > and
> > > >> > >> > > > > > > > >> binary
> > > >> > >> > > > > > > > >> > > > > > protocol,
> > > >> > >> > > > > > > > >> > > > > > > > > but I
> > > >> > >> > > > > > > > >> > > > > > > > > > did not hear anything about
> > > >> > >> transactions
> > > >> > >> > > > > support.
> > > >> > >> > > > > > Do
> > > >> > >> > > > > > > > we
> > > >> > >> > > > > > > > >> > have
> > > >> > >> > > > > > > > >> > > > some
> > > >> > >> > > > > > > > >> > > > > > > draft
> > > >> > >> > > > > > > > >> > > > > > > > > for
> > > >> > >> > > > > > > > >> > > > > > > > > > this purpose?
> > > >> > >> > > > > > > > >> > > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > > As I understand we have
> > several
> > > >> > >> problems:
> > > >> > >> > > > > > > > >> > > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > >    - thread and transaction
> > have
> > > >> hard
> > > >> > >> > > related
> > > >> > >> > > > > (we
> > > >> > >> > > > > > > use
> > > >> > >> > > > > > > > >> > > > > thread-local
> > > >> > >> > > > > > > > >> > > > > > > > > variable
> > > >> > >> > > > > > > > >> > > > > > > > > >    and thread name)
> > > >> > >> > > > > > > > >> > > > > > > > > >    - we can process only one
> > > >> > >> transaction
> > > >> > >> > at
> > > >> > >> > > > the
> > > >> > >> > > > > > same
> > > >> > >> > > > > > > > >> time
> > > >> > >> > > > > > > > >> > in
> > > >> > >> > > > > > > > >> > > > one
> > > >> > >> > > > > > > > >> > > > > > > thread
> > > >> > >> > > > > > > > >> > > > > > > > > (it
> > > >> > >> > > > > > > > >> > > > > > > > > >    mean we need hold thread
> > per
> > > >> > >> client. If
> > > >> > >> > > > > connect
> > > >> > >> > > > > > > 100
> > > >> > >> > > > > > > > >> thin
> > > >> > >> > > > > > > > >> > > > > clients
> > > >> > >> > > > > > > > >> > > > > > > to
> > > >> > >> > > > > > > > >> > > > > > > > 1
> > > >> > >> > > > > > > > >> > > > > > > > > >    server node, then need to
> > hold
> > > >> 100
> > > >> > >> > thread
> > > >> > >> > > > on
> > > >> > >> > > > > > the
> > > >> > >> > > > > > > > >> server
> > > >> > >> > > > > > > > >> > > > side)
> > > >> > >> > > > > > > > >> > > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > > > Let's discuss how we can
> > implement
> > > >> > >> > > > transactions
> > > >> > >> > > > > > for
> > > >> > >> > > > > > > > the
> > > >> > >> > > > > > > > >> > thin
> > > >> > >> > > > > > > > >> > > > > > client.
> > > >> > >> > > > > > > > >> > > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > > >
> > > >> > >> > > > > > > > >> > > > > > >
> > > >> > >> > > > > > > > >> > > > > >
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > > > --
> > > >> > >> > > > > > > > >> > > > > Sergey Kozlov
> > > >> > >> > > > > > > > >> > > > > GridGain Systems
> > > >> > >> > > > > > > > >> > > > > www.gridgain.com
> > > >> > >> > > > > > > > >> > > > >
> > > >> > >> > > > > > > > >> > > >
> > > >> > >> > > > > > > > >> > >
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >> > --
> > > >> > >> > > > > > > > >> > Sergey Kozlov
> > > >> > >> > > > > > > > >> > GridGain Systems
> > > >> > >> > > > > > > > >> > www.gridgain.com
> > > >> > >> > > > > > > > >> >
> > > >> > >> > > > > > > > >>
> > > >> > >> > > > > > > > >
> > > >> > >> > > > > > > >
> > > >> > >> > > > > > >
> > > >> > >> > > > > >
> > > >> > >> > > > >
> > > >> > >> > > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> > >
> > > >> >
> > > >>
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message