From dev-return-45928-apmail-ignite-dev-archive=ignite.apache.org@ignite.apache.org Fri May 3 05:10:05 2019 Return-Path: X-Original-To: apmail-ignite-dev-archive@minotaur.apache.org Delivered-To: apmail-ignite-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id CB09F19555 for ; Fri, 3 May 2019 05:10:04 +0000 (UTC) Received: (qmail 72282 invoked by uid 500); 3 May 2019 05:10:00 -0000 Delivered-To: apmail-ignite-dev-archive@ignite.apache.org Received: (qmail 72193 invoked by uid 500); 3 May 2019 05:09:59 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 72179 invoked by uid 99); 3 May 2019 05:09:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 May 2019 05:09:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id B851DC2CB5 for ; Fri, 3 May 2019 05:09:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.157 X-Spam-Level: X-Spam-Status: No, score=0.157 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FROM_EXCESS_BASE64=0.105, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id mETz9X8JP-Vo for ; Fri, 3 May 2019 05:09:55 +0000 (UTC) Received: from mail-ot1-f68.google.com (mail-ot1-f68.google.com [209.85.210.68]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 1E7B65F1CF for ; Fri, 3 May 2019 05:09:54 +0000 (UTC) Received: by mail-ot1-f68.google.com with SMTP id o39so4245499ota.6 for ; Thu, 02 May 2019 22:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=1raIZyECJfN3VgJE6Rjpfr0mMR0pMerlf52RUYaS8l8=; b=YYjpwqbMQoZh5TuIGOc1Fji2Ew3gAIRthG4ORDAV6E5eALnVwOHzOxFwMhi3CfyOrJ aCzPnDyxshcdhEAiCHOCorKKFKbD3rin42isqvPcKO+E8YvHQfg75fplL9CjcVV58KYa tpYz93ChY6BYqdsKikoKHFobF19ImsuCfWQVCrywCX3on51Wsa0FylVqjmLc31XLIVMx 08bdESxsdMaF4baALMzJL4tSVRF2BjLhOzx9DXs7sO+9245TxEC+uGI9VG9cZhnhDFTd 6KU3OY7Se1LvyKTelbdLibZNE5FRexapCi6Tz/i8dS3yGovvrrDmUa5VQKWG5A8d5cq6 QmJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=1raIZyECJfN3VgJE6Rjpfr0mMR0pMerlf52RUYaS8l8=; b=jwGtSnv+CfwqLgZfDtFKiyss/UrI0nu4JGrS3FGASj+QD2rlTQ2THIfz8JXbw1C1vu wLYDF4fFi5ggE7XFrhs3gTEOUQtr+Ys9d8Y+PMuLAY7c5eGuK542YIFPR36M2TaMCQ77 1jG28SgIfP/pZ1Fw/xO21Mz1SCNcEhYt3TY2nkQNxsyC2hmnWjxbfNqk0Ni/37vOHXj+ TMxrw7FUptNRJ74eWCEMqz6m695HPkLq8E0lKgs+IDNS7AG67G4BV8uycbht1djx5tfk o6qux7dRWLJKAuHvhWowSsSRtEAuIp9tzb3CvkDT/nIGjuvwQvE5tSufd7sYCWKJqU2J 3kSQ== X-Gm-Message-State: APjAAAUOW8d3qOqXFWnmzYrHaeg/A5F771Elg+K+zEHIZDmxpEjrG4al rqfQYCczelMzKHt489iYQGEqBfw6ymtPj+WvgFCOah9n X-Google-Smtp-Source: APXvYqxYm9DDovbE0RX8+hN5tbOd/JqZrwCTI9IU3qf9i9yqkxn2vdHcBhFE4PGwgVCNaFwPUfLq5f5vOmyJRZrHtUI= X-Received: by 2002:a05:6830:12d7:: with SMTP id a23mr5572229otq.289.1556860192426; Thu, 02 May 2019 22:09:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?0J/QsNCy0LvRg9GF0LjQvSDQmNCy0LDQvQ==?= Date: Fri, 3 May 2019 08:09:42 +0300 Message-ID: Subject: Re: Thin client: transactions support To: dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Alex, I went through IEP [1] and I have a couple of questions: 1. What is going to be used as transaction id? In a described protocol I see an int field for it. Should not it be GridCacheVersion corresponding to IgniteInternalTx#xidVersion? 2. OP_TX_END message assumes an empty response, but I think that errors during tx finish are possible and should be returned in a response. 3. In IEP it is stated that async processing of lock operations should be introduced on a client side to enable concurrent operations from different client threads. Do you have an idea how to achieve it? Also, a bit about a suspend/resume trait. I tried to think about it leaving away an existing transactions implementation in Ignite. As I understood we are going to resume a tx before each cache operation in the tx and resume the tx after the operation. All this to make an executing thread available for other operations (e.g. in other txs). >From the first glance it seems like an inversed logic. A straightforward way is to execute a cache operation within a particular transaction defined as an explicit tx id argument (e.g. cache.put(key, value, txid)). Can we do so? And leaving for now thin client API. I cannot say that one proposed in IEP is good or bad. I can only say that it ressembles current thick client API. And perhaps it should not. I think that we should consider similar APIs provided by other vendors and keep in mind that we have a bunch of client implementations for different languages. I suppose that we can return to it a little bit later. And I hope that we will do it. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3= A+transactions+support =D0=B2=D1=82, 30 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 13:24, Alex Plehan= ov : > > Hello, Igniters! > > I've update IEP [1] and implement PoC according to new approach (multiple > concurrent transactions per connection). > But to move forward another feature need to be implemented: suspend/resum= e > for pessimistic transactions (IGNITE-5714 [2]). Implementation of > suspend/resume is ready now and ticket in 'Patch available' status. Can a= ny > transactions expert help with review of IGNITE-5714? > > [1]: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+= transactions+support > [2]: https://issues.apache.org/jira/browse/IGNITE-5714 > > =D1=87=D1=82, 4 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 11:32, Alex Pleha= nov : > > > Vladimir, > > > > Ok, then I will rewrite IEP in the near future. > > > > =D1=87=D1=82, 4 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 11:14, Vladimir= Ozerov : > > > >> Hi Alex, > >> > >> I think we should be able to handle many transactions through a single > >> connection. This will make our protocol and client implementations muc= h > >> more efficient, and simplicity from developer's perspective is not our > >> goal. Consider normal nodes. We have server nodes and client nodes. Yo= u > >> may > >> span whatever number of transactions you need, but all of them are > >> coordinated through a single connection. The same should be applicable= to > >> thin clients. Protocol is already designed to handle this, as we pass > >> unique operation ID in order to distinguish one operation from another= . It > >> is true, though, that we will have to introduce a kind of "session" > >> concept, and pass additional identifier along with cache operations, b= ut > >> this doesn't sound like a problem to me. > >> > >> And provided that currently server-side transactions are bound to thre= ads > >> artificially, I would say that the first step in implementation of > >> transactions on thin clients should be decoupling server-side transact= ions > >> from threads. Without this we will have very inefficient implementatio= n, > >> when every new client transaction have to spawn a new thread. This is = slow > >> and introduces high memory pressure on a cluster node. We already work > >> this > >> way for MVCC transactions which are spawned from JDBC driver, and beli= eve > >> me, we do not want to replicated this bad practice to other clients :-= ) > >> > >> Vladimir. > >> > >> On Thu, Apr 4, 2019 at 10:08 AM Alex Plehanov > >> wrote: > >> > >> > Guys, so, do we need multiple concurrent transactions per connection= ? > >> > > >> > There are pros and cons for each approach. Difference between > >> approaches: > >> > > >> > One transaction at a time per connection: > >> > - This approach is used in RDBMS world and users got used to it > >> > - To use transactions concurrently users need to use different > >> connections > >> > and get these connections via something like a connection pool > >> > - Easy to implement (in fact, PoC is already done) > >> > > >> > Multiple concurrent transactions per connection: > >> > - At least for java thin client, we can implement transaction per > >> thread > >> > approach as implemented now for the thick client (perhaps other thin > >> > clients can implement the same abstraction) > >> > - There is also protocol change for all cache operations needed (to > >> bind > >> > cache operation to the transaction) > >> > - Significant changes to all implemented clients are needed > >> > - Implementation on the server side is more complex > >> > > >> > What do you think? > >> > > >> > > >> > =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 16:29, Alex = Plehanov : > >> > > >> > > Ilya, > >> > > > >> > > > We should be able to multiplex several transactions using a sing= le > >> > > Client connection. > >> > > In this case, we should significantly change cache operations synt= ax > >> (for > >> > > each implemented client), to bind each operation to the transactio= n. > >> > > > >> > > > I want to also ask if "Number of entries participating in > >> transaction > >> > > (may be approximate). 0 - default value." is needed. > >> > > I've tried to minimize API changes between thick and thin client t= o > >> > > simplify move from one to another. It's the only reason. But I agr= ee > >> with > >> > > you, the parameter is not very useful. > >> > > > >> > > > >> > > =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 14:48, Ily= a Kasnacheev < > >> ilya.kasnacheev@gmail.com>: > >> > > > >> > >> Hello! > >> > >> > >> > >> Pavel, I agree with you thorougly. We should be able to multiplex > >> > several > >> > >> transactions using a single Client connection. This means adding > >> > >> Transaction id parameter to every affected cache operation / SQL > >> > statement > >> > >> (if applicable) to make sure we do cache operations on relevant > >> > >> transaction. > >> > >> > >> > >> This is how other things work in Ignite, such as communication. W= e do > >> > not > >> > >> open dozens of connections, we multiplex operations asynchronousl= y > >> > through > >> > >> a single connection. > >> > >> > >> > >> I think that trying to pool Ignite connections will be highly > >> > >> inconvenient, > >> > >> since there is no existing infrastructure for such pooling (like > >> there > >> > >> exists for JDBC). > >> > >> > >> > >> I want to also ask if "Number of entries participating in transac= tion > >> > (may > >> > >> be approximate). 0 - default value." is needed. Does it actually = do > >> > >> anything in our tx protocol? Users of existing APIs are already > >> confused > >> > >> by > >> > >> this parameter, if we could get rid of it in thin client protocol= it > >> > would > >> > >> be nice clean-up. > >> > >> > >> > >> Regards, > >> > >> -- > >> > >> Ilya Kasnacheev > >> > >> > >> > >> > >> > >> =D0=B2=D1=82, 2 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 09:55, Pa= vel Tupitsyn : > >> > >> > >> > >> > Alex, > >> > >> > > >> > >> > > now we can only support one active transaction per connection > >> > >> > > >> > >> > I totally understand server-side and protocol limitations that = are > >> > >> causing > >> > >> > this. > >> > >> > But I have no idea how to support this in .NET Thin Client, for > >> > example. > >> > >> > > >> > >> > It is thread-safe and can handle multiple async operations in > >> > parallel. > >> > >> > But with TX support we have to somehow switch to single-threade= d > >> mode > >> > to > >> > >> > avoid unexpected effects. > >> > >> > > >> > >> > Any ideas? > >> > >> > > >> > >> > > >> > >> > On Mon, Apr 1, 2019 at 6:38 PM Alex Plehanov < > >> plehanov.alex@gmail.com > >> > > > >> > >> > wrote: > >> > >> > > >> > >> > > Dmitriy, thank you! > >> > >> > > > >> > >> > > Guys, I've created the IEP [1] on wiki, please have a look. > >> > >> > > > >> > >> > > [1] > >> > >> > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%= 3A+transactions+support > >> > >> > > > >> > >> > > > >> > >> > > =D1=87=D1=82, 28 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 14:3= 3, Dmitriy Pavlov >> >: > >> > >> > > > >> > >> > > > Hi, > >> > >> > > > > >> > >> > > > I've added permissions to account plehanov.alex > >> > >> > > > > >> > >> > > > Recently Infra integrated Apache LDAP with confluence, so i= t is > >> > >> > possible > >> > >> > > to > >> > >> > > > login using Apache credentials. Probably we can ask infra i= f > >> extra > >> > >> > > > permissions to edit pages should be added for committers. > >> > >> > > > > >> > >> > > > Sincerely, > >> > >> > > > Dmitriy Pavlov > >> > >> > > > > >> > >> > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 13= :37, Alex Plehanov < > >> > plehanov.alex@gmail.com > >> > >> >: > >> > >> > > > > >> > >> > > > > Vladimir, > >> > >> > > > > > >> > >> > > > > About current tx: ok, then we don't need tx() method in t= he > >> > >> interface > >> > >> > > at > >> > >> > > > > all (the same cached transaction info user can store by > >> > himself). > >> > >> > > > > > >> > >> > > > > About decoupling transactions from threads on the server > >> side: > >> > for > >> > >> > now, > >> > >> > > > we > >> > >> > > > > can start with thread-per-connection approach (we only ca= n > >> > support > >> > >> > one > >> > >> > > > > active transaction per connection, see below, so we need = one > >> > >> > additional > >> > >> > > > > dedicated thread for each connection with active > >> transaction), > >> > and > >> > >> > > later > >> > >> > > > > change server-side internals to process client transactio= ns > >> in > >> > any > >> > >> > > server > >> > >> > > > > thread (not dedicated to this connection). This change wi= ll > >> not > >> > >> > affect > >> > >> > > > the > >> > >> > > > > thin client protocol, it only affects the server side. > >> > >> > > > > In any case, we can't support concurrent transactions per > >> > >> connection > >> > >> > on > >> > >> > > > > the client side without fundamental changes to the curren= t > >> > >> protocol > >> > >> > > > (cache > >> > >> > > > > operation doesn't bound to transaction or thread and the > >> server > >> > >> > doesn't > >> > >> > > > > know which thread on the client side do this cache > >> operation). > >> > In > >> > >> my > >> > >> > > > > opinion, if a user wants to use concurrent transactions, = he > >> must > >> > >> use > >> > >> > > > > different connections from a connection pool. > >> > >> > > > > > >> > >> > > > > About semantics of suspend/resume on the client-side: it'= s > >> > >> absolutely > >> > >> > > > > different than server-side semantics (we don't need to do > >> > >> > > suspend/resume > >> > >> > > > to > >> > >> > > > > pass transaction between threads on the client-side), but > >> can't > >> > be > >> > >> > > > > implemented efficiently without implemented suspend/resum= e on > >> > >> > > > server-side. > >> > >> > > > > > >> > >> > > > > Can anyone give me permissions to create IEP on Apache wi= ki? > >> > >> > > > > > >> > >> > > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0=B2 = 11:59, Vladimir Ozerov < > >> > >> vozerov@gridgain.com>: > >> > >> > > > > > >> > >> > > > > > Hi Alex, > >> > >> > > > > > > >> > >> > > > > > My comments was only about the protocol. Getting curren= t > >> info > >> > >> about > >> > >> > > > > > transaction should be handled by the client itself. It = is > >> not > >> > >> > > protocl's > >> > >> > > > > > concern. Same about other APIs and behavior in case ano= ther > >> > >> > > transaction > >> > >> > > > > is > >> > >> > > > > > attempted from the same thread. > >> > >> > > > > > > >> > >> > > > > > Putting protocol aside, transaction support is complica= ted > >> > >> matter. > >> > >> > I > >> > >> > > > > would > >> > >> > > > > > propose to route through IEP and wide community > >> discussion. We > >> > >> need > >> > >> > > to > >> > >> > > > > > review API and semantics very carefully, taking > >> SUSPEND/RESUME > >> > >> in > >> > >> > > > count. > >> > >> > > > > > Also I do not see how we support client transactions > >> > efficiently > >> > >> > > > without > >> > >> > > > > > decoupling transactions from threads on the server side > >> first. > >> > >> > > Because > >> > >> > > > > > without it you will need a dedicated server thread for > >> every > >> > >> > client's > >> > >> > > > > > transaction which is slow and may even crash the server= . > >> > >> > > > > > > >> > >> > > > > > Vladimir. > >> > >> > > > > > > >> > >> > > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov < > >> > >> > > > plehanov.alex@gmail.com> > >> > >> > > > > > wrote: > >> > >> > > > > > > >> > >> > > > > > > Vladimir, what if we want to get current transaction = info > >> > >> (tx() > >> > >> > > > > method)? > >> > >> > > > > > > > >> > >> > > > > > > Does close() method mapped to TX_END(rollback)? > >> > >> > > > > > > > >> > >> > > > > > > For example, this code: > >> > >> > > > > > > > >> > >> > > > > > > try(tx =3D txStart()) { > >> > >> > > > > > > tx.commit(); > >> > >> > > > > > > } > >> > >> > > > > > > > >> > >> > > > > > > Will produce: > >> > >> > > > > > > TX_START > >> > >> > > > > > > TX_END(commit) > >> > >> > > > > > > TX_END(rollback) > >> > >> > > > > > > > >> > >> > > > > > > Am I understand you right? > >> > >> > > > > > > > >> > >> > > > > > > About xid. There is yet another proposal. Use some un= ique > >> > per > >> > >> > > > > connection > >> > >> > > > > > id > >> > >> > > > > > > (integer, simple counter) for identifying the > >> transaction on > >> > >> > > > > > > commit/rollback message. The client gets this id from= the > >> > >> server > >> > >> > > with > >> > >> > > > > > > transaction info and sends it back to the server when > >> trying > >> > >> to > >> > >> > > > > > > commit/rollback transaction. This id is not shown to > >> users. > >> > >> But > >> > >> > > also > >> > >> > > > we > >> > >> > > > > > can > >> > >> > > > > > > pass from server to client real transaction id (xid) = with > >> > >> > > transaction > >> > >> > > > > > info > >> > >> > > > > > > for diagnostic purposes. > >> > >> > > > > > > > >> > >> > > > > > > And one more question: what should we do if the clien= t > >> > starts > >> > >> a > >> > >> > new > >> > >> > > > > > > transaction without ending the old one? Should we end= the > >> > old > >> > >> > > > > transaction > >> > >> > > > > > > implicitly (rollback) or throw an exception to the > >> client? > >> > In > >> > >> my > >> > >> > > > > opinion, > >> > >> > > > > > > the first option is better. For example, if we got a > >> > >> previously > >> > >> > > used > >> > >> > > > > > > connection from the connection pool, we should not wo= rry > >> > about > >> > >> > any > >> > >> > > > > > > uncompleted transaction started by the previous user = of > >> this > >> > >> > > > > connection. > >> > >> > > > > > > > >> > >> > > > > > > =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3. =D0= =B2 11:02, Vladimir Ozerov < > >> > >> > vozerov@gridgain.com > >> > >> > > >: > >> > >> > > > > > > > >> > >> > > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not supp= ort > >> > them > >> > >> > yet, > >> > >> > > > and > >> > >> > > > > > > adding > >> > >> > > > > > > > them in future should not conflict with simple > >> START/END > >> > >> > > > > > infrastructure. > >> > >> > > > > > > > > >> > >> > > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov < > >> > >> > > > > vozerov@gridgain.com > >> > >> > > > > > > > >> > >> > > > > > > > wrote: > >> > >> > > > > > > > > >> > >> > > > > > > > > Hi Alex, > >> > >> > > > > > > > > > >> > >> > > > > > > > > I am not sure we need 5 commands. Wouldn't it be > >> enough > >> > to > >> > >> > have > >> > >> > > > > only > >> > >> > > > > > > two? > >> > >> > > > > > > > > > >> > >> > > > > > > > > START - accepts optional parameters, returns > >> transaction > >> > >> info > >> > >> > > > > > > > > END - provides commit flag, returns void > >> > >> > > > > > > > > > >> > >> > > > > > > > > Vladimir. > >> > >> > > > > > > > > > >> > >> > > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov < > >> > >> > > > > > plehanov.alex@gmail.com > >> > >> > > > > > > > > >> > >> > > > > > > > > wrote: > >> > >> > > > > > > > > > >> > >> > > > > > > > >> Sergey, yes, the close is something like silent > >> > rollback. > >> > >> > But > >> > >> > > we > >> > >> > > > > can > >> > >> > > > > > > > >> also implement this on the client side, just usi= ng > >> > >> rollback > >> > >> > > and > >> > >> > > > > > > ignoring > >> > >> > > > > > > > >> errors in the response. > >> > >> > > > > > > > >> > >> > >> > > > > > > > >> =D1=81=D1=80, 27 =D0=BC=D0=B0=D1=80. 2019 =D0=B3= . =D0=B2 00:04, Sergey Kozlov < > >> > >> > > > skozlov@gridgain.com > >> > >> > > > > >: > >> > >> > > > > > > > >> > >> > >> > > > > > > > >> > Nikolay > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > Am I correctly understand you points: > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > - close: rollback > >> > >> > > > > > > > >> > - commit, close: do nothing > >> > >> > > > > > > > >> > - rollback, close: do what? (I suppose noth= ing) > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > Also you assume that after commit/rollback we = may > >> > need > >> > >> to > >> > >> > > free > >> > >> > > > > > some > >> > >> > > > > > > > >> > resources on server node(s)or just do on clien= t > >> > started > >> > >> > TX? > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex Plehanov= < > >> > >> > > > > > > > plehanov.alex@gmail.com > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > wrote: > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > Sergey, we have the close() method in the th= ick > >> > >> client, > >> > >> > > it's > >> > >> > > > > > > > behavior > >> > >> > > > > > > > >> is > >> > >> > > > > > > > >> > > slightly different than rollback() method (i= t > >> > should > >> > >> > > > rollback > >> > >> > > > > if > >> > >> > > > > > > the > >> > >> > > > > > > > >> > > transaction is not committed and do nothing = if > >> the > >> > >> > > > transaction > >> > >> > > > > > is > >> > >> > > > > > > > >> already > >> > >> > > > > > > > >> > > committed). I think we should support > >> > >> try-with-resource > >> > >> > > > > > semantics > >> > >> > > > > > > in > >> > >> > > > > > > > >> the > >> > >> > > > > > > > >> > > thin client and OP_TX_CLOSE will be useful h= ere. > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > Nikolay, suspend/resume didn't work yet for > >> > >> pessimistic > >> > >> > > > > > > > transactions. > >> > >> > > > > > > > >> > Also, > >> > >> > > > > > > > >> > > the main goal of suspend/resume operations i= s to > >> > >> support > >> > >> > > > > > > transaction > >> > >> > > > > > > > >> > > passing between threads. In the thin client,= the > >> > >> > > transaction > >> > >> > > > > is > >> > >> > > > > > > > bound > >> > >> > > > > > > > >> to > >> > >> > > > > > > > >> > > the client connection, not client thread. I > >> think > >> > >> > passing > >> > >> > > a > >> > >> > > > > > > > >> transaction > >> > >> > > > > > > > >> > > between different client connections is not = a > >> very > >> > >> > useful > >> > >> > > > > case. > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > =D0=B2=D1=82, 26 =D0=BC=D0=B0=D1=80. 2019 = =D0=B3. =D0=B2 22:17, Nikolay Izhikov < > >> > >> > > > > > nizhikov@apache.org > >> > >> > > > > > > >: > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > > Hello, Alex. > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > We also have suspend and resume operations= . > >> > >> > > > > > > > >> > > > I think we should support them > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > =D0=B2=D1=82, 26 =D0=BC=D0=B0=D1=80=D1=82= =D0=B0 2019 =D0=B3., 22:07 Sergey Kozlov < > >> > >> > > > > > skozlov@gridgain.com > >> > >> > > > > > > >: > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > > Hi > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > Looks like I missed something but why we > >> need > >> > >> > > > OP_TX_CLOSE > >> > >> > > > > > > > >> operation? > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > Also I suggest to reserve a code for > >> SAVEPOINT > >> > >> > > operation > >> > >> > > > > > which > >> > >> > > > > > > > >> very > >> > >> > > > > > > > >> > > > useful > >> > >> > > > > > > > >> > > > > to understand where transaction has been > >> rolled > >> > >> back > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex > >> Plehanov < > >> > >> > > > > > > > >> > plehanov.alex@gmail.com > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > > wrote: > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > > Hello Igniters! > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > I want to pick up the ticket IGNITE-73= 69 > >> and > >> > >> add > >> > >> > > > > > > transactions > >> > >> > > > > > > > >> > support > >> > >> > > > > > > > >> > > > to > >> > >> > > > > > > > >> > > > > > our thin client implementation. > >> > >> > > > > > > > >> > > > > > I've looked at our current implementat= ion > >> and > >> > >> have > >> > >> > > > some > >> > >> > > > > > > > >> proposals > >> > >> > > > > > > > >> > to > >> > >> > > > > > > > >> > > > > > support transactions: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Add new operations to thin client > >> protocol: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > OP_TX_GET, 4000, Get current > >> transaction > >> > >> for > >> > >> > > > client > >> > >> > > > > > > > >> connection > >> > >> > > > > > > > >> > > > > > OP_TX_START, 4001, Start a new > >> > transaction > >> > >> > > > > > > > >> > > > > > OP_TX_COMMIT, 4002, Commit transac= tion > >> > >> > > > > > > > >> > > > > > OP_TX_ROLLBACK, 4003, Rollback > >> > transaction > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE, 4004, Close transacti= on > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > From the client side (java) new interf= aces > >> > >> will be > >> > >> > > > > added: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public interface ClientTransactions { > >> > >> > > > > > > > >> > > > > > public ClientTransaction txStart()= ; > >> > >> > > > > > > > >> > > > > > public ClientTransaction > >> > >> > > > > > txStart(TransactionConcurrency > >> > >> > > > > > > > >> > > > concurrency, > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation); > >> > >> > > > > > > > >> > > > > > public ClientTransaction > >> > >> > > > > > txStart(TransactionConcurrency > >> > >> > > > > > > > >> > > > concurrency, > >> > >> > > > > > > > >> > > > > > TransactionIsolation isolation, long > >> timeout, > >> > >> int > >> > >> > > > > txSize); > >> > >> > > > > > > > >> > > > > > public ClientTransaction tx(); // = Get > >> > >> current > >> > >> > > > > > connection > >> > >> > > > > > > > >> > > > transaction > >> > >> > > > > > > > >> > > > > > public ClientTransactions > >> > withLabel(String > >> > >> > lb); > >> > >> > > > > > > > >> > > > > > } > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public interface ClientTransaction ext= ends > >> > >> > > > > AutoCloseable { > >> > >> > > > > > > > >> > > > > > public IgniteUuid xid(); // Do we = need > >> > it? > >> > >> > > > > > > > >> > > > > > public TransactionIsolation > >> isolation(); > >> > >> > > > > > > > >> > > > > > public TransactionConcurrency > >> > >> concurrency(); > >> > >> > > > > > > > >> > > > > > public long timeout(); > >> > >> > > > > > > > >> > > > > > public String label(); > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > public void commit(); > >> > >> > > > > > > > >> > > > > > public void rollback(); > >> > >> > > > > > > > >> > > > > > public void close(); > >> > >> > > > > > > > >> > > > > > } > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > From the server side, I think as a fir= st > >> step > >> > >> > (while > >> > >> > > > > > > > >> transactions > >> > >> > > > > > > > >> > > > > > suspend/resume is not fully implemente= d) > >> we > >> > can > >> > >> > use > >> > >> > > > the > >> > >> > > > > > same > >> > >> > > > > > > > >> > approach > >> > >> > > > > > > > >> > > > as > >> > >> > > > > > > > >> > > > > > for JDBC: add a new worker to each > >> > >> > > > ClientRequestHandler > >> > >> > > > > > and > >> > >> > > > > > > > >> process > >> > >> > > > > > > > >> > > > > > requests by this worker if the > >> transaction is > >> > >> > > started > >> > >> > > > > > > > >> explicitly. > >> > >> > > > > > > > >> > > > > > ClientRequestHandler is bound to clien= t > >> > >> > connection, > >> > >> > > so > >> > >> > > > > > there > >> > >> > > > > > > > >> will > >> > >> > > > > > > > >> > be > >> > >> > > > > > > > >> > > > 1:1 > >> > >> > > > > > > > >> > > > > > relation between client connection and > >> > thread, > >> > >> > which > >> > >> > > > > > process > >> > >> > > > > > > > >> > > operations > >> > >> > > > > > > > >> > > > > in > >> > >> > > > > > > > >> > > > > > a transaction. > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Also, there is a couple of issues I wa= nt > >> to > >> > >> > discuss: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > We have overloaded method txStart with= a > >> > >> different > >> > >> > > set > >> > >> > > > > of > >> > >> > > > > > > > >> > arguments. > >> > >> > > > > > > > >> > > > Some > >> > >> > > > > > > > >> > > > > > of the arguments may be missing. To pa= ss > >> > >> arguments > >> > >> > > > with > >> > >> > > > > > > > >> OP_TX_START > >> > >> > > > > > > > >> > > > > > operation we have the next options: > >> > >> > > > > > > > >> > > > > > * Serialize full set of arguments and= use > >> > some > >> > >> > > value > >> > >> > > > > for > >> > >> > > > > > > > >> missing > >> > >> > > > > > > > >> > > > > > arguments. For example -1 for int/long > >> types > >> > >> and > >> > >> > > null > >> > >> > > > > for > >> > >> > > > > > > > string > >> > >> > > > > > > > >> > > type. > >> > >> > > > > > > > >> > > > We > >> > >> > > > > > > > >> > > > > > can't use 0 for int/long types since 0 > >> it's a > >> > >> > valid > >> > >> > > > > value > >> > >> > > > > > > for > >> > >> > > > > > > > >> > > > > concurrency, > >> > >> > > > > > > > >> > > > > > isolation and timeout arguments. > >> > >> > > > > > > > >> > > > > > * Serialize arguments as a collection= of > >> > >> > > > property-value > >> > >> > > > > > > pairs > >> > >> > > > > > > > >> > (like > >> > >> > > > > > > > >> > > > it's > >> > >> > > > > > > > >> > > > > > implemented now for CacheConfiguration= ). > >> In > >> > >> this > >> > >> > > case > >> > >> > > > > only > >> > >> > > > > > > > >> > explicitly > >> > >> > > > > > > > >> > > > > > provided arguments will be serialized. > >> > >> > > > > > > > >> > > > > > Which way is better? The simplest > >> solution is > >> > >> to > >> > >> > use > >> > >> > > > the > >> > >> > > > > > > first > >> > >> > > > > > > > >> > option > >> > >> > > > > > > > >> > > > > and I > >> > >> > > > > > > > >> > > > > > want to use it if there were no > >> objections. > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > Do we need transaction id (xid) on the > >> client > >> > >> > side? > >> > >> > > > > > > > >> > > > > > If yes, we can pass xid along with > >> > >> OP_TX_COMMIT, > >> > >> > > > > > > > OP_TX_ROLLBACK, > >> > >> > > > > > > > >> > > > > > OP_TX_CLOSE operations back to the ser= ver > >> and > >> > >> do > >> > >> > > > > > additional > >> > >> > > > > > > > >> check > >> > >> > > > > > > > >> > on > >> > >> > > > > > > > >> > > > the > >> > >> > > > > > > > >> > > > > > server side (current transaction id fo= r > >> > >> connection > >> > >> > > =3D=3D > >> > >> > > > > > > > >> transaction > >> > >> > > > > > > > >> > id > >> > >> > > > > > > > >> > > > > passed > >> > >> > > > > > > > >> > > > > > from client side). This, perhaps, will > >> > protect > >> > >> > > clients > >> > >> > > > > > > against > >> > >> > > > > > > > >> some > >> > >> > > > > > > > >> > > > > errors > >> > >> > > > > > > > >> > > > > > (for example when client try to commit > >> > outdated > >> > >> > > > > > > transaction). > >> > >> > > > > > > > >> But > >> > >> > > > > > > > >> > > > > > currently, we don't have data type > >> IgniteUuid > >> > >> in > >> > >> > > thin > >> > >> > > > > > client > >> > >> > > > > > > > >> > > protocol. > >> > >> > > > > > > > >> > > > Do > >> > >> > > > > > > > >> > > > > > we need to add it too? > >> > >> > > > > > > > >> > > > > > Also, we can pass xid as a string just= to > >> > >> inform > >> > >> > the > >> > >> > > > > > client > >> > >> > > > > > > > and > >> > >> > > > > > > > >> do > >> > >> > > > > > > > >> > > not > >> > >> > > > > > > > >> > > > > pass > >> > >> > > > > > > > >> > > > > > it back to the server with commit/roll= back > >> > >> > > operation. > >> > >> > > > > > > > >> > > > > > Or not to pass xid at all (.NET thick > >> client > >> > >> works > >> > >> > > > this > >> > >> > > > > > way > >> > >> > > > > > > as > >> > >> > > > > > > > >> far > >> > >> > > > > > > > >> > > as I > >> > >> > > > > > > > >> > > > > > know). > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > What do you think? > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > =D1=81=D1=80, 7 =D0=BC=D0=B0=D1=80. 20= 18 =D0=B3. =D0=B2 16:22, Vladimir > >> Ozerov < > >> > >> > > > > > > > >> vozerov@gridgain.com > >> > >> > > > > > > > >> > >: > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > > We already have transactions support= in > >> > JDBC > >> > >> > > driver > >> > >> > > > in > >> > >> > > > > > TX > >> > >> > > > > > > > SQL > >> > >> > > > > > > > >> > > branch > >> > >> > > > > > > > >> > > > > > > (ignite-4191). Currently it is > >> implemented > >> > >> > through > >> > >> > > > > > > separate > >> > >> > > > > > > > >> > thread, > >> > >> > > > > > > > >> > > > > which > >> > >> > > > > > > > >> > > > > > > is not that efficient. Ideally we ne= ed > >> to > >> > >> finish > >> > >> > > > > > > decoupling > >> > >> > > > > > > > >> > > > > transactions > >> > >> > > > > > > > >> > > > > > > from threads. But alternatively we c= an > >> > change > >> > >> > the > >> > >> > > > > logic > >> > >> > > > > > on > >> > >> > > > > > > > >> how we > >> > >> > > > > > > > >> > > > > assign > >> > >> > > > > > > > >> > > > > > > thread ID to specific transaction an= d > >> > >> > > "impersonate" > >> > >> > > > > thin > >> > >> > > > > > > > >> client > >> > >> > > > > > > > >> > > > worker > >> > >> > > > > > > > >> > > > > > > threads when serving requests from > >> multiple > >> > >> > users. > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM, Den= is > >> > Magda > >> > >> < > >> > >> > > > > > > > >> dmagda@apache.org> > >> > >> > > > > > > > >> > > > > wrote: > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > > Here is an original discussion wit= h a > >> > >> > reference > >> > >> > > to > >> > >> > > > > the > >> > >> > > > > > > > JIRA > >> > >> > > > > > > > >> > > ticket: > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > http://apache-ignite-developers.2346864.n4.nabble > >> > >> > > > . > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > > >> > com/Re-Transaction-operations-using-the-Ignite-Thin-Client- > >> > >> > > > > > > > >> > > > > > > > Protocol-td25914.html > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > -- > >> > >> > > > > > > > >> > > > > > > > Denis > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18 AM, > >> Dmitriy > >> > >> > > Setrakyan > >> > >> > > > < > >> > >> > > > > > > > >> > > > > > dsetrakyan@apache.org > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > wrote: > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > Hi Dmitriy. I don't think we hav= e a > >> > >> design > >> > >> > > > > proposal > >> > >> > > > > > > for > >> > >> > > > > > > > >> > > > transaction > >> > >> > > > > > > > >> > > > > > > > support > >> > >> > > > > > > > >> > > > > > > > > in thin clients. Do you mind tak= ing > >> > this > >> > >> > > > > initiative > >> > >> > > > > > > and > >> > >> > > > > > > > >> > > creating > >> > >> > > > > > > > >> > > > an > >> > >> > > > > > > > >> > > > > > IEP > >> > >> > > > > > > > >> > > > > > > > on > >> > >> > > > > > > > >> > > > > > > > > Wiki? > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > D. > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > On Tue, Mar 6, 2018 at 8:46 AM, > >> Dmitriy > >> > >> > > > > Govorukhin < > >> > >> > > > > > > > >> > > > > > > > > dmitriy.govorukhin@gmail.com> > >> wrote: > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > Hi, Igniters. > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > I've seen a lot of discussions > >> about > >> > >> thin > >> > >> > > > client > >> > >> > > > > > and > >> > >> > > > > > > > >> binary > >> > >> > > > > > > > >> > > > > > protocol, > >> > >> > > > > > > > >> > > > > > > > > but I > >> > >> > > > > > > > >> > > > > > > > > > did not hear anything about > >> > >> transactions > >> > >> > > > > support. > >> > >> > > > > > Do > >> > >> > > > > > > > we > >> > >> > > > > > > > >> > have > >> > >> > > > > > > > >> > > > some > >> > >> > > > > > > > >> > > > > > > draft > >> > >> > > > > > > > >> > > > > > > > > for > >> > >> > > > > > > > >> > > > > > > > > > this purpose? > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > As I understand we have severa= l > >> > >> problems: > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > - thread and transaction ha= ve > >> hard > >> > >> > > related > >> > >> > > > > (we > >> > >> > > > > > > use > >> > >> > > > > > > > >> > > > > thread-local > >> > >> > > > > > > > >> > > > > > > > > variable > >> > >> > > > > > > > >> > > > > > > > > > and thread name) > >> > >> > > > > > > > >> > > > > > > > > > - we can process only one > >> > >> transaction > >> > >> > at > >> > >> > > > the > >> > >> > > > > > same > >> > >> > > > > > > > >> time > >> > >> > > > > > > > >> > in > >> > >> > > > > > > > >> > > > one > >> > >> > > > > > > > >> > > > > > > thread > >> > >> > > > > > > > >> > > > > > > > > (it > >> > >> > > > > > > > >> > > > > > > > > > mean we need hold thread pe= r > >> > >> client. If > >> > >> > > > > connect > >> > >> > > > > > > 100 > >> > >> > > > > > > > >> thin > >> > >> > > > > > > > >> > > > > clients > >> > >> > > > > > > > >> > > > > > > to > >> > >> > > > > > > > >> > > > > > > > 1 > >> > >> > > > > > > > >> > > > > > > > > > server node, then need to h= old > >> 100 > >> > >> > thread > >> > >> > > > on > >> > >> > > > > > the > >> > >> > > > > > > > >> server > >> > >> > > > > > > > >> > > > side) > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > Let's discuss how we can imple= ment > >> > >> > > > transactions > >> > >> > > > > > for > >> > >> > > > > > > > the > >> > >> > > > > > > > >> > thin > >> > >> > > > > > > > >> > > > > > client. > >> > >> > > > > > > > >> > > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > > > > > > > > >> > >> > > > > > > > >> > > > > > > > >> > >> > > > > > > > >> > > > > > > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > -- > >> > >> > > > > > > > >> > > > > Sergey Kozlov > >> > >> > > > > > > > >> > > > > GridGain Systems > >> > >> > > > > > > > >> > > > > www.gridgain.com > >> > >> > > > > > > > >> > > > > > >> > >> > > > > > > > >> > > > > >> > >> > > > > > > > >> > > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > -- > >> > >> > > > > > > > >> > Sergey Kozlov > >> > >> > > > > > > > >> > GridGain Systems > >> > >> > > > > > > > >> > www.gridgain.com > >> > >> > > > > > > > >> > > >> > >> > > > > > > > >> > >> > >> > > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > > >> > > >> > > --=20 Best regards, Ivan Pavlukhin