kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: producer transaction and threading model
Date Wed, 20 Nov 2019 14:05:48 GMT
Spring framework works around this by providing a method like
doInTransaction( listofstuff).

Behind the scenes it manages a pool of transactional producers with ids
like  transaction_prefix + id.

So each call to dointransaction may initiate a transaction.

On Friday, November 15, 2019, Matthias J. Sax <matthias@confluent.io> wrote:

> That's correct. And each would need to use a different `transactional.id`.
>
>
> -Matthias
>
> On 11/14/19 11:17 AM, Anindya Haldar wrote:
> > Thanks for the information. Does that mean that each producer thread, in
> case it wants to have its own transactions, should use its own instance of
> KafkaProducer?
> >
> > Sincerely,
> > Anindya Haldar
> > Oracle Responsys
> >
> >
> >> On Nov 13, 2019, at 11:31 PM, Matthias J. Sax <matthias@confluent.io>
> wrote:
> >>
> >> That is not possible. A producer can only have a single open
> >> transaction. If your example, the call of thread-2 to start a new
> >> transactions would fail and an exception would be thrown.
> >>
> >> -Matthias
> >>
> >> On 11/13/19 10:52 AM, Anindya Haldar wrote:
> >>> The producer API doc (https://kafka.apache.org/23/
> javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html <
> https://kafka.apache.org/23/javadoc/index.html?org/apache/
> kafka/clients/producer/KafkaProducer.html>) says that KafkaProducer is
> thread safe.
> >>>
> >>> What I was trying to figure out is what the relationship is between a
> thread context and a transaction initiated in that thread. When multiple
> threads are using a single producer instance and initiating transactions
> are those transaction boundaries scoped to the thread context, or do they
> step on each others’ toes?
> >>>
> >>> Suppose two threads are sharing the same producer instance the
> following way:
> >>>
> >>>    thread-1 begins a transaction T1
> >>>    thread-1 starts sending messages in T!
> >>>    thread-2 begins a transaction T2
> >>>    thread-2 sends some messages in T2
> >>>    thread-2 gets and error and rolls back its transaction T2
> >>>
> >>> At this point does the rollback of transaction T2 by the thread-2
> affect the transaction T1 remaining open and used by thread-1 in any
> fashion? It is not clear to me from the API documentation. Will very much
> appreciate some insights into the behavior here. Similar questions also
> arise for commits by threads independently of each other.
> >>>
> >>> Sincerely,
> >>> Anindya Haldar
> >>> Oracle Responsys
> >>>
> >>>
> >>
> >
>
>

-- 
Sorry this was sent from mobile. Will do less grammar and spell check than
usual.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message