kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kunal Gupta <kunal.gu...@cube26.com>
Subject Re: adding more partition
Date Tue, 03 Jan 2017 07:16:35 GMT
For your Question 1 No, Message will be read only once by anyone of the
consumer in a consumer group

For your Question 2 Yes, Because each consumer group will read a message
exactly once. Kafka is meant for Guaranteed Message Processing

There are three ways of Guaranteed Message Processing 1) Exactly once 2) At
least Once 3) At most once
you can get information well in google.

If you set auto commit true then message will not be deleted, it all deals
with consumer offset for reading a topic. At which offset consumer group
has read the topic.

*Thanks, Kunal*
*+91-9958189589*
*Data Analyst*
*First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798
<http://dl.acm.org/citation.cfm?id=2790798> *
*Second Paper Publication : http://dl.acm.org/citation.cfm?id=2947432
<http://dl.acm.org/citation.cfm?id=2947432>*
*Blog:- **http://learnhardwithkunalgupta.blogspot.in
<http://learnhardwithkunalgupta.blogspot.in> *





On Tue, Jan 3, 2017 at 12:41 PM, Laxmi Narayan NIT DGP <nit.dgp673@gmail.com
> wrote:

> Hi ,
>
>  If I have kafka cluster along with replication .
>
> Say cluster of 3 kafka-server and replication factor of message with 2.
>
> Que 1:
>
> I have 1-consumer group but 2 consumer in one group reading from 1-Topic,
>
> If I enable auto commit true in consumer , will same message will be read
> twice ?
>
> Que2:
>
> If I have 2 consumer group reading from 1-Topic
>
> If I enable auto commit true in consumer , will same message will be read
> twice ?
>
>
> and If I set auto-commit true will message will be deleted ?
>
>
>
>
>
>
> *Regards,*
> *Laxmi Narayan Patel*
> *MCA NIT Durgapur (2011-2014)*
> *Mob:-    9741292048,8345847473    *
>
> On Tue, Jan 3, 2017 at 12:34 PM, Kunal Gupta <kunal.gupta@cube26.com>
> wrote:
>
> > I don't think so because more partition can led to unavailability but yes
> > can led to higher throughput. But it cause more problems like increasing
> > end to end latency, requires more open file handler and require more
> memory
> > at client side.
> >
> > *Thanks, Kunal*
> > *+91-9958189589*
> > *Data Analyst*
> > *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798
> > <http://dl.acm.org/citation.cfm?id=2790798> *
> > *Second Paper Publication : http://dl.acm.org/citation.cfm?id=2947432
> > <http://dl.acm.org/citation.cfm?id=2947432>*
> > *Blog:- **http://learnhardwithkunalgupta.blogspot.in
> > <http://learnhardwithkunalgupta.blogspot.in> *
> >
> >
> >
> >
> >
> > On Tue, Jan 3, 2017 at 12:25 PM, Laxmi Narayan NIT DGP <
> > nit.dgp673@gmail.com
> > > wrote:
> >
> > > Hi,
> > >
> > > Does adding more partitions to kafka will help in scaling ?
> > >
> > >
> > > *Regards,*
> > > *Laxmi Narayan Patel*
> > > *MCA NIT Durgapur (2011-2014)*
> > > *Mob:-    9741292048,8345847473    *
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message