kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laxmi Narayan NIT DGP <nit.dgp...@gmail.com>
Subject Re: adding more partition
Date Tue, 03 Jan 2017 07:25:53 GMT
where does kafka runs RAM or SSD ?

My question how to delete message frm Kafka topics.

Is message expire is only way ?




*Regards,*
*Laxmi Narayan Patel*
*MCA NIT Durgapur (2011-2014)*
*Mob:-    9741292048,8345847473    *

On Tue, Jan 3, 2017 at 12:46 PM, Kunal Gupta <kunal.gupta@cube26.com> wrote:

> For your Question 1 No, Message will be read only once by anyone of the
> consumer in a consumer group
>
> For your Question 2 Yes, Because each consumer group will read a message
> exactly once. Kafka is meant for Guaranteed Message Processing
>
> There are three ways of Guaranteed Message Processing 1) Exactly once 2) At
> least Once 3) At most once
> you can get information well in google.
>
> If you set auto commit true then message will not be deleted, it all deals
> with consumer offset for reading a topic. At which offset consumer group
> has read the topic.
>
> *Thanks, Kunal*
> *+91-9958189589*
> *Data Analyst*
> *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798
> <http://dl.acm.org/citation.cfm?id=2790798> *
> *Second Paper Publication : http://dl.acm.org/citation.cfm?id=2947432
> <http://dl.acm.org/citation.cfm?id=2947432>*
> *Blog:- **http://learnhardwithkunalgupta.blogspot.in
> <http://learnhardwithkunalgupta.blogspot.in> *
>
>
>
>
>
> On Tue, Jan 3, 2017 at 12:41 PM, Laxmi Narayan NIT DGP <
> nit.dgp673@gmail.com
> > wrote:
>
> > Hi ,
> >
> >  If I have kafka cluster along with replication .
> >
> > Say cluster of 3 kafka-server and replication factor of message with 2.
> >
> > Que 1:
> >
> > I have 1-consumer group but 2 consumer in one group reading from 1-Topic,
> >
> > If I enable auto commit true in consumer , will same message will be read
> > twice ?
> >
> > Que2:
> >
> > If I have 2 consumer group reading from 1-Topic
> >
> > If I enable auto commit true in consumer , will same message will be read
> > twice ?
> >
> >
> > and If I set auto-commit true will message will be deleted ?
> >
> >
> >
> >
> >
> >
> > *Regards,*
> > *Laxmi Narayan Patel*
> > *MCA NIT Durgapur (2011-2014)*
> > *Mob:-    9741292048,8345847473    *
> >
> > On Tue, Jan 3, 2017 at 12:34 PM, Kunal Gupta <kunal.gupta@cube26.com>
> > wrote:
> >
> > > I don't think so because more partition can led to unavailability but
> yes
> > > can led to higher throughput. But it cause more problems like
> increasing
> > > end to end latency, requires more open file handler and require more
> > memory
> > > at client side.
> > >
> > > *Thanks, Kunal*
> > > *+91-9958189589*
> > > *Data Analyst*
> > > *First Paper Publication : **http://dl.acm.org/citation.cfm?id=2790798
> > > <http://dl.acm.org/citation.cfm?id=2790798> *
> > > *Second Paper Publication : http://dl.acm.org/citation.cfm?id=2947432
> > > <http://dl.acm.org/citation.cfm?id=2947432>*
> > > *Blog:- **http://learnhardwithkunalgupta.blogspot.in
> > > <http://learnhardwithkunalgupta.blogspot.in> *
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Jan 3, 2017 at 12:25 PM, Laxmi Narayan NIT DGP <
> > > nit.dgp673@gmail.com
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Does adding more partitions to kafka will help in scaling ?
> > > >
> > > >
> > > > *Regards,*
> > > > *Laxmi Narayan Patel*
> > > > *MCA NIT Durgapur (2011-2014)*
> > > > *Mob:-    9741292048,8345847473    *
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message