kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devin Kramer <dkra...@rubiconproject.com.INVALID>
Subject Re: Kafka in python
Date Fri, 12 Jul 2019 17:43:18 GMT
If your consumer is always starting with the same consumer group ID it
should pick up where it left off.   This also assumes your consumer is
correctly sending its offsets to the __consumer_offsets topic and that the
time between it starting and stopping is not so long that whatever
retention you have set on that topic is expiring your consumers offset
value.   That being said Is there a technical reason your consumer can't
simply connect and stay connected and consuming as messages come in rather
then having to start up occasionally?  Although the later should work the
former is probably more inline with a standard kafka consumer, although I
may be speaking above my knowledge grade on that..

On Fri, Jul 12, 2019 at 8:23 AM Moustafa Mohamed Sadik <
mohamedsadik.moustafa@natixis.com> wrote:

> Hello,
> I am using kafka by python packages.
> Is it possible for server to retain messages when no consumer is available?
> I want to start producer and send message to the server (this part is ok).
> After that (when the sending to server is finished), I want to start a
> consumer and get the messages from the last read message.
> Actually when I do that, no message is shown in the consumer or when it is
> specified in the parameters, I got all messages from beginning.
> So how to get the message after the last consumed one when I start my
> consumer?
> Thanks
> MOUSTAFA Mohamed Sadik
> Apprenti ingénieur en informatique
> mohamedsadik.moustafa@natixis.com<mailto:mohamedsadik.moustafa@natixis.com
> >
> +33158553880-23880
> Natixis SA
> C1 - Public Natixis
> <http://www.natixis.com/natixis/jcms/rpaz5_78162/fr/natixis>
> L'integrite de ce message n'etant pas assuree sur internet, Natixis ne
> peut etre tenu responsable de son contenu. Toute utilisation ou diffusion
> non autorisee est interdite. Si vous n'etes pas destinataire de ce message,
> merci de le detruire et d'avertir l'expediteur. Ensemble, faisons un geste
> pour l'environnement : n'imprimons nos mails que si necessaire.
> Pour consulter l’ensemble des disclaimers de l’entreprise, veuillez
> consulter l’url :
> https://www.natixis.com/natixis/upload/docs/application/pdf/2017-09/disclaimer.pdf
> <
> https://www.natixis.com/natixis/upload/docs/application/pdf/2017-09/disclaimer.pdf
> >
> The integrity of this message cannot be guaranteed on the Internet.
> Natixis cannot therefore be considered responsible for the contents. Any
> unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, please
> delete it and notify the sender. Please respect the environment and print
> emails only when necessary.
> To consult the entire Natixis disclaimers, please use the following link :
> https://www.natixis.com/natixis/upload/docs/application/pdf/2017-09/disclaimer.pdf
> <
> https://www.natixis.com/natixis/upload/docs/application/pdf/2017-09/disclaimer.pdf
> >


Devin Kramer Director, Infrastructure Engineering


RUBICONPROJECT.COM <http://www.rubiconproject.com>  |  JOIN US
<http://rubiconproject.com/join-us/>  |  FOLLOW US

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message