kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marko Bonaći <marko.bon...@sematext.com>
Subject Re: Best way to commit offset on demand
Date Mon, 04 Jan 2016 11:26:12 GMT
How are you consuming those topics?

IF: I assume you have a consumer, so why not commit from within that
consumer, after you process the message (whatever "process" means to you).

ELSE: couldn't you have a dedicated consumer for offset commit requests
that you don't shut down between requests?

FINALLY: tell us more about your use case.

Marko Bonaći
Monitoring | Alerting | Anomaly Detection | Centralized Log Management
Solr & Elasticsearch Support
Sematext <http://sematext.com/> | Contact
<http://sematext.com/about/contact.html>

On Mon, Jan 4, 2016 at 12:18 PM, tao xiao <xiaotao183@gmail.com> wrote:

> Hi team,
>
> I have a scenario where I want to write new offset for a list of topics on
> demand. The list of topics is unknown until runtime and the interval
> between each commit is undetermined. what would be the best way to do so?
>
> One way I can think of is to create a new consumer and call
> commitSync(offsets) every time I want to commit. But it seems taking too
> much time to bootstrap the consumer. is there a lighter way to achieve
> this?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message