kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajasekar Elango <rela...@salesforce.com>
Subject Re: How to read in batch using HighLevel Consumer?
Date Tue, 04 Aug 2015 16:06:27 GMT
Here is an example on what sharninder suggested
http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/

Thanks,
Raja.

On Tue, Aug 4, 2015 at 12:01 PM, Sharninder <sharninder@gmail.com> wrote:

> You can't. Kafka is essentially a queue, so you always read messages one
> by one. What you can do is disable auto offset commit, read 100 messages,
> process them and then manually commit offset.
>
> --
> Sharninder
>
> > On 04-Aug-2015, at 9:07 pm, shahab <shahab.mokari@gmail.com> wrote:
> >
> > Hi,
> >
> > While we the producer can put data as batch in kafka server,  I couldn't
> > find any API (or any document) saying how we can fetch data as batch from
> > Kafka ?
> > Even when data is placed as batch in kafka server, still using High Level
> > consumer I can only read one by one, and I can not specify. for example,
> > read 100 items at once!
> >
> > Is this correct observation? or I am missing something?
> >
> > best,
> > /Shahab
>



-- 
Thanks,
Raja.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message