kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shahab <shahab.mok...@gmail.com>
Subject Re: How to read in batch using HighLevel Consumer?
Date Tue, 04 Aug 2015 19:04:57 GMT
Thanks a lot Shaminder for clarification and thanks Raja for pointing me to
the example.

best,
/shahab

On Tue, Aug 4, 2015 at 6:06 PM, Rajasekar Elango <relango@salesforce.com>
wrote:

> Here is an example on what sharninder suggested
>
> http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
>
> Thanks,
> Raja.
>
> On Tue, Aug 4, 2015 at 12:01 PM, Sharninder <sharninder@gmail.com> wrote:
>
> > You can't. Kafka is essentially a queue, so you always read messages one
> > by one. What you can do is disable auto offset commit, read 100 messages,
> > process them and then manually commit offset.
> >
> > --
> > Sharninder
> >
> > > On 04-Aug-2015, at 9:07 pm, shahab <shahab.mokari@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > While we the producer can put data as batch in kafka server,  I
> couldn't
> > > find any API (or any document) saying how we can fetch data as batch
> from
> > > Kafka ?
> > > Even when data is placed as batch in kafka server, still using High
> Level
> > > consumer I can only read one by one, and I can not specify. for
> example,
> > > read 100 items at once!
> > >
> > > Is this correct observation? or I am missing something?
> > >
> > > best,
> > > /Shahab
> >
>
>
>
> --
> Thanks,
> Raja.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message