storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raja.Aravapalli <Raja.Aravapa...@target.com>
Subject Re: storm-kafka integration - example application or a git link of a storm topology that consumes and process maintaining the kafka partition level ordering ?
Date Tue, 31 May 2016 15:23:03 GMT

Thanks a lot for your response lujinhong.

I have build a storm topology integrating storm-kafka integration, but my immediate bolt from
kafka-spout is not processing the tuples in order which is causing many records to ignore
without processing. Any help on what configuration of storm bolt I can take a look or change
to ensure my bolt processes the records in the order it receives ??

Or any other ways I can try to guarantee that my bolt is not-processing/processing in the
order it receives ?

Thanks a lot again!!!

Regards,
Raja.


From: jinhong lu <lujinhong2@gmail.com<mailto:lujinhong2@gmail.com>>
Reply-To: "user@storm.apache.org<mailto:user@storm.apache.org>" <user@storm.apache.org<mailto:user@storm.apache.org>>
Date: Monday, May 30, 2016 at 3:01 AM
To: "user@storm.apache.org<mailto:user@storm.apache.org>" <user@storm.apache.org<mailto:user@storm.apache.org>>
Subject: Re: storm-kafka integration - example application or a git link of a storm topology
that consumes and process maintaining the kafka partition level ordering ?

Storm-kafka is born to partition level ordering, it means every application is partition level
ordering(unless you buffer the message in memory and change the order), so no configuration
need to add for ordering.

Thanks,
lujinhong

在 2016年5月27日,23:39,Raja.Aravapalli <Raja.Aravapalli@target.com<mailto:Raja.Aravapalli@target.com>>
写道:

Hi

I know, storm doesn't guarantee total ordering gurantee for kafka topics, but see in many
documents, storm guarantees consumption/processing the messages maintaining the order at partition
level.

I am looking for a sample storm topology, that consumes/processes the messages of a kafka
topic maintaining the order of messages at a kafka partition level.. NOT Total Order!! ONLY
partition level ordering guarantee.

please share if you know any sample application. Thanks a lot!!



-Raja.

Mime
View raw message