kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aruna ramachandran <arunaeie...@gmail.com>
Subject Re: Doubts in Kafka
Date Tue, 08 Jan 2019 16:11:34 GMT
I need to process single sensor messages in serial (order of messages
should not be changed)at the same time I have to process 10000 sensors
messages in parallel please help me to configure the topics and partitions.

On Tue, Jan 8, 2019 at 9:19 PM Todd Palino <tpalino@gmail.com> wrote:

> I think you’ll need to expand a little more here and explain what you mean
> by processing them in parallel. Nearly by definition, parallelization and
> strict ordering are mutually exclusive concepts.
>
> -Todd
>
> On Tue, Jan 8, 2019 at 10:40 AM aruna ramachandran <arunaeienec@gmail.com>
> wrote:
>
> > I need to process the 10000 sensor messages in parallel but each sensor
> > message should be in order.If I create 10000 partition it doesn't give
> high
> > throughput .Order is guaranteed only inside the partition. How can
> > parallelize messages without changing the order pls help me to find the
> > solution.
> >
>
>
> --
> *Todd Palino*
> Senior Staff Engineer, Site Reliability
> Data Infrastructure Streaming
>
>
>
> linkedin.com/in/toddpalino
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message