kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anas A <anas.2...@gmail.com>
Subject Re: Increase number of topic in Kafka leads zookeeper fail
Date Wed, 18 May 2016 04:39:33 GMT
Hi there,

Thank you for your valuable support now I changed the method as one topic
with several partition for all devices and its solved my problem.



*Thanks & Regards,*


Anas A
DBA, Trinity Mobility
[image: facebook] <https://www.facebook.com/anas.24aj> [image: twitter]
<https://twitter.com/anas24aj> [image: linkedin]
<http://in.linkedin.com/in/anas24aj> [image: googleplus]
<https://plus.google.com/u/0/+anasA24aj/>
+917736368236
anas.24aj@gmail.com
Bangalore

On 16 May 2016 at 16:41, Tom Crayford <tcrayford@heroku.com> wrote:

> Hi there,
>
> Generally you don't use a single topic per device in this use case, but one
> topic with some number of partitions and the key distribution based on
> device id. Kafka isn't designed for millions of low volume topics, but a
> few high volume ones.
>
> Thanks
>
> Tom Crayford
> Heroku Kafka
>
> On Mon, May 16, 2016 at 5:23 AM, Anas A <anas.24aj@gmail.com> wrote:
>
> > We plan to use kafka as a message broker for IoT use case, where each
> > device is considered as unique topic. when I simulated 10 message per
> > second to 10 thousand topics zookeeper is getting bottle neck,all Kafka
> > monitoring tools fails to read the throughput values and number of topics
> > from JMX port because of that. will tuning zookeeper will solve the
> issues.
> > where In IoT use case there will be millions of device polling data to
> > millions of topics. I want to make sure the approach is perfect to go.
> > Please suggest.
> >
> >
> > *Thanks & Regards,*
> >
> >
> > Anas A
> > DBA, Trinity Mobility
> > [image: facebook] <https://www.facebook.com/anas.24aj> [image: twitter]
> > <https://twitter.com/anas24aj> [image: linkedin]
> > <http://in.linkedin.com/in/anas24aj> [image: googleplus]
> > <https://plus.google.com/u/0/+anasA24aj/>
> > +917736368236
> > anas.24aj@gmail.com
> > Bangalore
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message