kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: Kafka when volume is low
Date Sun, 16 Dec 2012 16:28:05 GMT
Hey Matan,

At LinkedIn we end up having lots of topics--some are very high volume and
some very very low volume. It works fine for low-volume topics.

At this time we don't support any kind of per-consumer throttling, that
would be a nice thing to have.

-Jay


On Sun, Dec 16, 2012 at 2:23 AM, Matan Safriel <matan@cloudaloe.org> wrote:

> Hi,
>
> I've read about Kafka being used for huge paces and volumes of messages. My
> scenario is a bit different, I'm looking for mostly only persistence of a
> message queue, where the queue is usually not huge.
> This scenario means that messages are not typically queued for long.
>
> Why Kafka? What I like about Kafka is that on those occasions where
> messages do accumulate beyond normal (e.g. when the receiving side is not
> accessible), then things are well behaved, as the messages are queued on
> disk. And that it materializes a form of resilient persistency leaner than
> a database.
>
> I'd keenly appreciate your feedback twofold. First and foremost, have you
> used or observed Kafka in low-traffic scenarios, is it efficient in memory
> footprint when volume is low etc, or do you foresee any prominent problem
> in that type of scenario?
>
> Secondly, assume that once in a while, messages do accumulate in large
> amounts on the queue, such as due to a network issue, and then when the
> receiving end is ready to consume them, can bandwidth utilization be capped
> such that the network isn't congested by that event.
>
> Thanks in advance,
> Matan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message