kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clark Haskins <cl...@kafka.guru>
Subject Re: Post on running Kafka at LinkedIn
Date Fri, 20 Mar 2015 21:27:58 GMT
Yep! We are growing :)

-Clark

Sent from my iPhone

> On Mar 20, 2015, at 2:14 PM, James Cheng <jcheng@tivo.com> wrote:
> 
> Amazing growth numbers.
> 
> At the meetup on 1/27, Clark Haskins presented their Kafka usage at the time. It was:
> 
> Bytes in: 120 TB
> Messages In: 585 million
> Bytes out: 540 TB
> Total brokers: 704
> 
> In Todd's post, the current numbers:
> 
> Bytes in: 175 TB (45% growth)
> Messages In: 800 billion (36% growth)
> Bytes out: 650 TB (20% growth)
> Total brokers: 1100 (56% growth)
> 
> That much growth in just 2 months? Wowzers.
> 
> -James
> 
>> On Mar 20, 2015, at 11:30 AM, James Cheng <jcheng@tivo.com> wrote:
>> 
>> For those who missed it:
>> 
>> The Kafka Audit tool was also presented at the 1/27 Kafka meetup:
>> http://www.meetup.com/http-kafka-apache-org/events/219626780/
>> 
>> Recorded video is here, starting around the 40 minute mark:
>> http://www.ustream.tv/recorded/58109076
>> 
>> Slides are here:
>> http://www.ustream.tv/recorded/58109076
>> 
>> -James
>> 
>>> On Mar 20, 2015, at 9:47 AM, Todd Palino <tpalino@gmail.com> wrote:
>>> 
>>> For those who are interested in detail on how we've got Kafka set up at
>>> LinkedIn, I have just published a new posted to our Engineering blog titled
>>> "Running Kafka at Scale"
>>> 
>>>   https://engineering.linkedin.com/kafka/running-kafka-scale
>>> 
>>> It's a general overview of our current Kafka install, tiered architecture,
>>> audit, and the libraries we use for producers and consumers. You'll also be
>>> seeing more posts from the SRE team here in the coming weeks on deeper
>>> looks into both Kafka and Samza.
>>> 
>>> Additionally, I'll be giving a talk at ApacheCon next month on running
>>> tiered Kafka architectures. If you're in Austin for that, please come by
>>> and check it out.
>>> 
>>> -Todd
> 

Mime
View raw message