kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hisham Mardam-Bey <his...@mate1inc.com>
Subject Re: Kafka is live in prod @ 100%
Date Tue, 06 Dec 2011 18:51:00 GMT
Very cool - thanks for sharing Taylor. I'll get some time to do the
same for our use case soon (=


On Tue, Dec 6, 2011 at 11:30 AM, Taylor Gautier <tgautier@tagged.com> wrote:
> I've already mentioned this before, but I wanted to give a quick shout to
> let you guys know that our newest game, Deckadence, is 100% live as of
> yesterday.
> Check it out at http://www.tagged.com/deckadence.html
> A little about our use case:
>   - Deckadence is a game of buying and selling - or rather trading -
>   cards.  Every user on Tagged owns a card.  There are 100M uses on Tagged,
>   so that means there are 100M cards to trade.
>   - Kafka enables real-time delivery of events in the game
>   - An end user browser makes a long-poll event http connection to receive
>   1:1 messages and 1:M messages from a specialized http server we built for
>   this purpose.  1:M messages are delivered from Kafka.
>   - Because of this design, we can publish a message anywhere inside our
>   datacenter and send it directly and immediately to any other system that is
>   subscribed to Kafka, or to an end-user browser
>   - Every update event for every card is sent to a unique topic that
>   represents the users card.
>   - When a user is browsing any card or list of cards - say a search
>   result - their browser subscribes to all of the cards on screen.
>   - The effect of this is that any changes to any card seen on-screen are
>   seen in real-time by all users of the game
>   - Our primary producers and consumers are PHP and NodeJS, respectively
> Well, I plan to write up more about this use case in the near future.  As
> you might have guessed, this is just about as far away from the original
> intent of Kafka as you could get - we have PHP that sends messages to
> Kafka.  Since it's not good to hold a TCP connection open in PHP, we had to
> do some trickery here.  There was no existing Node client so we had to
> write our own.  And since there are 100 million users registered on Tagged,
> that means we could have in theory 100M topics.  Of course in practice we
> have far fewer than that.  One of the main things we currently have to do
> is aggressively clean topics.  But basically we have many topics, few
> messages (relatively) per topic.  And order matters, so we had to deal with
> ensuring that we could handle the number of topics we would create, and
> ensure ordered delivery and receipt.
> In the future I have big plans for Kafka, another feature is currently in
> private test and will be released to the public soon (it uses Kafka in a
> more traditional way).  And we hope to have many more in 2012...

Hisham Mardam Bey
Director of Engineering | Mate1 Inc.
4200 St. Laurent Boulevard | Suite 550
Montreal, Quebec | H2W 2R2
t. +1.514.393.1414 x264

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

-=[ Codito Ergo Sum ]=-

View raw message