kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Surendranauth Hiraman <suren.hira...@sociocast.com>
Subject Re: Kafka for *critical* data
Date Fri, 01 Nov 2013 18:05:10 GMT
I'm a fan of kafka as well. We've been using 0.7.2 for about a year.

I recommend it strongly.

But I will point one thing. Not an issue with Kafka itself but when the
client side has failed, choosing what offset to reset to is not an exact
science. You will have to decide how much data loss or data replay is
sufficient for you.

But overall, for our requirements, the second issue has not been a major
problem.

-Suren



On Fri, Nov 1, 2013 at 12:14 PM, Olivier Pomel <oli@datadoghq.com> wrote:

> Richard - I can't speak about 0.8x, but we've been using Kafka in a most
> critical capacity on production since the early days @ Datadog.
>
> And...
>  - It never failed (!)
>  - It scaled predictably
>  - Its performance was consistent
>  - The simplicity of its design made it really easy to reason about
>
> We use lots of other great piece of technologies: Redis, Cassandra,
> Postgres, Elasticsearch, AWS. They're all widely accepted as "ready for
> production" and we love them, yet we found Kafka to be the part of our
> infrastructure that is the least likely to cause surprises.
>
> Hope this helps.
> Cheers,
> --
> Olivier Pomel | http://datadog.com
>
> (oh one small precision: we're not using dynamically redistributed
> high-level consumers)
>
>
> On Fri, Nov 1, 2013 at 9:30 AM, Richard Rodseth <rrodseth@gmail.com>
> wrote:
>
> > I'm excited about Kafka but want to be sure it is ready (or will be soon)
> > for a critical data pipeline.
> >
> > What are the showstoppers, if any? eg.
> >
> > https://issues.apache.org/jira/browse/KAFKA-156
> >
> > mentioned here:
> >
> >
> >
> http://stackoverflow.com/questions/12130481/is-kafka-ready-for-production-use/12764663#12764663
> >
> > I've looked at this list:
> > https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
> >
> > Comments?
> >
>



-- 
___________________________________________
Available at these partners:
[image: CloudFlare | shopify | Bigcommerce]

SUREN HIRAMAN, VP TECHNOLOGY
SOCIOCAST
Simple. Powerful. Predictions.

96 SPRING STREET, 7TH FLOOR
NEW YORK, NY 10012
O: (917) 525-2466 ext. 105
F: 646.349.4063
E: suren.hiraman@sociocast.com
W: www.sociocast.com

Increase Conversion Rates up to 500%. Go to www.sociocast.com and enter
your URL for a free trial!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message