kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Burroughs <chris.burrou...@gmail.com>
Subject Re: Unit test failure
Date Sat, 03 Dec 2011 16:32:45 GMT
On 11/30/2011 01:02 AM, Jay Kreps wrote:
> I think this is one of the core problems of this type of system--how to do
> good, realistic distributed testing. I have not seen anyone really succeed
> except via the "throw production load at it and see" approach so I would be
> interested to hear people's thoughts.

Cassandra has the same problem and has gone through several iterations
of distributed test systems.  The most recent is here:
https://github.com/riptano/cassandra-dtest  Time will tell if this
attempt is more successful.  The obvious prerequisite for something like
this is a set of scripts that make it easy to set up Kafka clusters of
different topologies for testing.

Another thing they do well is having a solid stress/perf tool
distributed with releases, everyone agrees it gives decent results, easy
to hack if you want to test something slightly different and thus it's
widely used.

View raw message