chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corbin Hoenes <>
Subject Re: Scribe vs. Chukwa
Date Mon, 04 Jan 2010 18:03:15 GMT

Thanks for the input.  I found another thread on the general hadoop list that pertains to
the the discussion.  I'll post here for future readers:

Your explanation helps a lot though.  I'd second the idea of have a Chukwa-vs page on the
website to help people understand what it can do for them.  We are trying to get our logs
into HDFS so I think chukwa will provide a lot of value in this regard.


On Dec 24, 2009, at 9:51 AM, Ariel Rabkin wrote:

> Howdy.
> It's a good question, and maybe we should have a
> Chukwa-vs-the-competition page on the website.
> Chukwa is designed to help collect system logs, and store them in
> HDFS.  Scribe is designed as more of a syslog++.  We went to some
> trouble to make sure our metadata and output formats were MapReduce
> friendly, whereas I think they leave that much more up to you.
> My understanding is that Scribe exposes a Thrift service, only. So if
> you want to use Scribe, you pretty much have to go all-in and modify
> your app and deployment to send logs via Thrift. Chukwa can just read
> whatever files or data sources are lying around.  We can pretty easily
> listen for incoming Thrift RPC calls -- I believe Jerome is currently
> coding that up.
> Chukwa comes with visualization and processing tools, whereas Scribe doesn't.
> Scribe might be slightly lower overhead -- I believe it's C++ and
> Chukwa is Java.  Chukwa isn't too big, either, so the absolute
> difference might not be significant.
> --Ari
> On Wed, Dec 23, 2009 at 3:06 PM, Corbin Hoenes <> wrote:
>> Does anyone know of a comparison between Chukwa and Scribe(
>> I am interested in why someone might choose Chukwa vs. Scribe and their pros/cons.
> -- 
> Ari Rabkin
> UC Berkeley Computer Science Department

Corbin Hoenes
skype: choenes

View raw message