chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <>
Subject how to generate a Chukwa SequenceFile
Date Tue, 19 Jan 2010 23:46:10 GMT

Is there an easy way (maybe using a utility class or the chukwa API) to
manually create a sequence file of chukwa records from a log file without
the need for HDFS?

My use case is this: I've got pig unit tests that read input sequence file
input using ChukwaStorage from local disk. I generated these files by
putting data into the cluster an waiting for the data processor to run.
We're looking to change the log format though, and I'd like to be able to
write and run the unit tests without putting the new data into the cluster.

If there were a command line way that I could do this that would be very
helpful. Or if anyone could point me to the relevant classes, I could write
such a utility and contribute it back.


View raw message