chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <>
Subject [jira] Updated: (CHUKWA-449) Clreate utility to generate a sequence files from a log file
Date Thu, 21 Jan 2010 23:58:54 GMT


Bill Graham updated CHUKWA-449:

    Attachment: CHUKWA-449-1.patch

Attaching CHUKWA-449-1.patch.

I've added a new method to TempFileUtil: 

public static void makeTestSequenceFile(File inputFile,
                                        Path outputFile,
                                        String clusterName,
                                        String dataType,
                                        String streamName,
                                        MapProcessor processor) throws IOException

I've also included a main method, with the following usage message:
Usage: java org.apache.hadoop.chukwa.util.TempFileUtil <inputFile> <outputFile>
[clusterName] [dataType] [streamName] [processorClass]
Description: Takes a plain text input file and generates a Hadoop sequence
             file contaning ChukwaRecordKey,ChukwaRecord entries
Parameters: inputFile      - Text input file to read
            outputFile     - Where to write the sequence file
            clusterName    - Cluster name to use in the records
            dataType       - Data type to use in the records
            streamName     - Stream name to use in the records
            processorClass - Processor class to use. Defaults to TsProcessor

I wasn't sure where to put this code, so let me know if there's a better home for it. Also,
since this is just a static helper utility there isn't a unit test.

> Clreate utility to generate a sequence files from a log file
> ------------------------------------------------------------
>                 Key: CHUKWA-449
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-449-1.patch
> See this thread:
> We should have a utility class that can generate a Chukwa sequence file from a raw log

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message