chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <>
Subject [jira] [Commented] (CHUKWA-632) Logs getting lost in hbase using TsProcessor
Date Tue, 20 Nov 2012 20:50:58 GMT


Hudson commented on CHUKWA-632:

Integrated in Chukwa-trunk #460 (See [])
    CHUKWA-632. Added getter method for startOffset in AbstractProcessor. (Abhijit Dhar via
Eric Yang) (Revision 1411810)

     Result = SUCCESS
eyang :
Files : 
* /incubator/chukwa/trunk/CHANGES.txt
* /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/

> Logs getting lost in hbase using TsProcessor
> --------------------------------------------
>                 Key: CHUKWA-632
>                 URL:
>             Project: Chukwa
>          Issue Type: Bug
>    Affects Versions: 0.5.0
>            Reporter: Abhijit Dhar
>            Assignee: Abhijit Dhar
>         Attachments: patch.txt
> When collector writes to hbase using TsProcessor, it is using timestamp as the key. If
there are multiple log statements with the same timestamp, which is the key, it looses them
in hbase.
> As a fix, I extended AbstractProcessor and built my key using timestamp and the AbstractProcessor.startOffset.
But, for that, it has to be protected or a getter on it in AbstractProcessor class to be able
to use it.
> Also, the way OutputCollector.collect method works, it might need to redo the way rowKey
is created.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message