crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Shi (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-212) Need target wrapper for HFileOuptutFormat
Date Thu, 01 Aug 2013 07:09:54 GMT


Chao Shi updated CRUNCH-212:

    Attachment: crunch-212-v1.patch

update patch to fix issues mentioned by Gabriel and Micah:
- use mini MR cluster to test with multiple regions (which is much slower than LocalJobRunner)
- comment a small paragraph to justify the use of HFileOutputFormatForCrunch
- remove the duplicate call of getDefaultWorkFile
> Need target wrapper for HFileOuptutFormat
> -----------------------------------------
>                 Key: CRUNCH-212
>                 URL:
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>            Reporter: Chao Shi
>         Attachments: crunch-212-draft2.patch, crunch-212-draft.patch, crunch-212-v0.patch,
> I need to import data to hbase from MR. I found HFileOutputFormat is ~5x more efficient
than HTableOutputFormat. So maybe we need a target wrapper for it.
> Future more, is it possible to call HBase to load it automatically after HFiles are generated?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message