hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: hbase bulk writes
Date Mon, 30 Nov 2009 22:41:18 GMT
Could you put your data in HDFS and load it from there with a MapReduce job?

J-D

On Mon, Nov 30, 2009 at 2:33 PM, Calvin <calvin.lists@gmail.com> wrote:
> I have a large amount of sequential ordered rows I would like to write to an
> HBase table.  What is the preferred way to do bulk writes of multi-column
> tables in HBase?  Using the get/put interface seems fairly slow even if I
> bulk writes with table.put(List<Put>).
>
> I have followed the directions on:
>   * http://wiki.apache.org/hadoop/PerformanceTuning
>   *
> http://ryantwopointoh.blogspot.com/2009/01/performance-of-hbase-importing.html
>
> Are there any other resources for improving the throughput of my bulk
> writes?  On
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapreduce/package-summary.htmlI
> see there's a way to write HFiles directly, but HFileOutputFormat can
> only
> write a single column famly at a time (
> https://issues.apache.org/jira/browse/HBASE-1861).
>
> Thanks!
>
> -Calvin
>

Mime
View raw message