hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Baranau <alex.barano...@gmail.com>
Subject Re: Inserting Random Data into HBASE
Date Wed, 01 Dec 2010 16:11:40 GMT
Well, if you are talking about importing large amount of (existing) data you
should look at bulk loads (
http://hbase.apache.org/docs/r0.89.20100924/bulk-loads.html).

In case you still want to measure write performance using Puts, then make
sure you use proper writeBufferSize and (if it is ok in your situation)
writeToWAL settings.

In general, this question (and related) raised a lot of times on mailing
list, try to search for relevant info (e.g. using search-hadoop.com service)
first and ask more specific questions (and please include more info about
your hardware, etc.).

Hope this helps,

Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase

On Wed, Dec 1, 2010 at 4:47 PM, rajgopalv <raja.fire@gmail.com> wrote:

>
> Hi,
> I have to test hbase as to how long it takes to store 100 Million Records.
>
> So i wrote a simple java code which
>
> 1 : generates random key and 10 columns per key and random values for the
> 10
> columns.
> 2 : I make a Put object out of these and store it in arrayList
> 3 : When arrayList's size reaches 5000 i do table.put(listOfPuts);
> 4 : repeat until i put 100 million records.
>
> And i run this java program as single threaded java program.
>
> Am i doing it right? is there any other way of importing large data for
> testing.? [ for now i'm not considering BULK data import/loadtable.rb etc.
> apart from this is there any other way ?]
>
>
> --
> View this message in context:
> http://old.nabble.com/Inserting-Random-Data-into-HBASE-tp30349594p30349594.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message