hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yossi Ittach" <yoss...@gmail.com>
Subject Re: HBase keeps crushing after load
Date Mon, 27 Oct 2008 15:54:58 GMT
Thank you very much - This indeed was the problem. I hijacked some company
resources and it works now :)


Vale et me ama
Yossi


On Mon, Oct 27, 2008 at 5:48 PM, Jonathan Gray <jlist@streamy.com> wrote:

> That kind of load on 5 virtual machines with only 2 cores and 4GB is not
> realistic.
>
> Typically you'd want a few independent machines with those kinds of specs.
>
> You're probably seeing thread starvation (only 2 cores but each
> regionserver
> is likely maxing one out) as well as swapping.  When that happens, all hell
> breaks loose.
>
> If you're trying to experiment and benchmark before you have the hardware
> for your cluster, you might try XLarge instances on Amazon EC2.
>
> JG
>
> From: Yossi Ittach [mailto:yossale@gmail.com]
> Sent: Monday, October 27, 2008 1:33 AM
> To: hbase-user@hadoop.apache.org
> Subject: HBase keeps crushing after load
>
> Hi All
>
> I've been benchmarking HBase for quite sometime now , and it keeps crushing
> under high load.
>
> The benchmark scheme is running on a dedicated host (320GB disk , 4 GB ram
> ,
> 2 dualcore processors) that runs VMware with 5 virtual stations:
> 1 - Hadoop NameNode
> 2 - Hbase Master
> 3 - Hadoop DataNode
> 4 - Hbase RegionServer
> 5 - Hbase RegionServer
>
> The benchmark is a program that loads the HBase with ~ 3.5M files , with
> avg
> size of 5k - total of about 30GB.
> HBase has 1 table: row=file name , column-family (with just one column)
> column = content (the file content)
>
> When I'm running a smaller benchmark (350K files ~ 3 GB) it works fine.
> When
> I run the full benchmark - hell breaks loose. After ~1M files , the
> regionservers start to report connection problems (region server can't
> connect to master , master can't connect to region server , ...) and/or
> shut
> themselves down one after another.
>
> I figured it may be some overload on the host server , but the resources
> are
> fine - a lot of cpu , memory and disk are available (even on peak times) .
>
> I attache part of the log files , if you find it helpful.
>
> Thank you very much!
>
> Vale et me ama
> Yossi
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message