hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amandeep Khurana <ama...@gmail.com>
Subject Re: Hbase can we insert such (inside) data faster?
Date Mon, 26 Oct 2009 10:00:39 GMT
1. You need odd number of servers for the zk quorum. 3-5 should be good
enough. In your case, even 1 is fine since the load is not much.
2. We used 7200rpm SATA drives.

On Mon, Oct 26, 2009 at 2:57 AM, Dmitriy Lyfar <dlyfar@gmail.com> wrote:

> Hi Amandeep,
>
> Thank you. I also forgot to mention that Zookeeper is managed by hbase on
> both nodes and
> quorum consists of two zookeepers per node.
> Could you tell me how much Zookeepers should I have per this configuration
> and how it usually should be?
> BTW, which hards disks did you use?
>
> 2009/10/26 Amandeep Khurana <amansk@gmail.com>
>
> > This is slow.. We get about 4k inserts per second per region server with
> > row
> > size being about 30kB. Using Vmware could be causing the slow down.
> >
> > Amandeep
> >
> > On Mon, Oct 26, 2009 at 2:04 AM, Dmitriy Lyfar <dlyfar@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > We are using hadoop + hbase (0.20.1) for tests now. Machines we are
> > testing
> > > on have following configuration:
> > > Vmware
> > > 4 core intel xeon, 2.27GHz
> > > Two hbase nodes (one master and one regionserver), 6GB RAM per each.
> > >
> > > Table has following definition:
> > >
> > > 12-byte string as Row
> > > Column family: C1 and 3 qualifiers: q1, q2, q3 (about 200 bytes per
> > record)
> > > Column family: C2 and 2 qualifiers q1, q2 (about 2-4KB per record)
> > >
> > > I've implemented simple java utility which parses our data source and
> > > inserts results into hbase (write buffer is 12MB, autoflush off).
> > > We got following results:
> > > ~450K records ~= 4GB of data.
> > > Total time of insertion is about 600-650 seconds or ~7 MB/second or 675
> > > rows
> > > per second, or 2ms per row.
> > >
> > > So the question is: is this time ok for such hardware or did I miss
> > > something important?
> > > Thank you.
> > >
> > > Regards, Dmitriy.
> > >
> >
>
>
>
> --
> Regards, Lyfar Dmitriy
> mailto: dlyfar@crystalnix.com
> jabber: dlyfar@gmail.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message