hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Mor <amit.mor.m...@gmail.com>
Subject Re: EC2 instance type recommendation ?
Date Tue, 16 Jul 2013 18:52:29 GMT
Thanks !
I was afraid to go into the SSD world with regards to VM, HDFS and HBase.
Doesn't it break the whole concept of sequential r/w ?
I've been using m1.xlarge and m2.4xlarge. Yet again , using the 68GB for a
JVM is, mmm, horrific. Worst of, i found that each of the 4 m1.xlarge disk
perform better than each of the 2 disks on m2.4xlarge
 On Jul 16, 2013 9:32 PM, "Bryan Beaudreault" <bbeaudreault@hubspot.com>

> You're right that EC2 does not provide great instance types for HBase.  In
> our research we found that most people seemed to be using c1.xlarge.  This
> still is not great, because 7GB of RAM for DataNode + RegionServer + OS is
> pitiful, but it works.  Of course that also makes it impossible to colocate
> TaskTrackers on the RegionServer nodes.  We also tried m1.xlarge in the
> beginning, and it's nice to have the memory but we quickly became CPU bound
> and thus moved to c1.xlarge.  Of course you're not going to get good disk
> performance on these ephemeral disks, but EBS is trouble in our experience.
> At HBaseCon, the pinterest engineers mentioned using hi1.4xlarge.  Those
> are very expensive though and unfortunately just have 2 SSD disks.  Perhaps
> they have benchmarks or can chime in as to whether they ventured to use EBS
> to supplement the 2 disks or what.
> On Tue, Jul 16, 2013 at 2:18 PM, Amit Mor <amit.mor.mail@gmail.com> wrote:
> > Hello, I am curious to hear the recommendations people have here for
> > running HBase on EC2 instances. I failed to find an instance that has a
> > good ratio/balance between CPU, # of disks and jvm acceptable heap Memory
> > to be used with (almost random) read intensive cluster. The major
> > bottleneck I found was the disk read latency that could even reach srvtm
> of
> > a second ... Combined with everything else ;-)
> >
> > Thanks,
> > Amit
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message