hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bradford Stephens <bradfordsteph...@gmail.com>
Subject Re: Slow Inserts on EC2 Cluster
Date Wed, 01 Sep 2010 16:38:39 GMT
Wow, thanks. I didn't consider that ... I try to avoid the cloud if at
all possible :)


On Wed, Sep 1, 2010 at 4:14 AM, Andrew Purtell <apurtell@apache.org> wrote:
>> From: Bradford Stephens
>> I'm banging my head against some perf issues on EC2. I'm
>> using .20.6 on ASF hadoop .20.2, and tweaked the ec2 hbase
>> scripts to handle the new version.
>> I'm trying to insert about 22G of data across nodes on EC2
>> m1.large instances [...]
> c1.xlarge provides (barely) adequate I/O bandwidth.
> Those periods of higher latency that you mention in the part of your mail that I clipped
are probably due to hypervisor stealing your resources to attend to a noisy neighbor with
a better reservation class.
> I would not consider EC2 a high performance platform, except for maybe their cluster
compute nodes which have been specially engineered for HPC using a completely different virtualization
and network architecture than the rest. EC2 is about bulk processing on a reasonable (subject
to definition) timeframe at cheap/elastic prices.
>  - Andy

Bradford Stephens,
Founder, Drawn to Scale

http://www.drawntoscalehq.com --  The intuitive, cloud-scale data
solution. Process, store, query, search, and serve all your data.

http://www.roadtofailure.com -- The Fringes of Scalability, Social
Media, and Computer Science

View raw message