hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andriy Kolyadenko" <cryp...@mail.saturnfans.com>
Subject Re: set number of map tasks for HBase MR
Date Sun, 11 Apr 2010 03:17:30 GMT
Hi,

thanks for quick response. I tried to do following in the code:

job.getConfiguration().setInt("mapred.map.tasks", 10000);

but unfortunately have the same result. 

Any other ideas?

--- amansk@gmail.com wrote:

From: Amandeep Khurana <amansk@gmail.com>
To: hbase-user@hadoop.apache.org, crypto5@mail.saturnfans.com
Subject: Re: set number of map tasks for HBase MR
Date: Sat, 10 Apr 2010 20:04:18 -0700

You can set the number of map tasks in your job config to a big number (eg:
100000), and the library will automatically spawn one map task per region.

-ak


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz


On Sat, Apr 10, 2010 at 7:59 PM, Andriy Kolyadenko <
crypto5@mail.saturnfans.com> wrote:

> Hi guys,
>
> I have about 8G Hbase table  and I want to run MR job against it. It works
> extremely slow in my case. One thing I noticed is that job runs only 2 map
> tasks. Is it any way to setup bigger number of map tasks? I sow some method
> in mapred package, but can't find anything like this in new mapreduce
> package.
>
> I run my MR job one a single machine in cluster mode.
>
>
> _____________________________________________________________
> Sign up for your free SaturnFans email account at
> http://webmail.saturnfans.com/
>




_____________________________________________________________
Sign up for your free SaturnFans email account at http://webmail.saturnfans.com/

Mime
View raw message