Well, my experience shows:
1. A local read can be >10ms and a remote put can be 1-2ms. Due to the
nature of LSM a read is always a scan to one or multiple files. A very
quick experiment can be, if you temporarily disable the check and only do
the put, will you see performance go up?
2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local"
put also involves RPC -- during WAL to HDFS.
Best Regards,
Wei
Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
Yorktown Heights, NY 10598
wtan@us.ibm.com; 914-945-4386
From: Prakash Kadel <prakash.kadel@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>,
Date: 02/18/2013 04:04 AM
Subject: Re: coprocessor enabled put very slow, help please~~~
its a local read. i just check the last param of PostCheckAndPut
indicating if the Put succeeded. Incase if the put success, i insert a row
in another table
Sincerely,
Prakash Kadel
On Feb 18, 2013, at 2:52 PM, Wei Tan <wtan@us.ibm.com> wrote:
> Is your CheckAndPut involving a local or remote READ? Due to the nature
of
> LSM, read is much slower compared to a write...
>
>
> Best Regards,
> Wei
>
>
>
>
> From: Prakash Kadel <prakash.kadel@gmail.com>
> To: "user@hbase.apache.org" <user@hbase.apache.org>,
> Date: 02/17/2013 07:49 PM
> Subject: coprocessor enabled put very slow, help please~~~
>
>
>
> hi,
> i am trying to insert few million documents to hbase with mapreduce.
To
> enable quick search of docs i want to have some indexes, so i tried to
use
> the coprocessors, but they are slowing down my inserts. Arent the
> coprocessors not supposed to increase the latency?
> my settings:
> 3 region servers
> 60 maps
> each map inserts to doc table.(checkAndPut)
> regionobserver coprocessor does a postCheckAndPut and inserts some rows
to
> a index table.
>
>
> Sincerely,
> Prakash
>
|