hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: coprocessor enabled put very slow, help please~~~
Date Mon, 18 Feb 2013 18:42:22 GMT
Well... 

if you look at the OP's code, there's a couple things I think could be causing a bit of the
overhead on the writes. 

Lets see if he fixes his code and see if there are any changes in performance. 

On Feb 18, 2013, at 11:56 AM, Wei Tan <wtan@us.ibm.com> wrote:

> Well, my experience shows:
> 
> 1. A local read can be >10ms and a remote put can be 1-2ms. Due to the 
> nature of LSM a read is always a scan to one or multiple files. A very 
> quick experiment can be, if you temporarily disable the check and only do 
> the put, will you see performance go up?
> 
> 2. In a lot of cases, RPC may NOT be the bottle neck. Remember a "local" 
> put also involves RPC -- during WAL to HDFS.
> 
> 
> Best Regards,
> Wei
> 
> Wei Tan 
> Research Staff Member 
> IBM T. J. Watson Research Center
> Yorktown Heights, NY 10598
> wtan@us.ibm.com; 914-945-4386
> 
> 
> 
> From:   Prakash Kadel <prakash.kadel@gmail.com>
> To:     "user@hbase.apache.org" <user@hbase.apache.org>, 
> Date:   02/18/2013 04:04 AM
> Subject:        Re: coprocessor enabled put very slow, help please~~~
> 
> 
> 
> its a local read. i just check the last param of PostCheckAndPut 
> indicating if the Put succeeded. Incase if the put success, i insert a row 
> in another table
> 
> Sincerely,
> Prakash Kadel
> 
> On Feb 18, 2013, at 2:52 PM, Wei Tan <wtan@us.ibm.com> wrote:
> 
>> Is your CheckAndPut involving a local or remote READ? Due to the nature 
> of 
>> LSM, read is much slower compared to a write...
>> 
>> 
>> Best Regards,
>> Wei
>> 
>> 
>> 
>> 
>> From:   Prakash Kadel <prakash.kadel@gmail.com>
>> To:     "user@hbase.apache.org" <user@hbase.apache.org>, 
>> Date:   02/17/2013 07:49 PM
>> Subject:        coprocessor enabled put very slow, help please~~~
>> 
>> 
>> 
>> hi,
>>  i am trying to insert few million documents to hbase with mapreduce. 
> To 
>> enable quick search of docs i want to have some indexes, so i tried to 
> use 
>> the coprocessors, but they are slowing down my inserts. Arent the 
>> coprocessors not supposed to increase the latency? 
>> my settings:
>>   3 region servers
>>  60 maps
>> each map inserts to doc table.(checkAndPut)
>> regionobserver coprocessor does a postCheckAndPut and inserts some rows 
> to 
>> a index table.
>> 
>> 
>> Sincerely,
>> Prakash
>> 
> 
> 


Mime
View raw message