hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajeshkumar J <rajeshkumarit8...@gmail.com>
Subject Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Date Wed, 11 Jan 2017 12:45:31 GMT
this is the log I am getting while running

2017-01-05 11:41:49,629 DEBUG
[B.defaultRpcServer.handler=15,queue=0,port=16020] ipc.RpcServer:
B.defaultRpcServer.handler=15,queue=0,port=16020: callId: 3 service:
ClientService methodName: Scan size: 23 connection: hostname:port
org.apache.hadoop.hbase.regionserver.LeaseException: lease '706' does not
exist
at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2491)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
2017-01-05 11:41:49,629 TRACE
[B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: callId: 2
service: ClientService methodName: Scan size: 29 connection: hostname:port
param: scanner_id: 706 number_of_rows: 2147483647 close_scanner: false
next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
true connection: hostname:port, response scanner_id: 706 more_results: true
stale: false more_results_in_region: false queueTime: 1 processingTime:
60136 totalTime: 60137

On Wed, Jan 11, 2017 at 4:45 PM, Rajeshkumar J <rajeshkumarit8292@gmail.com>
wrote:

> I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
> 10000. But it still fails.
>
> On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <carp84@gmail.com> wrote:
>
>> Hi Rajeshkumar,
>>
>> Simply speaking, in your case I'd suggest to set
>> hbase.client.scanner.caching
>> to a small value like 10 but not zero, and leave the
>> hbase.client.scanner.caching or hbase.rpc.timeout setting as default.
>> Below
>> are some detailed explanation:
>>
>> Setting hbase.client.scanner.caching to zero will cause the scan returns
>> no
>> data (please check whether you are getting a wrong result with this
>> setting), so please don't make such settings unless you intend to renew
>> the
>> scan lease. See RsRpcServices#scan source code for more details if
>> interested.
>>
>> For version 1.1.1 (actually after HBASE-16973), the default value of
>> hbase.client.scanner.caching
>> is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
>> until the result size reached hbase.client.scanner.max.result.size or
>> times
>> out. If you are using a sparse filter, then it's strongly suggested to
>> set hbase.client.scanner.caching
>> to some small value but not zero. See HBASE-16973 for more details if
>> interested.
>>
>> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
>> * For scan it's hbase.client.scanner.timeout.period taking effect
>> * For all other kinds of non-admin operations like
>> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
>> specified hbase.client.operation.timeout.
>> Check ClientScanner#scannerTimeout, ScannerCallable#call,
>> AbstractRpcClient#channelOperationTimeout and
>> AbstractRpcClient#callBlockingMethod for more details if interested.
>>
>> Best Regards,
>> Yu
>>
>> On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8292@gmail.com>
>> wrote:
>>
>> > Hi, As i got lease expire exception  i have increased the values of
>> these
>> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout.
>> Is
>> > there any disadvantage increasing these two values?
>> >
>> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
>> > rajeshkumarit8292@gmail.com>
>> > wrote:
>> >
>> > > I am using hbase 1.1.1.
>> > >
>> > >   I have following property value in hbase-site.xml and I didn't set
>> > value
>> > > hbase.client.scanner.caching property(so it must pick default value
>> for
>> > > this)
>> > >
>> > >         <property>
>> > >         <name>hbase.client.scanner.timeout.period</name>
>> > >         <value>60000</value>
>> > >         </property>
>> > >
>> > >         <property>
>> > >         <name>hbase.rpc.timeout</name>
>> > >         <value>60000</value>
>> > >         </property>
>> > > When scanning 30 tables in hbase(each table holds 45 million records)
>> I
>> > > got the following error
>> > >
>> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
>> > > org.apache.hadoop.hbase.regionserver.LeaseException:
>> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>> > exist
>> > >
>> > > But when I added hbase.client.scanner.caching property value as zero I
>> > > didn't get any error. Why this happened?
>> > > Even though I gave zero it supposed to pick the default value? How can
>> > > this process succeeds?
>> > >
>> > > Thanks
>> > >
>> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <elserj@apache.org>
>> wrote:
>> > >
>> > >> I am having an extremely difficult time parsing this one,
>> Rajeshkumar.
>> > >>
>> > >> If you still have a question/something you don't understand, I'd
>> > >> appreciate it if you could try to re-state it a little more clearly.
>> > >>
>> > >>
>> > >> Rajeshkumar J wrote:
>> > >>
>> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the
>> default
>> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have
set
>> the
>> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
>> > >>> succeeded. So only I am asking this. Also I didn't change any
>> property
>> > >>> whatever I used in process which failed in the process which
>> succeeded
>> > >>> except assigning Zero value for this property
>> > >>>
>> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<elserj@apache.org>
>> wrote:
>> > >>>
>> > >>> Most likely, since you gave a nonsensical value, HBase used a
>> default
>> > >>>> value instead of the one you provided. Since you have not shared
>> the
>> > >>>> version of HBase which you are using, I would recommend that
you
>> look
>> > at
>> > >>>> the code. It should be very obvious what value is being used
>> instead
>> > of
>> > >>>> the
>> > >>>> bogus value you provided.
>> > >>>>
>> > >>>>
>> > >>>> Rajeshkumar J wrote:
>> > >>>>
>> > >>>> I have tried setting hbase.client.scanner.caching property
to 0
>> with
>> > >>>>> above
>> > >>>>> two values as 60000. It runs fine. But
>> hbase.client.scanner.caching
>> > >>>>> property tells no of rows to be fetched for every scanner
but
>> here I
>> > >>>>> have
>> > >>>>> given zero. How this worked?
>> > >>>>>
>> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<elserj@apache.org>
>> > >>>>>  wrote:
>> > >>>>>
>> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically
for
>> > RPCs
>> > >>>>>
>> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
>> while
>> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC.
I believe
>> that
>> > >>>>>> the
>> > >>>>>> hbase.client.scanner.timeout.period is also used by
the
>> > >>>>>> RegionServers to
>> > >>>>>> define the lifetime of the Lease (the cause of the
LeaseException
>> > >>>>>> you're
>> > >>>>>> seeing).
>> > >>>>>>
>> > >>>>>> Generally, when you see these kinds of exceptions while
scanning
>> > data
>> > >>>>>> in
>> > >>>>>> HBase, it is just a factor of your hardware and current
>> performance
>> > >>>>>> (in
>> > >>>>>> other words, how long it takes to read your data).
I can't really
>> > >>>>>> give a
>> > >>>>>> firm answer because it is dependent on your system's
performance.
>> > You
>> > >>>>>> should be able to approximate the performance with
some
>> > >>>>>> back-of-the-envelope math.
>> > >>>>>>
>> > >>>>>> Some changes like https://issues.apache.org/jira
>> /browse/HBASE-13090
>> > >>>>>> and
>> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should
help
>> > reduce
>> > >>>>>> the
>> > >>>>>> need for you to tweak configuration properties in the
future.
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> Rajeshkumar J wrote:
>> > >>>>>>
>> > >>>>>> Hi,
>> > >>>>>>
>> > >>>>>>>         I have following property value as below
>> > >>>>>>>
>> > >>>>>>>            <property>
>> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
>> > >>>>>>>            <value>60000</value>
>> > >>>>>>>            </property>
>> > >>>>>>>
>> > >>>>>>>            <property>
>> > >>>>>>>            <name>hbase.rpc.timeout</name>
>> > >>>>>>>            <value>60000</value>
>> > >>>>>>>            </property>
>> > >>>>>>> When scanning 30 tables in hbase(each table holds
45 million
>> > >>>>>>> records) I
>> > >>>>>>> got
>> > >>>>>>>
>> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably
already
>> closed
>> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
lease does
>> > not
>> > >>>>>>> exist
>> > >>>>>>>
>> > >>>>>>> But when I change the value to
>> > >>>>>>>
>> > >>>>>>>           <property>
>> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
>> > >>>>>>>            <value>70000</value>
>> > >>>>>>>            </property>
>> > >>>>>>>
>> > >>>>>>>            <property>
>> > >>>>>>>            <name>hbase.rpc.timeout</name>
>> > >>>>>>>            <value>70000</value>
>> > >>>>>>>            </property>
>> > >>>>>>> It runs successfully.
>> > >>>>>>> Can any one tell me the reason for this failure
and also is
>> there
>> > any
>> > >>>>>>> factor to calculate these property values for any
data.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message