hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Li <car...@gmail.com>
Subject Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Date Wed, 11 Jan 2017 08:47:27 GMT
Hi Rajeshkumar,

Simply speaking, in your case I'd suggest to set hbase.client.scanner.caching
to a small value like 10 but not zero, and leave the
hbase.client.scanner.caching or hbase.rpc.timeout setting as default. Below
are some detailed explanation:

Setting hbase.client.scanner.caching to zero will cause the scan returns no
data (please check whether you are getting a wrong result with this
setting), so please don't make such settings unless you intend to renew the
scan lease. See RsRpcServices#scan source code for more details if
interested.

For version 1.1.1 (actually after HBASE-16973), the default value of
hbase.client.scanner.caching
is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
until the result size reached hbase.client.scanner.max.result.size or times
out. If you are using a sparse filter, then it's strongly suggested to
set hbase.client.scanner.caching
to some small value but not zero. See HBASE-16973 for more details if
interested.

Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
* For scan it's hbase.client.scanner.timeout.period taking effect
* For all other kinds of non-admin operations like
get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
specified hbase.client.operation.timeout.
Check ClientScanner#scannerTimeout, ScannerCallable#call,
AbstractRpcClient#channelOperationTimeout and
AbstractRpcClient#callBlockingMethod for more details if interested.

Best Regards,
Yu

On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8292@gmail.com>
wrote:

> Hi, As i got lease expire exception  i have increased the values of these
> properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is
> there any disadvantage increasing these two values?
>
> On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com>
> wrote:
>
> > I am using hbase 1.1.1.
> >
> >   I have following property value in hbase-site.xml and I didn't set
> value
> > hbase.client.scanner.caching property(so it must pick default value for
> > this)
> >
> >         <property>
> >         <name>hbase.client.scanner.timeout.period</name>
> >         <value>60000</value>
> >         </property>
> >
> >         <property>
> >         <name>hbase.rpc.timeout</name>
> >         <value>60000</value>
> >         </property>
> > When scanning 30 tables in hbase(each table holds 45 million records) I
> > got the following error
> >
> > WARN client.ScannerCallable [main]: Ignore, probably already closed
> > org.apache.hadoop.hbase.regionserver.LeaseException:
> > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> exist
> >
> > But when I added hbase.client.scanner.caching property value as zero I
> > didn't get any error. Why this happened?
> > Even though I gave zero it supposed to pick the default value? How can
> > this process succeeds?
> >
> > Thanks
> >
> > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <elserj@apache.org> wrote:
> >
> >> I am having an extremely difficult time parsing this one, Rajeshkumar.
> >>
> >> If you still have a question/something you don't understand, I'd
> >> appreciate it if you could try to re-state it a little more clearly.
> >>
> >>
> >> Rajeshkumar J wrote:
> >>
> >>> I am using hbase 1.1.1. Yes while setting zero it must pick the default
> >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
> >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> >>> succeeded. So only I am asking this. Also I didn't change any property
> >>> whatever I used in process which failed in the process which succeeded
> >>> except assigning Zero value for this property
> >>>
> >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<elserj@apache.org>  wrote:
> >>>
> >>> Most likely, since you gave a nonsensical value, HBase used a default
> >>>> value instead of the one you provided. Since you have not shared the
> >>>> version of HBase which you are using, I would recommend that you look
> at
> >>>> the code. It should be very obvious what value is being used instead
> of
> >>>> the
> >>>> bogus value you provided.
> >>>>
> >>>>
> >>>> Rajeshkumar J wrote:
> >>>>
> >>>> I have tried setting hbase.client.scanner.caching property to 0 with
> >>>>> above
> >>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
> >>>>> property tells no of rows to be fetched for every scanner but here
I
> >>>>> have
> >>>>> given zero. How this worked?
> >>>>>
> >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<elserj@apache.org>
> >>>>>  wrote:
> >>>>>
> >>>>> hbase.client.scanner.timeout.period is a timeout specifically for
> RPCs
> >>>>>
> >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
while
> >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe
that
> >>>>>> the
> >>>>>> hbase.client.scanner.timeout.period is also used by the
> >>>>>> RegionServers to
> >>>>>> define the lifetime of the Lease (the cause of the LeaseException
> >>>>>> you're
> >>>>>> seeing).
> >>>>>>
> >>>>>> Generally, when you see these kinds of exceptions while scanning
> data
> >>>>>> in
> >>>>>> HBase, it is just a factor of your hardware and current performance
> >>>>>> (in
> >>>>>> other words, how long it takes to read your data). I can't really
> >>>>>> give a
> >>>>>> firm answer because it is dependent on your system's performance.
> You
> >>>>>> should be able to approximate the performance with some
> >>>>>> back-of-the-envelope math.
> >>>>>>
> >>>>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090
> >>>>>> and
> >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help
> reduce
> >>>>>> the
> >>>>>> need for you to tweak configuration properties in the future.
> >>>>>>
> >>>>>>
> >>>>>> Rajeshkumar J wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>>         I have following property value as below
> >>>>>>>
> >>>>>>>            <property>
> >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> >>>>>>>            <value>60000</value>
> >>>>>>>            </property>
> >>>>>>>
> >>>>>>>            <property>
> >>>>>>>            <name>hbase.rpc.timeout</name>
> >>>>>>>            <value>60000</value>
> >>>>>>>            </property>
> >>>>>>> When scanning 30 tables in hbase(each table holds 45 million
> >>>>>>> records) I
> >>>>>>> got
> >>>>>>>
> >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already
closed
> >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease
does
> not
> >>>>>>> exist
> >>>>>>>
> >>>>>>> But when I change the value to
> >>>>>>>
> >>>>>>>           <property>
> >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> >>>>>>>            <value>70000</value>
> >>>>>>>            </property>
> >>>>>>>
> >>>>>>>            <property>
> >>>>>>>            <name>hbase.rpc.timeout</name>
> >>>>>>>            <value>70000</value>
> >>>>>>>            </property>
> >>>>>>> It runs successfully.
> >>>>>>> Can any one tell me the reason for this failure and also
is there
> any
> >>>>>>> factor to calculate these property values for any data.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message