hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajeshkumar J <rajeshkumarit8...@gmail.com>
Subject Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Date Wed, 11 Jan 2017 11:15:53 GMT
I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
10000. But it still fails.

On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <carp84@gmail.com> wrote:

> Hi Rajeshkumar,
>
> Simply speaking, in your case I'd suggest to set
> hbase.client.scanner.caching
> to a small value like 10 but not zero, and leave the
> hbase.client.scanner.caching or hbase.rpc.timeout setting as default. Below
> are some detailed explanation:
>
> Setting hbase.client.scanner.caching to zero will cause the scan returns no
> data (please check whether you are getting a wrong result with this
> setting), so please don't make such settings unless you intend to renew the
> scan lease. See RsRpcServices#scan source code for more details if
> interested.
>
> For version 1.1.1 (actually after HBASE-16973), the default value of
> hbase.client.scanner.caching
> is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
> until the result size reached hbase.client.scanner.max.result.size or
> times
> out. If you are using a sparse filter, then it's strongly suggested to
> set hbase.client.scanner.caching
> to some small value but not zero. See HBASE-16973 for more details if
> interested.
>
> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
> * For scan it's hbase.client.scanner.timeout.period taking effect
> * For all other kinds of non-admin operations like
> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
> specified hbase.client.operation.timeout.
> Check ClientScanner#scannerTimeout, ScannerCallable#call,
> AbstractRpcClient#channelOperationTimeout and
> AbstractRpcClient#callBlockingMethod for more details if interested.
>
> Best Regards,
> Yu
>
> On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8292@gmail.com>
> wrote:
>
> > Hi, As i got lease expire exception  i have increased the values of these
> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is
> > there any disadvantage increasing these two values?
> >
> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> > rajeshkumarit8292@gmail.com>
> > wrote:
> >
> > > I am using hbase 1.1.1.
> > >
> > >   I have following property value in hbase-site.xml and I didn't set
> > value
> > > hbase.client.scanner.caching property(so it must pick default value for
> > > this)
> > >
> > >         <property>
> > >         <name>hbase.client.scanner.timeout.period</name>
> > >         <value>60000</value>
> > >         </property>
> > >
> > >         <property>
> > >         <name>hbase.rpc.timeout</name>
> > >         <value>60000</value>
> > >         </property>
> > > When scanning 30 tables in hbase(each table holds 45 million records) I
> > > got the following error
> > >
> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> > exist
> > >
> > > But when I added hbase.client.scanner.caching property value as zero I
> > > didn't get any error. Why this happened?
> > > Even though I gave zero it supposed to pick the default value? How can
> > > this process succeeds?
> > >
> > > Thanks
> > >
> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <elserj@apache.org> wrote:
> > >
> > >> I am having an extremely difficult time parsing this one, Rajeshkumar.
> > >>
> > >> If you still have a question/something you don't understand, I'd
> > >> appreciate it if you could try to re-state it a little more clearly.
> > >>
> > >>
> > >> Rajeshkumar J wrote:
> > >>
> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the
> default
> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set
> the
> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> > >>> succeeded. So only I am asking this. Also I didn't change any
> property
> > >>> whatever I used in process which failed in the process which
> succeeded
> > >>> except assigning Zero value for this property
> > >>>
> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<elserj@apache.org>
> wrote:
> > >>>
> > >>> Most likely, since you gave a nonsensical value, HBase used a default
> > >>>> value instead of the one you provided. Since you have not shared
the
> > >>>> version of HBase which you are using, I would recommend that you
> look
> > at
> > >>>> the code. It should be very obvious what value is being used instead
> > of
> > >>>> the
> > >>>> bogus value you provided.
> > >>>>
> > >>>>
> > >>>> Rajeshkumar J wrote:
> > >>>>
> > >>>> I have tried setting hbase.client.scanner.caching property to 0
with
> > >>>>> above
> > >>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
> > >>>>> property tells no of rows to be fetched for every scanner but
here
> I
> > >>>>> have
> > >>>>> given zero. How this worked?
> > >>>>>
> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<elserj@apache.org>
> > >>>>>  wrote:
> > >>>>>
> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically
for
> > RPCs
> > >>>>>
> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
> while
> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I
believe
> that
> > >>>>>> the
> > >>>>>> hbase.client.scanner.timeout.period is also used by the
> > >>>>>> RegionServers to
> > >>>>>> define the lifetime of the Lease (the cause of the LeaseException
> > >>>>>> you're
> > >>>>>> seeing).
> > >>>>>>
> > >>>>>> Generally, when you see these kinds of exceptions while
scanning
> > data
> > >>>>>> in
> > >>>>>> HBase, it is just a factor of your hardware and current
> performance
> > >>>>>> (in
> > >>>>>> other words, how long it takes to read your data). I can't
really
> > >>>>>> give a
> > >>>>>> firm answer because it is dependent on your system's performance.
> > You
> > >>>>>> should be able to approximate the performance with some
> > >>>>>> back-of-the-envelope math.
> > >>>>>>
> > >>>>>> Some changes like https://issues.apache.org/
> jira/browse/HBASE-13090
> > >>>>>> and
> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should
help
> > reduce
> > >>>>>> the
> > >>>>>> need for you to tweak configuration properties in the future.
> > >>>>>>
> > >>>>>>
> > >>>>>> Rajeshkumar J wrote:
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>>>         I have following property value as below
> > >>>>>>>
> > >>>>>>>            <property>
> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> > >>>>>>>            <value>60000</value>
> > >>>>>>>            </property>
> > >>>>>>>
> > >>>>>>>            <property>
> > >>>>>>>            <name>hbase.rpc.timeout</name>
> > >>>>>>>            <value>60000</value>
> > >>>>>>>            </property>
> > >>>>>>> When scanning 30 tables in hbase(each table holds 45
million
> > >>>>>>> records) I
> > >>>>>>> got
> > >>>>>>>
> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably
already
> closed
> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
lease does
> > not
> > >>>>>>> exist
> > >>>>>>>
> > >>>>>>> But when I change the value to
> > >>>>>>>
> > >>>>>>>           <property>
> > >>>>>>>            <name>hbase.client.scanner.timeout.period</name>
> > >>>>>>>            <value>70000</value>
> > >>>>>>>            </property>
> > >>>>>>>
> > >>>>>>>            <property>
> > >>>>>>>            <name>hbase.rpc.timeout</name>
> > >>>>>>>            <value>70000</value>
> > >>>>>>>            </property>
> > >>>>>>> It runs successfully.
> > >>>>>>> Can any one tell me the reason for this failure and
also is there
> > any
> > >>>>>>> factor to calculate these property values for any data.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message