hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: RegionServer continues lease expired
Date Sat, 07 Mar 2015 06:23:45 GMT
You get the lease expired from any region hosted on this machine, or only
regions from a specific table? If the former, I'd say kill the machine and
request a new one. On AES, life is too short to diagnose crap VM's. If you
do that and the problem moves to another machine, it must the the region ;)

On Friday, March 6, 2015, Giusto Michele <m.giusto@reply.it> wrote:

>  Hi all,
>
> I am working on a Cloudera CDH5.3.1 cluster (so hbase-0.98.6+cdh5.3.1+74)
> and the cluster is running in AWS. The cluster is composed of 6 machines, 2
> masters and 4 slaves. I am having problem on a single slave node (the last
> one added, temporarily) where the local RegionServer, once it receives the
> first region (even without rows inside), continuously logs messages like:
>
> 2015-03-06 11:48:29,088 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1688 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:32,458 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1689 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:36,089 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1690 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:39,460 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1691 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:43,090 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1692 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:46,459 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1693 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
> 2015-03-06 11:48:50,089 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner 1694 lease
> expired on region
> test_table2,,1425636551770.739b80dfcac4fbc59ea6a6633b9e86a8.
>
>
>
> All the other services are working as expected apart from this problem. I
> am not able to understand why this is happening, the only difference I can
> see from logs/web UI is that this slave node is identified with its FQDN,
> while the others only with the hostname (no domain).
>
>
>
> I have checked on the other nodes and they are not reporting similar
> messages. Setting log level to TRACE didn’t show nothing else useful, at
> least for me.
>
>
>
> Finally, the original problem that lead me to find these messages is that
> when I map an HBase table as an external table on Hive/Impala, with one
> region of this table on the problematic RegionServer, the query remain
> pending forever. If the table is not using that RegionServer, queries work
> normally. It is important to note that the messages starts immediately
> after I create the HBase table, so I believe it is not Impala that creates
> problem.
>
> What does the messages imply? Can the reason of my Impala problem be
> related to these messages?
>
>
>
>
>
> Thanks in advance,
>
> Michele
>
>
>  Michele Giusto
>
> Target Reply
> Via Castellanza, 11
> 20151 - Milano - ITALY
> phone: +39 02 535761
> m.giusto@reply.it <javascript:_e(%7B%7D,'cvml','m.giusto@reply.it');>
> www.reply.it
>
> [image: Target Reply]
>
>
> ------------------------------
>
> --
> The information transmitted is intended for the person or entity to which
> it is addressed and may contain confidential and/or privileged material.
> Any review, retransmission, dissemination or other use of, or taking of any
> action in reliance upon, this information by persons or entities other than
> the intended recipient is prohibited. If you received this in error, please
> contact the sender and delete the material from any computer.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message