hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: how to check whether zookeeper quorum is working fine or not ?
Date Fri, 08 Jul 2011 15:32:58 GMT
When you looked at zk with zkdump or via the zkcli did it show max
connections as 2k?
St.Ack

On Fri, Jul 8, 2011 at 8:14 AM, praveenesh kumar <praveenesh@gmail.com> wrote:
> yeah..!!
>
> On Fri, Jul 8, 2011 at 8:43 PM, Stack <stack@duboce.net> wrote:
>
>> Is hbase managing zk?  Did you see zk restart when you restarted hbase?
>> St.Ack
>>
>> On Fri, Jul 8, 2011 at 8:09 AM, praveenesh kumar <praveenesh@gmail.com>
>> wrote:
>> > I restarted Hbase.. Is there any specific way to do that ?
>> >
>> > On Fri, Jul 8, 2011 at 8:35 PM, Stack <stack@duboce.net> wrote:
>> >
>> >> Did you restart your zk ensemble after making the change?
>> >> St.Ack
>> >>
>> >> On Fri, Jul 8, 2011 at 2:31 AM, praveenesh kumar <praveenesh@gmail.com>
>> >> wrote:
>> >> > Guys.. I have increased the zookeeper connections to 2000, still I
am
>> >> > getting the same zookeeper exception while running the pig scripts.
>> >> > Can anyone tell me what can be the reason ?
>> >> >
>> >> > Thanks,
>> >> > Praveenesh
>> >> >
>> >> > On Fri, Jul 8, 2011 at 12:16 AM, Barney Frank <
>> barneyfranks1@gmail.com
>> >> >wrote:
>> >> >
>> >> >> Why not just set the maxClientCnxcns to 0?  That would make it
>> >> unlimited,
>> >> >> right?
>> >> >>
>> >> >> On Thu, Jul 7, 2011 at 12:55 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>> >> >>
>> >> >> > To 1000.
>> >> >> >
>> >> >> > Click on 'zk dump' link from Master UI.
>> >> >> > You would know roughly how many connections were established.
>> >> >> > If 1000 is exhausted, increase more.
>> >> >> >
>> >> >> > See
>> >> >> >
>> >> >> >
>> >> >>
>> >>
>> http://zhihongyu.blogspot.com/2011/04/managing-connections-in-hbase-090-and.html
>> >> >> >
>> >> >> > On Thu, Jul 7, 2011 at 10:52 AM, praveenesh kumar <
>> >> praveenesh@gmail.com
>> >> >> > >wrote:
>> >> >> >
>> >> >> > > I did that.. I increased it to 200.. still facing the
same
>> error..
>> >> what
>> >> >> > > should be the ideal value ?
>> >> >> > >
>> >> >> > > On Thu, Jul 7, 2011 at 11:20 PM, Ted Yu <yuzhihong@gmail.com>
>> >> wrote:
>> >> >> > >
>> >> >> > > > Increase hbase.zookeeper.property.maxClientCnxns
to much bigger
>> >> >> value.
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > On Thu, Jul 7, 2011 at 10:46 AM, praveenesh kumar
<
>> >> >> > praveenesh@gmail.com
>> >> >> > > > >wrote:
>> >> >> > > >
>> >> >> > > > > My HBase and pig both are running fine for
me..
>> >> >> > > > > Only problem is coming whenever I am trying
to access big
>> hbase
>> >> >> table
>> >> >> > > > > containing too many columns and rows..
>> >> >> > > > > whenever I am trying to access hbase table
from pig... I am
>> >> getting
>> >> >> > > this
>> >> >> > > > > following error in the middle of the map process:
>> >> >> > > > >
>> >> >> > > > > org.apache.pig.tools.pigstats.SimplePigStats
- ERROR 2997:
>> >> Unable
>> >> >> to
>> >> >> > > > > recreate exception from backed error:
>> >> >> > > > > org.apache.hadoop.hbase.ZooKeeperConnectionException:
HBase
>> is
>> >> able
>> >> >> > to
>> >> >> > > > > connect to ZooKeeper but the connection closes
immediately.
>> This
>> >> >> > could
>> >> >> > > be
>> >> >> > > > a
>> >> >> > > > > sign that the server has too many connections
(30 is the
>> >> default).
>> >> >> > > > Consider
>> >> >> > > > > inspecting your ZK server logs for that error
and then make
>> sure
>> >> >> you
>> >> >> > > are
>> >> >> > > > > reusing HBaseConfiguration as often as you
can. See HTable's
>> >> >> javadoc
>> >> >> > > for
>> >> >> > > > > more information.
>> >> >> > > > > But if I access a small HBase table from pig..
its working
>> >> fine..
>> >> >> no
>> >> >> > > > > zookeeper exception is coming. I am able to
do all other
>> things
>> >> >> > > perfectly
>> >> >> > > > > fine.
>> >> >> > > > > This error made me think that there is some
problem in my
>> >> zookeeper
>> >> >> > > that
>> >> >> > > > is
>> >> >> > > > > why I was asking is there anyway to check whether
the
>> zookeeper
>> >> are
>> >> >> > > > working
>> >> >> > > > > fine or not ?
>> >> >> > > > >
>> >> >> > > > > My zookeeper log -- http://pastebin.com/RRdzZYui
>> >> >> > > > > My hbase-site.xml
>> >> >> > > > >
>> >> >> > > > > <configuration>
>> >> >> > > > > <property>
>> >> >> > > > > <name>hbase.rootdir</name>
>> >> >> > > > > <value>hdfs://ub13:54310/hbase</value>
>> >> >> > > > > <final>true</final>
>> >> >> > > > > </property>
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > > <property>
>> >> >> > > > > <name>hbase.zookeeper.quorum</name>
>> >> >> > > > > <value>ub13,ub12,ub11</value>
>> >> >> > > > > <final>true</final>
>> >> >> > > > > </property>
>> >> >> > > > >
>> >> >> > > > > <property>
>> >> >> > > > > <name>hbase.zookeeper.property.maxClientCnxns</name>
>> >> >> > > > > <value>200</value>
>> >> >> > > > > <final>true</final>
>> >> >> > > > > </property>
>> >> >> > > > >
>> >> >> > > > > <property>
>> >> >> > > > > <name>hbase.cluster.distributed</name>
>> >> >> > > > > <value>true</value>
>> >> >> > > > > <description>The mode the cluster will
be in. Possible values
>> >> are
>> >> >> > > > > false: standalone and pseudo-distributed setups
with managed
>> >> >> > Zookeeper
>> >> >> > > > > true: fully-distributed with unmanaged Zookeeper
Quorum
>> >> >> > > > > </description>
>> >> >> > > > > </property>
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > > On Thu, Jul 7, 2011 at 10:47 PM, Stack <stack@duboce.net>
>> >> wrote:
>> >> >> > > > >
>> >> >> > > > > > On Thu, Jul 7, 2011 at 8:54 AM, praveenesh
kumar <
>> >> >> > > praveenesh@gmail.com
>> >> >> > > > >
>> >> >> > > > > > wrote:
>> >> >> > > > > > > Can you please go through this and
tell me what can be
>> the
>> >> >> issue
>> >> >> > ?
>> >> >> > > > > > >
>> >> >> > > > > >
>> >> >> > > > > > Can you summarize?  (Life is short)
>> >> >> > > > > >
>> >> >> > > > > > If stuff works generally -- shell?  And
can you do the
>> hbase
>> >> >> > > > > > rowcounter mapreduce job successfully
-- but then when you
>> put
>> >> >> PIG
>> >> >> > in
>> >> >> > > > > > the picture it does not, it would seem
to be an issue with
>> >> your
>> >> >> PIG
>> >> >> > > > > > setup?
>> >> >> > > > > >
>> >> >> > > > > > St.Ack
>> >> >> > > > > >
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>

Mime
View raw message