hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yabo-Arber Xu <...@summba.com>
Subject Re: Again, HBase Data Lost!!
Date Mon, 08 Jun 2009 03:03:54 GMT
Hi J-D,

Thanks for your reply. We have a 10-node cluster installed with HBase/Hadoop
0.19.1. We wrote about 1.8M records into HBase, and it appeared fine. We can
use 'count' on shell to get the number. But after a while ( not sure exactly
when ) all nodes crashed right away. I checked the log and but did not find
anything particular at that moment. When I restart it again, all the rows in
the tables are lost but the tables are still there.

I have difficulty to connect IRC channel now ( will try it later ). For now,
I shared the master's log
here<http://cid-c31c56409ed94abd.skydrive.live.com/self.aspx/Public/Summba/hbase-cloudadmin-master-A6.log.2009-06-07>,
and the crash happens before the line "18:46:54 CST Starting master on A6".
You would notice that the period from "2009-06-07 17:00:57" to "2009-06-07
18:46:54" is empty ( due to the crash).

Thanks for your help!

Best,
Arber


On Sun, Jun 7, 2009 at 11:56 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Arber,
>
> From your email I have a hard time understanding exactly what happened
> on your cluster. But, before asking any question, I have to say that I
> never saw data just "disappearing" like this on the production cluster
> I've been managing for the last year.
>
> So, what happened? Did you lose the region server holding .META or
> -ROOT-? Was it after an importing job and all nodes crashed right
> away? If any crash, do you know why it happened?
>
> Giving us those details (and more) will help us solve your problem.
> You can also drop by the hbase IRC channel, there's always people
> there happy to take a look at your logs and debugging goes much
> faster.
>
> Thx,
>
> J-D
>
> On Sun, Jun 7, 2009 at 9:21 AM, Yabo-Arber Xu<arber.research@gmail.com>
> wrote:
> > Hi there,
> >
> > I had a hbase data lost couple of days ago due to the crash of HBase. At
> > that time i asked on the list and was told that it may be due to the lost
> of
> > META data on the master ( not flushed into disk during crash ). Just
> several
> > days after, this happened to me again, this time all the data is gone,
> while
> > all the tables are still there.  This time is on a 10-node hbase cluster.
> >
> > I checked the log but did not find anything strange. Could anybody shed a
> > light on this? Given such stability, i really worried whether we can use
> it
> > in production phase.
> >
> > Best,
> > Arber
> >
> > On Mon, May 25, 2009 at 10:25 AM, Yabo-Arber Xu <
> arber.research@gmail.com>wrote:
> >
> >> Hi there,
> >>
> >> I had a single-node cluster up and running. And yesterday the node
> crashed
> >> for unknown reason, and when I restart it, everything appears to work
> except
> >> that all the tables are LOST( UI says that there is no user tables)!! I
> >> checked the log file and didn't find any clue; while I found the tables
> >> files are still there on HDFS.
> >>
> >> Anybody has any clue?
> >>
> >> It's quite urgent. Any help will be really appreciated.
> >>
> >> Best,
> >> Arber
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message