hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gokulakannan M <gok...@huawei.com>
Subject RE: Query reg hbase and hadoop append
Date Tue, 04 Jan 2011 05:55:01 GMT
Hello,

 

      I want to merge the append feature from hadoop 0.20-append to the
hadoop 0.20.1 release which I am using currently.

 

      Can you please give the roadmap for append implementation in hadoop
0.20-append(JIRA issues or related details)? All I know is HDFS-200 only :(

 

Thanks and Regards,

Gokul

 

 

-----Original Message-----
From: Gokulakannan M [mailto:gokulm@huawei.com] 
Sent: Wednesday, December 29, 2010 10:53 AM
To: user@hbase.apache.org
Cc: saint.ack@gmail.com
Subject: RE: Query reg hbase and hadoop append

 

Thank you St.Ack. 

 

This helps a lot.. :)

 

 

 

-----Original Message-----

From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack

Sent: Tuesday, December 28, 2010 11:02 PM

To: user@hbase.apache.org; gokulm@huawei.com

Subject: Re: Query reg hbase and hadoop append

 

 

 

We have been concentrating on delivering an hbase that works with

 

hadoop's branch-0.20-append branch and CDH3.  We've not spent time on

 

proofing an hbase on top of hadoop 0.21.  I myself haven't tried it in

 

a while. There is reflection code around the sync call which tries to

 

figure which version of Hadoop we are on top of.  I'd imagine that

 

while it might work, it might not be as performant as when run against

 

hadoop 0.20 append since this is where we've been spending our time.

 

 

 

Here is our story on hadoop versions for the coming 0.90.0:

 

http://people.apache.org/~stack/hbase-0.90.0-candidate-2/docs/notsoquick.htm

l#hadoop

 

(Is this clear?  Should we add anything?).

 

 

 

As to why we're in this state where we are NOT up against the latest

 

version of hadoop, the story is a little involved.  I'll spare you the

 

torturous detail.  In short, the next major HBase release was to run

 

against hadoop 0.21 but when we learned, belatedly, that its release

 

was delayed with no guarantees regards when it would drop and that it

 

was not going to be installed (or tested) by the 'big boys' -- Y!, FB,

 

etc. -- we lost all enthusiasm for hadoop 0.21.  We figured no one

 

would run an hadoop 0.21 in production.

 

 

 

Our focus shifted to the append branch in 0.20 because this was a

 

relatively minor deviation from the pervasive hadoop 0.20.  We could

 

envision folks installing the append branch to get an hbase

 

that-did-not-lose-data and running this hadoop in production because

 

it was close enough to hadoop 0.20 and at least one of the big teams

 

was running it -- i.e. fb -- or something very close to it.  A bunch

 

of effort was sunk into making sure HBase worked on this branch and

 

that it was performant.

 

 

 

Cloudera's CDH3 since b2 has had an append.  Its provenance is the

 

hadoop 0.20-append branch.  Not hadoop 0.21.

 

 

 

Hope this helps,

 

St.Ack

 

 

 

 

 

 

 

On Tue, Dec 28, 2010 at 12:43 AM, Gokulakannan M <gokulm@huawei.com> wrote:

 

> hadoop 0.21.0 has been released with the stable hdfs append feature..

 

> 

 

> 

 

> 

 

> Should we do anything in hbase(modify / apply some patch) to use this hdfs

 

> append feature or hbase takes care of this in-built??

 

> 

 

> 

 

> 

 

> from web i've seen that this append feature(hdfs-265) is different from

what

 

> hbase expects(hdfs-200)..

 

> 

 

> 

 

> 

 

> can anyone pls explain the background???

 

> 

 

> 

 

> 

 

> 

 

> 

 

>  Thanks,

 

> 

 

>  Gokul

 

> 

 

> 

 

> 

 

> 

 

> 

 

> 

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message