hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jl...@streamy.com>
Subject RE: Many columns in 0.19
Date Mon, 09 Mar 2009 13:50:23 GMT
That is within the acceptable ranges for 0.19.  It's not recommended to go past 15 or so families
on 0.19, should be better on 0.20, but like Ryan said each family is stored separately so
you'll always be adding more overhead to the table for each additional family.  Total column
qualifiers/labels for 0.19 is several thousand, I've got row-families with about 50,000 columns
and don't see any problems.  The hope is, however, to be in the millions for 0.20.


> -----Original Message-----
> From: Michael Seibold [mailto:seibold@in.tum.de]
> Sent: Monday, March 09, 2009 2:06 AM
> To: hbase-user@hadoop.apache.org
> Subject: Re: Many columns in 0.19
> 
> Hi,
> 
> so how many column families per table and how many columns(=labels) per
> column family are feasible in HBase 0.19.0?
> 
> column families per table: ?
> columns per column family: ?
> 
> I want to use:
> --------------------
> column families per table: 10
> columns (=labels) per column family: 100 - 150
> 
> Will I hit HDFS performance issues with that?
> 
> Kind regards,
> Michael
> 
> 
> El lun, 09-03-2009 a las 01:50 -0800, Ryan Rawson escribió:
> > Don't forget, each column family is another file on disk, and file
> open.
> > Every column family is stored in it's own mapfile, and that increases
> the
> > load on HDFS.
> >
> > This particular restriction won't ever really go away (unless we
> introduce
> > locality groups, even then, each locality group = N families = 1
> file), but
> > in 0.20 it should be more feasable to have thousands of columns per
> family,
> > or more.
> >
> > -ryan
> >
> > On Mon, Mar 9, 2009 at 1:47 AM, Michael Dagaev
> <michael.dagaev@gmail.com>wrote:
> >
> > > Thank you, Ryan
> > >
> > > On Mon, Mar 9, 2009 at 10:28 AM, Ryan Rawson <ryanobjc@gmail.com>
> wrote:
> > > > Sadly this is still a limit.
> > > >
> > > > 0.20 should make things much better.
> > > >
> > > > -ryan
> > > >
> > > > On Mon, Mar 9, 2009 at 12:23 AM, Michael Dagaev <
> > > michael.dagaev@gmail.com>wrote:
> > > >
> > > >> Hi , all
> > > >>
> > > >>    I remember it was not recommended to add many columns (column
> > > >> qualifiers) in Hbase 0.18
> > > >> Does Hbase 0.19.0 still have this limitation?
> > > >>
> > > >> Thank you for your cooperation,
> > > >> M.
> > > >>
> > > >
> > >


Mime
View raw message