hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Re: hbase read performance tuning failed
Date Wed, 08 Jan 2014 01:51:04 GMT
Xiaofeng:
Take a look at this:
http://hbase.apache.org/book.html#hbase.regionserver.checksum.verify

Cheers


On Tue, Jan 7, 2014 at 5:36 PM, LEI Xiaofeng <leixf@ihep.ac.cn> wrote:

> I tried to set "BLOCKSIZE" to 0 when created table to tune the scan
> performance, but it did not work. Another thing is what do you mean by
> "turing HBase checksumming on"? I do not know how to do it. Could you give
> mw some detailed information?
>
> Thanks
>
>
> > -----原始邮件-----
> > 发件人: "Doug Meil" <doug.meil@explorysmedical.com>
> > 发送时间: 2014年1月8日 星期三
> > 收件人: "user@hbase.apache.org" <user@hbase.apache.org>, "lars hofhansl"
<
> larsh@apache.org>
> > 抄送:
> > 主题: Re: hbase read performance tuning failed
> >
> >
> > In addition to what Lars just said about the blocksize, this is a similar
> > question to another one that somebody asked, and it's always good to make
> > sure that you understand where your data is. As a sanity check, make sure
> > it's not all on one or two RSs (look at the hbase web pages or with tools
> > like Hannibal).
> >
> >
> > Also, you definitely want to to turn HBase checksumming on - and when you
> > do so you'll need to re-create the HFiles (e.g., you can't just change
> the
> > config and bounce the HBase cluster).  That's a significant reduction in
> > I/O.
> >
> > Likewise, if you are doing a full-scan, make sure that you select only
> the
> > attributes you need...
> >
> > See this for more:  http://hbase.apache.org/book.html#perf.reading
> >
> >
> >
> >
> >
> > On 1/7/14 1:24 PM, "lars hofhansl" <larsh@apache.org> wrote:
> >
> > >If increasing hbase.client.scanner.caching makes no difference you have
> > >another issue.
> > >How many rows do you expect your to return?
> > >
> > >On contemporary hardware I manage to scan a few million KeyValues (i.e.
> > >columns) per second and per CPU core.
> > >Note that for scan performance you want to increase the BLOCKSIZE.
> > >
> > >
> > >-- Lars
> > >
> > >
> > >
> > >________________________________
> > > From: LEI Xiaofeng <leixf@ihep.ac.cn>
> > >To: user@hbase.apache.org
> > >Sent: Monday, January 6, 2014 11:06 PM
> > >Subject: hbase read performance tuning failed
> > >
> > >
> > >Hi,
> > >I am running hbase-0.94.6-cdh4.5.0 and set up a cluster of 5 nodes. The
> > >random read performance is ok, but the scan performance is poor.
> > >I tried to increase "hbase.client.scanner.caching" to 100 to promote the
> > >scan performance but it made  no difference. And when I tried to make
> > >smaller blocks by setting "BLOCKSIZE" when created tables to get better
> > >random read performance it made no difference too.
> > >So, I am wondering if anyone could give some advice to solve this
> problem.
> > >
> > >
> > >
> > >Thanks
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message