hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CaiSijie <cai.si...@fujixerox.co.jp>
Subject Re: hbase performance period
Date Tue, 04 Nov 2008 02:26:02 GMT

Thank you for your replay.
My version of HBase is 0.18.0.  Yes, I read data in series.

But what I see is that reading 1st data cost least time and reading 128th
data cost most time. It means that time increase from reading 1st to 128th
data item. Then when reading 129th data item, time becomes less and similar
as reading 1st data item time. So the period is 128. 128th data, 256th data,
384th data... need most time and 1st data, 129th data, 257th data, 385th
data... need least time.

I have tested for many times but it always exists this phenomenon. I can be
sure that the mapfile index interval is 32.
And what i should say is that every item data is 100 KiloByte.

I am confused...

Sijie Cai 




stack-3 wrote:
> 
> The only thing that comes to mind is that by default in hadoop, the
> mapfile index interval is 128; every 128th entry in mapfile gets an
> entry in the mapfile index. Only, in hbase, we change the default
> interval to be 32. Check to make sure you are picking up
> hbase.io.index.interval of 32.
> 
> Otherwise, I'm not sure as to why you would see the below. Are you
> saying that there is a step every 128 intervals? That the 129th read
> takes longer than the read at position 1 and that the read at position
> 257 takes longer than the read at position 129?
> 
> The fact that it takes increasingly longer as you read from position 0
> up to 128 makes sense -- if the index interval is every 128 -- because
> we do serial search forward from the closest index position.
> 
> What version of hbase are you using?
> 
> You are doing your reads in series?
> 
> This is really interesting stuff. Can you dig in some more and try and
> figure whats going on?
> 
> Thanks Cai.
> 
> St.Ack
> 
> 

-- 
View this message in context: http://www.nabble.com/hbase-performance-period-tp20209042p20315108.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message