hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: 2 bucket caches?
Date Mon, 29 Jun 2015 14:43:45 GMT
I think you may want to think a bit about this… 

How far do you want to go with your memory management? 

'Off heap' is a new nifty way of saying application level swap and memory management.  So
what you are basically saying is that I have memory, local persistence, then HDFS persistence.

And your local persistence could be anything… (PCIe based flash, UltraDIMMs, RRAM (when
it hits the market), SSDs, even raided spinning rust… )

If you’re going in that direction, what is tachyon doing? 

If you want to do this… and I’m not saying its a bad idea, you’ll want to think a bit
more generic. Essentially its a layered hiearchy (memory, p1, p2, …) where p(n) is a pool
of devices which have a set of rules on how to propagate pages up or down the hierarchy. 





> On Jun 29, 2015, at 1:20 AM, Jean-Marc Spaggiari <jean-marc@spaggiari.org> wrote:
> 
> Hi,
> 
> Is it possible to have 2 bucket cache on a single region server?
> 
> Like L2 and L3? I would like to have L2 offheap and block evicted from L2
> going into L3 on SSD. So we already have something like that? Or should I
> open a JIRA?
> 
> hbase.bucketcache.ioengine can get only one value. Might be nice to have a
> flume-like approach...
> 
> hbase.bucketcache=myoffheap,myssddrive
> hbase.bucketcache.myoffheap.ioengine=offheap
> hbase.bucketcache.myssddrive.ioengine=file://my_ssd_mnt/there
> 
> And keep the order specified in hbase.bucketcache, so myoffheap=L2,
> myssddrive=L3, etc.?
> 
> Thanks,
> 
> JM

The opinions expressed here are mine, while they may reflect a cognitive thought, that is
purely accidental. 
Use at your own risk. 
Michael Segel
michael_segel (AT) hotmail.com






Mime
View raw message