hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Major compactions and OS cache
Date Thu, 17 Feb 2011 08:57:21 GMT
Ok, so in my fixed up version of the patch the DN validates the block token before handing
out the file location, so this is not arbitrary access, but it does mean that the hbase user
and the hdfs user must both have read permissions to the local DFS data directories for the
sharing to then work, and that elevates the hbase user to a special status indeed. 

Best regards,

    - Andy

Problems worthy of attack prove their worth by hitting back.
  - Piet Hein (via Tom White)


--- On Wed, 2/16/11, Ryan Rawson <ryanobjc@gmail.com> wrote:

> From: Ryan Rawson <ryanobjc@gmail.com>
> Subject: Re: Major compactions and OS cache
> To: "Jason Rutherglen" <jason.rutherglen@gmail.com>
> Cc: "Edward Capriolo" <edlinuxguru@gmail.com>, user@hbase.apache.org
> Date: Wednesday, February 16, 2011, 6:40 PM
> I can't say, I think there just isn't
> a push for it since mapreduce
> would not benefit from it as much nas HBase. Futhermore the
> patch
> proposals have to deal with HDFS security, and the one I'm
> testing
> just does not worry about security (and hence is a security
> hole
> itself).
> 
> HDFS is just a slow moving project alas.
> 
> On Wed, Feb 16, 2011 at 6:35 PM, Jason Rutherglen
> <jason.rutherglen@gmail.com>
> wrote:
> >> There is a patch that causes us to evict the block
> cache on close of
> >> hfile, and populate the block cache during
> compaction write out.  This
> >> is included in 0.90.
> >
> > That's good!
> >
> >> HDFS-347, which is a huge
> >> clear win but still no plans to include it in any
> hadoop version.
> >
> > Why's that?  It seems to be fairly logical.  Does it
> affect the
> > 'over-the-wire' protocol?
> >
> > On Wed, Feb 16, 2011 at 6:23 PM, Ryan Rawson <ryanobjc@gmail.com>
> wrote:
> >> There is a patch that causes us to evict the block
> cache on close of
> >> hfile, and populate the block cache during
> compaction write out.  This
> >> is included in 0.90.
> >>
> >> So that helps.  Fixing VFS issues is quite a bit
> longer term, since
> >> the on-wire format of HDFS rpc is kind of "fixed",
> petitioning for
> >> changes will be a little tricky. Again, see
> HDFS-347, which is a huge
> >> clear win but still no plans to include it in any
> hadoop version.
> >>
> >> -ryan
> >>
> >
> 


      

Mime
View raw message