gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferdy Galema <ferdy.gal...@kalooga.com>
Subject Re: auto flush
Date Wed, 04 Apr 2012 08:34:04 GMT
Hi Eric,

You are right on both issues. There should be an option to turn of
autoflushing when creating the HBaseStore. I think the gora properties file
is a nice place for this option.

And indeed a flush should flush all threadlocal instances. I implemented
the threadlocal code for the HBaseStore, because at the time there was no
synchronization at all for the HBaseStore. The solution for the flushing
problem would be to use a readwrite lock so that all threads can use their
HTable instance freely (read-lock) except when flushing, because that will
block actions until the flush is done (write-lock). That way all HTable
instances will get flushed. A nicer solution would be to see if there are
other (best-practices) ways to implement multithreading HBase client code.

I will shortly file Jira issues and patch them.

Ferdy.

On Wed, Apr 4, 2012 at 2:45 AM, Eric Newton <eric.newton@gmail.com> wrote:

> Hi Lewis,
>
> I changed gora-hbase in my local copy to always turn off auto flush, and
> then I flush as-needed.  It looks as though I might need to make sure that
> I flush with the same thread I write with, since it is using thread-local
> storage to pick up the correct client interface.  I don't see that as a
> major problem, it's just a surprising result of the way the connection was
> implemented.  I could be wrong, though.  I only spent about 10 minutes
> looking into it, in order to figure out why HBase was slower than I
> expected.
>
> -Eric
>
> On Tue, Apr 3, 2012 at 4:23 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi Eric,
> >
> > On Tue, Apr 3, 2012 at 4:04 PM, Eric Newton <eric.newton@gmail.com>
> wrote:
> >
> > > Is there any particular reason that gora-hbase uses auto flush on every
> > > HTable connection?
> > >
> >
> > I'm not using the HBase module at the moment so will wait for others to
> > chime in, however maybe you could comment on an alternative
> implementation
> > (or simply to remove auto flush making the flush configurable instead?) I
> > would be really interested to hear your comments.
> >
> >
> > >
> > > This makes Keith Turner's excellent goraci test run very slowly for
> > hbase.
> > >
> >
> > Well it's something we should defo look into then. Hopefully we can
> > actually get this test suite integrated into our CI build soon, until
> then
> > thanks for pointing out the problem.
> >
> >
> > >
> > > Also, I was unable to subscribe using user-subscribe@gora.apache.org.
> > >
> >
> > Yeah currently we don't have user list, all traffic has been coming
> through
> > dev@. I was actually going to wait until after our next release before
> > getting user@ sorted as most of the work going on has been development
> > since graduation. I'll progress with logging an issue with INFRA though,
> > thanks for pointing this out.
> >
> > Lewis
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message