nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kirby Bohling <>
Subject Re: Nutch dev. plans
Date Wed, 29 Jul 2009 14:55:28 GMT
2009/7/29 Doğacan Güney <>:
> Hey guys,
> Kirby, thanks for all the insightful information! I couldn't comment
> much as most of
> the stuff went right over my head :) (don't know much about OSGI).

A bit of a progress report to make sure I'm heading in the proper direction.

I'm learning I know a lot about Eclipse RCP, which hid lots of details
about OSGi from me.  No public code yet, I'm hoping that happens this

Got Felix downloaded and started an embedded OSGi environment
successfully.  I chose Felix because it's Apache licensed.  I'm not
clear if the CPL/EPL is acceptable to the ASF for inclusion and
distribution.  Sounds like Equinox is more full featured.  I'll
probably integrate with both just for sanity checking portability.

My quick research indicates that Hadoop isn't OSGi application
friendly, but can host an embedded OSGi environment.  I bogged down
attempting to integrate the bnd tool to run inside of Ant.  I think I
have that resolved so I can just wrap third party jar's in the ant
scripts.  So hopefully I can make more meaningful progress soon.

The current mental architecture I have is to make all the libraries in
./lib/*.jar end up in the top level classloader outside the OSGi
environment (I forget the technical OSGi name, I think it is the
System Classloader).  Then, turn the Nutch Core into a single bundle.
Turn each current plugin into an OSGi bundle.  Each plugin registers a
"service" which is a factory capable of creating whatever is currently
inside of the plugin.xml as an "implementation" attribute.  Modify the
core to use the factories inside of the extension point code.  I think
that is the minimally invasive way to get to OSGi.

I assumed that it would be easiest to get OSGi in and integrated with
minimal disruption.  If nothing else, we can use that as a staging
point to play with more invasive designs and architectures.  In the
long run, I think something more invasive makes more sense.  Hopefully
that is effective risk management rather then a waste of time.

Please course correct me if any of this seems a bad idea.

> Andrzej, would OSGI make creating plugins easier?

I'm not sure it will.  Hopefully we can accomplish that in some way or
another, especially out of tree plugins.  I know I have plenty of out
of tree plugins in my future at work.

> On Sun, Jul 26, 2009 at 19:09, Andrzej Bialecki<> wrote:
> [......snipping thread as it has gone too long.....]
> --
> Doğacan Güney

View raw message