aries-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremias Maerki <...@jeremias-maerki.ch>
Subject Re: [SPI Fly] Interest for a more dynamic plug-in/service discovery API?
Date Fri, 05 Oct 2012 13:58:58 GMT
Hi David

Thanks for taking the time!

On 05.10.2012 14:04:06 David Bosschaert wrote:
> Hi Jeremias,
> 
> I took some time to look at your code, just echoing what I think that
> it does to make sure that I understand things correctly.
> I think it implements a dynamic services mechanism that works both
> inside OSGi as well as outside of OSGi. Correct?

Yes.

> Services are declared using the META-INF/services resources, just like
> they are done with ServiceLoader. The client side doesn't use the
> java.util.ServiceLoader.load() API.

Right. That's to enable Java 5 compatibility and to allow for more
dynamics adding and removing plug-ins than java.util.ServiceLoader can
offer.

> In the case we're outside of OSGi there is a custom non-OSGi
> ServiceTracker/ServiceListener that reacts to services arriving /
> disappearing.

Right now, it doesn't handle anything disappearing (the equivalent of
ServiceLoader.reload()). It does one discovery run and sticks to the
discovered set, practically never calling ServiceListener.removed().

> When you're inside OSGi you can look up these services from the
> Service Registry as SPI-Fly has put them there.

Exactly. I was really pleased with SPI Fly's approach even if it made
some of my initial work obsolete (but it was flawed, too).

> Am I correctly summarizing? Did I miss anything?

Spot on.

> Next question is would it make sense to add this functionality to Aries?
> I think it does. To me many of the ideas in here match with the OSGi
> Connect RFP 145 (http://www.osgi.org/bugzilla/show_bug.cgi?id=145) and
> I think that, besides its practical use today, this code could be a
> valuable input to the standardization process of OSGi Connect. Overall
> the charter of OSGi Connect is to create a dynamic services
> environment that works both inside OSGi and out. To me the overall
> goal of your code seems similar.
> If we all agree that it would be suitable for this component to reside
> in Aries, I think we should strive to make it ultimately compliant
> with the OSGi Connect spec, when that's available.
> 
> Does this make sense to you?

As I understand it OSGi Connect's goal is to use a subset of the OSGi
framework (most importantly the service layer but not the module layer).
So you can use the OSGi ServiceTracker to lookup services. In that case,
my library isn't needed and probably not very useful, since it actually
strives not to use OSGi APIs at all. So, I'm not quite getting your
point here. I got about one too many hints that some people may have
reservations when introducing OSGi to a plain Java project ("Do we all
have to learn OSGi? Can I still use X in plain Java? etc."). OSGi,
unfortunately, is still not as widely adopted as I would like. I've
noticed how a low-level ServiceTracker can provoke reactions like: "Does
it have to be that complicated?" At least, until they get the power of
it. So, my main goal was to really just shield everyone from OSGi as
much as possible. Basically, I just wanted to provide an easy migration
path without the requirement to learn about OSGi beyond including
manifest metadata. If my thingy helps OSGi Connect, that's great but I
frankly don't see how. I'm probably still missing something.

Jeremias Maerki


> Best regards,
> 
> David
> 
> On 1 October 2012 14:24, David Bosschaert <david.bosschaert@gmail.com> wrote:
> > Thanks for reminding me, Jeremias. I downloaded it but then I got
> > distracted. Hope to have a look soon.
> >
> > David
> >
> > On 1 October 2012 03:27, Jeremias Maerki <dev@jeremias-maerki.ch> wrote:
> >> Hi David,
> >> did you have any chance to look at it already?
> >>
> >> BTW, I've just found a possible memory leak in the OSGi case when the
> >> client doesn't (or can't easily) remove the ServiceListener. Usually,
> >> there's only one ServiceListener per client class (static context) so
> >> it's not totally serious, but it prevents GC'ing the bundle. I'm working
> >> on that now.
> >>
> >> Cheers,
> >> Jeremias Maerki
> >>
> >>
> >> On 30.08.2012 17:29:47 David Bosschaert wrote:
> >>> Thanks Jeremias!
> >>>
> >>> I'll have a look at this soon.
> >>>
> >>> Cheers,
> >>>
> >>> David
> >>>
> >>> On 23 August 2012 15:39, Jeremias Maerki <dev@jeremias-maerki.ch>
wrote:
> >>> > Hi David
> >>> >
> >>> > Sorry for the delay. As life plays, you get side-tracked all the time,
> >>> > but since this is important to me I had to revisit eventually.
> >>> >
> >>> > I've simplified and cleaned up the code some more and added some tests.
> >>> > It's now ready for review:
> >>> > http://www.jeremias-maerki.ch/download/osgi/ch.jm.util.services-2.0.0.dev-src.zip
> >>> >
> >>> > So, if you still think this could be a good addition to Aries, I can
> >>> > rename the packages, switch to a Maven build (sigh) and prepare a
> >>> > package to vote upon for inclusion in Aries.
> >>> >
> >>> > What could we name it?
> >>> > - org.apache.aries.spifly.consumer
> >>> > - org.apache.aries.spifly.discovery
> >>> > - org.apache.aries.discovery
> >>> > - org.apache.aries.plugin.discovery
> >>> > - org.apache.aries.spi.catch ;-)
> >>> >
> >>> > You mentioned earlier that you don't see it as part of SPI-Fly. In
its
> >>> > current state it actually relies on a working implementation of RFP
167
> >>> > (SPI Service Loader support) because I removed the extender that looks
> >>> > for META-INF/services from 1.0 to 2.0. So, while it is not part of
RFC
> >>> > 167, it is still thighly connected to SPI-Fly, although not through
> >>> > package imports.
> >>> >
> >>> > On RFP 143: like I already mentioned, I do find it interesting. It
can
> >>> > serve as a different approach to solving the same problem. However,
my
> >>> > approach gets away with a 20KB JAR, compared to the 220KB of PojoSR
> >>> > (still small!). But my main argument is actually this: I need to find
a
> >>> > way to sell an easy way to the Apache XML Graphics project to achieve
> >>> > OSGi compatibility including plug-in support without throwing everything
> >>> > upside down. Practically none of the committers are familiar with OSGi.
> >>> > Introducing an OSGi-compatible service registry is generally cool for
> >>> > someone who knows OSGi already but the others would probably ask: why
do
> >>> > we need to learn about this? After all, the service registry can be
a
> >>> > bit verbose for a newbie with ServiceReference, BundleContext and such.
> >>> > I believe that selling OSGi Connect would be more difficult in such
a
> >>> > case than my small proposal.
> >>> >
> >>> > In the end, I'm looking for the best home for it. I think Aries is
a
> >>> > logical place.
> >>> >
> >>> > When looking at PojoSR I was wondering about use cases. Personally,
I'd
> >>> > rather have a full OSGi environment any time even if it means that
I
> >>> > have to be able to maintain a bundle cache/storage folder on the local
> >>> > file system (I actually wonder why these are not pluggable and
> >>> > virtualizable, yet, but that's a different story). Use case 4.2 (WARs)
> >>> > in the RFP draft makes a lot of sense, though. There you don't usually
> >>> > have any service dynamics. Or a command-line application that uses
some
> >>> > code that usually runs in a OSGi-enabled server. I have such a case
that
> >>> > I might explore eventually. But frankly, OSGi Connect doesn't create
too
> >>> > much attraction for me since I will usually want to work in a full
OSGi
> >>> > environment. And when I can't do that I'm likely happy with plain
> >>> > META-INF/services. I guess I can't correctly imagine the possibilities
> >>> > that are mentioned in use case 4.3 (Application Frameworks).
> >>> >
> >>> > Cheers,
> >>> > Jeremias Maerki
> >>> >
> >>> >
> >>> > On 13.07.2012 16:07:58 David Bosschaert wrote:
> >>> >> Hi Jeremias,
> >>> >>
> >>> >> Just FYI OSGi RFP 143 has just been published which relates precisely
> >>> >> to the issue of using a Service Registry without going full-blown
on
> >>> >> the modularity. Ultimately the idea is that there will be an OSGi
> >>> >> specification around this.
> >>> >> Your work might fit with this idea as well and I would be very
> >>> >> interested in your thoughts around this topic.
> >>> >>
> >>> >> See here: http://blog.osgi.org/2012/07/new-rfps-available-for-feedback.html
> >>> >> and here: https://www.osgi.org/bugzilla/show_bug.cgi?id=145
> >>> >>
> >>> >> Best regards,
> >>> >>
> >>> >> David
> >>> >>
> >>> >> On 13 June 2012 08:02, Jeremias Maerki <dev@jeremias-maerki.ch>
wrote:
> >>> >> > Hi David
> >>> >> >
> >>> >> > Thanks a lot for your feedback! PojoSR is interesting. You
could say
> >>> >> > that there is a certain overlap. If I understand this right,
PojoSR is
> >>> >> > suggesting to detect not-quite-OSGi bundles via
> >>> >> > ClassLoader.getResources("META-INF/MANIFEST.MF") rather than
to detect
> >>> >> > individual services via "META-INF/services/xy". It has a focus
on
> >>> >> > maintaining a service registry that is based on OSGi principles
(bundle
> >>> >> > activator and all) rather than working off the old JAR service
providers.
> >>> >> > It brings OSGi to plain Java rather than plain Java to OSGi.
;-) As an
> >>> >> > OSGi addict, that is very appealing to me. The problem is
selling it, I
> >>> >> > guess. Even selling OSGi compatibility in a plain Java project
requires
> >>> >> > addressing things like: "no, you don't have to change your
application
> >>> >> > to OSGi."
> >>> >> >
> >>> >> > So, both approaches have their strong points. Let's try to
put them next
> >>> >> > to each other:
> >>> >> >
> >>> >> > PojoSR:
> >>> >> > - ability to work OSGi-like in plain Java, no changes running
in real OSGi
> >>> >> > - runtime dependency on PojoSR, OSGi core and compendium in
plain Java
> >>> >> > - existing applications using META-INF/services have to change
their
> >>> >> > plug-in approach to OSGi-style services
> >>> >> > - existing plug-ins need to get OSGi metadata
> >>> >> >
> >>> >> > my approach:
> >>> >> > - developers are shielded from OSGi except for the build-time
dependency
> >>> >> > (OSGi core) and the build changes for OSGi metadata
> >>> >> > - one runtime dependency in plain Java (the ServiceListener
> >>> >> > infrastructure)
> >>> >> > - existing applications have to switch from ServiceLoader
(or Services
> >>> >> > in Apache XML Graphics) to a ServiceListener (ServiceTracker-like)
> >>> >> > for obtaining plug-ins
> >>> >> > - existing plug-ins need to add OSGi metadata as per RFC 167.
> >>> >> > - probably has a considerably lower learning curve for people
that
> >>> >> > haven't had contact with OSGi, yet.
> >>> >> >
> >>> >> > I guess one question is whether PojoSR works with SPI-Fly
in plain Java.
> >>> >> > That would make this very nice. But I suspect that might not
be so
> >>> >> > simple. If it doesn't work, all plug-ins have to be changed
to publish
> >>> >> > services through a BundleContext. That would make it impossible
to
> >>> >> > retro-fit an older plug-in just by adding OSGi metadata to
the JAR.
> >>> >> >
> >>> >> > Anyway, I'll put together a submission in the next few days
(probably
> >>> >> > next week due to other obligations). We can then see where
to go from
> >>> >> > there. I'd be happy to help with the spec however I can. I
guess my code
> >>> >> > will at least be a useful additional discussion base.
> >>> >> >
> >>> >> > Thanks,
> >>> >> > Jeremias Maerki
> >>> >> >
> >>> >> >
> >>> >> > On 12.06.2012 14:47:01 David Bosschaert wrote:
> >>> >> >> Hi Jeremias,
> >>> >> >>
> >>> >> >> Many thanks for the suggestion. You are highlighting a
valid point:
> >>> >> >> namely that people often have the need for services, like
the ones
> >>> >> >> supported in OSGi, but are unable to (or don't want to)
modularize
> >>> >> >> their project in order to be able to use OSGi.
> >>> >> >> There is currently some work underway in the OSGi expert
groups that
> >>> >> >> relates to this.
> >>> >> >>
> >>> >> >> The PojoSR project (http://code.google.com/p/pojosr/)
is a relatively
> >>> >> >> mature project that implements most of the OSGi Service
Registry and
> >>> >> >> also supports things like the BundleActivator without
doing the
> >>> >> >> modularity part of OSGi. So in a way it is similar to
what you have
> >>> >> >> written, if I'm not mistaken.
> >>> >> >>
> >>> >> >> In the OSGi Core Platform Expert Group there is currently
an effort
> >>> >> >> underway to create a specification inspired by PojoSR.
I will ensure
> >>> >> >> that during the specification work your work is also considered
as
> >>> >> >> input, if you wish [1].
> >>> >> >>
> >>> >> >> So in short - there will be a specification around this
in the not too
> >>> >> >> distant future, your work can be considered a stepping
stone to this
> >>> >> >> specification and it could even possibly become a compliant
> >>> >> >> implementation in the future. I assume that PojoSR will
also become
> >>> >> >> compliant but I think that there is always room for multiple
> >>> >> >> implementations of a spec.
> >>> >> >>
> >>> >> >> If you are interested in updating your implementation
to comply with
> >>> >> >> the spec in the future then I think that Apache Aries
would be a good
> >>> >> >> place to put your implementation right now from where
it can mature. I
> >>> >> >> think that it should not be a subcomponent of SPI-Fly
but rather a
> >>> >> >> top-level component of its own, within Aries.
> >>> >> >>
> >>> >> >> Best regards,
> >>> >> >>
> >>> >> >> David
> >>> >> >>
> >>> >> >> [1] or, if you'd rather do that yourself, let me know
:)
> >>> >> >>
> >>> >> >> On 8 June 2012 10:42, Jeremias Maerki <dev@jeremias-maerki.ch>
wrote:
> >>> >> >> > Hi there,
> >>> >> >> >
> >>> >> >> > due to my involvement with the Apache XML Graphics
project (FOP/Batik)
> >>> >> >> > where we make heavy use of META-INF/services, I've
got a big interest in
> >>> >> >> > how best to bring FOP and Batik into OSGi. My particular
requirement
> >>> >> >> > here is that FOP and Batik have to continue working
in plain Java but
> >>> >> >> > profit from the OSGi service registry for plug-in
discovery when
> >>> >> >> > possible.
> >>> >> >> >
> >>> >> >> > So, two years ago I came up with this:
> >>> >> >> > http://www.jeremias-maerki.ch/development/osgi/jar-services.html
> >>> >> >> >
> >>> >> >> > That works half-way good enough although I've never
felt comfortable
> >>> >> >> > enough to push it back to the XML Graphics project
as I've had a few
> >>> >> >> > flaws in the above implementation. So, by now SPI
Fly does a lot of
> >>> >> >> > things much better than my approach and it is following
an RFC that I
> >>> >> >> > find very useful.
> >>> >> >> >
> >>> >> >> > However, on the client side, RFC 167 is a bit heavy
on J2SE-1.6's
> >>> >> >> > ServiceLoader. Apache XML Graphics is still on J2SE-1.5
(yes, I know, I
> >>> >> >> > know) so it contains its own Services.java to find
plug-ins via
> >>> >> >> > META-INF/services. Furthermore, with ServiceLoader
you basically have to
> >>> >> >> > poll for changes.
> >>> >> >> >
> >>> >> >> > Finally getting to my point: I'd like to offer a
little API that brings
> >>> >> >> > some of the service dynamics you get with OSGi services.
This can
> >>> >> >> > already be seen on the page indicated above although
I've slightly
> >>> >> >> > modified the API. Essentially, to get notified about
new or disappearing
> >>> >> >> > plug-ins/services, you can register a service listener.
And that would
> >>> >> >> > look like this (client code):
> >>> >> >> >
> >>> >> >> >        ServiceTracker<ImageWriter> tracker
= Plugins.getServiceTracker(ImageWriter.class);
> >>> >> >> >        tracker.addServiceListener(new ServiceListener<ImageWriter>()
{
> >>> >> >> >
> >>> >> >> >            public void added(ImageWriter writer)
{
> >>> >> >> >                register(writer);
> >>> >> >> >            }
> >>> >> >> >
> >>> >> >> >            public void removed(ImageWriter writer)
{
> >>> >> >> >                unregister(writer);
> >>> >> >> >            }
> >>> >> >> >
> >>> >> >> >        });
> >>> >> >> >
> >>> >> >> > Where Plugins is:
> >>> >> >> >
> >>> >> >> > public class Plugins {
> >>> >> >> >
> >>> >> >> >    /**
> >>> >> >> >     * This is the services singleton.
> >>> >> >> >     */
> >>> >> >> >    private static final Services SERVICES = new Services();
> >>> >> >> >
> >>> >> >> >    public static void setServicesBackend(ServicesBackend
backend) {
> >>> >> >> >        SERVICES.setServicesBackend(backend);
> >>> >> >> >    }
> >>> >> >> >
> >>> >> >> >    public static <T> ServiceTracker<T>
getServiceTracker(Class<T> providerIntf) {
> >>> >> >> >        return SERVICES.getServiceTracker(providerIntf);
> >>> >> >> >    }
> >>> >> >> >
> >>> >> >> > }
> >>> >> >> >
> >>> >> >> > And the BundleActivator for the client looks like
this:
> >>> >> >> >
> >>> >> >> > public class Activator implements BundleActivator
{
> >>> >> >> >
> >>> >> >> >    private volatile ServicesOSGi services;
> >>> >> >> >
> >>> >> >> >    /** {@inheritDoc} */
> >>> >> >> >    public void start(BundleContext context) throws
Exception {
> >>> >> >> >        this.services = new ServicesOSGi(context);
> >>> >> >> >        Plugins.setServicesBackend(this.services);
> >>> >> >> >    }
> >>> >> >> >
> >>> >> >> >    /** {@inheritDoc} */
> >>> >> >> >    public void stop(BundleContext context) throws
Exception {
> >>> >> >> >        Plugins.setServicesBackend(null);
> >>> >> >> >        if (this.services != null) {
> >>> >> >> >            this.services.close();
> >>> >> >> >        }
> >>> >> >> >    }
> >>> >> >> >
> >>> >> >> > }
> >>> >> >> >
> >>> >> >> > What happens is this: by default "Services" starts
up with a plain-Java
> >>> >> >> > backend that simply looks up services through classic
means. When the
> >>> >> >> > BundleActivator is called, the backend is replaced
by ServicesOSGi that,
> >>> >> >> > instead, gets the plug-ins from the service registry.
In the background,
> >>> >> >> > any previously discovered plug-ins are "removed()"
and the new ones from
> >>> >> >> > the registry "added()".
> >>> >> >> >
> >>> >> >> > In the case of classic Java, the "removed()" method
is never called
> >>> >> >> > (probably doesn't ever need to be), i.e. no dynamics
there.
> >>> >> >> >
> >>> >> >> > To conclude: the goal is to profit from OSGi service
dynamics when
> >>> >> >> > discovering plug-ins while preserving the possibility
to run without
> >>> >> >> > OSGi API runtime dependencies and still in J2SE-1.5.
Furthermore, the
> >>> >> >> > OSGi-specific code shall be as minimal as possible
to shield those with
> >>> >> >> > no OSGi knowledge in the team from the learning curve.
> >>> >> >> >
> >>> >> >> > And now, I'm wondering if you're interested to adopt
this as a new part
> >>> >> >> > in SPI Fly as a useful alternative to using ServiceLoader.
If you don't
> >>> >> >> > want it, I'm going to publish it under Apache Extras.
> >>> >> >> >
> >>> >> >> > Thanks,
> >>> >> >> > Jeremias Maerki
> >>> >> >> >
> >>> >> >
> >>> >
> >>


Mime
View raw message