buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Assaf Arkin" <ar...@intalio.com>
Subject Re: Transitive dependencies, version matching and OSGi support
Date Thu, 08 May 2008 01:44:55 GMT
On Wed, May 7, 2008 at 6:19 PM, Shane Witbeck <shane@digitalsanctum.com>
wrote:

> I second the idea of going the way of OSGI. It's clearly the next step
> in Java's evolution.
>
> In terms of pre-1.3 dependency support, we resorted to doing something
> like the following to support a subset of compile dependencies for
> runtime packages:
>
> project.runtime.with(specs)


+1 for adding runtime dependencies.


>
>
> Because some of our environments had shared runtime classpaths, it
> wasn't always necessary to include certain libraries. I did this by
> extending Buildr::Project to support the different kinds of internal
> projects we have such as WebProject, StandaloneProject, BatchProject,
> etc.
>
> Something else that was needed in our projects was a way of putting
> together different packages for different environments. Again, by
> extending Buildr::Project, I introduced a way to package and deploy
> (via SSH) the various projects.
>
> I mention the ways that I've extended Buildr to possibly touch on new
> features to incorporate such as:
>
> 1. Being able to define different sets of dependencies for projects
> (in my case it was compile vs. runtime). I like the direction Assaf
> mentions with being able to merge dependency arrays and better
> transitive support. This is huge in terms of being able to make the
> buildfile less verbose.
> 2. A way to easily deploy (via SSH, etc.) packages


Can you expand on that?


>
> 3. Better integration with IDE's. It's a struggle to keep the
> dependencies in sync without having to regenerate the IDE files and in
> the process losing other project-specific IDE settings. Perhaps this
> is better solved via IDE-specific plugins for Buildr?


Good point.  It might be possible to retain those project-specific IDE
settings, though I don't know what it would look like.


>
> 4. In terms of additional language support, my vote is for Flex. With
> the recent open sourced SDK, I believe there will be a surge in usage
> among not only Java developers but developers of other languages as
> well.


We'll need someone who's using Flex to help develop it.

Also, this is something we don't have to tie to the release schedule.  We
finally have an API for adding new languages and delivering extensions using
Ruby Gems, so you can get people to use it without waiting for the next
release.  Separately, once we add it to the code base it will show up in the
next release, but I think independent Gem would be the way to start.

Assaf


>
>
> I'd be glad to elaborate further on the points I've brought up and
> look forward to contributing more.
>
> Thanks,
> Shane
>
>
> On Fri, May 2, 2008 at 5:41 PM, Assaf Arkin <arkin@intalio.com> wrote:
> > Now that 1.3 is out, it's time to start talking about future releases.
> >   We've got the 1.3.x line of releases, for dealing with outstanding
> issues,
> >  new ones we'll find along the way, and building on new features that
> showed
> >  up in 1.3.
> >
> >  Separately, there's 1.4, the next major release and a place to make more
> >  significant changes.  We just have to decide what those would be.
> >
> >  I propose that we focus 1.4 on improving the way we handle dependencies.
> >
> >  Specifically what I have in mind are transitive dependencies, version
> >  matching, and OSGi support.  So I'll start by talking about some things
> I've
> >  been looking into recently.
> >
> >
> >  For starters, adding a dependency method to artifact (which also
> includes
> >  packages) that returns an array of all the known dependencies for that
> >  artifacts.  Ruby arrays are pretty powerful, you can do things like:
> >
> >  # Does it have a particular dependency?
> >  foo.dependencies.any? { |dep| ... }
> >  # Add new dependencies
> >  bar.dependencies << ....
> >  # Merge foo's dependencies into bar
> >  bar.dependencies |= foo.dependencies
> >
> >  One problem that comes up frequently are broken dependency lists, they
> may
> >  be missing a dependency, or reference the wrong version, or include one
> you
> >  don't want to use.  Mutable arrays would make it easy to fix these
> broken
> >  dependency lists, for example:
> >
> >  # Change the artifact in memory
> >  artifact( spec ).exclude( dont-want-this ).include( but want this )
> >  # And use it
> >  compile.with spec
> >
> >  It will be even better to introduce a Dependencies class that extends
> array
> >  and adds all sorts of convenience methods and lazy expansion, similar to
> >  Rake's FileList.  Then you could do things like:
> >
> >  # Find dependency by its specification
> >  foo.dependencies.has?( spec )
> >  # Exclude all dependencies that match a pattern
> >  foo.dependencies.exclude!( pattern )
> >  # Expand all transitive dependencies and return flat list
> >  foo.dependencies.expand
> >  # Replace one with another
> >  foo.dependencies.replace!( dont-want, want )
> >
> >  The API for methods like compile.with and compile.dependencies remain
> the
> >  same, just changing the underlying implementation.
> >
> >  Packages would acquire dependencies from the project, for example,
> >  package(:jar) would add all the dependencies from compile.dependencies,
> >  which of course you can fix to your heart's content.
> >
> >
> >  Version matching makes it possible to point at the most recent version
> that
> >  satisfies a particular version requirement, so >=1.2 to get 1.2.0 or
> later,
> >  or ~>1.2.3 to get any version between 1.2.3 and 1.3 (excluding).
> >
> >
> >  OSGi is a set of big specs that boil a lot of oceans, the part I'm
> >  particularly interested in, is managing dependencies.  When we started
> >  working on Buildr, OSGi was used almost exclusively by Eclipse.  That
> >  changed.  We're seeing more and more servers and frameworks adopting
> OSGi.
> >
> >  Imagine that we had OSGi support.  You will specify all the dependencies
> >  once in the buildfile.  Those will be used to build the project, and
> also
> >  incorporated into the package manifest.  Other projects could use these
> >  packages, reading the dependency list from the package manifest.  More
> >  important, when you get to deploy those packages, the runtime can use
> that
> >  information to bring in the right dependencies.
> >
> >  That's one of those things that, once you use it, you won't want to go
> back.
> >
> >
> >  The problem is, OSGi and Maven are incompatible with each other.  I'll
> try
> >  to summarize the differences:
> >
> >  1. The OSGi package identifier is typically the same as the file name.
>  The
> >  Maven package identifier combines group and artifact identifiers, only
> the
> >  later is used as the file name.
> >
> >  2.  Both use four-part version numbers.  The first three are numerical
> (e.g.
> >  1.2.3) and managed the same way, the fourth one is the qualifier.  OSGi
> >  qualifiers are alphanumeric and compared lexically, so R1 comes before
> R2;
> >  using number is not advised, 10 comes before 2.  Maven qualifiers are
> >  numeric for releases, and alphanumeric for pre-releases; the qualifier
> 10
> >  comes after 2, and 2 comes after R2.
> >
> >  3.  The rules for version matching are therefore not the same.  The same
> >  rule could match different versions, depending on whethr you use OSGi or
> >  Maven.
> >
> >  There's also the issue that OSGi keeps the meta-data inside the package,
> >  while Maven requires the additional POM file, and if both are present we
> >  have to decide which one to rely on.
> >
> >  It's possible to some extent to support both at the same time, but there
> >  will always be that impedence mismatch.  If we do a good enough job, it
> will
> >  probably only affect edge cases, but we need to be aware.
> >
> >  It will definitely be easier to pick one of the two, either OSGi or
> Maven,
> >  and add transitive support and version matching based on that model,
> then do
> >  translation when necessary (e.g. when reading POM file, generating OSGi
> >  manifest).
> >
> >  If I had to start all over again, I would go for OSGi with Maven
> >  compatibility layer for three simple reasons:
> >
> >  1. Runtime deployment
> >  2. Simpler to implement, and if your build doesn't quite work,
> troubleshoot
> >  and fix.
> >  3. Works the same way as most other packaging schemes (e.g. Gems,
> Debian,
> >  RPM).
> >
> >
> >  Comments, ideas?
> >
> >  Assaf
> >
>
>
>
> --
> -Shane
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message