buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Witbeck" <sh...@digitalsanctum.com>
Subject Re: Transitive dependencies, version matching and OSGi support
Date Thu, 08 May 2008 01:19:28 GMT
I second the idea of going the way of OSGI. It's clearly the next step
in Java's evolution.

In terms of pre-1.3 dependency support, we resorted to doing something
like the following to support a subset of compile dependencies for
runtime packages:

project.runtime.with(specs)

Because some of our environments had shared runtime classpaths, it
wasn't always necessary to include certain libraries. I did this by
extending Buildr::Project to support the different kinds of internal
projects we have such as WebProject, StandaloneProject, BatchProject,
etc.

Something else that was needed in our projects was a way of putting
together different packages for different environments. Again, by
extending Buildr::Project, I introduced a way to package and deploy
(via SSH) the various projects.

I mention the ways that I've extended Buildr to possibly touch on new
features to incorporate such as:

1. Being able to define different sets of dependencies for projects
(in my case it was compile vs. runtime). I like the direction Assaf
mentions with being able to merge dependency arrays and better
transitive support. This is huge in terms of being able to make the
buildfile less verbose.
2. A way to easily deploy (via SSH, etc.) packages
3. Better integration with IDE's. It's a struggle to keep the
dependencies in sync without having to regenerate the IDE files and in
the process losing other project-specific IDE settings. Perhaps this
is better solved via IDE-specific plugins for Buildr?
4. In terms of additional language support, my vote is for Flex. With
the recent open sourced SDK, I believe there will be a surge in usage
among not only Java developers but developers of other languages as
well.

I'd be glad to elaborate further on the points I've brought up and
look forward to contributing more.

Thanks,
Shane


On Fri, May 2, 2008 at 5:41 PM, Assaf Arkin <arkin@intalio.com> wrote:
> Now that 1.3 is out, it's time to start talking about future releases.
>   We've got the 1.3.x line of releases, for dealing with outstanding issues,
>  new ones we'll find along the way, and building on new features that showed
>  up in 1.3.
>
>  Separately, there's 1.4, the next major release and a place to make more
>  significant changes.  We just have to decide what those would be.
>
>  I propose that we focus 1.4 on improving the way we handle dependencies.
>
>  Specifically what I have in mind are transitive dependencies, version
>  matching, and OSGi support.  So I'll start by talking about some things I've
>  been looking into recently.
>
>
>  For starters, adding a dependency method to artifact (which also includes
>  packages) that returns an array of all the known dependencies for that
>  artifacts.  Ruby arrays are pretty powerful, you can do things like:
>
>  # Does it have a particular dependency?
>  foo.dependencies.any? { |dep| ... }
>  # Add new dependencies
>  bar.dependencies << ....
>  # Merge foo's dependencies into bar
>  bar.dependencies |= foo.dependencies
>
>  One problem that comes up frequently are broken dependency lists, they may
>  be missing a dependency, or reference the wrong version, or include one you
>  don't want to use.  Mutable arrays would make it easy to fix these broken
>  dependency lists, for example:
>
>  # Change the artifact in memory
>  artifact( spec ).exclude( dont-want-this ).include( but want this )
>  # And use it
>  compile.with spec
>
>  It will be even better to introduce a Dependencies class that extends array
>  and adds all sorts of convenience methods and lazy expansion, similar to
>  Rake's FileList.  Then you could do things like:
>
>  # Find dependency by its specification
>  foo.dependencies.has?( spec )
>  # Exclude all dependencies that match a pattern
>  foo.dependencies.exclude!( pattern )
>  # Expand all transitive dependencies and return flat list
>  foo.dependencies.expand
>  # Replace one with another
>  foo.dependencies.replace!( dont-want, want )
>
>  The API for methods like compile.with and compile.dependencies remain the
>  same, just changing the underlying implementation.
>
>  Packages would acquire dependencies from the project, for example,
>  package(:jar) would add all the dependencies from compile.dependencies,
>  which of course you can fix to your heart's content.
>
>
>  Version matching makes it possible to point at the most recent version that
>  satisfies a particular version requirement, so >=1.2 to get 1.2.0 or later,
>  or ~>1.2.3 to get any version between 1.2.3 and 1.3 (excluding).
>
>
>  OSGi is a set of big specs that boil a lot of oceans, the part I'm
>  particularly interested in, is managing dependencies.  When we started
>  working on Buildr, OSGi was used almost exclusively by Eclipse.  That
>  changed.  We're seeing more and more servers and frameworks adopting OSGi.
>
>  Imagine that we had OSGi support.  You will specify all the dependencies
>  once in the buildfile.  Those will be used to build the project, and also
>  incorporated into the package manifest.  Other projects could use these
>  packages, reading the dependency list from the package manifest.  More
>  important, when you get to deploy those packages, the runtime can use that
>  information to bring in the right dependencies.
>
>  That's one of those things that, once you use it, you won't want to go back.
>
>
>  The problem is, OSGi and Maven are incompatible with each other.  I'll try
>  to summarize the differences:
>
>  1. The OSGi package identifier is typically the same as the file name.  The
>  Maven package identifier combines group and artifact identifiers, only the
>  later is used as the file name.
>
>  2.  Both use four-part version numbers.  The first three are numerical (e.g.
>  1.2.3) and managed the same way, the fourth one is the qualifier.  OSGi
>  qualifiers are alphanumeric and compared lexically, so R1 comes before R2;
>  using number is not advised, 10 comes before 2.  Maven qualifiers are
>  numeric for releases, and alphanumeric for pre-releases; the qualifier 10
>  comes after 2, and 2 comes after R2.
>
>  3.  The rules for version matching are therefore not the same.  The same
>  rule could match different versions, depending on whethr you use OSGi or
>  Maven.
>
>  There's also the issue that OSGi keeps the meta-data inside the package,
>  while Maven requires the additional POM file, and if both are present we
>  have to decide which one to rely on.
>
>  It's possible to some extent to support both at the same time, but there
>  will always be that impedence mismatch.  If we do a good enough job, it will
>  probably only affect edge cases, but we need to be aware.
>
>  It will definitely be easier to pick one of the two, either OSGi or Maven,
>  and add transitive support and version matching based on that model, then do
>  translation when necessary (e.g. when reading POM file, generating OSGi
>  manifest).
>
>  If I had to start all over again, I would go for OSGi with Maven
>  compatibility layer for three simple reasons:
>
>  1. Runtime deployment
>  2. Simpler to implement, and if your build doesn't quite work, troubleshoot
>  and fix.
>  3. Works the same way as most other packaging schemes (e.g. Gems, Debian,
>  RPM).
>
>
>  Comments, ideas?
>
>  Assaf
>



-- 
-Shane

Mime
View raw message