buildr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Assaf Arkin" <>
Subject Re: a strong suggestion
Date Wed, 04 Jun 2008 01:08:37 GMT
On Tue, Jun 3, 2008 at 12:50 PM, Shane Witbeck <> wrote:
> I'm in agreement about just keeping a closer watch on dependencies but
> are you confident about being able to keep up with the changing
> dependencies?

We don't have to.  If we have a tight dependency list, it doesn't
matter that new releases of these dependencies come out, and we can
release on our own schedule.  For example, if Rake 0.9 comes out
tomorrow, it won't affect Buildr 1.3.1, since it uses Rake 0.8.1 and
will keep doing so forever.

The problem only happens if we release with a broken dependency,
either relying on something that's broken, or not being specific
enough across all the transitive dependencies.

> For the initial issue with 1.3.0 it took 3-4 weeks to
> get the fixes in the 1.3.1 release. This is why I eluded to simply
> documenting the the issues (and workarounds) if the release schedules
> can't keep up with the dependency-related fixes.

The release process we have right is to only make official Apache
releases and mirror those on RubyForge, and it's too much of a process
for a one-line fix, like changing a broken version dependency.


> I'd be willing to help test against windows and leopard but as of
> right now I'm still stuck at 1.2.10 until the 1.3.x releases are more
> stable.
> Shane
> On Tue, Jun 3, 2008 at 2:44 PM, Assaf Arkin <> wrote:
>> On Sat, May 31, 2008 at 6:45 AM, Shane Witbeck <> wrote:
>>> Being a buildr user for almost a year now and having a vested interest
>>> in seeing Buildr continue to be successful, I'd like to make a strong
>>> suggestion.
>>> It seems that now, just after the 1.3.1 release, there's another
>>> dependency issue around Antwrap, hoe, and rubyforge. So yet again the
>>> standard way of installing is broken and will most likely discourage a
>>> lot of newcomers to Buildr knowing they have to constantly work around
>>> the dependency issues. I realize this is a caveat of a remote
>>> dependency system but I really think we need to do a better job of
>>> either documenting these issues (on the main Buildr site) or start
>>> assembling installs to insulate ourselves from the dependency-related
>>> woes. In my opinion, this should be the top priority above all other
>>> planned features.
>>> Has anyone else considered making the installs easier and less prone
>>> to these issues?
>>> --
>>> -Shane
>> Let's go over some of the problems we had, and available/proposed solutions.
>> First, installing Ruby itself.  Leopard users have their work cut out
>> for them, and Windows has a nice one-click installer, Linux is a bit
>> more tricky due to Debian packaging messing with Gems.  JRuby is
>> fairly trivial to install and works the same way on all operating
>> systems.
>> For 1.2 this was a bit of an issue, mostly learning the magic
>> incantation by trial.  For 1.3 we have very thorough installation
>> instructions, and three install scripts available for Linux, OSX and
>> JRuby.
>> Second, installing/upgrading Buildr (ignoring dependencies for a
>> moment).  Since this is an open source project, absolutely essential
>> that we make it easy to install from source.  I think we solved that
>> quite nicely using the rake install task.  You can also use that for
>> official releases, which are all tagged in SVN (and as I just found
>> out, we can also tag for Git).
>> It's as simple as:
>> svn co
>> cd buildr
>> rake install
>> Most people don't like installing from source (myself included), and
>> the gem command is a huge convenience over that (cleans up temporary
>> files, upgrades all your gems in one go, etc).  Again ignoring
>> dependency issues we ran ito, I think it's a good enough distribution
>> models.  Since Gems 1.1 came out earlier this year, it's been simple,
>> fast, and just works.
>> Third, are the dependencies.  Those are a pain to install
>> individually, so both gem install/upgrade and rake install handle
>> those for us.
>> However, we did end up with several annoyances.  First, were issues
>> with RubyForge itself (earlier this year), solved since.  It's always
>> annoying with other people's infrastructure messes with your
>> distribution, but they haven't been worse than anything else
>> (including Apache's own).
>> Second
>> RubyForge
>> RJB
>> Hoe
>> Before we propose solutions, I went and review the setup problems we
>> encountered going all the way back to 1.2.  If memory (and e-mail
>> archives) serves me right:
>> Installing Ruby. Ruby comes pre-installed on Leopard, there's a decent
>> one-click installer for Windows, but some Linux (*cough*debian*cough*)
>> that require several manual steps to get a working setup.
>> Using older versions of RubyGems. Anything pre-1.0 is slow, buggy, and
>> annoying by asking you to pick the right platform on each
>> multi-platform Gem. That does away once you upgrade to 1.0/1.1, which
>> unfortunately is not yet the default on most Ruby installations.
>> Broken Gem setup.  There are some ways in which you can break your Gem
>> environment, leading to unhelpful error messages.
>> RubyForge issues with their Gem index, since solved. While annoying to
>> have a release break by someone else's infrastructure, overall
>> RubyForge is working as well as the best infrastructure out there, I
>> don't think we can improve on that.
>> RJB is easy to install on Windows (DLL), but requires setting
>> JAVA_HOME and having build tools on other platforms.
>> Broken dependencies.
>> #1 and #2 we so far solved by writing a better installation guide, and
>> also providing easy to install scripts for Linux, OSX and JRuby
>> (anyone care to add one for Windows?)  It also attempts to deal with
>> #5.
>> The most recent slew of problems comes out of broken dependencies.
>> There are two of those, and let me explained why they happened.  The
>> first one was RJB doing a 1.1.3 release for all platforms except
>> Windows.  Didn't show up during pre-release testing because we don't
>> test Gem installs on all platforms.
>> The second was a new release of the Hoe and Rubyforge Gems while we
>> were waiting to green light our release.  Antwrap depends on any
>> version of Hoe, so it finds the most recent one, which depends on the
>> most recent version of Rubyforge Gem, but Buildr is tested and
>> released against an older version of Rubyforge.  So they try to load
>> two different versions of the same dependency.
>> The solution for this one was very simple, put the Rubyforge
>> dependency ahead of the Antwrap dependency, so Buildr's version takes
>> precedence.  It's an easy fix, the problem was not realizing it will
>> happen, not sure how it escaped me, but I should have noticed the Hoe
>> dependency while setting up the version dependencies for 1.3.1.
>> So two things we learned is: 1) we need more help testing releases on
>> Windows, and 2) pay more attention to dependencies and write a task to
>> verify those before making a release.
>> We currently have three(!) ways to install Buildr.  Because it's an
>> open source project, we care that people be able to install Buildr
>> from source, which is what the rake install task is far.  And that one
>> already has a fix for the dependency problem, but only if you're
>> installing from trunk/head, which doesn't guarantee a stable release.
>> (If you're installing from a release tag, you get the same broken
>> dependencies)
>> The other one is gem install/upgrade which is way more convenient than
>> installing from source, but has the problem that any release may be
>> broken.  The install scripts all use gem install/upgrade, so subject
>> to the same problem.
>> So the most obvious solution is to just fix the dependencies, pay more
>> attention in the future, and make a new release.  I personally don't
>> see a problem pushing a new RubyForge release ( to just fix
>> the dependency issue, without doing a parallel Apache release.
>> The other, we can sometimes work around this with the install scripts,
>> but that's really painful and you still get a broken Gem.  If the
>> install script reverts to an older version of Rubyforge, your next
>> attempt at gem upgrade will break Buildr again.  And the next release
>> we'll change the scripts and break older versions (not everyone
>> upgrades at once) again.
>> We can host our own Gem server containing just the dependencies we
>> tested Buildr against, and nothing gets pushed there until we tested
>> against it.  That won't work either, because once 1.3.2 goes out,
>> which depends on RubyForge 1.0 (in fact, trunk/head already does),
>> we'll break 1.3.1.  And the problem was not technical but human error,
>> so most likely we'll just shift the mistakes elsewhere.
>> Last, we can try a one-click installer for a self-contained version of
>> Buildr.  This one will include all the dependencies in the
>> distribution, so it doesn't depend on anything available (or not) in
>> Gem servers.  It can also combine all the niceties of the script.  For
>> Windows an executable, for Linux I guess we could use Debian
>> packages/RPMs.
>> One thing I looked at briefly was RubyScript2Exe, but I'm not sure how
>> it will work in practice:
>> Does anyone want to have a look at that?
>> Any other idea how we could get simple/slick installation?
>> Assaf
> --
> -Shane

CTO, Intalio

View raw message