uima-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marshall Schor <...@schor.com>
Subject Re: Feedback requested on build approach and tooling
Date Mon, 09 Aug 2010 19:56:31 GMT
 Thanks, Thilo, for your responses.  I have just a couple of small thing to put
inline below...

I encourage others to post their experience(s).


On 8/9/2010 2:58 PM, Thilo Götz wrote:
> Since I've been griping to Marshall about the build for the last few
> months (mostly off-list), I'll respond below.  Let me start by saying
> that I now regret very much that we decided a few years ago to switch
> our build to maven.  I agreed with that decision at the time.  However,
> after many days and weeks lost to maven magic, I now think that was the
> wrong thing to do.  A few weeks ago I strongly considered building an
> alternative, Ant based build system for those parts of UIMA that I'm
> interested in.  I didn't, mostly because I had other things to do. I
> still think it's a good idea.
> If anybody's interested in what I don't like about maven, there's a blog
> that says it a lot better than I ever could.  I agree with almost everything
> he says about maven, although maybe I wouldn't have put it quite so
> strongly :-)
> http://kent.spillner.org/blog/work/2009/11/14/java-build-tools.html
> More inline comments below.
> On 8/9/2010 18:03, Marshall Schor wrote:
>>  The current build tooling, now released and on maven central, seems to be working.
>> It improves many things:
>> *  We now can use the maven-release-plugin, which enables our use of the NEXUS
>> repository; this process is used by ~ 50 Apache projects, and we now can follow
>> the Apache conventional way of doing releases that produce Maven artifacts, see
>> http://www.apache.org/dev/publishing-maven-artifacts.html
>> * RAT (audit reports we need and used to run manually) are now automated, and
>> will fail the build if something's wrong here.
>> * Several deficiencies of our release process, such as some Jars not including
>> required License/Notice files, are now avoided, because we use a standard Apache
>> process for insuring these are included.
>> * We now use the standard Apache-wide parent-pom to insure we have releases
>> consistent with other Apache projects.
>> * We no longer have any inter-project dependencies on the SVN checkout working
>> directories, except for "aggregate" projects which build sets of projects, and
>> our xxx-distr projects, which distribute these.  We used to require checking out
>> the uimaj node to get tooling needed for building (docbooks, etc.), and then
>> check out the sandbox into a particular "parallel" working directory - this is
>> no longer required.
> That's the one improvement I really care about.
>> * The source releases didn't match SVN.  The standard Apache-wide parent-pom
>> builds these now, and they match.
>> * We separated our parent poms into ones providing inheritance-based factoring,
>> and ones providing aggregation. 
>> * We changed the way we use maven properties.  We previously were setting
>> various properties related to releases in a way which generated warnings from
>> Maven 3 (and m2eclipse, which embedds maven 3).  We now set versions in a way
>> that is much more consistent with the Maven "way".
>> * Many more things were automated.  These included deriving the OSGi versions of
>> "versions" which don't have a -SNAPSHOT but instead have a ".SNAPSHOT".
>> -----------------------
>> Several people have been very helpful in getting improvements to the build
>> documentation done.
>> The main barrier to using our build system at the moment seems to be the
>> one-time setup required, which has the following unusual things:
>> 1) you have to run Maven 3 beta-1.  Maven 2.2.1 (the last stable version of
>> maven that will do "releases" correctly with the new NEXUS release process) has
>> bugs in property resolutions, that affect our build, and that were fixed in
>> 3-beta-1.
> I'll submit that basing the build on beta software is not a good idea.  Maven 2
> was bad enough, thank you very much.  The build is our most basic
> infrastructure, second only to svn.  When it's broken, everything's broken.
> It's broken far too often anyway.  We're doing ourselves no favors by building
> with a beta version of maven.

I agree in general, but not in this specific case.  Maven 3 beta-1 has not
failed for me, and fixes solid failures that happen with maven 2.2.1.
>> 2) Maven 3 beta 1 fixed some errors that exposed a flaw in the docbkx plugin -
>> there's a one line fix to do there, plus you have to rebuild it (mvn install) to
>> get a working copy in your local maven repo.  The docbook builds might work
>> without this, but they will be slow, because the part that's broken is causing
>> the xml validation to go out to the internet to find DTDs (instead of finding
>> them in the local file system).
> I'd rather have a slow docbook build than this error prone procedure that will
> likely no longer work in a couple of weeks.  I don't build the docbooks unless
> I absolutely have to, because of the Saxon and other downloads that always
> required me to start the build 3 times, because it wouldn't continue after
> the downloads. 
That whole process was due to our private docbook build tool chain, and is now
> So a little more slowdown in the docbook build is perfectly
> acceptable to me.
>> Docbkx was chosen because it allowed us to stop using our own home-grown docbook
>> publishing tool chain, which in turn caused lots of SVN checkout issues (we used
>> to have to checkout multiple projects from multiple SVN places, into a
>> particular directory structure, in order to build).
>> 3) Use of m2eclipse is optional.  If you do use m2eclipse, you have to use a
>> particular version, and update that version.  This is to work around a bug in
>> m2eclipse which wasn't properly recognizing "generated-sources" which several of
>> our projects use, now.  m2eclipse in the version that fixes this, won't work
>> properly with Eclipse 3.6 (Helios) - at least as of the last time I checked, so
>> the one-time-setup advises using Eclipse 3.5.2.
> Use of m2eclipse is optional only if you don't want to use Eclipse.  That's
> not really a choice.  I haven't programmed Java in Emacs for about 10 years.
> BTW, last time m2eclipse crapped out on me, I tried using mvn eclipse:eclipse
> instead.  No luck.
> I do happen to use Helios, and I have yet to find a plugin that doesn't work
> with it.  Way to go, Maven.
>> The choice to use m2eclipse seems (nevertheless) good, because it is currently
>> getting the most "investment", including a new "book" on it, as compared to the
>> maven-eclipse-plugin (which we used to use), and it has a lot more functionality
>> in Eclipse. 
>> The new build system is documented on our web site here:
>> http://uima.apache.org/building-uima.html
>> It has instructions for building without Eclipse, just using command line
>> things, and also instructions for using Eclipse via m2eclipse.
> I followed those instruction 3 or 4 times, and once they seemed to work
> for about 5 minutes.  I have given up.  As I told Marshall privately,
> I'll wait for a few non-Maven experts like me to confirm that they can
> build, make and commit changes over a couple of days (on Linux, if possible)
> before I'll try again.
> Marshall, I realize that you put in an enormous amount of work here,
> and that some things are probably better than they were.  However, the
> basic build *must* work without complications.  The rest is important,
> too, but it's a secondary concern.
> If somebody opens an issue and I want to look at the source code
> and try some changes, I don't want to have to send queries back
> to the list and ask for help before I can even look at the source
> code in eclipse.  The Maven Eclipse plugin was annoying, but it
> was a know evil and worked most of the time.  And when it didn't
> you could hand-massage the classpath or whatever.  Still better
> than no Eclipse at all.
>> For those interested, there's more background information here:
>> http://uima.apache.org/maven-design.html, and a somewhat older document on our
>> wiki describing the motivations for improving the build system, here:
>> https://cwiki.apache.org/UIMA/redesign-for-more-standards-including-maven.html
>> The one-time-setups may simplify over time, as parts of the tooling we depend on
>> are upgraded with new releases.
>> Please post feedback on this build process - is this a good direction, and
>> whether or not you are able to do builds!
> Please do.  If I'm the only one with problems, I'll shut up and try to figure
> out why that is.
> --Thilo
>> -Marshall

View raw message