aries-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Nodet <gno...@gmail.com>
Subject Re: [DISCUSSION] Modifications to Aries release process
Date Fri, 06 Jul 2012 14:01:04 GMT
Git only has problems if you copy only a subtree (that's actually what is
done when tagging during the release process for a given module).  If you
branch the whole tree, it works perfectly with git and git-svn.  So I don't
think it would be worse on that side.

On Fri, Jul 6, 2012 at 3:51 PM, Holly Cummins <
holly.k.cummins@googlemail.com> wrote:

> Oh yes, I agree, that's the third approach, and it has a lot of
> advantages in terms of the stability of trunk. The disadvantage of
> that approach is that it doesn't seem to play well with the git
> mirroring. It also seems really wrong to me that we'd have to release
> mainline code from a branch, but that's just my personal feeling.
> There are advantages and disadvantages to each approach, and I don't
> think any of them are totally clean or totally simple, unfortunately.
>
> On Fri, Jul 6, 2012 at 2:31 PM, Guillaume Nodet <gnodet@gmail.com> wrote:
> > If the problem is to not destabilize trunk, one solution is to create a
> > branch and do all the release work in that branch.
> > That way, people are not impacted by the releases in progress.
> >
> > On Fri, Jul 6, 2012 at 3:24 PM, Holly Cummins <
> > holly.k.cummins@googlemail.com> wrote:
> >
> >> Re-opening a slightly old conversation, just to ensure we have a
> >> record of the advantages and disadvantages of each approach to
> >> releasing groups of bundles. (I'm also updating the 'Releasing Aries'
> >> web page with the pros and cons of each way of doing it.)
> >>
> >> On Thu, Jun 28, 2012 at 6:12 PM, Guillaume Nodet <gnodet@gmail.com>
> wrote:
> >> > I think you made a wrong assumption which is that the staging repo has
> >> > to be backed by a single svn revision (which is btw not really the
> >> > case because it comes from several tags anyway).
> >> > What I'd suggest is doing the api bundles release and have them
> >> > uploaded to nexus, then upgrade the implementations  and all other
> >> > components to use those releases, commit,  release those other
> >> > projects, THEN, close the staging repo and call for a vote.
> >> > So we'd have a single staging repo / vote with a set of testable
> bundles
> >> in it.
> >>
> >> This approach will break trunk unless we do an extra step.
> >>
> >> Say we release an API bundle, at version 1.0.0. The release plugin
> >> will change the version of the API bundle in trunk to be
> >> 1.0.1-SNAPSHOT. Normally, you'd just upgrade everything to depend on
> >> 1.0.1-SNAPSHOT of the API bundle (and if we weren't doing release by
> >> bundle, the release plugin would automatically update the dependencies
> >> for us). However, it's important that things which depend on the API
> >> bundle continue depending on 1.0.0, unless there's a good reason to
> >> change (like they use a new method). Otherwise, we lose a lot of the
> >> benefits of semantic versioning. Why? If we add a new method to the
> >> API, the package version changes to 1.1, and anything compiling
> >> against 1.0.1-SNAPSHOT won't resolve against 1.0.0, even if it would
> >> actually *work* fine against 1.0.0. In order to avoid restricting our
> >> modularity, we need to depend on 1.0.0 unless the 1.0.1-SNAPSHOT
> >> methods are actually being used.
> >>
> >> So, continuing the example, in trunk, as part of the release, we've
> >> switched to depending on 1.0.0 of the API. This compiles in the tagged
> >> release, as long as bundles are compiled in a specific order. In
> >> trunk, it won't compile at all, because nothing in trunk is building
> >> API-1.0.0, and version 1.0.0 isn't in a maven repository yet. Instead,
> >> trunk is building 1.0.1-SNAPSHOT. So to patch trunk up, we need to add
> >> an extra step in which we switch back to depending on 1.0.0-SNAPSHOT,
> >> and then a second extra step once the release is promoted, to depend
> >> on 1.0.0.
> >>
> >> I think this is sensible to do occasionally, but it's error prone and
> >> makes more work for the release manager. Obviously, the big advantage
> >> of it is that the release gets voted through over a period of a few
> >> days, rather than a month or two, for a big release. :)
> >>
> >> Holly
> >>
> >>
> >>
> >> > On Thu, Jun 28, 2012 at 6:45 PM, Holly Cummins
> >> > <holly.k.cummins@googlemail.com> wrote:
> >> >> Hi Guillaume,
> >> >>
> >> >> Thanks for your comments. Here are some of my thoughts ...
> >> >>
> >> >> On Thu, Jun 28, 2012 at 3:19 PM, Guillaume Nodet <gnodet@gmail.com>
> >> wrote:
> >> >>> I think one problem, considering the current votes, is that it's
> >> >>> really difficult to test anything.   Releasing api bundles with
no
> >> >>> implementation to test is definitely not helping imo.
> >> >>
> >> >> I know what you mean about the testing, and I'm not totally sure what
> >> >> the best answer is. I know what I'm releasing comes from trunk, and
> is
> >> >> being tested by the Jenkins builds, so I'm pretty confident it works
> >> >> in a real system. However being tested in more environments and by
> >> >> more systems is obviously a Good Thing.  I think the best way to test
> >> >> the API bundles is with the current -SNAPSHOT bundles of the
> >> >> implementation, either in something like the blog sample or some
> other
> >> >> working system. If we weren't moving from 0.x to 1.0 you could also
> >> >> test micro releases alongside existing impl bundles to ensure
> >> >> everything resolves and works as claimed.
> >> >>
> >> >>> Holly, just a question: is there a specific reason why are you
doing
> >> >>> the release in multiple votes ? It would be simpler to just release
> >> >>> everything in one go and wait for a longer time because there are
> more
> >> >>> things to check, or at least, release the api + implementation
so
> that
> >> >>> we can actually try something.  Just my 2 cents.
> >> >>
> >> >> I agree that this sort of 'extended incremental' release is a bit
> >> >> awkward, and I was wondering when someone would ask what on earth I
> >> >> was doing :). IMO it's the cleanest way to work with with
> >> >> release-by-bundle (which I know you disagree with). If I release
> >> >> everything in one go, there's a problem with the dependencies between
> >> >> bundles. At the moment in trunk, almost every dependency is a
> SNAPSHOT
> >> >> dependency. In the past we've updated all bundles to use non-SNAPSHOT
> >> >> (but not yet released) versions in a branch, and I could even do
> >> >> something similar without using a branch by briefly having the trunk
> >> >> builds produce 1.0.0 artefacts. However, I think this creates a
> >> >> greater burden for testers. If there are compilation-order
> >> >> dependencies between parts of a release which don't share a top-level
> >> >> pom, everyone verifying a script has to compile them in the right
> >> >> order. I count 118 bundles to release, so that's a lot of bundles to
> >> >> get in the right order, and I didn't think any PMC member would want
> >> >> to try. :) I guess this could be automated with a verification script
> >> >> which either hardcodes or calculates the dependency graph, but it
> >> >> seemed to me like more work for everyone and more risk for the
> >> >> release. My hope was that if verifying individual mini-releases was
> >> >> easy enough, doing multiple ones wouldn't be a problem (and in fact
> >> >> would nicely distribute any effort, making it easier to vote).
> >> >>
> >> >> I know at this stage some of you are thinking "and *this* is why
> >> >> release by bundle is a bad idea!", and that's not really a debate I
> >> >> want to re-open. Among other things, I think any re-engineering of
> our
> >> >> poms at this stage will further delay the release.
> >> >>
> >> >> The good news is I believe this problem will almost entirely go away
> >> >> for 1.0.x and 1.x releases, because the impl bundle will, in most
> >> >> cases, depend on an *already* released version of its API bundle or
> >> >> another Aries component. This means a bunch of related bundles could
> >> >> be released at the same time, without compile issues, or a meaningful
> >> >> release really could consist of just a single bundle. That's true
> >> >> modularity and it should give both us and our users big benefits.
> >> >>
> >> >>
> >> >>>
> >> >>> On Mon, Jun 25, 2012 at 8:07 PM, Daniel Kulp <dkulp@apache.org>
> wrote:
> >> >>>>
> >> >>>> Honestly, with the change to using Nexus, the SHA1 and MD5
checks
> are
> >> >>>> completely pointless.   Nexus generates them itself based on
what's
> >> >>>> uploaded.  The "is it a valid signature" part of the GPG testing
is
> >> also
> >> >>>> pointless as Nexus won't let you close the repo unless the
> signatures
> >> are
> >> >>>> valid.   The only check you really need to do is to make sure
the
> key
> >> that
> >> >>>> was used is "trusted" by you.   (aka: was it really Holly who
> >> deployed those
> >> >>>> artifacts)    So the monontonous parts of checking that stuff
is
> >> really
> >> >>>> irrelevant at this point.  (providing we trust that infra has
Nexus
> >> >>>> sufficiently locked down and secure)
> >> >>>>
> >> >>>>
> >> >>>> I actually don't have a big issue with the difficulting in
getting
> >> votes.
> >> >>>> I'm involved in another community that has a PMC that is easily
4
> >> times the
> >> >>>> size of this one, yet we still have difficulting getting votes
> there.
> >> >>>> While not ideal, life events can cause priority shifts and
such so
> >> people
> >> >>>> may not be able to be as responsive.
> >> >>>>
> >> >>>> My bigger problem is that the entire per bundle release process
and
> >> symantic
> >> >>>> versioning crap has put a HUGE burden on the release manager.
> That
> >> makes
> >> >>>> it much harder to get quality releases out and makes it less
likely
> >> that
> >> >>>> anyone will step up to get "minor fixes" released.   The only
> reason I
> >> >>>> stepped up with the 0.3.1 bp stuff is that *MY*  customers
are
> being
> >> >>>> affected by it.   Like wise for the proxy stuff.   If *my*
> customers
> >> were
> >> >>>> not affected, I don't think I would have spent the time and
effort.
> >>  If
> >> >>>> the process for getting fixes and releases out to users was
smaller
> >> and
> >> >>>> easier, I have no problem doing them.   For CXF, we do full
> releases
> >> on 3
> >> >>>> branches every other month or so.   But that's because it's
EASY to
> >> do.
> >> >>>>
> >> >>>> If it was up to me, I'd toss out the entire versioning thing
with
> 1.0
> >> and go
> >> >>>> back to per module versioning thing.   So my fix to proxy would
>  have
> >> >>>> involved checking out all of "proxy", fixing it, and releasing
all
> of
> >> proxy
> >> >>>> as a proxy "0.3.1", even the modules that haven't changed.
  It's
> >> just a
> >> >>>> huge hassle to track down which bundles have changed, which
haven't
> >> which
> >> >>>> version numbers need to be updated, etc....   If it's not quick
and
> >> easy to
> >> >>>> do releases as a release manager, very few people are going
to step
> >> up to do
> >> >>>> it.     It may not be 100% "proper OSGi", but IMO, getting
fixes
> and
> >> such to
> >> >>>> the users is more important than that.    But that's my opinion.
> >> >>>>
> >> >>>>
> >> >>>> Dan
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> On Saturday, June 23, 2012 03:27:07 PM Holly Cummins wrote:
> >> >>>>> Hi all,
> >> >>>>>
> >> >>>>> Now that Jeremy's taken the time to write up our release
> verification
> >> >>>>> process, I'd like to propose we change it. :) I think it's
too
> >> onerous
> >> >>>>> on the pmc, which therefore also inhibits our ability to
be
> >> responsive
> >> >>>>> to our users.
> >> >>>>>
> >> >>>>>
> >> >>>>> ------------------------------- Why what we have isn't
working for
> >> the
> >> >>>>> community -------------------------------
> >> >>>>>
> >> >>>>> I believe our users would like more frequent releases.
We've had
> >> >>>>> several keen requests and tweets and comments on the aries-user
> >> >>>>> mailing list wishing we'd release more often. For example:
> >> >>>>>
> >> >>>>> * "Desperately waiting for an Aries release after loooong
time.."
> >> >>>>> * "The problem with Aries is they seem to be too busy coding
to
> >> >>>>> release anything."
> >> >>>>> * "Compared to other projects (like Karaf and Camel) Aries
> releases
> >> >>>>> tend to take quite some time."
> >> >>>>> * "It's 2012 now and Aries 0.3 is almost a year old. Is
there any
> >> >>>>> chance of a new Aries JPA release any time soon? "
> >> >>>>> * "Looks like Apache Aries has made no visible progress
since Jan
> >> >>>>> 2011, if the time stamps on the maven central artefacts
are to be
> >> >>>>> believed."
> >> >>>>>
> >> >>>>> ------------------------------- Why what we have isn't
working
> for us
> >> >>>>> -------------------------------
> >> >>>>>
> >> >>>>> Both Dan and I are trying to do releases at the moment,
and
> >> struggling
> >> >>>>> to get enough PMC votes. Dan's release is to back port
a
> show-stopper
> >> >>>>> proxy fix, so a release there is particularly pressing
- he's got
> a
> >> >>>>> non-binding +infinity vote, but that's all. My test support
> release
> >> >>>>> vote has been open for about 64 hours, and only got one
vote so
> far
> >> >>>>> (thanks David B!). Obviously testsupport is less exciting
than
> proxy,
> >> >>>>> but that bundle does block more interesting releases.
> >> >>>>>
> >> >>>>> Why aren't people voting? My guess is that it's too much
work to
> do
> >> >>>>> the full set of verifications described at
> >> >>>>> http://aries.apache.org/development/verifyingrelease.html.
There
> are
> >> >>>>> seven steps, and while they don't actually take that long
to
> >> complete,
> >> >>>>> it's enough of a burden that we tend to leave the voting
to
> someone
> >> >>>>> else unless we really care about a release. I'm as guilty
of this
> as
> >> >>>>> anyone - I think a release is a good idea, but I'm spending
enough
> >> >>>>> time working on the 1.0.0 release that I don't want to
take time
> out
> >> >>>>> to vote on another release. I suspect Dan might feel exactly
the
> same
> >> >>>>> about my 1.0.0 bundles. :)
> >> >>>>>
> >> >>>>> With release-by-bundle, there's a lot of verifications.
Excluding
> the
> >> >>>>> sandbox code, we have 123 bundles to release in 1.0.0.
At three
> votes
> >> >>>>> per bundle, that means the PMC need to do 369 MD5 checks,
369 PGP
> >> >>>>> checks, 369 RAT checks, and so on, just to get 1.0.0 out
the door.
> >> >>>>> This just doesn't seem like it scales. Batching the bundle
> releases
> >> >>>>> together eases some of this burden, but not all.
> >> >>>>>
> >> >>>>> ------------------------------- What I propose
> >> >>>>> -------------------------------
> >> >>>>>
> >> >>>>> I suggest we move to a more trust-based system, where PMC
members
> >> >>>>> carefully check releases if they want, but where in general
> they're
> >> >>>>> voting on the principle of the release, rather than the
mechanics
> of
> >> >>>>> the archives. In particular, they don't feel compelled
to do
> checks
> >> >>>>> before voting. If PMC members could say "Our users need
this
> >> function,
> >> >>>>> so +1", or "I know Holly has done sensible things in the
past, so
> +1"
> >> >>>>> or even "Do I want to check the SHAs on a test support
bundle?
> >> Really?
> >> >>>>> +1" it would get our releases moving better, and also save
work
> for
> >> >>>>> all of us.
> >> >>>>>
> >> >>>>> (At the moment I think what's happening is people are thinking
> "Do I
> >> >>>>> want to check the SHAs on a test support bundle? Really?"
and then
> >> >>>>> skipping the +1 bit. :)  )
> >> >>>>>
> >> >>>>> To ensure that at least *someone* has run the checks, the
release
> >> >>>>> manager could include the output of the seven checks in
an email
> to
> >> >>>>> the list. I think this level of checking is perfectly compatible
> with
> >> >>>>> the minimum Apache process, which is that the release manager
> signs
> >> >>>>> the artefacts and three PMC members vote +1
> >> >>>>> (http://www.apache.org/dev/release-publishing.html#voted).
> >> >>>>>
> >> >>>>> What do people think?
> >> >>>>>
> >> >>>>> Holly
> >> >>>> --
> >> >>>> Daniel Kulp
> >> >>>> dkulp@apache.org - http://dankulp.com/blog
> >> >>>> Talend Community Coder - http://coders.talend.com
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> ------------------------
> >> >>> Guillaume Nodet
> >> >>> ------------------------
> >> >>> Blog: http://gnodet.blogspot.com/
> >> >>> ------------------------
> >> >>> FuseSource, Integration everywhere
> >> >>> http://fusesource.com
> >> >
> >> >
> >> >
> >> > --
> >> > ------------------------
> >> > Guillaume Nodet
> >> > ------------------------
> >> > Blog: http://gnodet.blogspot.com/
> >> > ------------------------
> >> > FuseSource, Integration everywhere
> >> > http://fusesource.com
> >>
> >
> >
> >
> > --
> > ------------------------
> > Guillaume Nodet
> > ------------------------
> > Blog: http://gnodet.blogspot.com/
> > ------------------------
> > FuseSource, Integration everywhere
> > http://fusesource.com
>



-- 
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
FuseSource, Integration everywhere
http://fusesource.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message