aries-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillaume Nodet <gno...@gmail.com>
Subject Re: [DISCUSSION] Modifications to Aries release process
Date Fri, 06 Jul 2012 13:31:56 GMT
If the problem is to not destabilize trunk, one solution is to create a
branch and do all the release work in that branch.
That way, people are not impacted by the releases in progress.

On Fri, Jul 6, 2012 at 3:24 PM, Holly Cummins <
holly.k.cummins@googlemail.com> wrote:

> Re-opening a slightly old conversation, just to ensure we have a
> record of the advantages and disadvantages of each approach to
> releasing groups of bundles. (I'm also updating the 'Releasing Aries'
> web page with the pros and cons of each way of doing it.)
>
> On Thu, Jun 28, 2012 at 6:12 PM, Guillaume Nodet <gnodet@gmail.com> wrote:
> > I think you made a wrong assumption which is that the staging repo has
> > to be backed by a single svn revision (which is btw not really the
> > case because it comes from several tags anyway).
> > What I'd suggest is doing the api bundles release and have them
> > uploaded to nexus, then upgrade the implementations  and all other
> > components to use those releases, commit,  release those other
> > projects, THEN, close the staging repo and call for a vote.
> > So we'd have a single staging repo / vote with a set of testable bundles
> in it.
>
> This approach will break trunk unless we do an extra step.
>
> Say we release an API bundle, at version 1.0.0. The release plugin
> will change the version of the API bundle in trunk to be
> 1.0.1-SNAPSHOT. Normally, you'd just upgrade everything to depend on
> 1.0.1-SNAPSHOT of the API bundle (and if we weren't doing release by
> bundle, the release plugin would automatically update the dependencies
> for us). However, it's important that things which depend on the API
> bundle continue depending on 1.0.0, unless there's a good reason to
> change (like they use a new method). Otherwise, we lose a lot of the
> benefits of semantic versioning. Why? If we add a new method to the
> API, the package version changes to 1.1, and anything compiling
> against 1.0.1-SNAPSHOT won't resolve against 1.0.0, even if it would
> actually *work* fine against 1.0.0. In order to avoid restricting our
> modularity, we need to depend on 1.0.0 unless the 1.0.1-SNAPSHOT
> methods are actually being used.
>
> So, continuing the example, in trunk, as part of the release, we've
> switched to depending on 1.0.0 of the API. This compiles in the tagged
> release, as long as bundles are compiled in a specific order. In
> trunk, it won't compile at all, because nothing in trunk is building
> API-1.0.0, and version 1.0.0 isn't in a maven repository yet. Instead,
> trunk is building 1.0.1-SNAPSHOT. So to patch trunk up, we need to add
> an extra step in which we switch back to depending on 1.0.0-SNAPSHOT,
> and then a second extra step once the release is promoted, to depend
> on 1.0.0.
>
> I think this is sensible to do occasionally, but it's error prone and
> makes more work for the release manager. Obviously, the big advantage
> of it is that the release gets voted through over a period of a few
> days, rather than a month or two, for a big release. :)
>
> Holly
>
>
>
> > On Thu, Jun 28, 2012 at 6:45 PM, Holly Cummins
> > <holly.k.cummins@googlemail.com> wrote:
> >> Hi Guillaume,
> >>
> >> Thanks for your comments. Here are some of my thoughts ...
> >>
> >> On Thu, Jun 28, 2012 at 3:19 PM, Guillaume Nodet <gnodet@gmail.com>
> wrote:
> >>> I think one problem, considering the current votes, is that it's
> >>> really difficult to test anything.   Releasing api bundles with no
> >>> implementation to test is definitely not helping imo.
> >>
> >> I know what you mean about the testing, and I'm not totally sure what
> >> the best answer is. I know what I'm releasing comes from trunk, and is
> >> being tested by the Jenkins builds, so I'm pretty confident it works
> >> in a real system. However being tested in more environments and by
> >> more systems is obviously a Good Thing.  I think the best way to test
> >> the API bundles is with the current -SNAPSHOT bundles of the
> >> implementation, either in something like the blog sample or some other
> >> working system. If we weren't moving from 0.x to 1.0 you could also
> >> test micro releases alongside existing impl bundles to ensure
> >> everything resolves and works as claimed.
> >>
> >>> Holly, just a question: is there a specific reason why are you doing
> >>> the release in multiple votes ? It would be simpler to just release
> >>> everything in one go and wait for a longer time because there are more
> >>> things to check, or at least, release the api + implementation so that
> >>> we can actually try something.  Just my 2 cents.
> >>
> >> I agree that this sort of 'extended incremental' release is a bit
> >> awkward, and I was wondering when someone would ask what on earth I
> >> was doing :). IMO it's the cleanest way to work with with
> >> release-by-bundle (which I know you disagree with). If I release
> >> everything in one go, there's a problem with the dependencies between
> >> bundles. At the moment in trunk, almost every dependency is a SNAPSHOT
> >> dependency. In the past we've updated all bundles to use non-SNAPSHOT
> >> (but not yet released) versions in a branch, and I could even do
> >> something similar without using a branch by briefly having the trunk
> >> builds produce 1.0.0 artefacts. However, I think this creates a
> >> greater burden for testers. If there are compilation-order
> >> dependencies between parts of a release which don't share a top-level
> >> pom, everyone verifying a script has to compile them in the right
> >> order. I count 118 bundles to release, so that's a lot of bundles to
> >> get in the right order, and I didn't think any PMC member would want
> >> to try. :) I guess this could be automated with a verification script
> >> which either hardcodes or calculates the dependency graph, but it
> >> seemed to me like more work for everyone and more risk for the
> >> release. My hope was that if verifying individual mini-releases was
> >> easy enough, doing multiple ones wouldn't be a problem (and in fact
> >> would nicely distribute any effort, making it easier to vote).
> >>
> >> I know at this stage some of you are thinking "and *this* is why
> >> release by bundle is a bad idea!", and that's not really a debate I
> >> want to re-open. Among other things, I think any re-engineering of our
> >> poms at this stage will further delay the release.
> >>
> >> The good news is I believe this problem will almost entirely go away
> >> for 1.0.x and 1.x releases, because the impl bundle will, in most
> >> cases, depend on an *already* released version of its API bundle or
> >> another Aries component. This means a bunch of related bundles could
> >> be released at the same time, without compile issues, or a meaningful
> >> release really could consist of just a single bundle. That's true
> >> modularity and it should give both us and our users big benefits.
> >>
> >>
> >>>
> >>> On Mon, Jun 25, 2012 at 8:07 PM, Daniel Kulp <dkulp@apache.org> wrote:
> >>>>
> >>>> Honestly, with the change to using Nexus, the SHA1 and MD5 checks are
> >>>> completely pointless.   Nexus generates them itself based on what's
> >>>> uploaded.  The "is it a valid signature" part of the GPG testing is
> also
> >>>> pointless as Nexus won't let you close the repo unless the signatures
> are
> >>>> valid.   The only check you really need to do is to make sure the key
> that
> >>>> was used is "trusted" by you.   (aka: was it really Holly who
> deployed those
> >>>> artifacts)    So the monontonous parts of checking that stuff is
> really
> >>>> irrelevant at this point.  (providing we trust that infra has Nexus
> >>>> sufficiently locked down and secure)
> >>>>
> >>>>
> >>>> I actually don't have a big issue with the difficulting in getting
> votes.
> >>>> I'm involved in another community that has a PMC that is easily 4
> times the
> >>>> size of this one, yet we still have difficulting getting votes there.
> >>>> While not ideal, life events can cause priority shifts and such so
> people
> >>>> may not be able to be as responsive.
> >>>>
> >>>> My bigger problem is that the entire per bundle release process and
> symantic
> >>>> versioning crap has put a HUGE burden on the release manager.   That
> makes
> >>>> it much harder to get quality releases out and makes it less likely
> that
> >>>> anyone will step up to get "minor fixes" released.   The only reason
I
> >>>> stepped up with the 0.3.1 bp stuff is that *MY*  customers are being
> >>>> affected by it.   Like wise for the proxy stuff.   If *my* customers
> were
> >>>> not affected, I don't think I would have spent the time and effort.
>  If
> >>>> the process for getting fixes and releases out to users was smaller
> and
> >>>> easier, I have no problem doing them.   For CXF, we do full releases
> on 3
> >>>> branches every other month or so.   But that's because it's EASY to
> do.
> >>>>
> >>>> If it was up to me, I'd toss out the entire versioning thing with 1.0
> and go
> >>>> back to per module versioning thing.   So my fix to proxy would  have
> >>>> involved checking out all of "proxy", fixing it, and releasing all of
> proxy
> >>>> as a proxy "0.3.1", even the modules that haven't changed.   It's
> just a
> >>>> huge hassle to track down which bundles have changed, which haven't
> which
> >>>> version numbers need to be updated, etc....   If it's not quick and
> easy to
> >>>> do releases as a release manager, very few people are going to step
> up to do
> >>>> it.     It may not be 100% "proper OSGi", but IMO, getting fixes and
> such to
> >>>> the users is more important than that.    But that's my opinion.
> >>>>
> >>>>
> >>>> Dan
> >>>>
> >>>>
> >>>>
> >>>> On Saturday, June 23, 2012 03:27:07 PM Holly Cummins wrote:
> >>>>> Hi all,
> >>>>>
> >>>>> Now that Jeremy's taken the time to write up our release verification
> >>>>> process, I'd like to propose we change it. :) I think it's too
> onerous
> >>>>> on the pmc, which therefore also inhibits our ability to be
> responsive
> >>>>> to our users.
> >>>>>
> >>>>>
> >>>>> ------------------------------- Why what we have isn't working for
> the
> >>>>> community -------------------------------
> >>>>>
> >>>>> I believe our users would like more frequent releases. We've had
> >>>>> several keen requests and tweets and comments on the aries-user
> >>>>> mailing list wishing we'd release more often. For example:
> >>>>>
> >>>>> * "Desperately waiting for an Aries release after loooong time.."
> >>>>> * "The problem with Aries is they seem to be too busy coding to
> >>>>> release anything."
> >>>>> * "Compared to other projects (like Karaf and Camel) Aries releases
> >>>>> tend to take quite some time."
> >>>>> * "It's 2012 now and Aries 0.3 is almost a year old. Is there any
> >>>>> chance of a new Aries JPA release any time soon? "
> >>>>> * "Looks like Apache Aries has made no visible progress since Jan
> >>>>> 2011, if the time stamps on the maven central artefacts are to be
> >>>>> believed."
> >>>>>
> >>>>> ------------------------------- Why what we have isn't working for
us
> >>>>> -------------------------------
> >>>>>
> >>>>> Both Dan and I are trying to do releases at the moment, and
> struggling
> >>>>> to get enough PMC votes. Dan's release is to back port a show-stopper
> >>>>> proxy fix, so a release there is particularly pressing - he's got
a
> >>>>> non-binding +infinity vote, but that's all. My test support release
> >>>>> vote has been open for about 64 hours, and only got one vote so
far
> >>>>> (thanks David B!). Obviously testsupport is less exciting than proxy,
> >>>>> but that bundle does block more interesting releases.
> >>>>>
> >>>>> Why aren't people voting? My guess is that it's too much work to
do
> >>>>> the full set of verifications described at
> >>>>> http://aries.apache.org/development/verifyingrelease.html. There
are
> >>>>> seven steps, and while they don't actually take that long to
> complete,
> >>>>> it's enough of a burden that we tend to leave the voting to someone
> >>>>> else unless we really care about a release. I'm as guilty of this
as
> >>>>> anyone - I think a release is a good idea, but I'm spending enough
> >>>>> time working on the 1.0.0 release that I don't want to take time
out
> >>>>> to vote on another release. I suspect Dan might feel exactly the
same
> >>>>> about my 1.0.0 bundles. :)
> >>>>>
> >>>>> With release-by-bundle, there's a lot of verifications. Excluding
the
> >>>>> sandbox code, we have 123 bundles to release in 1.0.0. At three
votes
> >>>>> per bundle, that means the PMC need to do 369 MD5 checks, 369 PGP
> >>>>> checks, 369 RAT checks, and so on, just to get 1.0.0 out the door.
> >>>>> This just doesn't seem like it scales. Batching the bundle releases
> >>>>> together eases some of this burden, but not all.
> >>>>>
> >>>>> ------------------------------- What I propose
> >>>>> -------------------------------
> >>>>>
> >>>>> I suggest we move to a more trust-based system, where PMC members
> >>>>> carefully check releases if they want, but where in general they're
> >>>>> voting on the principle of the release, rather than the mechanics
of
> >>>>> the archives. In particular, they don't feel compelled to do checks
> >>>>> before voting. If PMC members could say "Our users need this
> function,
> >>>>> so +1", or "I know Holly has done sensible things in the past, so
+1"
> >>>>> or even "Do I want to check the SHAs on a test support bundle?
> Really?
> >>>>> +1" it would get our releases moving better, and also save work
for
> >>>>> all of us.
> >>>>>
> >>>>> (At the moment I think what's happening is people are thinking "Do
I
> >>>>> want to check the SHAs on a test support bundle? Really?" and then
> >>>>> skipping the +1 bit. :)  )
> >>>>>
> >>>>> To ensure that at least *someone* has run the checks, the release
> >>>>> manager could include the output of the seven checks in an email
to
> >>>>> the list. I think this level of checking is perfectly compatible
with
> >>>>> the minimum Apache process, which is that the release manager signs
> >>>>> the artefacts and three PMC members vote +1
> >>>>> (http://www.apache.org/dev/release-publishing.html#voted).
> >>>>>
> >>>>> What do people think?
> >>>>>
> >>>>> Holly
> >>>> --
> >>>> Daniel Kulp
> >>>> dkulp@apache.org - http://dankulp.com/blog
> >>>> Talend Community Coder - http://coders.talend.com
> >>>
> >>>
> >>>
> >>> --
> >>> ------------------------
> >>> Guillaume Nodet
> >>> ------------------------
> >>> Blog: http://gnodet.blogspot.com/
> >>> ------------------------
> >>> FuseSource, Integration everywhere
> >>> http://fusesource.com
> >
> >
> >
> > --
> > ------------------------
> > Guillaume Nodet
> > ------------------------
> > Blog: http://gnodet.blogspot.com/
> > ------------------------
> > FuseSource, Integration everywhere
> > http://fusesource.com
>



-- 
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
FuseSource, Integration everywhere
http://fusesource.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message