lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Rowe <sar...@gmail.com>
Subject Re: [DISCUSS] Lucene/Solr release management
Date Wed, 01 Jun 2016 18:11:18 GMT

> On May 31, 2016, at 9:29 PM, Chris Hostetter <hossman_lucene@fucit.org> wrote:
> 
> if keeping track of what's already been done (and making it easier to make 
> notes on problematic bits as they happen) then it seems like just cloning 
> the "ReleaseTodo" page before the release, and adding a big <h1>**** HERE 
> ****</h1> that moves down the page as the RM steps through things, and 
> <font color=RED>NOTE: ...</font> to things above the HERE marker that had

> problems ... then when the release is done, folks can discuss/edit the 
> original ReleaseTodo page based on the red notes.
> 
> If that sounds hackish to folks i would agree with them -- but it 
> seems only slightly more hackish to me then tracking all of this in some 
> other todo list tool, or a giant spreedsheet, and wouldn't require porting 
> the existing docs into some other tool.

The main thing I was hoping to achieve was not requiring RMs to figure how to substitute the
right stuff into the example commands themselves, but rather have some software do it for
them.  The other stuff would be nice too, but are of lesser importance IMHO.  (In-situ notes
would be helpful for remembering what needs fixing; and a checklist would be helpful for the
interrupted RM: on some releases, some steps never got completed.)

I agree that copy/pasting from a display format into the command line is not all that helpful,
but that’s already the current situation.

> My alternative straw man suggestion would be to make imcremental 
> progress on simplifying the steps by writing small scripts to automated as 
> many of the steps as possible, and merge those scripts when/where 
> possible as we get more confident in them -- with the end goal being that 
> ASF jenkins could run the whole thing with a few simple paramaterized jobs 
> that are run on demand by RMs... the "create new major branch" job, 
> the "create new minor branch" job, and the "create RC" job.

+1 to increase automation.  

Many of the current manual tasks are individually fairly simple, so scripting them (individually
anyway) has seemed in the past to me like overkill.

I’m not sure we want Jenkins to do this stuff.

> I know i've suggested this before, and folks who have been RMs many times 
> have told me it's a bad idea to trust automation for all of this -- but we 
> should at least be able to automate *some* of it.  
> 
> Even if folks don't agree with the idea of letting scripts commit things 
> or pushing RCs to the dist repo, there's still very little reason i can 
> think of not to have scripts that *generate* & echo the exact commands 
> based on the type of build so it's trivial for the RM to run them 
> manually.

+1 to this step.  This is better than a spreadsheet (or similar) generating them, because
inspection is still possible prior to running, and running is simpler (type in the script
name vs. copy/paste a bunch of commands).

> if having a "checklist" is something RMs think would really be helpful -- 
> why not at least automate the creation / processing of the checklist as 
> much as possible with some simple scripts? 
> 
> Imagine having a simple JSON file in our repo listing all the steps and 
> some metadata about when/why they matter, along with a 
> release-checklist.py script...
> 
> release-checklist.py 6.1.0 RC0
>  - makes a copy of the checklist JSON in some file that is .gitignore'd
>  - updates the JSON to note that we're working on 6.1.0, RC0
>  - deletes any stuff from the JSON that's only applicable for an X.0.0 
>  - deletes any stuff that's only applicable when there are previous RCs
> release-checklist.py next
>  - echo's out the next thing the RM should do
>  - or, when possible, runs a script for the next step, which should echo
>    out what it did and how to check/test the results
>  - if the last item on the checklist isn't marked "done" do nothing,
>    instead remind the RM what the last step was so they can check it
>    (or re-run it manually if it was a script and they had to manually 
>    fix something)
> release-checklist.py done
>  - update the JSON to note the last step is "done"
> 
> 
> ...maybe initially the JSON file is just a bunch of links to individual 
> wiki pages/sub-section with info on what to run -- but gradually we can 
> convert those into little helper scripts that are run with the $version 
> and $rc_num passed in, so they can echo out the *exact* commands for the 
> user w/o risk of typos, (and maybe eventually even exec some of those 
> commands directly)
> 
> Some of these steps could even update the JSON checklist with additional 
> metadata for use by later steps -- ie: the step that runs 
> "buildAndPushRelease.py" could record the current SHA on the branch being 
> built, so that a later step which does the "git tag ..." (or echo's out 
> hte "git tag..." command for the RM to run manually) could include that 
> and save the RM from needing to to look it up and edit the command.

+1.  Some notes:

1. I think if used this will become the de facto source of RM documentation, so it will have
to be able to do a full listing of all steps, maybe also the steps that *aren’t* included
too for the given release type, so that the RM can see/verify the full context.

2. The JSON file shouldn’t be stored in the source tree; several steps look for/assure clean
checkouts.  Maybe a configurable location with a default uner /tmp/releases/X.Y.Z/ (already
used by at least one release script now).

3. Some things can be safely done out of order, while others have prerequisites.  Maybe the
script could somehow make these dependencies visible?  Skipping and out of order completion
(and progress for some manual multi-step things) should be supported as well.

--
Steve
www.lucidworks.com


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message