rave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Franklin, Matthew B." <mfrank...@mitre.org>
Subject RE: Quality assurance steps for Rave
Date Thu, 12 Apr 2012 01:59:56 GMT
>-----Original Message-----
>From: Marlon Pierce [mailto:marpierc@iu.edu]
>Sent: Tuesday, April 10, 2012 12:01 PM
>To: dev@rave.apache.org
>Subject: Re: Quality assurance steps for Rave
>
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I started a checklist of tests at
>http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
>have seen from the [Rave Wiki] messages.

Good set of tests.  I had been going through most of these on my own before each release,
but that will become impossible to keep up with as the functionality grows.

>
>* Assuming the user interface tests are done manually, updating the wiki
>every time would be cruddy. Is there good test/assurance process
>management software out there?  Should we do this with an online (Google)
>spreadsheet?

I say create a blocking Pre-release test task in Jira and have subtasks for each "section"
of the tests. (we can use copy so it isn't such a manual task)

>
>* Do we want instead to actively maintain selenium tests?  I started this but
>let it drop because these only worked in Firefox, and Firefox's frequent
>updates broke the tests.  The tests themselves would have to be actively
>maintained--if you change or add a user interface feature, you need to also
>update the test.

I think if we develop the simplest possible tests to protect against regression, it could
work without too much effort.  We need to make it an explicit part of the process though.
 It has also been my experience that recorded test cases don't work without some modification.
 I have also had decent success in keeping code-based (not selenese) tests working....

>
>* Are there other recommendations beside Selenium?
>
>
>Marlon
>
>
>On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>> +1 with Marlon on having a checklist of features. I know going forward this
>list can grow and we may need more people to verify the build.
>>
>> Based on my experience doing 0.10 release, Matt and others have done a
>great job putting all the steps together into scripts. Thanks!
>>
>> After the code freeze announcement, Release manager can tag the current
>code and verification can be done on that particular tag. One developer
>(release manager also) can verify the code tag for the feature list and if very
>thing looks good can do the release based on the tag. Matt can comment
>more as i have to still understand when the pom versions are updated.
>>
>> Thanks
>> Raminder
>>
>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>
>> I'd like to propose the following.
>>
>> * Develop a list of Rave features that should be tested, put this on the wiki,
>and update it every time a new feature is added.  I'm happy to get this
>started.  Recommendations for tools to automate some of this are welcome.
>>
>>
>> * Regular testing of the above feature list (not just in preparation for
>releases).  We would need a record of this (who tested, when, etc).  If we do
>this manually, we could keep records by updating the wiki, posting to the dev
>list, etc.  Recommendations for better tools are of course welcome.
>>
>>
>> * All features should be tested before release by at least one person.  The
>Release Manager should coordinate this. We could continue to do this as part
>of the current Release Candidate process, but it may be better to have an
>intermediate SVN tag that can go through QA before we start an official vote.
>This would allow us to keep the trunk open for commits while we test and
>avoid canceled releases.
>>
>>
>> I'm not a QA expert, so comments welcome.
>>
>>
>>
>> Marlon
>>
>>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
>Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cAW
>wg
>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP+
>MmvI
>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
>H/
>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0M
>5CIIJ
>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0=
>=06BC
>-----END PGP SIGNATURE-----

Mime
View raw message