rave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marlon Pierce <marpi...@iu.edu>
Subject Re: Quality assurance steps for Rave
Date Fri, 13 Apr 2012 12:18:33 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Great, I will try it out--


Marlon


On 4/13/12 7:43 AM, Jasha Joachimsthal wrote:
> Jasha Joachimsthal
> 
> Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
> US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll free)
> 
> www.onehippo.com
> 
> 
> On 12 April 2012 03:59, Franklin, Matthew B. <mfranklin@mitre.org> wrote:
> 
>>> -----Original Message-----
>>> From: Marlon Pierce [mailto:marpierc@iu.edu]
>>> Sent: Tuesday, April 10, 2012 12:01 PM
>>> To: dev@rave.apache.org
>>> Subject: Re: Quality assurance steps for Rave
>>>
> I started a checklist of tests at
> http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as you
> have seen from the [Rave Wiki] messages.
>>>
>>> Good set of tests.  I had been going through most of these on my own
>>> before each release, but that will become impossible to keep up with as the
>>> functionality grows.
>>>
> 
> * Assuming the user interface tests are done manually, updating the wiki
> every time would be cruddy. Is there good test/assurance process
> management software out there?  Should we do this with an online (Google)
> spreadsheet?
>>>
>>> I say create a blocking Pre-release test task in Jira and have subtasks
>>> for each "section" of the tests. (we can use copy so it isn't such a manual
>>> task)
>>>
> 
> * Do we want instead to actively maintain selenium tests?  I started this
>>> but
> let it drop because these only worked in Firefox, and Firefox's frequent
> updates broke the tests.  The tests themselves would have to be actively
> maintained--if you change or add a user interface feature, you need to
>>> also
> update the test.
>>>
>>> I think if we develop the simplest possible tests to protect against
>>> regression, it could work without too much effort.  We need to make it an
>>> explicit part of the process though.  It has also been my experience that
>>> recorded test cases don't work without some modification.  I have also had
>>> decent success in keeping code-based (not selenese) tests working....
>>>
> 
> * Are there other recommendations beside Selenium?
>>>
> 
>> I've just created a test project [1] with JBehave [2] which uses Selenium
>> under the hood.
> 
>> Scenario: User creates a new account and logs in into the portal
>> When I go to "http://localhost:8080/portal"
>> Then I see the login page
>> When I follow the new account link
>> Then I get the new account form
>> When I fill in the form with username "newuser" password "password"
>> confirmpassword "password" email "newuser@example.com"
>> And I submit the new account form
>> Then I see the login page
>> And A message appears "Account successfully created"
>> When I fill in the login form with username "newuser" password "password"
>> Then I see my portal page with the add new widgets box
> 
> 
>> you can run it with mvn clean install or run the NewUserStories class from
>> your IDE.
>> It does not clean up the newly created user so you cannot run it twice
>> successfully without cleaning up the user.
> 
>> [1] https://github.com/jashaj/PortalTests
>> [2] http://jbehave.org/
> 
>>  >
> 
> Marlon
> 
> 
> On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
>>>>> +1 with Marlon on having a checklist of features. I know going forward
>>> this
> list can grow and we may need more people to verify the build.
>>>>>
>>>>> Based on my experience doing 0.10 release, Matt and others have done
a
> great job putting all the steps together into scripts. Thanks!
>>>>>
>>>>> After the code freeze announcement, Release manager can tag the current
> code and verification can be done on that particular tag. One developer
> (release manager also) can verify the code tag for the feature list and
>>> if very
> thing looks good can do the release based on the tag. Matt can comment
> more as i have to still understand when the pom versions are updated.
>>>>>
>>>>> Thanks
>>>>> Raminder
>>>>>
>>>>> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
>>>>>
>>>>> I'd like to propose the following.
>>>>>
>>>>> * Develop a list of Rave features that should be tested, put this on
>>> the wiki,
> and update it every time a new feature is added.  I'm happy to get this
> started.  Recommendations for tools to automate some of this are welcome.
>>>>>
>>>>>
>>>>> * Regular testing of the above feature list (not just in preparation
for
> releases).  We would need a record of this (who tested, when, etc).  If
>>> we do
> this manually, we could keep records by updating the wiki, posting to the
>>> dev
> list, etc.  Recommendations for better tools are of course welcome.
>>>>>
>>>>>
>>>>> * All features should be tested before release by at least one person.
>>>  The
> Release Manager should coordinate this. We could continue to do this as
>>> part
> of the current Release Candidate process, but it may be better to have an
> intermediate SVN tag that can go through QA before we start an official
>>> vote.
> This would allow us to keep the trunk open for commits while we test and
> avoid canceled releases.
>>>>>
>>>>>
>>>>> I'm not a QA expert, so comments welcome.
>>>>>
>>>>>
>>>>>
>>>>> Marlon
>>>>>
>>>>>
>>
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPiBmZAAoJEEfVXEODPFIDLmsIAJhErl0l1Ryyv5RJSCaa6vt9
VvlII/FpaRjYAlAr/NRnvUhgRrY78YSBjKsJqapTOLEbLDqkbBSozo1VPoESPX21
T1ztNQQlhn572PjLSlvhwWHFDWh/esNGZ1Zga8g2lTEo8kfNfPVqZVue3cLi5b1N
lxK+4aOqTR2L57w8xg4q5DmclxwwKR62RkVR8POW+kUF2efp5GyIbqCXy/dAmnrj
9uGU71gUexN5+cRueKvfUUa5FVXLEtJPnuLr9o8HJsLXDxokmqL+8VVZdoDoFbbR
b6RI7KUIfSZVA6gWMXuQrOwRn1r4D1ZZIHMHO+8ZIgW3Xauk2CSYpSPlESdwhGo=
=3M7b
-----END PGP SIGNATURE-----

Mime
View raw message