rave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jasha Joachimsthal <j.joachimst...@onehippo.com>
Subject Re: Quality assurance steps for Rave
Date Fri, 13 Apr 2012 12:04:03 GMT
On 13 April 2012 13:52, Franklin, Matthew B. <mfranklin@mitre.org> wrote:

> >-----Original Message-----
> >From: Jasha Joachimsthal [mailto:j.joachimsthal@onehippo.com]
> >Sent: Friday, April 13, 2012 7:43 AM
> >To: dev@rave.apache.org
> >Subject: Re: Quality assurance steps for Rave
> >
> >Jasha Joachimsthal
> >
> >Europe - Amsterdam - Oosteinde 11, 1017 WT Amsterdam - +31(0)20 522 4466
> >US - Boston - 1 Broadway, Cambridge, MA 02142 - +1 877 414 4776 (toll
> free)
> >
> >www.onehippo.com
> >
> >
> >On 12 April 2012 03:59, Franklin, Matthew B. <mfranklin@mitre.org> wrote:
> >
> >> >-----Original Message-----
> >> >From: Marlon Pierce [mailto:marpierc@iu.edu]
> >> >Sent: Tuesday, April 10, 2012 12:01 PM
> >> >To: dev@rave.apache.org
> >> >Subject: Re: Quality assurance steps for Rave
> >> >
> >> >-----BEGIN PGP SIGNED MESSAGE-----
> >> >Hash: SHA1
> >> >
> >> >I started a checklist of tests at
> >> >http://wiki.apache.org/rave/ReleaseManagement/QualityAssurance, as
> >you
> >> >have seen from the [Rave Wiki] messages.
> >>
> >> Good set of tests.  I had been going through most of these on my own
> >> before each release, but that will become impossible to keep up with as
> the
> >> functionality grows.
> >>
> >> >
> >> >* Assuming the user interface tests are done manually, updating the
> wiki
> >> >every time would be cruddy. Is there good test/assurance process
> >> >management software out there?  Should we do this with an online
> >(Google)
> >> >spreadsheet?
> >>
> >> I say create a blocking Pre-release test task in Jira and have subtasks
> >> for each "section" of the tests. (we can use copy so it isn't such a
> manual
> >> task)
> >>
> >> >
> >> >* Do we want instead to actively maintain selenium tests?  I started
> this
> >> but
> >> >let it drop because these only worked in Firefox, and Firefox's
> frequent
> >> >updates broke the tests.  The tests themselves would have to be
> actively
> >> >maintained--if you change or add a user interface feature, you need to
> >> also
> >> >update the test.
> >>
> >> I think if we develop the simplest possible tests to protect against
> >> regression, it could work without too much effort.  We need to make it
> an
> >> explicit part of the process though.  It has also been my experience
> that
> >> recorded test cases don't work without some modification.  I have also
> had
> >> decent success in keeping code-based (not selenese) tests working....
> >>
> >> >
> >> >* Are there other recommendations beside Selenium?
> >>
> >
> >I've just created a test project [1] with JBehave [2] which uses Selenium
> >under the hood.
>
> +1 for JBehave
>
> >
> >Scenario: User creates a new account and logs in into the portal
> >When I go to "http://localhost:8080/portal"
> >Then I see the login page
> >When I follow the new account link
> >Then I get the new account form
> >When I fill in the form with username "newuser" password "password"
> >confirmpassword "password" email "newuser@example.com"
> >And I submit the new account form
> >Then I see the login page
> >And A message appears "Account successfully created"
> >When I fill in the login form with username "newuser" password "password"
> >Then I see my portal page with the add new widgets box
> >
> >
> >you can run it with mvn clean install or run the NewUserStories class from
> >your IDE.
>
> I would think we would want to do this from a different profile, right?
>

We need to think about the right setup of the integration tests. It would
be good if an experienced tester could help with that. I just wanted to
show an option how to test the portal without clicking through it manually.


>
> >It does not clean up the newly created user so you cannot run it twice
> >successfully without cleaning up the user.
> >
> >[1] https://github.com/jashaj/PortalTests
> >[2] http://jbehave.org/
> >
> > >
> >> >
> >> >Marlon
> >> >
> >> >
> >> >On 4/6/12 3:32 PM, Raminderjeet Singh wrote:
> >> >> +1 with Marlon on having a checklist of features. I know going
> forward
> >> this
> >> >list can grow and we may need more people to verify the build.
> >> >>
> >> >> Based on my experience doing 0.10 release, Matt and others have done
> >a
> >> >great job putting all the steps together into scripts. Thanks!
> >> >>
> >> >> After the code freeze announcement, Release manager can tag the
> >current
> >> >code and verification can be done on that particular tag. One developer
> >> >(release manager also) can verify the code tag for the feature list and
> >> if very
> >> >thing looks good can do the release based on the tag. Matt can comment
> >> >more as i have to still understand when the pom versions are updated.
> >> >>
> >> >> Thanks
> >> >> Raminder
> >> >>
> >> >> On Apr 6, 2012, at 2:53 PM, Marlon Pierce wrote:
> >> >>
> >> >> I'd like to propose the following.
> >> >>
> >> >> * Develop a list of Rave features that should be tested, put this on
> >> the wiki,
> >> >and update it every time a new feature is added.  I'm happy to get this
> >> >started.  Recommendations for tools to automate some of this are
> >welcome.
> >> >>
> >> >>
> >> >> * Regular testing of the above feature list (not just in preparation
> for
> >> >releases).  We would need a record of this (who tested, when, etc).  If
> >> we do
> >> >this manually, we could keep records by updating the wiki, posting to
> the
> >> dev
> >> >list, etc.  Recommendations for better tools are of course welcome.
> >> >>
> >> >>
> >> >> * All features should be tested before release by at least one
> person.
> >>  The
> >> >Release Manager should coordinate this. We could continue to do this as
> >> part
> >> >of the current Release Candidate process, but it may be better to have
> an
> >> >intermediate SVN tag that can go through QA before we start an official
> >> vote.
> >> >This would allow us to keep the trunk open for commits while we test
> and
> >> >avoid canceled releases.
> >> >>
> >> >>
> >> >> I'm not a QA expert, so comments welcome.
> >> >>
> >> >>
> >> >>
> >> >> Marlon
> >> >>
> >> >>
> >> >-----BEGIN PGP SIGNATURE-----
> >> >Version: GnuPG/MacGPG2 v2.0.16 (Darwin)
> >> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> >> >
> >>
> >>iQEcBAEBAgAGBQJPhFlJAAoJEEfVXEODPFIDPZ4H/2FM9G6b6mJuJgkong4cA
> >W
> >> >wg
> >>
> >>TMj8Vks5ykOsRspkqIff4Q8ItzkVrrq0hKb+1p2gaaGj8nG6/BhKnh+CaSsuwfhC
> >>
> >>QqeucJdj4/0RreCx+eQ9M0A17Y9O21yZxKuKQaiPHHWbv32NZWHsuycUPDP
> >+
> >> >MmvI
> >>
> >>mC5/w9+N2jQf2O7y6X0fR6ydfyggJrhBbM0PNY3yp6O7dHb91KqUz4Vm5giXl1
> >> >H/
> >>
> >>OzG0NEDv/EweIbTCeEJWOs2oDZGZgkRCT7z0LoJXXWNHifUSSJMuUDwPu0
> >M
> >> >5CIIJ
> >>
> >>tX16D/JdC2ik9FtarCHeQV101uEJYrC3Mx4F9Lb/PO0Hj+ePOBrha9rMRnSOcK0
> >=
> >> >=06BC
> >> >-----END PGP SIGNATURE-----
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message