rave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Gooch <eodgo...@gmail.com>
Subject Re: Usability evaluation report
Date Fri, 14 Dec 2012 18:15:14 GMT
+1 danke schon

On Fri, Dec 14, 2012 at 5:50 AM, Matthias Niederhausen <
matthias.niederhausen@t-systems-mms.com> wrote:

> Hello Stan,
> sorry for getting back so late, but I was ensnared in yet another user
> study
> (with some different focus) in China for the last few weeks. Hopefully, I
> will be able to post some results we got as a "side-effect" of this new
> study soon.
> I will try to address the issues you mentioned, also considering our latest
> findings:
> 1) In the Omelette project, we have created several means to add widgets
> directly to a workspace using recommendations. While the quality of
> recommendations is a tricky topic, users like the ability to add new
> widgets
> without leaving their workspace. One tractable option for this might be a
> search field on the top of the page offering a direct result list so I can
> click a widget to add it to the workspace. I am inspired by the twitter
> search field here, which offers icons and additional data on the result
> (e.g., type of result) immediately.
> On the other hand, virtually all of our users did not expect the page
> layout
> to be restricted to some fixed number of columns (that has to be edited
> separately). Nearly all of them tried to instinctively drag&drop widgets
> like icons on a desktop or views in Eclipse and became frustrated when they
> could not do so. So, the default layout of a page should be something that
> dynamically allocates space (maybe using a dynamic number of columns).
> 2) I am very glad to see that you agree on the feedback usability issues.
> One particular idea that I have had is that upon opening a workspace, the
> widget frames should be overlayed by the big widget icon until they have
> loaded and some "loading" indicator. This way, users will immediately know
> that their browser has not crashed and where which widget is.
> 3) Thank you for your concerns regarding the formalties of the report. This
> was the first usability report I have ever created and I will be glad to
> improve the structure of future reports with your hints. I have now heard
> of
> Techsmith Morae for the first time and it looks very promising. In our
> studies, however, we have also used "discount" approaches (Google Docs
> questionaires, measuring times by stopwatch, recording screen, audio and
> camera using the tool XSplit).
> All the best,
> Matthias
> -----Urspr√ľngliche Nachricht-----
> Von: Drozdetski, Stan A. [mailto:drozdetski@mitre.org]
> Gesendet: Dienstag, 13. November 2012 23:15
> An: dev@rave.apache.org
> Cc: Matthias Niederhausen
> Betreff: RE: Usability evaluation report
> Matthias, thank you for sharing this with the community! Very helpful to
> see
> not just the summary of findings, but also your entire protocol.
> I'm picking out two major issue groups in need of improvement:
> 1) Navigation to/from widget store + workflow for adding gadgets In our
> internal testing, we've seen quite a wide variety of mental models that
> people approach gadget-based interfaces with. Even folks with experience in
> other portal systems may not instinctively know which parts of the page are
> modifiable, and how to adjust the page contents vs page layout. We've tried
> video tutorials and first-time-use hints, but can't quite claim a silver
> bullet just yet.
> Another challenge is the separation between the widget store and the actual
> layout; it simply puts a barrier in the way of direct interface
> manipulation. I've been thinking through some solutions that show the user
> what's available without taking them off the page, but screen real estate
> becomes an issue. If you or anyone else have seen innovative solutions,
> please do share with the list.
> 2) Feedback for actions
> Yes, no feedback (or feedback that the user can't see) is a usability death
> spell. Same goes for long loading time. Thank  you for noting that, and I
> agree with your recommendations 100%.
> Couple of unsolicited thoughts on the report itself - in case you're
> interested in what worked/didn't work for us at MITRE, or if there are
> other
> usability nerds on the list:
> - We find it very helpful to sort findings by severity (saving the per-task
> findings for the protocol section). Attaching recommendations, as you have
> done, is very helpful.
> - At least in the US, everyone is very sensitive about protecting the
> identity of participants. We try to take out any personally-identifiable
> information - even gender - out of all parts of the report, and refer to
> participants as P1, P2, and so on.
> - For studies with small to medium numbers of participants, we prefer to
> use
> ratios (3 out 10 users) in reporting, instead of percentages (30%). If
> anyone still has questions about sample sizes compared to traditional
> marketing approaches - well, there's an educational opportunity.
> Your notes and time-on-task measurements are very detailed - what recording
> methods do you use? We used to be very heavy into TechSmith Morae, but with
> the time demands of agile projects find ourselves moving more and more
> towards discount usability methods...
> Stan Drozdetski

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message