What is usability testing? How is it different
from - and similar to - other forms of user input, such as product
reviews or focus groups? What makes usability testing uniquely useful?
What must be specified before you can begin?
Usability testing is a method for assessing the ease with which
products are learned and used. The underlying model for virtually
all usability tests is that real users carry out real work with
a product. The "product" in this model can be a shrink-wrapped application
or a website, a working product or a prototype, or even a series
of screen designs mocked up on paper. The important concept is that
in usability testing, users are asked to do something realistic
with a product, and to do enough of it to approximate the experience
they would have with the real product in the real world. This is
a key difference between usability testing and other forms of user
input.
A second important characteristic of usability testing is that it
generates quantifiable data. For example, usability testing can
tell you how long it takes, on the average, for users to perform
typical tasks or reach particular usage goals with your product.
This measure, called Time on Task, is frequently used as a yardstick
for both the intuitiveness of a user interface and the efficiency
with which the product is used. Other types of quantitative measurements
capture how users evaluate the product subjectively, such as how
the product made them feel and whether they believed the product
was simple or complex. A third type of data comes from direct observations
of user behavior made by a trained usability research team during
testing. When catalogued, these observations provide data which
point to specific features of the user interface that made a task
take longer or a product be perceived as more difficult than others.
A third and very important characteristic of usability testing is
that it propels everyone on the product team to agree on what is
meant by "usability." Without this sort of definitional framework,
usability remains an opinion-based, ambiguous concept that is often
not defined or measured at all.
The starting point for planning a usability test
is a definition of the questions the study should answer. The second
step is to define the kinds of users who should participate in the
study and a set of usage tasks appropriate to both the users and
the questions under study. Next, it is necessary to agree on a set
of usability measures with good validity for these users, their
tasks, and the environment in which the product will be used.
The fourth and final planning step is the development of a detailed
study design which takes all of this information into account, along
with providing a way to deliver data-driven answers to specific
questions about the product. Because it can involve considerable
interchange with the client and up-front research into the capabilities
of the product, ergosoft undertakes detailed study design after
contract award.
In parallel with the development of the detailed study design and
other preparatory activities, study participants are recruited to
fit the user profile. Users meeting our client's criteria are recruited
through market research firms, contract professional employment
agencies, and direct postings to local media. An abundance of users
of all types exists in Austin. When testing begins, participants
are brought one at a time to ergosoft's fully-equipped usability
lab, where they carry out the test tasks and provide usability data.
After testing ends, data are analyzed (using statistical tests where
appropriate), videotapes of testing sessions are carefully reviewed,
and the research team is thoroughly debriefed. The very large amount
of information these activities produce is then studied by the ergosoft
research team and applied to answering the specific questions identified
before the study began.
Performance Data
Time on Task is routinely recorded during usability studies conducted
by ergosoft. This measure of user performance provides an
excellent yardstick for both the intuitiveness of a user interface
and the efficiency with which the product is used.
Subjective Impressions
After each task users are asked to rate the product on a variety
of attributes related to the his or her subjective experience of
usability. Examples include perceived effort, visual complexity,
and the perceived importance of the task. Differences among tasks
with regard to the various attributes are informative concerning
specific properties of the user interface that affect the user's
experience of usability. For example, on some tasks users will experience
difficulty due to the visual complexity of the display and on others
they will experience difficulty due to cognitive factors such as
high mental effort and memory. Another aspect of users' experience
is the perceived importance of the task being performed. Some types
of usability problems Ü for example, navigation difficulty Ü can
also make users feel that the task is unimportant. Ratings of impressions
like these suggest design recommendations for improving the user's
subjective experience of the product.
Behavioral Observations
Observations on users' immediate responses and comments are made
by ergosoft's interdisciplinary team of trained psychologists,
human factors engineers, and usability professionals during testing.
Like the other usability data ergosoft collects, this information
allows us to recommend design changes to improve usability.
Videotaping
All testing sessions are videotaped. This permits an examination
of user behavior in more detail if needed. Session videotapes may
also be used to create a highlight tape if desired. Highlight videotapes
graphically illustrate the problems users have with products and
so make highly useful visual aids for driving home the need for
change.
Results provide complete and detailed answers
to all of the questions identified as questions the study should
answer. In addition, ergosoft's usability testing methods routinely
provide the following information:
- A detailed list of usability
problems with the product under study.
- An overall data-based opinion
about the usability of the product or website.
- Recommendations for short-term,
relatively inexpensive user interface improvements that will
noticeably improve usability. These typically deal with language,
prompts, dialogs, some aspects of navigation, the functional
options provided, keyboard interaction techniques, screen layout,
use of color, audio feedback, visual feedback, the use of user
interface controls, and the addition or clarification of help
information, FAQs, "tool tips," and other forms of on-line assistance.
- Recommendations for longer-term,
more comprehensive improvements that would significantly differentiate
the usability of the product or website from others. These typically
deal with deep and pervasive changes to the information architecture
which may be required to make the website correspond more closely
with the user's mental model of the information space; this
correspondence will in turn dramatically simplify and improve
navigation.
|
|
ergosoft
laboratories ©2001-2003
|
|
|
