IMAT5209 Human Interaction in Systems Design – Assignment
IMAT5209 Human Interaction in Systems Design
Assignment
Usability Evaluation of an
Interactive System
Deadline: 12:30 Friday 7 May 2020
Learning Outcomes
This assignment is designed to provide practical experience of
analysing usability requirements, and carrying out an analysis of
usability requirements and priorities, performing a systematic
usability evaluation using a standard method, and producing a
report of your fndings.
Submission and Marking Procedure
Part A: Individual/Group assignments. It is worth 25% of the total
mark for the module.
Part B: This is an individual assignment. It is worth 75% of the
total mark for the module.
Your report should be submitted electronically as a Word document
via the Turnitin link for Assignment Two under Assessment on
Blackboard, and should be submitted in hardcopy form at FOTAC.
The submission deadline is 12:30 on Friday 7 May 2020.
Task
Your frm of interaction design consultants is trying to build up a
portfolio of impressive work, to enable it to pitch for business
convincingly in the future.
Your task is to produce a set of usability requirements and a
usability evaluation of an interactive system, plus a report of
your results, by applying a systematic evaluation methodology. You
have a completely free choice of what interactive system you
evaluate.
IMAT5209 Human Interaction in Systems Design – Assignment
Carrying out the Assignment
Producing the usability evaluation assignment will involve
1. Choosing an interactive system to study.
2. Identifying the use cases or aspects of the functioning of the
system to be considered, and briefly describing them in your
documentation. (These don’t need to be a complete set of use
cases; for very complicated systems focusing on one part of
what they do is just fne. However you should give a clear
indication of what subset of the functionality of the system
you are considering, and what you are not considering. If in
doubt, cover less functionality in more detail.)
3. Defne a set of usability requirements, considering what the
design really needs to get right to achieve a good user
experience, and defning the requirements precisely enough
that it would be possible to measure the system’s
performance.
4. Choosing an evaluation methodology. You should apply a
standard evaluation methodology such as user testing,
cognitive walkthrough, or heuristic evaluation.
5. Defning an evaluation procedure. This will include stating one
or several user tasks to be tested or considered with exact
descriptions of the scenario and the goal the user is trying to
achieve, as well as what the evaluator will do to collect results
and produce an evaluation. The evaluation procedure needs to
be described in full, separately from the description of the
results.
6. Carrying out the evaluation. This will involve applying the
procedure and documenting what happens, and what the
procedure fnds. (If applying your procedure looks like an
excessive amount of work, or producing an excessively large
volume of documentation, ask advice; we would prefer an
evaluation giving detailed insight into part of the functionality
to an evaluation with broad coverage but a thinner or more
superfcial analysis.)
7. Deriving fndings about the usability of the interactive system
from the results of the usability evaluation. This should include
consideration of how strong and how general the conclusions
are.
IMAT5209 Human Interaction in Systems Design – Assignment
Written Submission
Your report should comprise the following elements:
Part ONE: The interactive system and its users. A brief
statement of what the interactive system is and what it does –
sufcient to make the rest of the report comprehensible; plus
a description of the user populations and the assumptions it is
reasonable to make about the capabilities of the users. The
word count should be between 150 and 400 words – longer
only if really needed.
Part TWO: The use cases. Brief accounts of the use cases
considered, plus a statement of what you are not considering,
if you are only looking at part of the system. A use case
diagram is optional. The word count should be between 100
and 300 words.
Part THREE: The usability requirements. Brief but exact
statements of fve precisely focused, testable usability
requirements. The word count should be between 300 and
600 words.
Part FOUR: The evaluation methodology. An exact
description of the evaluation procedure to be followed,
including what the methodology you are using, exact
descriptions of user tasks being considered, instructions to be
given to users in user testing, or the set of guidelines used in
heuristic evaluation. In principle, you need step by step
descriptions of the correct procedures for completing the use
cases you are considering in a heuristic evaluation or
cognitive walkthrough, or asking people to perform in a user
trial. However if the tasks are long and complicated with a
variety of possible paths, something briefer might be needed;
consider what is cost-effective, and ask advice if in doubt. The
word count should be between 300 and 800 words, plus
documentation of instructions or task steps if included.
Part FIVE: The evaluation. The results of applying the
evaluation procedure: what you saw test subjects doing,
measurements of their performance, answers to questions and
so on; or evidence for violation of particular design guidelines;
or descriptions of how and why beginning users might go
wrong in particular places, etc. The word count should be
between 1000 and 2500 words.
Part SIX: The fndings of the evaluation. The fndings of
your evaluation about the usability of the interactive system.
Include comments on how the fndings relate to the results of
the evaluation procedure, and ideally about how strong the
evidence is, as well as judgements of how serious you think
the usability problems are. An itemized bullet point structure
is likely to be easier to read than long paragraphs of text. This
should also include an appraisal of the strengths and
IMAT5209 Human Interaction in Systems Design – Assignment
weaknesses and successes and failures of the evaluation
process. The word count should be between 500 and 1000
words.
Appendix. Your notes made during observations of user trials,
while conducting a heuristic evaluation, etc, should be
included in an appendix. Handwritten notes should be
scanned or photocopied. Acknowledgements of any help
received, any information we should have when assessing the
assignment, etc.
Guidance
Some advice on how to do the assignment.
Writing
DO NOT write more than you need to. Do not bother with
unnecessary introductions or generalities about usability or human
computer interaction. This is just unwelcome extra work for both
you and your tutor. Brief means brief. Terse is good. However you do
need to be detailed and exact about your procedure and your
results and fndings. What Part One needs will depend on the choice
of system, but please take seriously thinking about the assumptions
you can make about the capabilities of the users. Include word
counts for sections.
The Usability Requirements
This section of the assignment is essentially separate from the
usability evaluation. You DO NOT need to test your usability
requirements. It’s good if you do (and you need to think about how
you could), but (1) you are welcome to use a usability evaluation
method that doesn’t address them, and (2) we don’t want you to
limit your imagination to requirements that you can easily test in
your assignment.
Your task is to create a set of fve usability requirements
covering different aspects of the human computer interaction for
your chosen interactive system. The set of usability requirements
should include at least one with a performance measure, and at
least one with a preference measure. You need to consider which
subset of your user population each requirement applies to;
including one or two for disabled users is also good if they are likely
users of your chosen system.
A usability requirement should include
Statement of the usability attribute
IMAT5209 Human Interaction in Systems Design – Assignment
Statement of how it will be measured
Statement of the criteria that will represent attainment of the
specifcation
Statement of the subset of users to which the specifcation
applies
Statement of the pre-conditions of measurement (e.g period of
training)
The criteria of attainment should include
The worst case: The lowest acceptable level of performance
The planned case: The target level of performance
The best case: The highest level of performance that can be
hoped for in favourable conditions
The now level: The level of performance achieved by the
current system to be replaced, if applicable.
The usability attributes to be evaluated should be precisely focused
on particular user tasks and be clearly and narrowly defned aspects
of the usability of the system for the task. (Look at the handout on
Some Usability Factors for hints.) These ought to be a wide variety,
to demonstrate your ability to consider different aspects of usability.
The measures should be clearly defned so the reader can see
exactly will be counted or measured, and should be feasible; you will
get credit for well-chosen valid measures. The subset of users to be
covered by the usability requirements needs to be clearly stated
and should be varied, and the preconditions for making the
measurement should be clear. The criteria of attainment should
ideally be sensible.
Each individual usability requirement will be judged on having
A usability attribute that is precise and task-focused, and
important for the success of the system. Going beyond the
most obvious options will be appreciated.
A measurement that is sufciently precisely defned, is
feasible, and is likely to be valid.
Clear and sensible statements of the subset of users and the
preconditions of measurement, that are restrictive enough to
make the measurement meaningful.
Statements of attainment criteria that are clear and ideally
not crazy.
The set of usability requirements will be judged on having
A broad and well-chosen coverage of different aspects of
usability.
Both performance and preference measures.
Requirements for potential test subjects who represent a
sensible range of users.
IMAT5209 Human Interaction in Systems Design – Assignment
The Usability Evaluation
This does NOT NEED to have anything to do with your usability
requirements.
You are expected to apply a systematic evaluation method.
That is, you need to do a user trial, or a heuristic evaluation, or a
cognitive walkthrough. (Make sure you know what a heuristic
evaluation or a cognitive walkthrough actually is, before claiming to
be doing one.) If you want to use a different approach to doing a
usability evaluation, ask advice.
We are looking for thorough and detailed evaluations, and especially
fndings about exactly where there are actual or potential usability
problems. If you think you are doing a disproportionately amount of
work, or writing an enormous amount, then you should aim to be
thorough and detailed, and compromise on how much of the system
you cover.
If you do a user trial, you should aim to observe carefully what your
subjects do and where they make mistakes or fnd things confusing
and awkward, and report sources of problems as exactly as possible.
Timings for tasks and subjective satisfaction ratings in debriefng are
valuable but less interesting than actual usability problems.
Remember that having carefully designed, realistic tasks is
important, and that unless you want to look at exploration or
browsing the task should have clear end points and success criteria.
The exact wording of instructions matters. So these need to be
described exactly in your documentation. Don’t be over-directive:
provide clear goals and enough information about the scenario, but
don’t tell people what to do.
Heuristic evaluations are likely to be more successful when using
more detailed and concrete sets of guidelines than just Nielsen’s ten
broad categories of usability problems. We recommend Nielsen’s
113 design guidelines for web homepages for doing heuristic
evaluations of websites.
For cognitive walkthroughs, you need to describe the procedure
including the questions to be considered at each step, and show
evidence that the questions have been systematically used in the
evaluation.
This is an individual assignment, and we are expecting you to work
alone (apart from test subjects if you do a user trial). However some
activities might be done better with more than one person doing
them. You may recruit assistance, but if you have help, you need to
describe this clearly in your report.
IMAT5209 Human Interaction in Systems Design – Assignment
Choice of Interactive System
The assignment gives you a completely free choice of what
interactive system you consider; but it needs to be a real, existing
interactive system that you have access to and can study.
Possibilities include software applications such as programming
language development environments or case tools or games or
photo editing systems; e-commerce websites or museum websites
or government websites; one of DMU’s web-based systems for
students or staff; electronic devices such as remote controls for
televisions or DVD players, or digital cameras, or car radios; or
control panels for appliances such as microwave ovens or home
heating systems; or a self-service system such as an automatic
ticket vending machine. You may, if you wish, choose to evaluate
two very similar and directly competing products, and assess ways
in which one is superior to the other.
It’s perfectly okay to decide to evaluate a part of a big or
complicated system, or consider a limited set of use cases. When in
doubt, go into more detail about less of the system.
The one piece of advice we can give is to choose something that is
complicated or difcult to use, or is used to carry out complicated
tasks, and preferably has obvious usability problems. Studying more
complicated and less frequently used features of a system is likely
to be more fruitful than focusing on the standard functions people
use all the time. Standard features of highly optimised systems that
large numbers of people use, like Amazon, don’t make for
interesting evaluations.
You may choose to interpret ‘interactive system’ very broadly and
present a usability evaluation of a static information display, but this
would require a sophisticated and detailed analysis of how people
use it for practical tasks, and these tasks would need to be
complicated enough to give you something to analyse. Ask advice if
you consider this.
The post IMAT5209 Human Interaction in Systems Design appeared first on My Assignment Online.