Usability Test Report for Bournemouth University IT
Services
| Date of Report: Date of Test: Location of Test: |
21 May 2018 11 May 2018 Bournemouth, UK |
Prepared for: BU Digital Vision Steering Group
Phone Number:
Email:
Prepared by: Kouao Kedja
Phone Number:
Email: s5075958@bournemouth.ac.uk
Table of Contents
Table of Contents………………………………………………………………………………………………2
1. Executive Summary ……………………………………………………………………………………….3
2. Introduction …………………………………………………………………………………………………..4
3. Methodology………………………………………………………………………………………………….4
3.1. Evaluation methods ……………………………………………………………………………………………….. 4
3.2. Heuristic Evaluation ……………………………………………………………………………………………….. 4
3.3. Think Aloud …………………………………………………………………………………………………………… 5
4. Findings ……………………………………………………………………………………………………….6
4.1. Heuristic Evaluation ……………………………………………………………………………………………….. 6
4.2. Think Aloud …………………………………………………………………………………………………………. 16
5. Recommendations ……………………………………………………………………………………….19
5.1. Improved navigation……………………………………………………………………………………………… 19
5.2. Design recommendations ……………………………………………………………………………………… 19
5.3. Design prototype………………………………………………………………………………………………….. 20
Appendices ………………………………………………………………………………………………………. i
Appendix A: University IT Services: United Kingdom …………………………………………………………..i
Appendix B: Resources Templates for Think Aloud Sessions ……………………………………………..ii
Appendix C: Computing Environment…………………………………………………………………………….. vii
Appendix D: User metrics ……………………………………………………………………………………………. viii
3/30
1. Executive Summary
BU IT Services website (itservices.bournemouth.ac.uk) is the online informative and support
resource of the university IT service desk, for students and staff, on IT related issues or concerns.
Therefore, its users have the right to expect the most efficient and complete service from the
website. This report originate from an evaluation of the website usability in order to identify any
eventual issues having a significant impact on the user’s experience.
We applied two evaluation methods to assess the website usability, which involved expert
inspections through a Heuristic Evaluation and users tests during Think Aloud sessions.
The main outcomes of the evaluations highlighted navigation and search problems, significant
numbers of dead links, design inconsistencies and language issues. The expert inspection listed
the issues bellow:
1. Users do not have a context when viewing an article
2. Significant number of dead and inaccurate links
3. Audiovisual category and its sub-categories have a different design and layout
4. All child articles have the same icon
5. No precise reason for denied accesses and no suggested alternative
6. No help on login pages
7. Difficult navigation for long pages
8. Significant number of PDF files as webpages
9. Missing help and documentation
10. Search results and news are not user friendly
11. Duplicated catalogue of service on home page
12. Use of acronyms without explaining their meanings
The second evaluation consisted in an observation of four different users, asked to perform a set
of tasks while commenting their actions and giving their feedback. The website’s users
complained about the complexity of several tasks. This complexity became worst when they
encountered dead links, restricted resources and misleading instructions. As the website do not
provides sufficient guidance or any help and documentation on how it works, only their
recognition, recall and cognitive abilities allowed the user to find their way on certain tasks.
Finally, the users confirmed the need of an improved search feature, even if the current one
provided average satisfaction. This resulted in a shared opinion on the overall website with both
extremely positive and negative feedbacks.
Despite these issues, our tests highlighted the average satisfaction of users about the quality of
the contents and their organisation around distinct topics, even with some cross-references. The
minimalistic and colourful design of the website was also a positive point for the website.
At the end of this evaluation, we identified several recommendations to improve the usability of
the website, related to:
1. An improvement of the search function with ordering and filtering options
2. A complete redesign of the home page with the search and the most popular links
emphasised
3. A careful verification/correction of hyperlinks and buttons to avoid loops and dead ends
4. A inclusion of contextual elements to provide users with context about their current
location in the website architecture
5. A redesign long pages through a pagination mechanism to reduce extensive scrolling
This report will review all the aforementioned topics in more details.
4/30
2. Introduction
BU IT Services website (itservices.bournemouth.ac.uk) is the online informative and support
resource of the university IT service desk, for students and staff on IT related issues or concerns.
This evaluation aims to review the renovated BU IT service website in order to identify and report
usability issues, and to provide recommendations for improvements and changes that will
increase its users’ satisfaction. This because, instead of adding to the burden of user who seek
help and support, it is very important for an IT service website to be easy to use, efficient and
effective(Buyya et al. 2008).
3. Methodology
3.1. Evaluation methods
There is a large range of methods to evaluate the effectiveness, efficiency, and satisfaction of the
user experience of a website. However, each method have their pros and cons, and depends on
the context of the evaluation: time and budget constraints, stage of the product in its lifecycle,
users’ goals, etc.
For BU IT service website, the evaluation methods applied were involving expert reviews through
Heuristic Evaluation, and users’ tests and feedback through Think Aloud sessions. Both methods
are discount usability engineering methods, which allows the combination of usability experts and
real user’s opinions to ensure an accurate evaluation of the product.
3.2. Heuristic Evaluation
The choice of the Heuristic Evaluation (HE), as expert inspection method, is due to its ability to
rely on accepted usability principles for a website evaluation. It is also a fast and effective
usability inspection method, which we can combine with other usability testing techniques.
However, to be effective, this method need to aggregate multiple experts’ evaluations, as an
individual evaluator will only be able to find a limited percentage of issues, which is between 20%
and 51% according to Jacobsen et al. (1998)
Involving multiple evaluators was not possible for our report because of time and budget
constraints. Therefore, only the author of this report participated to the evaluation, based on
Nielsen’s ten heuristics(Nielsen 1995).
The author evaluated the website under the following conditions:
Table 1: HE evaluator Profile
| Gender | Male |
| Age range | +30 |
| Computer usage | +26 hours/week |
Table 2: HE computing environment
| URL: | https://itservices.bournemouth.ac.uk/ |
| Computer platforms: | HP Intel i5-6500 @ 3.20 Ghz (4CPUs) – Built-in 24 inch display – 8Gb Ram |
| Browser tested: | Chrome 66.0.3359.170 |
| Screen resolution: | 1920×1080 |
| Operating system: | Windows 7 Enterprise 64-bits |
| Connection speed: | PING: 7ms / Download: 94.79 Mbps / Upload: 95.97 Mbps1 |
1 http://www.speedtest.net/result/7300579148
5/30
Table 3: Nielsen’s 10 Heuristics:
| 1. Visibility of system status | 6. Recognition rather than recall |
| 2. Match between system and the real world | 7. Flexibility and efficiency of use |
| 3. User control and freedom | 8. Aesthetic and minimalist design |
| 4. Consistency and standards | 9. Help users recognize, diagnose, and recover from errors |
| 5. Error prevention | 10. Help and documentation |
3.3. Think Aloud
In addition to the website inspection by an expert, the usability evaluation also consisted in a
cognitive task analysis. During “Think Aloud” sessions, we observed four (04) participants while
they were individually performing specific tasks on the website, according to eight (08) defined
scenarios.
What happened during the usability test
The author of this report conducted the usability test of the website at BU Library (Talbot
Campus, 1rst Floor) on Friday 11th, May 2018.
During the usability evaluation, we asked each participant to spend a minimum of twenty
minutes with the site. During their sessions, participants:
| |
Completed a user background questionnaire Answered questions about initial site impressions Performed real-world tasks on the site while thinking aloud Answered questions about their overall satisfaction Completed a System Usability Scale(SUS) questionnaire |
Who we tested
Due to time and budget constraints, only four postgraduate students from Bournemouth
University volunteered to participate to the website evaluation.
Table 4: “Think Aloud” participants’ profiles
| Audience Type Staff 0Postgraduate 3Undergraduate 1TOTAL (participants) 4 |
Computer Usage 0to 10 hrs. wk. 011 to 25 hrs. wk. 126+ hrs. wk. 3TOTAL (participants) 4 |
| Age 18-25 326-30 1+30 0TOTAL (participants) 4 |
Gender Women 3Men 1TOTAL (participants) 4 |
What participants did
During the usability evaluation, we asked to participants to complete eight scenarios. These
scenarios derive from the similarities between the services and Frequently Asked Questions
(FAQ) from five universities’ IT services website in United Kingdom (Appendix A).
For each given scenario, each participant decided on their own the sequence of sub-tasks to
perform, without any hint from the test facilitator. They had to decide solely based on what was on
their screen while thinking aloud, and only use the search function as last resort.
6/30
Table 5: Think aloud scenarios
| # | Scenarios |
| 1 | Changing students password |
| 2 | Connecting to projectors/screens on-campus |
| 3 | Top-up print account |
| 4 | Printing from personal computer |
| 5 | Accessing online timetable information |
| 6 | Accessing student’s storage space from personal computer |
| 7 | Connecting to wireless network |
| 8 | Get help on general/specific IT issue by filling a request form |
What data we collected
During the test, we collected:
| | The task completion rate: Defines if the participant succeeded in finding the information required to reach its goal easily, whit difficulty/help, or failed. The numbers of errors: The evaluation excluded slip errors and only considered lapse, and mistakes. This because, slips often occur because of factors that are external to the |
| |
system (external distraction, fatigue, etc.), thus imputable to participants or their
environment. However, it is possible to mitigate or reduce slip errors with specific design
methods (HG48).
| | The System Usability Scale: An industry standard for classifying the ease of use of the website with 10 standard questions asked to participants at the end of their sessions. Task completion time: We collected this metric to assess the complexity of the scenario and the steps required for its completion. |
| |
Test environment & Tools
The computing environment and location of the tests were the same as for the Heuristic
Evaluation. In addition, we used a Nimbus Screen Capture2, a free plugin for web browsers, to
record the screen and the voices of participants and the facilitator.
4. Findings
4.1. Heuristic Evaluation
Thirteen usability issues emerged from our Heuristic Evaluation of BU IT service website. These
issues occurred when some characteristics of the website do not complied with Nielsen’s ten
usability heuristics. In the summary table (Table 6), we also used two scales to measure the
severity rating of each issue (0= not a usability problem; 4=catastrophic usability problem) and
the ease-of-fix rating of that issue (0= extremely easy to fix; 4= requires a significant effort to fix).
Navigation issues (#1, #3, and #7), multiple dead links (#2) and design inconsistencies (#3, #4,
and #8) are the most severe issues identified during this evaluation. Missing guidance (#5, #6,
#9, and #12) for user is less severe, but still crucial issue. Fortunately, these issues are easy to
fix without extensively altering the look and feel of the website, as well as its navigation.
2 Nimbus Screen Capture is available from https://nimbusweb.me/screenshot.php
7/30
Table 6: Heuristic Evaluation summary
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 1 | Users do not have a context when viewing an article |
4 | 1 | #1 #3 #7 |
Visibility of system status ; User control and freedom ; Flexibility and efficiency of use |
| 2 | Significant number of dead and inaccurate links |
4 | 1 | #3 #5 | Error prevention and freedom ; User control |
| 3 | Audiovisual category and its sub-categories have a different design and layout |
4 | 2 | #4 #7 |
Consistency and standards; Flexibility and efficiency of use |
| 4 | All child articles have the same icon |
3 | 2 | #2 #4 |
Match between system and the real world ; Consistency and standards |
| 5 | No precise reason for denied accesses and no suggested alternative |
3 | 1 | #5 #9 |
Error prevention ; Help users recognize, diagnose, and recover from errors |
| 6 | No help on login pages | 2 | 0 | #5 #9 |
Error prevention ; Help users recognize, diagnose, and recover from errors |
| 7 | Difficult navigation for long pages |
2 | 0 | #1 #6 #7 |
Visibility of system status ; Recognition rather than recall ; Flexibility and efficiency of use |
| 8 | Significant number of PDF files as webpages |
2 | 1 | #3 #4 #5 |
User control and freedom ; Consistency and standards ; Error prevention |
| 9 | Missing help and documentation |
2 | 1 | #10 | Help and documentation |
| 10 | Search results and news are not user friendly |
2 | 1 | #4 #7 | Consistency and standards ; Flexibility and efficiency of use |
| 11 | Duplicated catalogue of service on home page |
1 | 1 | #8 | Aesthetic and minimalist design |
| 12 | Use of acronyms without explaining their meanings |
1 | 0 | #6 | Recognition rather than recall |
| Severity rating: 1=Not a usability issue; 2=Minor usability issue; 3=Major usability problem; 4= Severe Usability problem Ease of fix rating: 0= extremely easy to fix; 4= requires a significant effort to fix |
1) Users do not have a context when viewing an article
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 1 | Users do not have a context when viewing an article |
4 | 1 | #1 #3 #7 |
Visibility of system status ; User control and freedom; Flexibility and efficiency of use |
a. Issue
Only the page title (and wrong icon) gives information on the topic covered in its content. Users
miss any visual aid on the page location within the site’s hierarchy. There is a lack of contextual
information to give proper answers to the following questions: Where am I? Where can I go?
Should I go there? In addition, there is no contextual sub-menu to allow the user to jump straight
right section on long articles. These issues violate the three usability principles of “Visibility of
system status”, “User control and freedom” and “Flexibility and efficiency of use” (heuristics #1,
#3 and #7).
8/30
b. Evidence
c. Recommendations
Breadcrumbs for all website (Martin, 2006),
Quick links for long pages,
Highlight selected letter for “A to Z”
2) Significant numbers of dead links
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 2 | Significant number of broken and inaccurate links |
4 | 1 | #3 #5 |
User control and freedom ; Error prevention |
a. Issue
A large number of hyperlinks and clickable buttons lead to unavailable knowledge articles,
restricted resources or broad topic pages instead of a specific article on the topic. These are
against the principles #3 and 5# as they take users to wrong destinations or dead-ends against
their will.
Research tools and database icon
Which category?
Which letter?
M or N?
You need to scroll
down until you find
your question…
9/30
b. Evidence
c. Recommendations
Step one: Automatically check hyperlinks redirections using a webmaster tool3 to flag dead links
Step two: Individually check redirections to find mistakes and take corrective actions
Step three: Create missing contents and update links accordingly
3) Audiovisual section and its sub-sections have a different design and layout
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 3 | Audiovisual category and its sub-categories have a different design and layout |
4 | 2 | #4 #7 |
Consistency and standards; Flexibility and efficiency of use |
a. Issue
The audivisual category uses a navigation model based on tiles and pictograms/icons. This
model differs from others categories, which use lists of hyperlinks. That is against the principle #4
of consistency and standards. In addition, all articles in this section are in PDF format, thus
breaking the navigation flow and representing the second most harmful design mistake in web
design (Nielsen 2011).
3 W3C Link Checker : https://validator.w3.org/checklink
| Click |
| Links and results |
A page on BU main website
The right link is here
Click
eduroam information are further below
10/30
b. Evidence
c. Recommendations
Step one: Create HTML equivalent for each PDF file in the audiovisual category, also include
original PDF file as a download link in page content.
Step Two: Create list of audiovisual services as per other categories, in compliance with the
website design guidelines
4) All child articles have the same icon
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 4 | All child articles have the same icon |
3 | 2 | #2 #4 | Match between system and the real world; Consistency and standards |
a. Issue
All articles have the same illustration icon, which is the icon for the “Research Tools and
Database” services category, visible on the home page tiles.
b. Evidence
| Clic |
| Audiovisual Catego |
VS Other ries |
| Clic |
| Clic |
| Research Tools and databases icon |
Article icons (different
service categories)
Account & access
Network & connection
Files, Drives and Storage
11/30
c. Recommendations
Replace icons to make sure that each child article uses its parent category icon.
5) Significant numbers of dead links
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 5 | No precise reason for denied accesses and no suggested alternative |
3 | 1 | #5 #9 |
Error prevention ; Help users recognize, diagnose, and recover from errors |
a. Issues
Errors messages are short use technical language and do not provide eventual solutions. This
violate principles #5 and #9, as users cannot understand the errors too avoid it in the future, nor
take corrective actions.
b. Evidence
c. Recommendation
Use plain language for concise error explanations. Also, provide alternative solutions to users if
any.
6) No help on login pages
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 6 | No help on login pages | 2 | 0 | #5 #9 |
Error prevention ; Help users recognize, diagnose, and recover from errors |
a. Issues
There is no indication on the username format. Student cannot assume if it is their student ID only
or student email ([student ID]@bournemouth.ac.uk). The only given option is to reset the
password
| Click |
According to the article, resource is available for students too. However, clicking on the
Knowledge Base button opens a new tab displaying an error message; which is unclear
and do not provide any alternative for the user.
12/30
b. Evidence
c. Recommendations
Add contextual hint on the user name format, as in the password reset page.
7) Difficult navigation for long pages
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 7 | Difficult navigation for long pages |
2 | 0 | #1 #6 #7 |
Visibility of system status ; Recognition rather than recall ; Flexibility and efficiency of use |
a. Issues
For long articles, there is no context except paragraph titles. There is also no way to jump directly
to the right section or to go to the top unless you reach the bottom of the page. This means that
users have to scroll the page to find what they want, while relying on their memory about the
location of the information. All these facts violate the principles #1, 6 and #7.
Where is the error?
13/30
b. Evidence
c. Recommendations
Grouping the sections of the page at the top will provide users with easy scanning. Adding a fixed
“go to top” button will allow users to quickly return to sections’ list.
8) Significant number of PDF files as webpages
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 8 | Significant number of PDF files as webpages |
2 | 1 | #3 #4 #5 |
User control and freedom ; Consistency and standards ; Error prevention |
a. Issues
Many links lead to PDF files without warning users. This is against the principles #3, #4 and #5;
because it breaks the navigation flow, preventing users to return to the website without the use of
the browser “previous page” function, PDF also have their own design, and user may only open a
PDF file if they intended to.
b. Evidence
How to find the right question?
How to go back to the top?
Where am I?
The menu link “IT for
Students” open a PDF
file in a new tab
14/30
c. Recommendations
To solve this issue, website managers have to create html version of each PDF files, update links
accordingly and propose the PDF files as separate downloadable resources.
9) Missing help and documentation
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 9 | Missing help and documentation |
2 | 1 | #10 | Help and documentation |
a. Issue
There is no help and documentation section to explain the use of the feature of the website. It is
against the heuristic #9 about providing users with guidance on the use of the system
b. Evidence
c. Recommendations
Create “Help and Documentation” articles and a link to the footer.
10) Search results and news are not user friendly
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 10 | Search results and news are not user friendly |
2 | 1 | #4 #7 | Consistency and standards ; Flexibility and efficiency of use |
a. Issues
Search results miss ordering and filtering capabilities, whereas news only miss categorisation. In
a website were all articles have their categories, it is hard for user to see which news, or which
result, relates to the category of its interest. It is not compliant with the consistency of the website
and limit the flexibility and efficiency of the use of the search results (heuristics #4 and #7).
b. Evidence
Missing help and
documentation on website
footers
There is no way to sort or
filter search results (What is
the most relevant? Is that the
right category? Etc.)
Which category
for each news?
15/30
c. Recommendations
Add filtering and sorting options to search results page.
Add categories or tags to news, and add them to events cards.
11) Duplicated catalogue of service on home page
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 11 | Duplicated catalogue of service on home page |
1 | 1 | #8 | Aesthetic and minimalist design |
a. Issue
The menu left menu item (level 1) have the same functions as labels like the services card.
However, the left menu opens a sub-menu links, whereas the card opens the page that list all the
categories. This is a duplication on features and a violation of the principle #8 on minimalist
design.
b. Evidence
c. Recommendations
Improve left menu bar functionalities with “Catalogue of Service” cards functionalities and delete those cards
12) Use of acronyms without explaining their meanings
| # | Issue | Severity rating |
Ease of fix rating |
Heuristic number |
Broad heuristic |
| 12 | Use of acronyms without explaining their meanings |
1 | 0 | #6 | Recognition rather than recall |
a. Issues
Some article uses acronyms that do not have any explanation on the same page. That increases
the probability that the user may not recognize the meaning of the acronyms, which violates the
heuristic on recognition and recall (#6)
Same labels but not same
functions
16/30
b. Evidence
c. Recommendations
Make sure to provide the explanation on any acronym the first time they appear on each page.
4.2. Think Aloud
The test sessions were the opportunities for participants to give their opinion about the website.
Before the sessions, we discovered that all participants knew about the website even if they did
not used it before. All were also expecting the website to provide efficient supports on IT related
issues, guessed that it was for current students and staff, and assumed that is was managed the
IT service desk of the Bournemouth University.
The Table 7 below summarises participants’ comment for the given scenarios:
Table 7: Summary of user’s comments
| Scenarios | User’s Comments |
| Changing students password | No comment |
| Connecting to projectors/screens on campus |
1. The terms “a/v” and “audiovisuals” are not clear to users 2. All “Audiovisial” articles are PDF files 3. “Audiovisual” navigation is complicated |
| Top-up print account | No comment |
| Printing from personal computer |
4. The “Printer setup” menu is not very clear for users 5. The website do not addresses the high complexity of this task correctly |
| Accessing online timetable information |
6. The Timetable status icon looks like a clickable item and is therefore misleading. |
| Accessing student’s storage space from personal computer |
7. Article on the website do not address the issue clearly and have many dead links and loops. |
| Connecting to wireless network |
8. Article is not easy to read or understand are not easy to read and understand |
| Get help on general/specific IT issue by filling a request form |
9. Users feel lost when the links redirect them to external websites |
The heuristic evaluation identified most of the issues raised by user’s, confirming the evaluation
findings. However, two comments are worth considering in addition to HE findings:
What is “SNOW”?
There is not any
explanation on this
page
17/30
1) The Timetable status icon looks like a clickable item and is therefore misleading.
a. Issue
On home page, users tried to click on the services live status icons and expected to reach the
corresponding services. This placeholder’s size and position of on the homepage suggest that
users can interact with the icons, while they cannot.
b. Evidence
c. Suggestion
Redesigning and repositioning the placeholder avoid click attempts.
2) The website do not correctly addresses the high complexity of tasks
a. Issue
When a scenarios to perform a complex sequence of tasks (operations on H drives), the website
failed to provide clear guidance to user. Instead, the presence of dead links and loops raise the
complexity of the scenario.
b. Evidence
Click
The page says that the
service is available for
students and staff, but only
provide links for staff
More dead and protected
links.
Click
| Nothing happens when users click on this area |
18/30
c. Recommendation
In addition to high quality and valid contents and links (mentioned in heuristic evaluation section),
all articles should have a complexity level (from easy, to hard/complex) so that users may
evaluate the efforts required to solve an issue.
3) Usability metrics
In addition to user’s comments, we also collected usability metrics during think aloud sessions.
The results, summarized in Table 8, shows that:
a. The scenarios, which involve the use of personal devices with university equipment
(printer, projector and storage), are the most difficult with the lowest completion rate,
highest errors occurrences and highest duration. Therefore, website articles must
carefully address these issues.
b. Most errors are due to misleading labels, dead links and navigation loops.
Table 8: Usability metrics summary
| Scenarios | Description | Completion rate |
Average errors |
Completion Time (seconds) |
| #1 | Changing students password | 62% | 1.25 | 91.67 |
| #2 | Connecting to projectors/screens on campus |
37% | 2.5 | 216.5 |
| #3 | Top-up print account | 87% | 0.25 | 38.25 |
| #4 | Printing from personal computer |
75% | 1.5 | 115.25 |
| #5 | Accessing online timetable information |
87% | 0.25 | 52.5 |
| #6 | Accessing student’s storage space from personal computer |
25% | 3 | 266 |
| #7 | Connecting to wireless network |
87% | – | 43.5 |
| #8 | Get help on general/specific IT issue by filling a request form |
62% | 1.25 | 87 |
| Average | 65% |
Finally, the System Usability Scale (SUS), shows that web-experienced users found the BU IT
Service website pretty well designed and useful, whereas average users and beginners thinks
that it need major improvements.
Table 9: System Usability Scale results
| User | Gender | Level | Age range |
Computer Usage (hours) |
SUS Score |
SUS Score Meaning |
Comments on participant persona |
| #1 | Male | Post Graduate | 18-25 | +26 | 77.5 | Good | Tech-savvy |
| #2 | Male | Post Graduate | 26-30 | +26 | 82.5 | Excellent | Average Technology user |
| #3 | Female | Post Graduate | 18-25 | +26 | 45 | Awful | Non Tech Savvy |
| #4 | Male | Under Graduate | 18-25 | 11-25 | 60 | Poor | Beginner and enthusiastic |
19/30
5. Recommendations
Even if the current system is functional with multiple qualities, there are still rooms for
improvements. This section will cover areas were the next improvements shall be made to
accommodate most users’ needs and solve identified issues. An interactive website prototype is
available on InVision to demonstrate the design solutions.
5.1. Improved navigation
The new site map originate from the findings of the evaluation methods and the analysis of few
knowledge bases and IT support website. It aims to improve the navigation the by naturally
grouping the contents and offering alternates paths.
5.2. Design recommendations
| Location | Recommendations |
| 1. All Website | 1. Change navigation menu with Home | News | All Services I Get Support | FAQ | About us. Active menu should always have a darker background 2. Replace left menu by twitter feed to show latest updates and remove twitter button 3. Use Glyphs instead of icons for Services status 4. Limit page scrolling to 1200 pixels; add pagination for longer articles. 5. Use Blue, Purple and Grey shades from official 2013 colour palette, and White colours for gaps and empty spaces. 6. Use Bitter font family for texts and labels. |
| 2. Homepage | 7. Instead of general information on IT Services, include quick links to articles grouped by categories, to provide help from the start. 8. Increase search area and horizontally align to centre of screen. Use small search area for all other pages. 9. Add sorting and filtering for search results. Add also advanced search option (Search in news, categories, by topic, by keyword, text search, etc.) 10. Add the two latest news cards before page footer. |
| 3.All Services | 11. Display categories cards with flat monochromatic icons for minimalistic design purpose 12. Add a list view option 13. Add A to Z as alternative view for articles, with pagination and darker background for active letter |
| 4. Help Articles |
14. Add location based breadcrumb (article category>article title) to provide users with context. 15. Add complexity level to article header. 16. Suggest related articles on articles pages. |
| 5. Others (*) | 16. Add tags and categories to Events 17. Review all links and articles’ contents. |
IT Services
Home
News
All services A to Z
IT Helpdesk
FAQ
How it works
About us
Helpdesk Home
Self Help (Search)
Get Help
Services status
Search
Catalogue
Live status
Incidents
Statistics
Request Help
Track progress
20/30
5.3. Design prototype
Following this evaluation, an interactive design prototype for BU IT service website is available
online on InVision® and accessible through the following link: https://invis.io/FUJ2MYPN59S. The
prototype presents different screens following the suggestions above, except for suggestions 16
and 17 because of time constraints.
The prototype implements a heavily redesigned home page, improved navigation and search,
neutral couloirs and flat designs. It also implements few contextual elements (breadcrumbs,
related contents, tags and categories), to provide the user with location within the website
structure.
a) The new Homepage
| Most popular links |
| Emphasised Search area on Home page |
| Twitter feed |
| Service Now management Portal |
New navigation menu with highlighted active item
Latest news
21/30
b) Services catalogue
c) Services category
Small search box on all pages
Alternative listing
Location-based breadcrumbs
Pagination
22/30
d) Article page
REFERENCES
Buyya, R., Yeo, C. S., and Venugopal, S., 2008. Market-oriented cloud computing: Vision, hype,
and reality for delivering it services as computing utilities. In: High Performance Computing
and Communications, 2008. HPCC’08. 10th IEEE International Conference on. Ieee, 5–13.
Jacobsen, N. E., Hertzum, M., and John, B. E., 1998. The evaluator effect in usability studies:
Problem detection and severity judgments. In: Proceedings of the Human Factors and
Ergonomics Society Annual Meeting. SAGE Publications Sage CA: Los Angeles, CA, 1336–
1340.
Nielsen,J., 2011. Top 10 Mistakes in Web Design [online]. Available from:
https://www.nngroup.com/articles/top-10-mistakes-web-design/ [Accessed 14 May 2018].
Nielsen, J., 1995. Alertbox : current issues in Web usability. [online]. Available from:
https://www.nngroup.com/articles/ten-usability-heuristics/ [Accessed 21 May 2018].
Related articles suggestions
Article tags
Location-based breadcrumbs
i/viii
Appendices
Appendix A: University IT Services: United Kingdom4
| BOURNEMOUTH UNIVERSITY (Evaluated website) (https://itservices.bournemouth.ac.uk/) 1. Accounts & Access 2. Audiovisual 3. Email services 4. Files, drives & storage 5. Finance & accounting tools 6. IT Learning Centre 7. Networks & connectivity 8. Printers, hardware & software 9. Research tools & databases 10. Tools and Resources 11. Telephone services 12. Website services 13. Technology enhanced learning UNIVERSITY of OXFORD (Ranked 1rst) (https://www.it.ox.ac.uk/services) 1. Business systems and administration support 2. Connect and communicate 3. Data services 4. Desktops and devices 5. Help, advice and consultancy 6. Research support 7. Security 8. Teaching and learning UNIVERSITY of CAMBRIDGE (Ranked 2nd) (https://www.itservices.cam.ac.uk/) 1. User Accounts & Security 2. Email, Telephony & Collaboration 3. Support, Policies & IT Training 4. Devices, Applications, Networks & Printing 5. Teaching & Learning Resources 6. Central Systems & Management Reporting 7. Research Support, Data & Computing 8. Development & Website Services |
IMPERIAL COLLEGE LONDON (Ranked 3rd) (https://www.imperial.ac.uk/admin-services/ict/) 1. Connect and communicate 2. Computers and printing 3. Research support 4. Administrative systems 5. Business Analytics 6. Teaching and learning UNIVERSITY COLLEGE LONDON (UCL) (Ranked 4th) (http://www.ucl.ac.uk/isd/services) 1. User IDs & Passwords 2. Email & Calendar 3. Computers at UCL 4. Print, Copy & Scan 5. Get Connected 6. Learning & Teaching 7. Communicate & Collaborate 8. Stay Secure 9. Software & Hardware 10. File Storage, Sharing & Collaboration 11. Telephony services 12. Websites, Apps & Databases 13. Digital Media Services 14. Research IT 15. Accessibility & Disability IT Support LONDON SCHOOL OF ECONOMICS AND POLITICAL SCIENCE (Ranked 5th) (https://info.lse.ac.uk/staff/divisions/imt/help/online guides-faqs) 1. Accounts & Passwords 2. Computers & Printing 3. WiFi & Networks 4. Email 5. File Storage: Onedrive for Business & H: Space 6. Sharing & Collaboration: LSE Filedrop & SharePoint 7. Software Guides 8. Remote Access 9. Assistive Technologies 10. Telecoms: Skype for Business, phones & mobiles 11. Teaching & Learning: Moodle, LSE for You & Lecture Recording 12. Information Security 13. Microsoft Office 365 Resources |
4 Times Higher Education – World University Rankings 2018 (United Kingdom)
https://www.timeshighereducation.com/world-university-rankings/2018/world-ranking#survey-answer
ii/viii
Appendix B: Resources Templates for Think Aloud Sessions
1. Introduction to Testing with Moderator Interaction
2. Consent Form (Adult)
3. What to Test/Note taker’s Guide
4. System Usability Scale (SUS)
iii/viii
iv/viii
v/viii
vi/viii
vii/viii
Appendix C: Computing Environment
http://www.speedtest.net/result/7300579148
viii/viii
Appendix D: Usability metrics
Sessions’ recordings are available on the link below:
https://www.youtube.com/playlist?list=PLPvYzeZc9UebOMyqUGYC8yXtiZ9qTJoxt
| Task | Desciption | Completion rate | Avg / task |
Errors | Avg | Completion Time (sec) | Avg | |||||||||
| P#1 | P#2 | P#3 | P#4 | P#1 | P#2 | P#3 | P#4 | P#1 | P#2 | P#3 | P#4 | |||||
| #1 | Changing students password |
2 | 1 | 2 | 0 | 62% | 0 | 1 | 1 | 3 | 1.25 | 18 | 170 | 87 | 91.67 | |
| #2 | Connecting to projectors/screens on-campus |
1 | 2 | 0 | 0 | 37% | 2 | 1 | 3 | 4 | 2.5 | 372 | 61 | 216.5 | ||
| #3 | Top-up print account |
1 | 2 | 2 | 2 | 87% | 1 | 0 | 0 | 0 | 0.25 | 49 | 32 | 30 | 42 | 38.25 |
| #4 | Printing from personal computer |
1 | 1 | 2 | 2 | 75% | 2 | 1 | 3 | 0 | 1.5 | 197 | 75 | 109 | 80 | 115.3 |
| #5 | Accessing online timetable information |
2 | 2 | 2 | 1 | 87% | 0 | 0 | 0 | 1 | 0.25 | 24 | 20 | 31 | 135 | 52.5 |
| #6 | Accessing student’s storage space from personal computer |
1 | 1 | 0 | 0 | 25% | 2 | 3 | 3 | 4 | 3 | 212 | 320 | 266 | ||
| #7 | Connecting to wireless network |
2 | 2 | 2 | 1 | 87% | 0 | 0 | 0 | 0 | 0 | 13 | 43 | 37 | 81 | 43.5 |
| #8 | Get help on general/specific IT issue by filling a request form |
2 | 0 | 2 | 1 | 62% | 0 | 4 | 0 | 1 | 1.25 | 26 | 108 | 127 | 87 | |
| Average completion rate | 1.3125 | 65% |
The post BU Digital Vision Steering Group appeared first on My Assignment Online.