I’m currently working on an IOS and Android mobile app for a UK bank which is due to go live mid 2015. As part of this project I was involved in organising lab based usability testing sessions which took place at Space Interactive Media Labs in Manchester.

Many of the main stakeholders came along to observe the usability testing sessions from behind the one way glass. Ten participants were to perform the usability tests over two days. Each one to one session lasted for about an hour. The facilitator put the participant at ease by running through what they could expect and then asked the participant to perform three or four scenario based activities and ten to fifteen short tasks which would take up to an hour. The observers were made up of the product owners, UX designer, functional analyst, marketing executive, research analyst and development managers. Each of the observers had a specific job to do with the help of supporting cheat sheets to help them with the task. Eight participants were observed over two days using the in development app on an Android device.

Clear trends started to emerge early on in the testing sessions, showing areas of the application that users didn’t understand or where they were struggling to perform a task. The most apparent finding was that all participants struggled to set up their password during the registration process. The password criteria that had been used was clearly too complicated for users to set up on a mobile device. This went to the top of the priority list to fix.

It became obvious that the terminology being used for key sections of the app was not understood by users in the way that we had expected. Also the way terminology was interpreted was specific to different user groups, hence why it’s important to determine who the main user groups are and source those participants for your testing sessions.

It was interesting that two distinct types of users emerged, those that read and those that tap. The ‘readers’ meticulously read everything on a screen and tried to understand it before doing anything else, these tended to be older users who were also comfortable with the terminology used in the market sector. Then there were the ones who tap, they didn’t read text that was longer than a button label on menu and form screens and were at ease tapping around an app to discover what was there. The ‘tappers’ tended to be younger users who browsed without apprehension or fear. This user group didn’t understand some of the labels that had been used for key sections where market sector terminology had been used. The language used wasn’t complicated but they were words and phrases that had become common place internally within an organisation and after a while people don’t question what they mean anymore or whether lay people understand them.

The main things I took away from the sessions are as follows:

• The information architecture in conjunction with the terminology used are a crucial part a users understanding of and ability to use an app
• Keep it simple. Users found multiple calls to action confusing
• An onboarding experience done badly is like having a NO ENTRY sign right at the beginning of your app
• Entering information on a mobile device is different to entering it on a computer, the keypad is a lot smaller and it’s easier for a user to make mistakes therefore a user journey that has worked on a desktop may not be suitable on a mobile
• A lot of findings don’t take a lot of expensive development time to fix and will make an app infinitely better by removing major barriers to the users understanding or ability to use the app.
• Stakeholders understand the issues found has they have seen them at first hand and therefore buy in to make changes
• People from Manchester provide very straight forward feedback which is great when user testing

Why use a lab?

Formal lab based user testing is a great way to involve key stakeholders in the user testing process. A lab provides a dedicated environment where the whole team can immerse themselves in a project and focus on the task in hand and what’s important to achieve. It’s a lot more relaxed as you haven’t got office life whizzing by outside a meeting room and the risk of someone interrupting or an observer being pulled out on an urgent matter. It also builds a greater sense of team work, people behave differently when they’re out of the office and you get to know your team members better. Participants don’t see any of the observers so they’re more relaxed and the observers can’t influence them in anyway. One of my colleagues in marketing said he’d found the experience the most insightful thing he’d been part of since being at the company.

I’m interested to hear your experiences of both lab and guerrilla user testing, so please feel free to leave a comment.