Usability Testing for a Telehealth Portal
Goal:
The goal of this project was two-fold:
Measure the overall user experience of the telehealth platform, including usability, design, content, brand, and emotion.
Identify specific opportunities to improve the user experience from the patient perspective
Process:
Conducted a usability test and survey with participants to measure the overall user experience. Analyzed qualitative and quantitative data to provide recommendations.
Role:
UX Researcher on a team of 3
Responsibility:
Conducted initial research, ran half of the usability tests, helped analyze data
Methods:
Usability Test (Moderating) and Survey
Tools:
Qualtrics and Excel
Overview
A telehealth platform client was seeking ways to identify issues with its current site. Our team was hired to help the client test common use cases of the platform and provide quantitative data to identify usability issues. Our team conducted task-based usability study that incorporated a survey between tasks. The survey collected data on usability, design, emotion, and brand. The end result was a hybrid of qualitative and quantitative findings presented as a scorecard.
Process
1 - Discovery
After consulting with the client and conducting a review of the platform, we determined 5 tasks to use for the usability test. The tasks were prepared to represent a broad range of activities a customer may perform on the mobile website, covering different features and functionality. From the discovery research, we determined a mix of ages and telehealth familiarity were important traits for the participants. We then put together a moderation guide and screener for the recruitment of participants.
2 - User Research
For this study, we took a hybrid approach to the research. The usability test consisted of 17 participants completing the 5 tasks and filling out a survey about their experience. The participants switched between the test on the mobile website and then the survey on their computer. All of this was conducted and moderated over zoom.
Qualitative Research - Usability Testing
We conducted 17 usability tests to get the qualitative data. We observed the participants ask they completed the 5 tasks to the best of their ability. The test was broken down into three parts:
Background questions - used to gain a better picture of the participants experience with telehealth
The tasks - scenarios were created for each task and the participants used the talk-aloud method while completing the tasks. After each tasks participants were asked about their experience and how it compared to their expectations of the task.
Tasks were evaluated based on ease of completing the task, the time it took to complete, satisfaction with the outcome of the task, and if the website was designed to support completion of the task.
Wrap-up questions - these questions dug deeper to understand the overall experience and how the site compared to their expectations.
Quantitative Research - Survey
The survey used quantitative metrics to measure a few aspects:
Prior to any tasks, participants rated their expectations on a scale of 1-5 (5 being the highest). Ratings included four dimensions; easy to use, useful information, organization, and look & feel. After completing the tasks, they were asked to rate the site based on these criteria again.
Participants were asked to weigh four elements: usability, emotion, design, and brand in terms of what was most important to them. They were given 100 points to distribute across the four categories based on what matters most to them. This was used to calculate a personalized score for the client.
3 - Data Analysis
The quantitative analysis supported and paired well with the qualitative analysis. The analysis resulted in answering the following questions for the client:
how am I doing?
what were the major observations?
what are the recommendations?
Qualitative Analysis - using the data from the tasks and post-task questions, we used thematic analysis to determine issues within each task.
Quantitative Analysis - Through survey analysis, a number of factors were measured and determined:
An overall experience score
The difference between expectations and actual experiences
Overall task analysis and task evaluation breakdown by ease, time, satisfaction, and design
UX priorities for the client - usability, brand, emotion, design/content
4 - Client Presentation
At the end of the project, we presented a “scorecard” to present the results. This “scorecard” communicated our findings and the recommendation.
Limitations & Constraints
This project did face a few contraints and challenges:
This study took place at the height of the pandemic when it was unsafe to be in person. Due to that, conducting the test over zoom proved challenging. The software used would not support screensharing on mobile and desktop and our team had to adjust from testing on a mobile device to testing a mobile website. As a result some of the finding may not fully represent the mobile experience.
There were a number of participants who were not familiar with zoom or screen sharing technology. This caused us to drop a few participants in the middle of the study.
Outcome & Reflection
After the final report and data was delivered to the client there were aspects of this project that I would change. First, I would have suggested that we familiarize the client with the level of data analysis we were conducting. While the data is informative and helpful, I am not sure the client was able to understand everything that was presented. Second, if time permitted, I would have preferred to switch to a different software to conduct the study. This would allow the most accurate results for the client.
Overall, the client was satisfied and grateful for the work. They were impressed with all of the recommendations - from the small color change suggestions to the larger usability issues. I am proud of the final product and