Evaluate the usability of the Airpals website's Order flow and gather feedback on user’s expectations after placing an order.
During the kickoff meeting, we acquainted ourselves with the Airpals team and took the time to understand their goals and objectives for the project. We gathered crucial insights into their target users and business strategy. The Airpals team requested that we prioritize usability testing on specific key areas of their website and review their Google Analytics data to identify any pain points that should be included in our study. Additionally, they expressed a desire for us to gain an understanding of user frustrations when using their website.
Meeting outcome
Defining target participants + developing a recruitment strategy
Using the information obtained during the kick-off meeting and the existing personas developed by the Airpals team, we established our target participant profile. Given that our target participants needed to be employed and over 18 years old, we determined that the most effective recruitment strategy would be to email a screener to the Pratt Students distribution list and post it on our social media accounts.
Defining the research objective
Our research objective was to obtain both qualitative insights into user perceptions and behaviors, as well as quantitative data, with the intention of generating robust evidence to support our project goal.
Designing the test
To conduct our usability tests, we created a script that included a scenario, 2 tasks, post-task questions, questionnaire, and post-test questions, questionnaire. The post-task and post-test questions enabled us to collect qualitative data, while the post-task and post-test questionnaire facilitated the gathering of quantitative insights, as specified by the client and defined in our research objective.
Pre-test questions, task, scenario, post-task questions and post-test questions
Conducting pilot sessions
To refine the script and questions, we conducted 5 pilot tests. During these tests, we closely observed participant behavior and identified any issues or areas for improvement in the script. For instance, we discovered that some of the task instructions were vague or repetitive and required revisions to enhance clarity. Moreover, we recognized that the second task outlined in the script did not provide significant value to the research and made the test longer than the desired duration. As a result, we chose to eliminate the second task for the usability sessions conducted with the recruited participants.
Recruiting participants
The screener was carefully crafted to ensure that only participants who matched the target demographic would qualify for the study. Out of 18 responses received, we selected 11 participants who best matched the target participant profile to participate in the usability test. Although 11 participants were recruited, 8 were retained as the best fit for the user group, and the results reflect the responses of those retained.
Scheduling sessions
We reached out to the 11 selected participants with additional information and provided them with a Doodle link to indicate their availability. Using the participants' preferred time slots, we scheduled Zoom meetings for the remote usability testing sessions. To ensure the smooth running of each 30-minute session, we assigned a moderator and a notetaker to oversee the testing.
Conducting sessions + Collecting data
We conducted 11 remote moderated user testing sessions over zoom. This allowed us to observe and record participant behavior as they shared their screen and navigated through the order form to complete the task. As previously stated, we used a combination of verbal questions and a Google Form questionnaire to collect both qualitative and quantitative data.
Screenshot from a user testing session conducted on Zoom
Review + Group data based on usability issues
After completing the usability testing sessions, we reviewed the data collected through the notetaker Google Form. To analyze the feedback, we utilized a digital grouping spreadsheet and applied a thematic analysis approach to classify the data based on common user frustrations, behaviors, or pain points observed during the testing.
Identify key usability issues
We assessed the issues identified in the previous step based on their severity and frequency, considering how badly the issue affected the usability of the form and how often it occurred during the test sessions. We then prioritized them accordingly to identify key issues and determine solutions for them.
Snapshot of the digital grouping Google Sheet
Report + Presentation
Based on our analysis, we identified key issues and recommendations which we presented to the client through a comprehensive report and presentation.
"I want to express my sincere gratitude for all the hard work and dedication you put into this project. Your contributions will be of tremendous help, and we're excited to implement your feedback."