Enhancing the order form experience for the Airpals website
USABILITY STUDY
PROJECT TIMELINE
6 weeks (Mar - May 2023)
RESEARCH METHOD USED
Moderated Remote User Testing
TOOLS USED
Figma, Google Forms, Google Sheets, Google Docs, Zoom, Doodle
TEAM OF USABILITY EXPERTS
Anne Kuo, Mary Haws, Sara Sarmiento, Priyanka Rajesh Jain
Overview
Airpals is a B2B courier service that offers same-day and scheduled delivery in NYC and parts of Jersey City. Their goal is to simplify courier delivery and make work life easier for users by adding modern updates and empathy to the industry. To improve the pick-up and delivery scheduling order form on their website, our team of 5 usability experts conducted 11 moderated user tests. We used the findings to provide recommendations to enhance user experience and improve the order form's flow.
Project Goal
Evaluate the usability of the Airpals website's Order flow and gather feedback on user’s expectations after placing an order.
My contributions
-
Analyzed Google analytics data to identify pain points on the Airpals website.
-
Worked closely with the team to design and script a user test that aligned with research objectives.
-
Conducted participant recruitment, moderated 2 remote user testing sessions and took notes for other 2 sessions.
-
Collaborated with the team to analyze findings and develop recommendations.
-
Created mockup and prototype, and drafted findings and recommendations for the “Review” page of the order form and the error messages displayed on the form.
-
Actively took part in drafting the final presentation and report.
-
Presented the findings and recommendations to the client.
What was our process?
Step 1: Project kickoff meeting with the client
During the kickoff meeting, we acquainted ourselves with the Airpals team and took the time to understand their goals and objectives for the project. We gathered crucial insights into their target users and business strategy. The Airpals team requested that we prioritize usability testing on specific key areas of their website and review their Google Analytics data to identify any pain points that should be included in our study. Additionally, they expressed a desire for us to gain an understanding of user frustrations when using their website.
Meeting outcome
-
Received information regarding existing user personas created by the Airpals team which would feed into our target participant definition.
-
Determined 2 key areas of focus for our study: the pick-up and delivery scheduling form and the order confirmation email.
Step 2: Preparation
DEFINING TARGET PARTICIPANTS + DEVELOPING A RECRUITMENT STRATEGY
Using the information obtained during the kick-off meeting and the existing personas developed by the Airpals Team, we established our target participant profile. Given that our target participants needed to be employed and over 18 years old, we determined that the most effective recruitment strategy would be to email a screener to the Pratt Students distribution list and post it on our social media accounts.
DEFINING THE RESEARCH OBJECTIVE
Our research objective was to obtain both qualitative insights into user perceptions and behaviors, as well as quantitative data, with the intention of generating robust evidence to support our project goal.
DESIGNING THE TEST
To conduct our usability tests, we created a script that included a scenario, 2 tasks, post-task questions, questionnaire, and post-test questions, questionnaire. The post-task and post-test questions enabled us to collect qualitative data, while the post-task and post-test questionnaire facilitated the gathering of quantitative insights, as specified by the client and defined in our research objective.
Pre-test questions, task, scenario, post-task questions and post-test questions
CONDUCTING PILOT SESSIONS
To refine the script and questions, we conducted 5 pilot tests. During these tests, we closely observed participant behavior and identified any issues or areas for improvement in the script. For instance, we discovered that some of the task instructions were vague or repetitive and required revisions to enhance clarity. Moreover, we recognized that the second task outlined in the script did not provide significant value to the research and made the test longer than the desired duration. As a result, we chose to eliminate the second task for the usability sessions conducted with the recruited participants.
RECRUITING PARTICIPANTS
The screener was carefully crafted to ensure that only participants who matched the target demographic would qualify for the study. Out of 18 responses received, we selected 11 participants who best matched the target participant profile to participate in the usability test. Although 11 participants were recruited, 8 were retained as the best fit for the user group, and the results reflect the responses of those retained.
Recruited participant demographics
Step 3: Data Collection
SCHEDULING SESSIONS
We reached out to the 11 selected participants with additional information and provided them with a Doodle link to indicate their availability. Using the participants' preferred time slots, we scheduled Zoom meetings for the remote usability testing sessions. To ensure the smooth running of each 30-minute session, we assigned a moderator and a notetaker to oversee the testing.
CONDUCTING SESSIONS + COLLECTING DATA
We conducted 11 remote moderated user testing sessions over zoom. This allowed us to observe and record participant behavior as they shared their screen and navigated through the order form to complete the task. As previously stated, we used a combination of verbal questions and a Google Form questionnaire to collect both qualitative and quantitative data.
Screenshot from a user testing session conducted on Zoom
Step 4: Data Analysis
REVIEW + GROUP DATA BASED ON USABILITY ISSUES
After completing the usability testing sessions, we reviewed the data collected through the notetaker Google Form. To analyze the feedback, we utilized a digital grouping spreadsheet and applied a thematic analysis approach to classify the data based on common user frustrations, behaviors, or pain points observed during the testing.
IDENTIFY KEY USABILITY ISSUES
We assessed the issues identified in the previous step based on their severity and frequency, considering how badly the issue affected the usability of the form and how often it occurred during the test sessions. We then prioritized them accordingly to identify key issues and determine solutions for them.
Snapshot of the digital grouping Google Sheet
Step 5: Communication
REPORT + PRESENTATION
Based on our analysis, we identified key issues and recommendations which we presented to the client through a comprehensive report and presentation.
Overall Findings
Key recommendations
Recommendation #1: Streamline the flow for inputting parcel details
Recommendation #2: Clarify pick-up and drop-off instructions
Conclusion
We are confident that the recommendations we have provided will assist Airpals in improving the order form's usability and enhancing the overall user experience.
"I want to express my sincere gratitude for all the hard work and dedication you put into this project. Your contributions will be of tremendous help, and we're excited to implement your feedback."
- Joshe Ordonez, Airpals Founder
Next steps + Key learnings
Next steps
-
The option to place a one-time order as a guest, rather than requiring users to sign in.
-
Expanding the service to include regular users instead of only catering to businesses.
Key learning
Client management
Initially, our client had agreed to help us recruit 30% of the participants and had assumed that they could attend the sessions for the participants they had recruited and conduct a mock test session with our team. We discovered this misunderstanding during our mid-project check-in meeting, which was likely due to inadequate communication of the project's scope and limitations. To address the issue, we sent an email that clarified the project's scope and limitations, which helped to clear up the confusion. This experience taught me the importance of clarifying project scope and limitations at the outset of a project.