It was a part of the Voyager Project during the Alibaba – Lazada migration time. We revamped the design for the Lazada Android application and wanted to test the design in Indonesia, Malaysia, Thailand, and Vietnam. We targeted 20 participants in each country – 10 Lazada existing customers and 10 non-existing customers.
The UX Design team was accountable and responsible for the Vietnam market. We worked collaboratively with different departments (CX, CS, IT, Finance, Legal, Admin, UX Design, and UX Research) to conduct the research sessions. The team started with a Research Guide from our Lead UX Researcher. We recruited participants, repaired legal contracts & cash intensives, set up research laps, learned the test scripts, did a trial test with my UX team, interviewed the participants, and organized & documented the findings. Below is my summary of our 2 weeks of preparing and conducting the usability test in Vietnam.
The primary part of the research was to learn:
- How do users navigate to find the products they need from the Homepage
- How do they scan, filter, and browse the Search Results Page and Category Page
- How do they interact with the Product Page
- How do they interpret the product tiles
- Their thoughts and experiments toward the test app compared to the current app.
The secondary part was to learn the insights of:
- Their online shopping behaviors
- The applications that they use daily.
- Their expectations for an application that’s personalized just for them.
Finding & Screening Participants:
Our team developed a list of demographic questionnaires for screening sessions to identify suitable participants for our studies. We coordinated with our local Customer Experience (CX) team to access the customer’s database to call for a necessary number of participants.
Meeting Confirmations:
Once we got a list of 22 suitable participants and their time slots, the local Customer Support (CS) team was in charge of following up with the participants a few hours before our meetings. The team was also responsible for updating the timetable for the 5-day usability test.
We took the opportunity to utilize all facilities as we could to set up spaces for moderators, participants, and observers. Two cameras were used, one was focused on how participants interacted with the test phone, and the other was for an overviewing view. All interview sessions were recorded and live-streamed via our internal video meeting web application. Interactions with test phones and card-sorting activities were recorded and documented.
Usability Test Guide:
There were scripts with questions and scenarios given by our UX Research Lead. Everyone on the team had a way to understand the document. The pictures above were my way of understanding the questionnaires and activities. I cut the questions, scenarios, and tasks into pieces, then mapped them together in ways that I could communicate naturally with the participants in Vietnamese.
Trial Session:
We had 3 UX designers and 1 UX researcher leading the usability test sessions in 1 week. We ran a trial test before the beginning of formal sessions. We had members of the team standing in as the participants, moderators, and observers. The purposes were to get everyone clear on the research goals, scenarios, and activities of the sessions and get aligned on the user flows from the Homepage to the Product Page.
Introduction:
I welcomed and informed the participants that we wanted to study their experiences with the new-look of our product and explained to them the goals and purposes of the study. I also informed them about the legal agreement and the sessions being video-recorded.
Observe, Listen and Ask “why”:
My tasks were mostly guiding the participants through the test flows and activities. I let them do most of the talking. I was just listening, observing, taking notes, and asking “why”. I kept encouraging them to share their thoughts when they interacted with the test phone. There were many times our conversations led to unrelated topics. The challenge was to lead the conversations back on the test flows.
Record and Take Notes:
I took photos of card-sorting activities. I recorded the sessions using Lookback as a backup for the team. I also used Screen Recording on my computer as a personal backup, just in case I missed something. In addition to recording the sessions, I also created quick notes with a bulleted list of findings after every session.
Share:
When the sessions were over, we gathered together to discuss and share our findings. We segmented them into 3 different groups – Homepage, Search Results & Category, and Product Page. We discussed which findings were more related to our research goals. We then drafted out the key findings (both positive and negative) on PowerPoint to share with other teams.
Refine and Define:
I went through my notes again and watched the records again to find more answers related to the research goals. I started to rewrite the sections that were clearest to me and added background information to form up stories of all participants. By rewriting the findings, I began to see patterns of information that helped me to define our user’s personas. I also began to see the different journeys of our users when they do online shopping.
Direct:
Apart from learning about our users’ experiences with the product features, I also gained insights into their behaviors, interests, and online activities. I then discussed the insights with our Lead UX Researcher and other UX Designers. By doing this, my designer mind began to form context and use cases for our product. The challenge from here was to get PMs and the UX Design team to discuss and come up with design directions and wireframes to enhance our product features.
It was my first time conducting a usability test. It was an interesting experience. I made lots of mistakes. I have discovered the connections between the users and our product. I finally know the reasons for who I design for and why my design should go in such directions to meet the user’s needs & wants.
Danchi, the designer