The client is the world’s most successful US/UK university admissions support consultancy, with their students 4x more likely to gain acceptance to the Ivy League, Oxford, Cambridge and more.
Their tutors and mentors are the students at, or graduates from, Harvard, Stanford, Princeton, Yale, Columbia, MIT, Oxford, Cambridge, LSE, UCL and more. They have some of the most compelling academic and extracurricular résumés of any group of tutors and mentors in the world.
Problem
Currently users are asked to leave feedback after each online tutoring session they have with the tutors. The feedback form consists of the 1-5 scale to grade the experience and free entry form field for the additional comments.
There are two main problems: first, the distribution of feedback seems impossibly narrow, almost as if 5 is normal, 4 is bad and less than 4 is catastrophically terrible.
Second, users use free form field to leave comments that are not directly related to the feedback. Example: Can we have a look at x question for the next lesson. Additionally with the growing volume it is getting hard to process all the free form feedback. Free form feedback is confidential and is not shared with tutors. Only internal Crimson staff get access to this information, whereas overall rating summary is shared to all user groups.
Business needs
Being able to process user feedback quickly and use it to efficiently rate tutors to get a detailed picture of the tutors’ performance across different metrics.
Research and Analysis
Given the situation, I used the unmoderated contextual inquiry method for this purpose.
The student is 15 years old. He uses a phone to take the lesson and the micro-survey. Outside school, the student uses Snapchat, IG and YouTube in addition to other platforms.
Research surfaced that the students had doubts about the lesson that he would like the tutor to address in the next session. He currently uses the free form to post these questions. The student is unaware that the message posted in the feed form is not seen by the tutor.
The micro-survey appears just after the lesson has culminated. The current feedback system uses numbers (1 to 5) or stars for measuring purposes in the micro survey. (Acquiescence bias where the user tends to agree with the question, is observed)
Are we making it easy for users to give feedback?
Currently users are asked to leave feedback after each online tutoring session they have with the tutors.
The feedback form consists of the 1-5 scale to grade the experience and free entry form field for the additional comments.
Is the feedback relevant to us?
Problem
There are two main problems: first, the distribution of feedback seems impossibly narrow, almost as if 5 is normal, 4 is bad and less than 4 is catastrophically terrible.
Second, users use free form field to leave comments that are not directly related to the feedback. Example: Can we have a look at x question for the next lesson. Additionally with the growing volume it is getting hard to process all the free form feedback. Free form feedback is confidential and is not shared with tutors. Only internal staff get access to this information, whereas overall rating summary is visible to all.
Can we measure the feedback for the right purpose?
Business needs
Being able to process user feedback quickly and use it to efficiently rate tutors to get a detailed picture of the tutors’ performance across different metrics.
No product is devoid of experience.
It is therefore important to solutionise wearing two different thinking hats, product designer and experience designer.
As a product designer, I make amendments to the user flows reflected in the individual screens.
As an experience designer I look at the overall repercussions of the design changes on the user experience and aim to minimise the collateral damage caused.
Problem 1
The distribution of feedback seems impossibly narrow, almost as if 5 is normal, 4 is bad and less than 4 is catastrophically terrible.
Solution
Replace numbers with emojis
Feedback emojis are one of the fastest-growing and popular feedback collection method that is now used by businesses of all sizes.
Emogi are an everyday artefact with which we relate common emotions and reactions. Assigning an empathy character to a number will better resonate with the user’s mindset and encourage an honest feedback when required (when score is 3 or below).
This may also sit well with the digital native users who use emojis regularly as a means to sum up words and communicate emotions.
Emojis are extensively used by people of all ages. More than 5 billion emojis are sent daily on Facebook Messenger. Students are naturally tuned to using emojis and use them without hesitation or giving it a second thought
Since emoji feedback doesn’t look like a survey and doesn’t need more than a few seconds, the survey no longer feels formal
Emoji feedback is user-friendly. It requires minimal work from the user. They just have to select an emoji to describe their initial reaction without having to deal with numbers
The design of the emogi can be customised to suit the other elements in the design system
Get feedback from students without asking them to participate in a survey. They will select an emoji even if you don’t ask them as it suits their mental model
Emogis remove the ambiguity and confusion on the attributes of numbers. High level emotions are easy to capture with emogi as against a rating system with stars or numbers
Problem 2
Users use free form field to leave comments that are not directly related to the feedback.
Solution
Explain purpose of feedback and allow for alternate means for students to communicate with the tutors
Inform the student that the feedback is strictly confidential and is not visible to the tutor to eliminate hesitation and fear.
Provide alternate means of communication between student with the tutor.
Problem 3
Growing volume it is getting hard to process all the free form feedback.
Solution
Restrict availability of free form to feedbacks below threshold (unsatisfactory or below) to limit the number of feedbacks received and ensure the feedbacks are relevant to the purpose set out.
If satisfactory, poor or very poor, the feedback box appears that will ask the student for a more detailed response and scope of improvement.
With evidence that users want to rectify their problems as soon as they arise over leave a feedback after a positive interaction, a commonly seen trait in the industry, No detailed feedback is sought if the score is 4 or more.
In the current trend of experience design, CSAT is associated with products like UBER. These services do not ask for a detailed feedback unless the score is between 1 and 3.
Wireframes
Reflections
This task was a great opportunity to look at the experience the product provides while meeting the expectations of the users.
As I began designing for the students, I realised there was a lot of scope to improve the current feedback collection method.
Emogis are no longer informal or comical and have been adopted by several platforms and businesses as a means of feedback collection.
The use of emogi shifts the conversation from purely a metric to a reaction sought after the session. This I believe this is a more user-centered process because it aims to mimic the current mental models of the student. It is therefore that these small changes make a significant difference to the student’s experience with the product.
I love designing with the student in mind. Talking to young minds gives an insight in to the digital that is yet to emerge. Overall, I enjoyed the task and look forward to the opportunity of working with Crimson Education.
Depending on the length and the occurrence of the interaction, CSAT may feel like a repetitive task for the user. It is therefore important for the exercise to be short and purposeful. Skimming back on the number of clicks required to perform a task reduced cognitive load and increases engagement.
UX writing needs to indicate clearly — feedback solicited is on the tutor and not the content or any other variables of the session.
Overall, CSAT is a great tool that should be used for both relationship and touch point metric. However blending this exercise with infrequently but periodically run NPS metric between modules may be more advisable to gain feedback on the over all performance of the tutor. This is likely to be more reliable in the long run where the student has had the time to process and reflect upon the content of the module.
It is also an opportunity to solicit feedback on various aspects of the programme all at one go from the user, better utilising their time for the purpose.