Banner showing multiple screens of the poroposed redesign.

Freelance Gig

ACADEMIC Project UX RESEARCH ux DESIGN

Revamping the 5-star evaluation process for sellers providing services on online platform. The project's scope enabled us to introduce features for sellers to present improvements based on shoppers feedback.

01.

Overview

Understanding the Project Context

My Role

UX Research: Secondary Research, User Interviews, User Surveys, User persona development, User Scenario Storyboard, Affinity Diagramming, Video-making
UX Design: Ideation, Lo-fi Wireframes, Mid-fi Mockups, User Testing, Hi-fi Prototypes, Video making

Project DEtails

Duration: 1 months, Aug '23 - Sept '23
Type: Academic Design Project
Team: Harshavardhan Thokala, Graham Allgood, Prithvi Manjunatha, Sowymyashree Bharathan

Project Context

As part of the UX Design Graduate Studio, a core course in Purdue University's UX Design master's program, we were tasked with redesigning the evaluation process for freelancers on online platforms. The goal was to create a fair and balanced evaluation system that benefits both buyers and sellers, by reimagining the conventional 5-star rating system.

The main objectives of this project were to ensure potential sellers understand the seller’s journey. Design a evaluation system to assess sellers’ work fairly and enable them to use ratings for improvement and demonstrate progress to potential future sellers.

02.

Problem

How Might We Improve the Evaluation Process on Etsy to Ensure Gig Workers Receive Fair Assessments of their Service?

People are increasingly accepting the idea of availing services from sellers through online platforms. However, the method of evaluating sellers through reviews and ratings appears inconsistent, lacks detail, and does not accurately reflect the received service, thereby impacting the sellers' business.

The goal was to evaluate and propose a redesign of the effort required of the buyers to review sellers fairly while still providing constructive and accurate information about their service. Additionally, there is a need to anticipate how the shared information could potentially be exploited against the sellers.

03.

Solution

How do Sellers and Buyers Benefit from Our Etsy’s Redesigned 5-Star Rating System?

Feature #1

Comprehensive Rating Breakdown

The goal of the 'Comprehensive Rating Breakdown' feature is to enhance the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process, such as Product Description, Shipping, Product Quality, and Customer Service.

Feature #2

Guided Feedback

The goal of the 'Guided Feedback' feature is to educate and empower users about the vital role their reviews play in the Etsy community, aiming to guide buyers toward a more thoughtful and precise evaluation of their purchases. This is achieved by providing clear and contextual guidelines for each rating level within the feedback pop-up.

Feature #3

Rating Optimization

The goal of this feature is to create a toolkit within the Etsy platform that offers comprehensive guidance and strategies to help sellers enhance their ratings. This toolkit could include tips on improving product quality, enhancing customer service, optimizing shipping processes, and engaging effectively with buyers. Additionally, it could provide insights into interpreting customer feedback to identify areas for improvement, empowering sellers to take targeted actions and ultimately boost their ratings on Etsy.

Feature #4

Performance Progress Banner

The goal of this feature is to demonstrate the improvement in a seller's ratings over time. For example, it highlights enhancements in shipping ratings, such as from 3 to 4, over a designated time span. When a buyer views a product from that seller, a banner will display the shipping rating improvement achieved within the last 6 months, showcasing the seller's commitment to enhancing their service.

Watch the Pitch Video on YouTube

04.

Research

Exploring the Problem Space

Through a 2-step primary and secondary research process, we identified key issues that sellers and buyers typically face on freelancing platforms. Our initial insights into the problem space came from secondary research, where research papers and case studies in this domain provided a strong understanding of the prevalent issues on online freelancing websites.

Secondary Research

We conducted a review of existing academic research and industry case studies to gain insights into the problem space.

User Interviews

Through interviews with four Etsy sellers from across the United States, we delved deeper into Etsy's current review system and Star Seller Program.

User Surveys

Through user surveys, we gathered
insights from sellers across the world, which helped us comprehend issues from a wider perspective.

Our findings revealed that the 5-star rating system is still considered the most effective option for purchasing a product compared to binary and 10-star ratings. This is because it has a strong cognitive fit, offering users a clear understanding of the rating system and its associated meanings. Additionally, we explored weaknesses of the 5-star rating system, such as differences in mental models, and proposed information guidelines to ensure that buyers, sellers, and the platform share a common understanding of what each star rating represents.

Affinity Diagram

After conducting interviews and surveys with the main target user group, we affinity-mapped the insights to categorize the data and identify common issues that sellers were experiencing with the platform. We then synthesized the data to develop themes aimed at solving user problems and improving the rating and reviewing experience for both the seller and buyer user groups.

Affinity Diagram

Affinity Diagram of User Interviews

Pain Points and Causes

After gathering information on the most commonly experienced pain points by users, we focused on identifying the main reasons causing these issues.

Lack of Motivation from Buyers to Rate their Experience

There is a lack of motivation to provide ratings or reviews, as buyers perceive it as an additional task.

Buyers and Sellers View 5-Ratings Scale Differently

Perceptions of ratings vary depending on the buyer's background and previous experiences.

Absence of Way to Demonstrate Progress Based on Reviews

Etsy focuses exclusively on enhancing the buyer experience.

Buyers Unaware of the Impact of their Ratings on Sellers

Etsy’s primary emphasis has been on encouraging buyers to provide a rating rather than educating them about its impact.

Unclear Guidelines on how to Improve Ratings

Lack of assistance from Etsy leaves sellers to explore the solution to improve their ratings.

One Aspect of Buying Experience Affecting the Overall Rating

Negative experiences have a stronger impact on ratings than positive ones.

Mental Model Diagram and Identifying Common Problems
Mental Model Diagram

Mental Model Diagram

User Persona - Seller and Buyer group

From our insights and analysis of research data, we identified two main user groups: sellers (small business owners) and buyers. To better understand their needs, we developed personas for these stakeholders, which were crucial for efficiently identifying and communicating user needs. The personas, along with user scenario storyboard, further guided our process in identifying user needs and pinpointing key issues.

Seller User Persona

Sellers Use Persona

Buyer User Persona

Buyers Use Persona

User Scenario Storyboard - Sellers' and buyers' navigating the platform

To visualize the steps sellers and buyers take to achieve their goals on the online freelancing platform, we developed storyboards for each of the involved stakeholders. This process helped pinpoint issues that users might encounter while interacting with the product, allowing us to learn about user behaviors, motivations, and opportunities for our redesign.

User Scenario Storyboard for a Seller Persona

Storyboard - Seller's Perspective

User Scenario Storyboard for a Buyer Persona

Storyboard - Buyer's Perspective

05.

Ideation

Exploring and Validating Redesign Proposals through Iteration Process and User Testing

Low Fidelity Wireframes
Lo-fi wireframe of the proposed features
Lo-fi wireframe of the proposed features
Lo-fi wireframe of the proposed features

To translate our findings from primary and secondary research, we developed solutions for the identified pain points through low-fidelity wireframes. Narrowing down to the most feasible, functional, and intuitive design, the wireframes presented above showcase a few of our redesigns.

Mid Fidelity Wireframes
Mock-up of the redesigned review and ratings interface

Mock-up of the redesigned review and ratings interface

Pop-up message prompting buyers to communicate their issue with seller

Pop-up message prompting buyers to communicate their issue with seller

Left: Etsy providing guidelines for sellers to improve their ratingsRight: Etsy informing buyers about sellers imorovement based on reviews

Left: Etsy providing guidelines for sellers to improve their ratings
Right: Etsy informing buyers about sellers imorovement based on reviews

Evaluating the proposed solution

To validate our proposed redesign and further discover opportunites fro improvements, we conducted an unmoderated usability testing with sellers on Etsy platform using the mid fidelity mock-ups. The following insights were provided by the participants influencing our final design proposal:
- Breaking down the initial review screen based on categories and progressively disclosing the rating and review tasks.
- Including a progress bar informing buyers about their current state in the reviewing process.
- Reorder reviews interface to categorize tasks better.
- Users feared losing review progress by selecting "Contact Seller."
- Users were unsure about the next steps after the star information screen.

06.

Design

Redesign Suggestions Based on User Testing Data

Feature 1: Comprehensive Rating Breakdown

The goal of the "Comprehensive Rating Breakdown" feature is to elevate the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process such as Product Description, Shipping, Product Quality, and Customer Service.

Comprehensive Rating User Flow

Comprehensive Rating Breakdown

Feature 2: Guided Feedback

The goal of the "Guided Feedback" feature is to educate and empower users about the vital role their reviews play in the Etsy community aiming to steer buyers towards a more thoughtful and precise evaluation of their purchases. By providing clear and contextual guidelines for each rating level within the feedback pop-up.

Guided Feedback Interfaces

Guided Feedback

Feature 3: Rating Optimization

The goal of this feature is to Create a toolkit within the Etsy platform that offers comprehensive guidance and strategies to help sellers enhance their ratings. This toolkit could include tips on improving product quality, enhancing customer service, optimizing shipping processes, and engaging effectively with buyers. Additionally, it could provide insights into interpreting customer feedback to identify areas for improvement, empowering sellers to take targeted actions and ultimately boost their ratings on Etsy.

Rating Optimization Screen

Rating Optimization

Feature 4: Performance Progress Banner

The goal of this feature is to exhibit the improvement in a seller's ratings over time. For example, it highlights enhancements in shipping ratings from for instance 3 to 4 over a designated time span. When a buyer views a product from that seller, a banner will display the shipping rating improvement achieved within the last 6 months, showcasing the seller's commitment to enhancing their service.

Performance Progress Banner

Performance Progress Banner