Optimizing ExpertVoice Product Reviews Research

Role

Product Research Intern

Skills

User Interview, Survey, Stakeholder Mgmt

Timeline

2024 Summer, 6 weeks

Team

2 Designers, 1 Engineers, 1 Researcher, 2 PM, 1 BA
As the first and only product researcher on a design team of 7 at ExpertVoice, I led a project to improve product reviews. Using a mixed-method approach, I combined insights from qualitative interviews with data from quantitative surveys to explore ways to improve both the quality and quantity of reviews. The findings informed strategic recommendations for prioritizing review features, with a projected 30% increase in review quantity and a 20% boost in review quality.
1
Project Context
2
Methodology
3
Key Takeaways
4
Next Step
PROJECT CONTEXT

Product Reviews for ExpertVoice

ExpertVoice is an e-commerce platform for the pro-deal industry, where experts in specific fields purchase products with company-provided discounts.
In E-commerce space, product reviews is a critical and heavily influences consumer behavior, with over 90% of users relying on reviews to make purchasing decisions. On ExpertVoice, users spend an average of 20 minutes reading at least 11 reviews before buying, so improving review quality and quantity can enhance product understanding, boost conversion rates, and drive revenue growth.

Project Goal

The project aims to address the decline in review quality and quantity on ExpertVoice, with a consistent YOY decrease of 10-30% in quality and in the past six months and 36.26% YOY decrease in quantity just in June. In this project, I wanted to uncover: How can we improve the review system to enhance the consumer shopping experience, product understanding, and user engagement?

Result

This research, completed in early September, aimed to increase review quantity by 30% and quality by 20%, impacting over 302k users. Some of the recommendations have already been implemented, including an AI-generated product summaries, which began internal testing later that month.
My Research Process

Research Goals and Questions

METHODOLOGY

Desk Research

I began this project by collaborating with a business analyst to review internal research and Tableau reports. To understand best practices, I explored online resources like Baymard Research. Additionally, I conducted a competitive analysis to see how other platforms strategically design and utilize product reviews.

See Report

User Interview

To understand experts’ needs, motivations, and challenges, I conducted 30-minute interviews with 10 participants. I developed an interview protocol and collaborated with a UX writer to recruit participants through Pendo on ExpertVoice. After that, I transcribed the recordings and analyzed the data using affinity diagramming.

See Report

Workshop with Stakeholders

To prioritize findings, form hypotheses for further testing, and gather remaining questions, I organized a workshop with key stakeholders. Through 3 structured activities, we consolidated our understanding of the problem space and narrowed the scope. As a result, we developed three hypotheses and identified four key questions for the next steps.

See Figjam Board

Survey

Using the outputs from the workshop, I designed a short survey to validate the findings and hypotheses for general EV users while also addressing the remaining questions. The survey, created on Lyssna, consisted of three sections. I deployed the survey on the ExpertVoice website through Pendo, targeting to 150 responses to achieve a 95% confidence level with an 8% margin of error.

See Survey
CHALLENGE 1

Initiating the project: align with and manage key stakeholders

After reviewing previous internal research and Tableau reports, I found no studies directly related to our current review feature, despite months of declining product review metrics. Recognizing this gap, I saw an opportunity to initiate this project, which also aligns with the company’s recent strategic focus on enhancing experts' shopping experience on the consumer side.
Organizing stakeholders for the project was challenging, but with support from my mentor and manager, I brought together product managers, designers, and engineers who had previously worked on product reviews. I engaged them to gather their insights on user needs and potential improvements for the review system.
For the rest of the project, I collaborated with this group of stakeholders to prioritize interests, formalize hypothesis, and ideate solutions regularly in the form of workshops.
CHALLENGE 2

Planning the research and creating standardized process

While the design team already had some research templates, as the first Product Researcher on the design team, I aimed to establish a standard operating procedure for user research to set an example for future projects, building on the team’s existing templates.
This process included thoroughly documenting all research materials, planning every step in advance, and clearly defining expectations with stakeholders. I ensured they understood their roles, what I needed from them, and what they would receive in return, fostering transparency and collaboration throughout the research process.

Key Takeaways

Experts on ExpertVoice value insightful reviews, especially for time-sensitive decisions, as reviews play a crucial role in their purchasing process.

Experts are internally motivated reviewers who trust ExpertVoice and enjoy recommending products, so we should leverage their internal motivations and foster a cycle of engagement and exchange.
Revised Jobs To Be Done

When I show I’m an expert, I want my voice to be heard, so I feel validated in sharing my knowledge, helping others make informed decisions, and contributing to a trusted community.

RESULT

Experts value quick answers and evidence to their questions. While experts prioritize product quality and durability, other factors vary by category.

🔍 Insight:
In the interviews, experts emphasized the importance of verifying that a product will meet their specific needs, which requires detailed and contextually relevant reviews. Survey results show that 63% of experts prioritize learning about the product's quality, durability, and value.

Top 3 reasons why experts read reviews

Top 4 factors experts care about

While these factors are important across all product categories, the significance of other factors depends on the category. Below is an example of they they vary:

Top factors for Health & Nutrition

Top factors for Apparel & Accessories

When reading reviews, experts start by filtering based on the number of reviews and ratings but then dive deeper into details, focusing on the most helpful reviews and comments before making a decision. One interviewee mentioned: "The rating is a quick, easy see and the comments are the supportive material." (P7)

Prioritization of existing/wishlist features created based on interview results

🔑 Solution
Prioritize detailed descriptions, enhance review readability, and apply dynamic attributes by category.
RESULT

After using a product, experts want to share their experiences and insights through reviews to help others in the community make better decisions and feel a sense of reciprocity.

🔍 Insight:
As quoted here: “I look at the community overall and and think about the people who've written reviews that I've read. That's why, in part, I bought that product. I feel like I should reciprocate” (P4). Essentially, what’s motivating people is the sense of reciprocity, where people are receiving and giving things from the community.

What motivates you to write reviews on ExpertVoice?

Promoting a cycle of exchange

Moreover, a lack of reminders might be the reasons for low review quantity. It usually takes at least a few days or even a few months for experts to finish experiencing the products, so they will forget. They prefer follow up emails or in-site reminders to remind them.
🔑 Solution
Motivate experts to write more reviews by promoting a sense of reciprocity and achievement. Collaborate with brands to improve their engagement with experts’ reviews. Strengthen reminders to encourage experts to write reviews after making a purchase.
RESULT

Experts don’t communicate: low engagement with the review system is turning people away from ExpertVoice.

🔍 Insight:
While 69% of the survey respondents believed that ExpertVoice reviews is more valuable and beyond than other platforms, they also prefer other platform for their availability of diverse reviews (43%), review quantity (41%), and reviewer credibility (37%). However, other platforms also suffered from inaccurate or biased reviews (54%), reviewer credibility issues (46%), and lack of details (31%). Therefore, if the review number is too low, they choose to collect reviews elsewhere when purchasing on ExpertVoices, and we lose customers.

Other platforms experts use for reviews

Experts are turning away from low engagement

Still, from the interviews, experts noted that ExpertVoice reviews are valuable because they are written by vetted experts. By leveraging this strength, we can highlight the value of ExpertVoice reviews, making them more impactful for both current users and potential new users.
🔑 Solution
Design strategies to boost review engagement and build a cohesive expert community for communication. Help users see the value of ExpertVoice reviews by quantifying the expertise.
NEXT STEP

Project Outcome & Impact

The outcome of my project was well-received, as I had the opportunity to present my findings to company executives and at a weekly town hall, where many colleagues expressed their appreciation. The project received positive feedback from stakeholders, who appreciated the insights and enjoyed collaborating throughout the process. Many expressed a greater interest in incorporating research into their work moving forward. My product manager is already working to integrate some of the recommendations into the product roadmap, highlighting the value and impact of this research in driving meaningful improvements.

Metrics

Completed in early September, the research aimed to increase review quantity by 30% and quality by 20%, directly impacting over 302k users. Some recommendations, such as AI-generated product summaries, have already been implemented and entered internal testing later that month, marking the first steps toward improving the review experience and driving meaningful results.

What should we do next?

As the next steps for the project, I recommended enhancing the social features of reviews to foster more interaction and engagement, improving the review writing experience to make it easier and more intuitive for users. Additionally, boosting incentives for writing reviews can encourage more participation, while promoting the unique value of our expert-vetted reviews can attract and retain users by building trust and credibility.

My takeaway

Building Stakeholder Engagement

When I first started this project, unlike other interns assigned to direct product teams, I had to create my own group of stakeholders and manage them. When I invited relevant stakeholders to the kickoff meeting, they didn’t provide the feedback I needed and often didn’t follow through on action items afterward, which was frustrating.
To address this, I sought guidance from my mentor and manager, reached out to experienced researchers for advice, and implemented clearer action items while setting better expectations with the team. Although it wasn’t entirely successful—since this project wasn’t part of their planned workload—I took ownership and drove it to completion.

Overcoming Imposter Syndrome

As the first and only product researcher at a company with low research maturity, I often suffered from imposter syndrome, questioning whether my work was valued or impactful. Without established research processes or a clear framework, I had to carve out my role and justify the importance of my work. To overcome this, I focused on small wins—sharing meaningful insights, collaborating closely with stakeholders, and showing how research could drive decisions. With time, I learned to trust my expertise and realized that being in uncharted territory was an opportunity to shape the research culture. The positive feedback from stakeholders reminded me that my contributions mattered and built my confidence to keep advocating for research.

Other Projects

Negotium: Evaluating User Engagement and Content Quality
Understanding and evaluating investors’ needs with Earning Report feature