Melissa ShiEnterprise research · Evaluative research · Product strategy
Evaluative Research on Cost Analysis Tool
Resetting product direction two months before release through targeted research and actionable evidence.
Problem
After product-owner and subject-matter-expert turnover, the team was approaching release without a shared understanding of users, value, or success.
Response
I sequenced stakeholder interviews, heuristic evaluation, usability testing, and workflow analysis to identify risks and turn evidence into product decisions.
Impact snapshot
Research turned release uncertainty into an actionable direction
The work answered long-running questions about the tool's value, reopened communication with primary users, and gave the team a prioritized backlog for continued development.
Design context
A usability request revealed a product alignment problem
The Cost Analysis Tool helped a production site see where value was eroding and where costs could be reduced. Development started in 2021, but ownership changes left the team without a reliable view of the intended users, their work, or the product's definition of success.
Unclear audience
Different stakeholders named different primary users and access needs.
Unclear value
The team could not consistently explain how users would apply the tool in cost decisions.
Unclear readiness
A planned release date existed, but shared success criteria and workflow evidence did not.
Before evaluating screens, did the team agree on the problem the product needed to solve?
Decision 1
Diagnose alignment before usability
A conventional usability test would have measured an interface against unstable assumptions. I began with six stakeholder interviews to establish where the product vision diverged.
Test the product story first
I asked the product owner and subject-matter experts to describe the goal, users, development stage, success criteria, and expected access. Comparing the answers exposed a stark vision gap.
This reframed the assignment from validating a nearly finished interface to determining whether the product was ready to deliver the intended value.
Decision 2
Earn room for deeper research
The team initially wanted a fast expert review. I used that constraint as a first step, not the final answer.
Start with a heuristic evaluation
The review identified immediate interaction issues while helping me learn the product well enough to design realistic tasks. A structured findings log made the risks visible and traceable.
Use early evidence to expand the study
The findings demonstrated what expert review could reveal and what it could not. That distinction built credibility and helped secure access to users for formal usability testing.
Convert disagreement into research questions
The gaps became four decision-focused objectives:
- Evaluate MVP usability
- Assess whether value drivers were sufficient
- Understand tool usefulness in real work
- Determine appropriate access
Decision 3
Focus limited research where it could change decisions
Eight business teams used the underlying process, but the schedule could not support interviewing every group. I selected participants for decision coverage rather than broad representation alone.
Role coverage
Two Role A and four Role B participants represented the people expected to interpret and act on cost information.
Fresh perspective
One new subject-matter expert challenged inherited assumptions and added domain context.
Mixed evidence
Task performance exposed breakdowns, SUS provided a benchmark, and follow-up interviews clarified value and workflow needs.
The combination was deliberate: observation showed what people could do, the scale made usability comparable, and interviews explained why the tool did or did not fit their work.
Decision 4
Turn findings into product direction
The study produced more than an issue list. I connected interface problems, missing capabilities, access needs, user characteristics, and workflows so the team could decide what to build next.
Prioritize across three lenses
I combined heuristic and usability findings in one backlog, then ranked them by user impact, business priority, and technical feasibility. This separated urgent usability fixes from feature enhancements and new capabilities.
Replace assumed access with observed needs
Testing validated parts of the stakeholder model and revealed a restricted user level for people who needed selected data and editing rights without full access.
Show where the product changed the work
I mapped the current business process, the tool's present contribution, and a future workflow if user requests were met. The comparison made value gaps and automation opportunities concrete.
Results and reflection
The team left with evidence it could act on
Prioritized development
The team received an actionable backlog, and several recommendations were underway within two months.
Continued investment
Evidence about usefulness and future value supported the case for continued product development and funding.
Shared product model
User profiles, access needs, and current and future workflows gave the team a common basis for decisions.
Reopened communication
The study reconnected the project team with primary users and created a repeatable way to collect and prioritize feedback.
My main takeaway was that rigor is not the number of methods used. It is choosing each method for the uncertainty it resolves, then connecting the evidence to a decision the team can make.
Keep exploring