UX Value As A Tool For Prioritizing Bugs
This article was written by Silvana Churruca between 2011-2014. Silvana is a UX Designer & Researcher with a strong multidisciplinary background: starting with Graphic Design, Arts and Communications, and later specializing in Cognitive Systems and Interactive Media, Product and Project Design Management and Communication Design Theory. Follow Silvana on her LinkedIn.
In daily agile development experience, sometimes as a UX responsible, it isn't easy to follow the track to everything that has happened with the product and ensure that the user’s voice is there when needed. In my own experience, bug assessment is a tricky one, especially if you are in a product's early development stage. To find a quick and practical way to measure a bug relevance from a user experience point of view and business/product goals, I develop this matrix for my own. After many bugs, this turns out to work quite well, and the result is efficient both for me as the PO (Product Owner). So, Why don’t you try it yourself?
What is it?
- UXValue is a number resulting from the relation between Severity and Impact.
- Answers the follow questions:
- How problematic is the issue? Impedes the completion of a task or adds difficulty to the user task) Significantly affects the brand or product image?
- And finally, is the issue part of a critical process/functionality for the product? Or is it a secondary/tertiary feature?.
- UXValue has four dimensions: Critical, High, Medium, and Low. And can be expressed on a 100-point scale.
Why use it?
- It helps to focus attention on the bugs that matter from a user AND business point of view.
- Better adapt UX to quick, agile iteration dynamics helping include UX metrics on a more fluid daily work.
- Follow a methodical process that allows other UX team members to measure any product bugs, following the same standard criteria.
How to use it?
UX value is a relation between Severity and Impact. Severity understood as a combination of Impact on the task/goal users is performing or chasing and the Impact this could have on the Product/Brand equity, revenue, or profitability. The impact is defined as the relevance the task/process has as part of overall product experience.
UX Value Levels and principles:
- Critical: That which affects a first level process (high Impact) and makes it impossible to perform a task is always a vital bug.
- High: That which affects a second level process (medium Impact) and makes it impossible to perform a task is a high bug.
- Anything that affects a third level process (low impact) and makes it impossible to perform a task is a medium bug.
- Anything that affects a first level process (high Impact) and doesn’t impede performing a task is a medium bug.
- Anything that affects a third level process (low impact) and that doesn’t prevent perform a task is a low bug.
- Anything that affects a second level process (medium Impact) that doesn’t prevent performing a task is a low bug.
Calculating UX Value…
Crossing severity and impact factors using the above-defined logic (UX value level-principles), the following are the basic score combinations (first you will need to calculate severity and impact, see next)
ESTIMATING SEVERITY AND IMPACT
Severity understood as a combination of Impact on the task/goal users is performing or chasing and the Impact this could have on the Product/Brand equity, revenue, or profitability.
- 4 (critical): An essential issue of severity would result in users’ being unable to complete their intended task, as well as an immediate, noticeable, and significant negative impact on an organization’s brand equity, revenue, or profitability.
- 3 (high): A high-severity usability issue is one that would likely result in users’ not being able to complete their intended task and, from a business perspective, negatively affect an organization’s brand, revenue, or profitability.
- 2 (medium): A medium-severity usability issue would likely impede or frustrate users significantly, but probably would not prevent users from eventually accomplishing a task. It might also negatively affect an organization’s brand, revenue, or profitability.
- 1 (low): A low-severity usability issue would likely present some momentary or transient difficulty or confuse users, but would not prevent users from accomplishing their task. It would have no discernible effect on an organization’s brand, revenue, or profitability.
The impact is defined as the relevance the task/process has as part of overall product experience and product strategy.
Impact's critical point is: PO and UX team must agree in an impact-list of functionalities, processes, and tasks for the product, so the UX team and PO will be applying the same criteria when defining de impact. The impact must be very close to the product strategy goals, and that’s why some items on the list may change over time. This is not a static list!.
- 3 (high): The issue affects a critical process, task, or feature (first level) in importance for the product (from business strategy).
- 2 (medium): The issue affects a secondary importance process, task, or feature to the product (from business strategy).
- 1 (low): The issue affects a third level importance process, task/functionality to the product (low relevance) (from the point of view of business strategy).
UX VALUE into a 100-Points scale:
UX Value could also be expressed on a 0-100 scale to make the prioritizing task even easier for items within the same dimension. I use the conversion table you see in the image, considering the frequency and amount of bugs for each level observed on the actual product. Still, I think the best is to adjust to each product reality, so there are more values for the most common bugs level.
Having a 100-points scale also helps you adjust the bug position considering other factors as the frequency of occurrence. For example, if 1 issue results in de High UXValue, but the frequency is not much higher, you can apply a low value to the High range (closer to 75).
Primarily I use “UXvalue” naming to make clear the measure goes beyond usability. For an agile environment, it is crucial that this value reflects the user's point of view and the product/business goals and be compatible with the release planning. In traditional usability bibliography, severity ratings are used to determine, as the name suggests, the severity that a usability issue (usability problem) can have on the product experience. This severity it is defined mainly as the relation between three factors: impact on user experience, impact on business goals, and predicted frequency of occurrence (measuring the user experience, Tullins, and Albert, 2008). The literature on these metrics is abundant, and some authors use different scales or factors. I take what I found was practical for the agile experience that allows me to estimate bugs daily without affecting other tasks' attention or the team velocity. That’s why I decide to use a different name for it, as my model has some differences with the models I based on, so we avoid misunderstandings. Suggestions and improvements are welcome!