This article was written by Silvana Churruca between 2011-2014. Silvana is a UX Designer & Researcher with a strong multidisciplinary background: starting with Graphic Design, Arts and Communications, and later specializing in Cognitive Systems and Interactive Media, Product and Project Design Management and Communication Design Theory. Follow Silvana on her LinkedIn.
In daily agile development experience, sometime as a UX responsible it is quite difficult to follow the track to everything that has happened with the product and make sure that the user’s voice is there when needed. In my own experience bug assessment is a tricky one, especially if you are in a early development stage of a product. So trying to find a quick and practical way to measure a bug relevance from user experience point of view but also business/product goals, I develop this matrix for my own. After many bugs turn out to work quite good and result very practical both for me as the PO (Product Owner). So, Why don’t try it yourself?
- UXValue is a number resulting from the relation between Severity and Impact.
- Answers the follow questions:
How problematic is the issue? Impedes complete a task, or just difficult the user task), Significantly affect the brand or product image?
And finally, is the issue part of a key process/functionality for the product? or it is a secondary/tertiary feature?.
- UXValue has 4 dimensions: Critical, High, Medium and Low. And can be expressed on a 100-point scale.
Why use it?
- Helps to focus attention on the bugs that really matter from a user AND business point of view.
- Better adapt UX to quick agile iteration dynamics helping include UX metrics on a more fluid daily work.
- Follow a methodical process that allow other UX team members to measure any product bugs, following same common criteria.
How to use it?
UX value is a relation between Severity and Impact. Severity understood as combination of Impact on the task/goal users is performing or chasing and the Impact this could have on the Product/Brand equity, revenue, or profitability. Impact defined as the relevance the task/process have as part of an overall product experience.
UX Value Levels and principles:
- Critical: That which affects a first level process (high Impact) and makes it impossible to perform a task is always critical bug.
- High: That which affects a second level process (medium Impact) and makes it impossible to perform a task is hight bug.
- Anything that affects a third level process (low impact) and makes it impossible to perform a task is medium bug.
- Anything that affects a first level process (high Impact) and and doesn’t impede perform a task is medium bug.
- Anything that affects a third level process (low impact) and that doesn’t prevent perform a task is low bug.
- Anything that affects a second level process (medium Impact) that doesn’t prevent perform a task is low bug.
Calculating UX Value…
Crossing severity and impact factors using the above defined logic (UX value level-principles) the follow are the basic score combinations (first you will need to calculate severity and impact, see next)
ESTIMATING SEVERITY AND IMPACT
Severity understood as combination of Impact on the task/goal users is performing or chasing and the Impact this could have on the Product/Brand equity, revenue, or profitability.
- 4 (critical): An issue of critical severity would definitely result in users’ being unable to complete their intended task, as well as an immediate, noticeable, and significant negative impact on an organization’s brand equity, revenue, or profitability.
- 3 (high): A high-severity usability issue is one that would likely result in users’ not being able to complete their intended task and, from a business perspective, negatively affect an organization’s brand, revenue, or profitability.
- 2 (medium): A medium-severity usability issue would be likely to significantly impede or frustrate users, but probably would not prevent users from eventually accomplishing a task. It might also negatively affect an organization’s brand, revenue, or profitability.
- 1 (low): A low-severity usability issue would likely present some type of momentary or transient difficulty or result in confusion for users, but would not prevent users from accomplishing their task. It would have no discernible effect on an organization’s brand, revenue, or profitability.
Impact is defined as the relevance the task/process have as part of an overall product experience and product strategy.
Key point with Impact is: PO and UX team must agree in a impact-list of functionalities, process and tasks for the product, so UX team and PO will be applying same criteria when define de impact. Impact must be very close to the product strategy goals, and that’s way some items on the list may change over time. This is not a static list!.
- 3 (high): The issue affects a key process, task or feature (first level) in importance for the product (from the point of view of business strategy).
- 2 (medium): The issue affects a secondary importance process, task or feature to the product (from the point of view of business strategy).
- 1 (low): The issue affects a third level importance process, task / functionality to the product (low relevance) (from the point of view of business strategy).
UX VALUE into a 100-Points scale:
UX Value could be also expressed on 0-100 scale in order to o make the prioritizing task, even easier for items within a same dimension. I use the conversion table you see in the image, considering the frequency and amount of bugs for each level observed on the actual product, but I think the best is to adjusted to each product reality, so there are more values for the most common bugs level.
Having a 100-points scale also helps you to adjust the position of the bug considering other factors as the frequency of occurrence. For example, if 1 issue result to de High UXValue, but the frequency is not much higher, then you can apply a low value into the High range (closer to 75).
Primarily I use “UXvalue” naming in order to make clear the measure goes beyond usability. For an agile environment is quite important that this value reflects not only users point of view but also, the product/business goals and be compatible with the release planning.in traditional usability bibliography, severity ratings are used to determinate, as name suggest, the severity that an usability issue (usability problem) can have on the product experience. This severity it is defined mainly as the relation between three factors: impact on user experience, impact on business goals and predicted frequency of occurrence (measuring the user experience, Tullins and albert, 2008). Literature on these metrics is abundant and some authors use different scales or factors, I take what I found was practical for the agile experience that allow me to estimate bugs daily without affecting other tasks attention or the team velocity. That’s why I decide to use a different name for it, as my model have some differences with the models I based on, so we avoid misunderstandings. Suggestions and improvements are welcome!