Let's talk
UX strategy and measurable UX quality
Making UX quality measurable – and therefore steerable.
For product management and product leadership at vendors of B2B and expert software, and for project leads introducing an expert system. We define what good usability concretely means in your system, measure it with established instruments and translate findings into principles, rules and goals a team can carry. The result: UX moves from a matter of taste to a steering variable.
In short
UX strategy at Beabstracto means that UX quality in complex expert software is defined, measured and translated into decisions. We distinguish pragmatic quality – task success, efficiency, error rate – from hedonic quality in Hassenzahl's sense – stimulation and identification –, capture both with instruments such as the User Experience Questionnaire (UEQ), the System Usability Scale (SUS) and the HEART framework, add adoption and usage metrics, and build benchmarks, quality rituals and OKR inputs from them. Our clients are vendors of B2B and expert software and organisations introducing an expert system.
We work as UX strategist and UX lead at the interface of product management, engineering, UX research and the business department – in Munich, remotely and on site by arrangement.
At a glance
- Role
- UX strategist, UX lead, UX consultant – also as external reinforcement of a product team or as business analyst with a UX focus
- Services
- UX audit and benchmark, measurement system (KPIs, OKRs, quality rituals), decision system (UX tensions, design principles, design rules), implementation support with adoption metrics
- Industries
- Vendors of banking, tax and reporting software, DMS/ECM and enterprise software, industrial software and control systems; implementation projects at asset management companies (KVGs), depositaries and industrial companies
- Methods
- UEQ and UEQ-S, SUS, Google HEART, usability tests with task success, time on task and error rate, UX benchmarks over time, OKRs, UX audit checklists, Figma prototypes
- Location
- Remote, Munich, on site by arrangement
- Languages
- German, English, Portuguese
Why does UX quality in expert software fail – even though everyone wants it?
Rarely because of the team's ability. Usually because nobody has defined what "good enough" means, so UX cannot be prioritised against features. These are the situations we hear about most often.
UX is decided by opinion
Product management, development and design each have their own idea of what users need. Without a shared measure, the loudest voice or the latest customer complaint wins. Every discussion starts again from scratch, screen by screen.
Tickets instead of metrics
The only indicator of usability is the support volume. That shows pain, but not cause – and it shows nothing about the users who quietly keep an Excel list next to the system.
New module, no baseline
A reporting module or a configuration tool goes live and nobody measured beforehand. Whether it runs better or worse than the old way can no longer be proven after go-live.
Adoption remains a guess
Licences are bought, training delivered. How many key users and case workers actually use the system for the intended process – and how many work around it – the project lead does not know. The steering committee asks anyway.
SLA risk visible only when the deadline breaks
Regulated workflows have fixed deadlines. The system often shows the backlog only when it can no longer be recovered. Leading indicators such as time to error detection or rework rate are missing from the product and from the reporting.
OKRs without UX, UX without OKRs
The company steers with goals and metrics, but usability does not appear in them. So UX remains an acceptance step at the end – not part of the goal system in which capacity is decided.
The instruments we measure with
No success figures of our own – those remain confidential. Instead, the key facts about the instruments and models we work with.
- 26 items, 6 scales
- User Experience Questionnaire (UEQ): attractiveness, perspicuity, efficiency, dependability, stimulation, novelty; short form UEQ-S with 8 items
- 10 items, 0–100
- System Usability Scale (SUS): ten statements, one overall score, technology-agnostic – suited to comparisons across releases and systems
- 5 dimensions
- HEART: Happiness, Engagement, Adoption, Retention, Task success – with the Goals → Signals → Metrics process
- 6 → 8
- Six UX tensions, decided by team vote, became eight design principles and concrete rules for a regulatory reporting module
Laugwitz, Held, Schrepp 2008; ueq-online.org
Brooke 1996
Rodden, Hutchinson, Fu, CHI 2010
Case withholding-tax reclaim software (/en/cases/raquest)
What we deliver
Four building blocks, individually or as a whole – depending on whether you want to know where your product stands, how to steer UX permanently, or how an implementation succeeds measurably.
Where the system stands today – evidenced, not felt
We establish a baseline for your expert system or a single module: usability tests with real expert users and realistic tasks, task success, time on task and error rate, plus a standardised questionnaire (UEQ or SUS) for perceived quality. An expert review along the interaction principles of ISO 9241-110 and Nielsen's heuristics adds causes to the measurement. The result is a baseline against which every release can be compared.
- Test plan with tasks, success criteria and recruitment profile of the expert users
- Benchmark report: task success, time on task, error rate, UEQ or SUS scores with interpretation
- Expert review with prioritised findings (severity × frequency)
- UX audit checklist tailored to your product, for repetition by the team
- Recommendation on which measures are suited to ongoing observation
Where the system stands today – evidenced, not felt
We establish a baseline for your expert system or a single module: usability tests with real expert users and realistic tasks, task success, time on task and error rate, plus a standardised questionnaire (UEQ or SUS) for perceived quality. An expert review along the interaction principles of ISO 9241-110 and Nielsen's heuristics adds causes to the measurement. The result is a baseline against which every release can be compared.
- Test plan with tasks, success criteria and recruitment profile of the expert users
- Benchmark report: task success, time on task, error rate, UEQ or SUS scores with interpretation
- Expert review with prioritised findings (severity × frequency)
- UX audit checklist tailored to your product, for repetition by the team
- Recommendation on which measures are suited to ongoing observation
Bringing UX quality into the organisation's goal system
A benchmark alone changes nothing if it ends in a presentation. We build the measurement system that carries it: a KPI framework following Goals → Signals → Metrics that covers pragmatic quality, hedonic quality, adoption and usage; OKRs into which UX metrics flow as inputs; and quality rituals – UX reviews, design critiques, recurring benchmark rounds – that ensure measuring leads to deciding, not just measuring. We use analytics data where it exists; where it does not, we start with tests and questionnaires.
- KPI framework: goals, signals and metrics per user group and core process
- Instrumentation plan: which events the product should log, agreed with engineering and data protection
- OKR proposals with UX key results, aligned with product management
- Ritual design: formats, roles, decision rules for UX reviews and benchmark rounds
- Dashboard concept for product leadership and steering committee
So that not every screen is renegotiated
In regulated software, many UX questions are business decisions: speed or safety, guidance or free exploration, automation or manual control. We make these tensions explicit as UX tensions, let the team decide, and derive design principles and concrete rules from them – for instance: block invalid actions early, allow controlled correction afterwards; make SLA status and blockers visible without overloading the interface. Every UI decision thus becomes traceable to persona, process, critical moment and principle.
- Critical moments along the core processes, rated by frequency and damage on failure
- UX tensions with documented team vote
- Design principles with rationale and examples
- Design rules as testable statements for reviews, design system and functional concept
- Decision log: which rule justifies which interface decision
Measure adoption from go-live, not guess it later
Anyone introducing an expert system needs an answer, at the latest in the steering committee, to whether it is being adopted. Before go-live we define what adoption means – which process is completed by which role inside the system rather than beside it –, measure the as-is state before the switch, and track adoption, usage, errors and rework from the first day in operation. This connects UX strategy with our implementation support: functional concept, UAT, key-user training, go-live and hypercare get measures instead of gut feeling.
- Adoption definition per role and process, agreed with business department and project lead
- Baseline measurement before go-live: throughput time, errors, parallel tools
- Adoption and usage metrics for hypercare and regular operation
- SLA risk indicators for deadline-bound workflows
- Reporting format for steering committee and product leadership
Domain language
These terms shape conversations about UX quality with product management, UX research and the business department. We use them precisely so that measurement and decision speak the same language.
- Pragmatic and hedonic quality
- User Experience Questionnaire (UEQ, UEQ-S)
- System Usability Scale (SUS)
- HEART: Happiness, Engagement, Adoption, Retention, Task success
- Goals → Signals → Metrics
- Task success, time on task, error rate
- Adoption and usage
- Baseline and UX benchmark
- Quality rituals: UX review, design critique, benchmark round
- OKRs and UX key results
- UX tension, design principle, design rule
- Critical moments and SLA risk indicators
- Usability per ISO 9241-11: effectiveness, efficiency, satisfaction
- Interaction capability per ISO/IEC 25010:2023
- Pragmatic quality
- The part of UX quality that relates to the task: do users complete their task successfully, efficiently and with few avoidable errors? Measurable via task success, time on task, error rate and the UEQ scales perspicuity, efficiency and dependability.
- Hedonic quality
- In Hassenzahl's sense, the part of UX quality that goes beyond the task: stimulation (does the product feel interesting, inspiring?) and identification (does it fit how users see themselves and their work?). In expert software it decides trust – and trust decides adoption.
- UEQ
- User Experience Questionnaire by Laugwitz, Held and Schrepp (2008): 26 semantic differential items on six scales – attractiveness, perspicuity, efficiency, dependability, stimulation, novelty. Covers pragmatic and hedonic quality; the short form UEQ-S has eight items. Freely available, with a benchmark data set.
- SUS
- System Usability Scale by John Brooke (1996): ten statements with five-point agreement, computed into a score from 0 to 100. Technology-agnostic and fast – good for comparisons between releases, weak at explaining causes.
- HEART
- Metrics framework by Rodden, Hutchinson and Fu (Google, CHI 2010): Happiness, Engagement, Adoption, Retention, Task success. For each dimension, goals, signals and metrics are defined – instead of starting from whatever data happens to exist.
- Adoption
- The share of intended users who actually complete a process in the system – not merely log in. We define adoption per role and core process and measure it against the state before the implementation.
- Benchmark
- A repeatable measurement of UX quality with fixed tasks, fixed instruments and a fixed user profile, whose first run forms the baseline. Only the second run turns numbers into a statement about change.
- UX tension
- A pair of legitimate goals the product cannot fully satisfy at the same time – such as speed versus safety or transparency versus abstraction. Decided and documented by the team before anyone argues about interfaces.
- Design principle
- The decided side of a UX tension, phrased as a precedence rule: "safety before speed", "guidance before free exploration". A principle answers how the system should behave by default and is translated into testable design rules.
- Task success
- The share of users who complete a defined task fully and correctly without help. The most basic pragmatic metric – in deadline-bound expert processes more important than any satisfaction scale.
How we work
Five steps that recur in every UX strategy – at a product vendor as much as at an organisation introducing an expert system. The scope depends on product and question.
- 1
Clarify goals and user groups
With product management, product leadership or project lead we clarify which decisions the measurement should later support – prioritisation, release approval, steering committee. We prioritise user groups by frequency of use and business impact and define the core processes against which quality is measured.
- 2
Establish the baseline
Usability tests with expert users on realistic tasks, complemented by UEQ or SUS and, where available, usage data. An expert review supplies the causes behind the numbers. Result: a benchmark report that says where the system stands and what costs the most.
- 3
Decide tensions, derive principles and rules
From processes and critical moments we name the product's UX tensions. The team decides them by vote; we formulate design principles and translate them into rules that can be checked in reviews, functional concept and design system.
- 4
Set up the measurement system and rituals
KPI framework following Goals → Signals → Metrics, instrumentation plan with engineering, UX key results for the OKRs, formats for UX reviews and benchmark rounds. We run the first rounds together with the team until the ritual works without us.
- 5
Measure, compare, adjust
After the release or the go-live we repeat the benchmark with the same tasks and instruments, track adoption and usage, and evaluate the change against the baseline. Whatever has not moved becomes the next hypothesis – not the next opinion.
What we learned
From three years of UX work as an employee at a vendor of document management software and from our UX consulting for the vendor of withholding-tax reclaim software – without confidential figures.
- 01
Measure instead of arguing
A UX benchmark with a baseline turns design debates into decisions. It need not be academic; it must be repeatable and tied to goals the product team pursues anyway – for instance as an OKR input. As soon as benchmark results appear in the goal system, UX becomes prioritisable against features. Before that, it is a wish.
- 02
Pragmatic quality first, hedonic quality not forgotten
In expert applications, what counts first is whether users complete their tasks efficiently, safely and with few avoidable problems. But a product that feels dated or contradictory loses trust – and trust decides adoption and usage. That is why both dimensions belong in the benchmark and both in prioritisation, even though the hedonic one never takes precedence in expert software.
- 03
Let tensions be decided before anyone argues about a UI
Safety or speed, guidance or exploration: in regulated systems these are business decisions, not design questions. Naming them as pairs of opposites and letting the team decide yields principles nobody questions screen by screen anymore. Principles become rules, rules become interface behaviour – and every decision stays traceable.
- 04
Define measures before implementation, claim no results
A strategy and prototype phase delivers no SLA rates. It delivers the indicators by which the implementation will later be evaluated: time to the right report, time to detection of blocking errors, context switches, rework, support demand. Whoever fixes these beforehand can measure honestly after go-live – and need not invent anything in the case study.
- 05
Rituals beat one-off projects
A single benchmark is a snapshot. Only UX reviews, design critiques and recurring benchmark rounds turn it into a curve a team can orient itself by. Rituals need formats, roles and decision rules – otherwise they become a status round in which things are measured but nothing is decided.
- 06
Consistency is a substitute for training
Every interaction pattern users only have to learn once saves training effort and support requests. Design rules that land in the design system therefore act directly on adoption metrics and hypercare load – at the vendor as much as at the user organisation that configures its own screens and workflows.
Experience
Public cases in which UX quality was measured or built up as a decision system. We do not publish internal benchmark values.
DMS/ECM, B2B SaaS
UX benchmarks, quality rituals and OKRs for a workflow designer
Three years of UX/UI design as an employee at a vendor of document management software: benchmarks oriented on the UEQ and Hassenzahl's model, usability tests, UX audit checklists and benchmark results as OKR inputs across several product teams. UX became a measurable quantity in the organisation's goal system.
→Withholding-tax reclaim software
From UX tensions to design principles and a KPI framework for a reporting module
UX consulting for the vendor of withholding-tax reclaim software: twelve critical moments, six UX tensions decided by team vote, eight design principles with rules, plus success indicators and a KPI framework for the implementation phase – defined before implementation, measured afterwards.
→Professional references
We orient our work on these sources; we claim no certification and no institutional relationship.
Bettina Laugwitz, Theo Held, Martin Schrepp, in: USAB 2008, LNCS 5298, Springer · 2008
Construction and Evaluation of a User Experience Questionnaire
The UEQ with 26 items on six scales; freely available with handbook and benchmark data set. Our standard instrument for pragmatic and hedonic quality.
Marc Hassenzahl, in: Funology – From Usability to Enjoyment, Kluwer · 2003
The Thing and I: Understanding the Relationship Between User and Product
Establishes the distinction between pragmatic and hedonic quality (stimulation, identification) on which AttrakDiff and the UEQ scales build.
John Brooke, in: Usability Evaluation in Industry, Taylor & Francis · 1996
SUS: A 'Quick and Dirty' Usability Scale
Ten items, one score from 0 to 100, technology-agnostic. We use SUS when a quick overall score comparable across systems is needed.
Kerry Rodden, Hilary Hutchinson, Xin Fu (Google), CHI 2010, ACM · 2010
Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications
The HEART framework and the Goals → Signals → Metrics process; our template for KPI frameworks that connect adoption and task success.
ISO/TC 159/SC 4 · 2018, confirmed 2023
ISO 9241-11:2018 – Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts
Defines usability as effectiveness, efficiency and satisfaction in a context of use – the conceptual basis for task success, time on task and error rate.
ISO/IEC JTC 1/SC 7 · 2023
ISO/IEC 25010:2023 – Systems and software engineering – SQuaRE – Product quality model
Product quality model with nine characteristics, including interaction capability (formerly usability). Helps to place UX quality alongside reliability and security in a vendor's quality model.
Michael Minge, Laura Riedel, Mensch und Computer 2013; English version Minge, Thüring, Wagner, Kuhr, Springer 2017 (TU Berlin) · 2013 / 2017
meCUE – A modular questionnaire for measuring user experience
Modular, freely available questionnaire from Germany with modules for product perceptions, emotions, consequences of use and overall judgement – an alternative to the UEQ when emotions and consequences of use should be captured separately.
ISO/TC 159/SC 4 · 2020
ISO 9241-110:2020 – Ergonomics of human-system interaction – Part 110: Interaction principles
Seven interaction principles against which we check expert reviews and design rules – such as suitability for the task, self-descriptiveness and robustness against use errors.
Frequently asked questions
- How do you measure UX quality?
- With two kinds of data. Behavioural data from usability tests and, where available, from the product: task success, time on task, error rate, adoption, usage. And perceived quality via standardised questionnaires such as UEQ or SUS. Both are collected with fixed tasks and a fixed user profile so that the measurement is repeatable and a second run shows change.
- What is the difference between pragmatic and hedonic quality?
- Pragmatic quality asks: do users complete their task efficiently, safely and with few avoidable problems? Hedonic quality, in Hassenzahl's sense, asks: does the product feel stimulating and does it fit how users see their work – coherent, modern, trustworthy? In expert software the pragmatic side takes precedence; the hedonic side co-decides trust and therefore adoption.
- Do we need an analytics tool for this?
- No, not at the start. A baseline comes from usability tests and questionnaires with expert users. Product analytics is added when you want to observe adoption and usage continuously; for that we work out an instrumentation plan with engineering and data protection that logs only the events a defined metric needs.
- How does this relate to OKRs?
- UX metrics become key results or their inputs: for instance a task-success target for a core process or an adoption target for a new module. Usability thus appears in the goal system in which capacity is decided – and becomes prioritisable against features instead of being accepted at the end.
- Do you work with our product management or against it?
- With it. UX strategy only works when product management co-formulates the goals and owns the metrics. We bring method, instruments and an outside view; priorities continue to be set by the product team – now with findings instead of opinions.
- We already have a UX team. Why external UX strategy?
- For the part that gets left behind in daily work: baseline, measurement system, decision rules, OKR integration. We work together with your UX research and design, not around them, and hand over so that rituals and benchmarks keep running without us.
- We are introducing an expert system rather than building one – does this fit?
- Yes. Then we measure adoption, usage, errors and rework against the state before the switch and connect this with our implementation support: functional concept, UAT, key-user training, go-live and hypercare get defined measures by which the steering committee can steer.
- What is a UX tension, and why should the team vote on it?
- A pair of legitimate goals – speed versus safety, transparency versus abstraction – that cannot both be fully met. In regulated software these are business decisions. Once the team decides them consciously, design principles emerge that are not renegotiated screen by screen.
- Do you publish your clients' benchmark values?
- No. Internal benchmark values, client names and product internals remain confidential. In case studies we describe before and after qualitatively and name only the instruments and indicators used for measurement.
- For intermediaries: which profile are you looking for when you contact us?
- UX strategist, UX lead or senior UX consultant for complex B2B and expert software, with UX research, usability tests, benchmarks, OKR work, design system and Figma; German at negotiation level, English fluent. Remote, in Munich or on site by arrangement.
Let's talk about what good UX concretely means in your system.
An initial conversation without a sales pitch. We listen, look at your product or your implementation project and tell you honestly where measurement helps – and where it does not.
Book an initial conversation