Alpha usability testing: the method before you code
Florent Dabernat
May 9, 2026
2 min read
In this article
Before launching an HR dashboard still in development, an alpha test helps catch major points of confusion and clarify vocabulary while it's still easy and cheap to change everything.
Building a solid test plan
A test plan lays out the context and objectives, the hypotheses to check, the profiles and sample (in practice, five participants per key persona is generally enough), the scenarios and tasks to complete, the metrics tracked (success rate, time, errors, SEQ/SUS, verbatims) and the logistics, recording and consent included. Success criteria must be defined before the sessions, never after the fact.
Running the sessions well
In moderated sessions, the facilitator always reminds participants that the product is being tested, not the person, a clarification that puts participants at ease and improves the quality of feedback. Participants complete tasks while thinking aloud. The observer doesn't step in to help, except in case of a complete block, and notes both behaviors and key verbatims. A short ease-of-use questionnaire (SEQ) is given after each task to capture in-the-moment perception objectively.
On an alpha test of an HR dashboard run with five participants and four tasks, the average SEQ (5.2/7) was enough to identify three labels to fix before final development, avoiding costly back-and-forth later.
Analyzing and prioritizing results
Identified issues are classified by frequency and severity, then prioritized based on impact and the effort required to fix them. The final report highlights the main findings, backed by concrete evidence (screenshots, verbatims), and offers actionable recommendations, along with a retest plan to verify the fixes actually solved the identified issues.
On a real case, this process raised the SUS score from 62 to 78 after fixes, with completion time down 23% and a notable drop in support tickets related to the feature tested.
Frequently asked questions
-
Are five participants really enough for a reliable user test?
- For a qualitative alpha test aimed at catching major issues, yes: beyond five participants per profile, newly discovered issues become marginal relative to the extra effort invested.
-
What if a participant gets completely stuck during a task?
- The observer can step in minimally to unblock the session and keep collecting data on subsequent tasks, while carefully noting the sticking point for analysis.
-
Do SEQ and SUS measure the same thing?
- No: the SEQ (Single Ease Question) assesses the perceived ease of a specific task right after completing it, while the SUS (System Usability Scale) measures the product's overall usability at the end of the session.