IEC 62366 Usability Engineering: Formative vs. Summative Evidence

The distinction that determines whether your usability file passes

Usability engineering has a way of appearing straightforward until a Notified Body reviewer starts asking questions. The framework under IEC 62366-1 is built around a fundamental distinction — formative evaluation versus summative evaluation — and many manufacturers blur it in ways that create real problems at audit.

Formative evaluation is iterative. You run it during design and development to find problems and fix them. It can be informal, small-scale, and exploratory. Its job is to improve the design. Summative evaluation is the final validation: a controlled study with representative users performing realistic tasks with the finished (or near-final) device, generating objective evidence that the design is safe to use. Summative evaluation feeds your Usability Evaluation Report and, ultimately, your technical file. Confusing the two — or worse, presenting formative sessions as summative evidence — is one of the most common usability engineering gaps Notified Bodies flag.

What Notified Bodies have been looking for since 2021

Since the MDR transition gathered pace, Notified Body feedback on usability files has become more consistent and more pointed. A few patterns have emerged across multiple manufacturers:

The summative study design is frequently underpowered. Reviewers have been questioning study sample sizes, task selection rationale, and the representativeness of the user population. If your summative study used five participants from your own engineering team, expect questions. The guidance from IEC 62366-1 on representative users is not just procedural — reviewers are checking whether the users in your study actually resemble the users on the label.

Use-related risk analysis is often disconnected from the rest of the risk management file. IEC 62366-1 requires you to identify use errors that could cause harm and show that your design addresses them. Notified Bodies have been checking that the critical tasks in your summative study can be traced back to the use-related risks in your ISO 14971 file. If those two documents don't reference each other, the connection is missing in a way that reviewers notice.

Hazard-related use scenarios (previously called "critical tasks" in older versions of the standard) require specific documentation. The 2015 revision of IEC 62366-1 changed terminology and tightened requirements around what needs to be studied versus observed. Teams working from older procedures or older guidance documents sometimes find their summative study doesn't address the right task set.

Building the usability file

The usability file is the collected record of everything you did under IEC 62366-1. It should include your usability engineering plan, your user profile documentation, your task analysis, records of all formative evaluation activities (even if informal), your summative evaluation protocol and report, and the traceability between use-related risks and the tasks addressed in summative testing.

One thing that catches teams off guard: there is no requirement that formative evaluation be conducted in a lab or be formally documented at the level of summative evaluation. But there is a requirement that it happened, that you learned something from it, and that what you learned is reflected in the design. A usability file that jumps straight from user profile to summative study with nothing in between will draw questions about whether iterative improvement actually occurred.

The MDCG and usability

The European Commission and Notified Bodies have been clear that IEC 62366-1 application is expected under MDR. MDCG guidance and Notified Body position papers have reinforced that usability engineering is not optional for software-based devices or devices with a significant user interface — and that "significant" is interpreted broadly. If a user interaction with your device could contribute to a harm, it falls within scope.

For combination products or devices with a companion app, the scope question is worth resolving explicitly in your usability engineering plan. Reviewers want to see that you've considered the full use scenario, not just the hardware interface.

Where this connects to the rest of your technical file

Usability engineering does not live in isolation. Your use-related risk analysis connects to your ISO 14971 risk management file. Your summative evaluation conclusions connect to your clinical evaluation — if a use error could contribute to a safety-relevant outcome, the clinical evaluation should address whether that outcome is adequately mitigated. Your instructions for use are downstream from your usability file: IFU content that addresses identified use errors needs to trace back to those errors being identified.

Getting the connections documented is as important as getting the individual studies right. A strong summative study with no visible connection to the rest of the technical file will still generate findings.