Analysis in usability evaluation

Abstract

To improve current knowledge on analysis in usability evaluation, we present the findings of a survey study involving155 usability practitioners who have reported on their latest usability evaluation. The study is based on a previous pilot interview study. Analysis was found to be informal and pragmatic, conducted as part of fast-paced usability evaluations. Analysis structure and support from the usability research of the last 15 years hardly seem to have had any direct impact on analysis practices, even though general knowledge resources such as heuristics and guidelines are much used. Collaboration in analysis is common, and there is considerable awareness on collaboration as a means to improve reliability. Making redesign suggestions is a tightly integrated part of analysis, and also an aspect of evaluation that is seen as challenging. Severity classifications are commonly used, both for usability problems and redesign suggestions. On basis of the findings we summarize six key implications for usability practitioners, to inspire future practice, and six implications for usability researchers, to guide future research. The study is supported by the Norwegian research project R2D2 Networks (http://r2d2networks.origo.no) and the European COST project TwinTide (http://twintide.org).