Analysis of usability evaluation data: An interview study with usability professionals

Abstract

While the planning and implementation of usability evaluations are well described in the HCI literature, the analysis of the evaluation data is not. This implies that we know little of the current practice of usability professionals on this later phase of usability evaluation, and that we have an insufficient basis for research on methods and tools to support analysis. We present interviews with 11 usability professionals on how they conduct analysis, including the resources, collaboration, creation of recommendations, and prioritization involved. The interviews provide three main findings: • Analysis seem to be conducted accordning to pragmatic and informal approaches • Collaboration among analysts seem to be fairly frequent. Two prevalent purposes of such collaboration was (a) identification of more problems and generation of better design suggestions, and (b) quality control. • The analysis process seem to involve both problem identification redesign suggestions, even though the latter is mainly unsupported by existing usability evaluation methods. We discuss how future researh could be to better support analysis. Also, we present three questions that usability professionals may consider during analysis.