Usability evaluation of four functional identical versions of DES (Database of empirical studies)

Abstract

Simula research laboratory, dept. of software engineering, has hired four different contractors to develop one version each of the department's new Database of Empirical Studies (DES). The four versions have been developed according to the same requirements specification.

Researchers at SINTEF ICT have compared the usability of the four DES-versions through expert evaluations (pilot) and user tests with 18 users. The users were highly representative of the DES user population.

The same general finding was made in both the expert evaluations and use tests. The systems differed with regard to usability. DES-versions A and D received the best overall usability scores. B and C received the lowest. The differences were statistically significant.

It should be noted that DES-version B was best for two of the three tasks. However, it's login (Task 1) had sufficiently low usability for this DES-version to receive the lowest global usability score. It is expected that a redesign of the B login (assumed to be a quick fix) would have made this DES-version the most usable.

Qualitative usability problems associated with the four DES-versions are also presented.

Oppdragsgiver Simula Research Laboratory, Department of Software Engineering