Factors distinguishing evaluation techniques in HCI

Evaluation techniques are essential for determining the usability, efficiency, and user experience of interactive systems in human-computer interaction (HCI). There are various evaluation techniques employed in HCI, each with its own characteristics and considerations. Some common evaluation techniques include:

• Heuristic evaluation

• Usability testing

• Cognitive walk-through

• Surveys and questionnaires

• Field studies

However, not every evaluation technique is equivalent. For successful evaluations, it is essential to understand the factors that distinguish them.

This answer will explore some key factors and their effects on different evaluation techniques:

Key factors

Seven key factors distinguish evaluation techniques, as shown in the following illustration.

Stage of evaluation

The stage at which evaluation is carried out can significantly impact the evaluation process.

• Design evaluations focus on early stage concepts, prototypes, and iterations to inform design decisions and identify usability issues.

• On the other hand, implementation evaluations assess the final product or system's usability, performance, and user experience.

Style of evaluation

Evaluation techniques can be classified as either laboratory-based or field-based.

• Laboratory evaluations involve testing the system in controlled environments. It helps assess the system's functionality.

• Field evaluations occur in real-world settings, providing insights into how users interact with the system in their natural environments.

Level of subjectivity or objectivity

Evaluation techniques can either be:

• Highly subjective (e.g., field studies relying on user opinions and experiences)

• More objective (e.g., usability testing with performance metrics and task completion rates).

The level of subjectivity or objectivity desired can influence the choice of evaluation technique.

Type of measures

Evaluation techniques provide different measures to assess usability and user experience. These measures may include:

• Quantitative data includes numerical data such as task completion time or error rates.

• Qualitative data includes user satisfaction ratings, user feedback, or usability problem identification.

Level of information

Evaluation techniques differ in the level of information they provide.

• High-level techniques offer a broader perspective, focusing on overall system performance, usability, and user satisfaction.

• Low-level techniques provide detailed insights into specific aspects, such as usability issues or interface interactions, allowing for targeted improvements.

Level of interference

The desired level of interference can affect the choice of technique.

• Some evaluation techniques, such as usability testing, involve direct user interaction, potentially influencing their behavior and introducing bias.

• On the other hand, expert reviews and cognitive walkthroughs do not directly involve users and have a lower level of interference.

Available resources

Evaluation techniques require varying resources, including time, budget, equipment, and expertise.

• Some methods, such as surveys or heuristic evaluations, can be conducted with limited resources because they require fewer participants and minimal equipment. They can be performed remotely or with a small group of evaluators.

• Others, like large-scale usability testing, may require more substantial investments in time, participants, specialized equipment, and experienced evaluators.

Conclusion

Several factors distinguish HCI evaluation techniques and should be considered when choosing an evaluation technique. By selecting the appropriate evaluation technique, designers can ensure that their systems are easy to use and efficient.