Formal models, usability and related work in IR

Annelise M Pejtersen
Centre for Human Machine Interaction
Risų National Laboratory

Denmark
Mark D Dunlop
Centre for Human Machine Interaction
Risų National Laboratory

Denmark
Raya Fidel
Graduate School of Library and Information Science, University of Washington
USA
annelise.mark.pejtersen@risoe.dk mark.dunlop@risoe.dk fidelr@u.washington.edu

Experimentation on the usability and effectiveness of web search engines, and the web as a whole, can lead to misleading results if the evaluation work is carried out too early (e.g., planning to evaluate the functionality of the web in advance of a test of the interface readability) or without setting a suitable set of constraints, or boundary, around the experiments (e.g. not suitably controlling the work domain and strategies when evaluating interface readability). The boundary along which the cut is made varies considerably, depending on the aim of the experiment. Therefore a suitable framework for identifying the experimental boundary conditions must be applied. This short paper proposes the use of the framework shown in Figure 1 for defining different types of evaluation boundaries when evaluating the worth of web search engines and World Wide Web information seeking in general.

...continued...

pdf_wee.gif (111 bytes) PDF full paper

Reference

A.M. Pejtersen, M. D. Dunlop, and R. Fidel, "A Use Centred Framework for Evaluation of the Web", Proceedings of SIGIR 1999 Workshop on Evaluation of Web Document Retrieval edited by M. Agosti and M. Melucci, August 1999.

oldmdd.jpeg (6948 bytes)