Petrelli, D. (2008). On the role of user-centred evaluation in the advancement of interactive information retrieval. Information Processing & Management 44(1), 22-38.

doi:10.1016/j.ipm.2007.01.024

This article is part of an IP&M special issue on user-centered evaluated for interactive information retrieval. This particular article makes the uncontroversial (I hope) argument that information-seeking research, information retrieval systems research, and interactive information retrieval research need to be more integrated. It argues that both formative and summative, qualitative and quantitative methods need to be used at various stages in the process. More formal, fixed-task approaches can be used in the formative stages to improve a prototype design, while more naturalistic approaches (that will tend to be more qualitative) should be used at later stages both to understand use in the particular context and to ensure that the design supports tasks the user brings with her as well as those that the designers can predict.

Of course the hard part is getting all the research communities on board with this kind of framework.