Undertaking sociotechnical evaluations of health information technologies

Kathrin M Cresswell, Aziz Sheikh


There is an increasing international recognition that the evaluation of health information technologies should involve assessments of both the technology and the social/organisational contexts into which it is deployed.

There is, however, a lack of agreement on definitions, published guidance on how such ‘sociotechnical evaluations’ should be undertaken, and how they distinguish themselves from other approaches. We explain what sociotechnical evaluations are, consider the contexts in which these are most usefully undertaken, explain what they entail, reflect on the potential pitfalls associated with such research, and suggest possible ways to avoid these.


Evaluation; health information technology; socio-technical

Full Text:



Coiera E. Building a National Health IT System from the middle out. Journal of the American Medical Informatics Association 2009;16:271–3. http://dx.doi.org/10.1197/jamia.M3183. PMid: 19407078; PMCid:PMC2732241.

Greenhalgh T and Stones R. Theorising big IT programmes in healthcare: strong structuration theory meets actor-network theory. Social Science and Medicine 2010;70:1285–94. http://dx.doi.org/10.1016/j.socscimed.2009.12.034. PMid:20185218.

Catwell L and Sheikh A. Evaluating eHealth interven-tions: the need for continuous systemic evaluation. PLoS Medicine 2009;6:e1000126. http://dx.doi.org/10.1371/journal. pmed.1000126. PMid:19688038; PMCid:PMC2719100.

Berg M, Aarts J and van der Lei J. ICT in health care: socio-technical approaches. Methods of Information in Medicine 2003;42:297–301. PMid:14534625.

Cresswell K, Worth A and Sheikh A. Actor-network theory and its role in understanding the implementation of information technol-ogy developments in healthcare. BMC Medical Informatics and Decision Making 2010;10:67. http://dx.doi.org/10.1186/1472-6947-10-67. PMid:21040575; PMCid:PMC2988706.

Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. Available from: http://www.iom.edu/~/media/Files/Report%20Files/2011/Health-IT/HIT%20and%20Patient%20Safety.pdf. Last accessed: 16/07/2013.

Cherns A. Principles of sociotechnical design revisited. Human Relations 1987;40:153–61. http://dx.doi.org/10.1177/001872678704000303.

Clegg CW. Sociotechnical principles for system design. Applied Ergonomics 2000;31:463–77. http://dx.doi.org/10.1016/S0003-6870(00)00009-0.

Harrison MI, Koppel R and Bar-Lev S. Unintended consequences of information technologies in health care—an interactive socio-technical analysis. Journal of the American Medical Informatics Association 2007;14:542–9. http://dx.doi.org/10.1197/jamia. M2384. PMid:17600093; PMCid:PMC1975796.

Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Medicine 2011;8:e1000387.

Sheikh A, Cornford T, Barber N, Avery A, Takian A, Lichtner V, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from a prospective national evaluation in “early adopter” hospitals. British Medical Journal 2011;343:d6054. http://dx.doi. org/10.1136/bmj.d6054. PMid:22006942; PMCid:PMC3195310.

Robertson A, Cresswell K, Takian A, Petrakaki D, Crowe S, Cornford T, et al. Implementation and adoption of nationwide electronic health records in secondary care in England: quali-tative analysis of interim results from a prospective national evaluation. British Medical Journal 2010;341:c4564.

Takian A, Petrakaki D, Cornford T, Sheikh A and Barber N. Building a house on shifting sand: methodological consider-ations when evaluating the implementation and adoption of national electronic health record systems. BMC Health Services Research 2012;12:105–29. http://dx.doi.org/10.1186/1472-6963-12-105. PMid:22545646; PMCid:PMC3469374.

Westbrook J, Braithwaite J, Georgiou A, Ampt A, Creswick A, Coiera E, et al. Multimethod evaluation of information and com-munication technologies in health in the context of wicked prob-lems and sociotechnical theory. Journal of the American Medical Informatics Association 2007;14:746–55. http://dx.doi.org/10.1197/jamia.M2462. PMid:17712083; PMCid:PMC2213479.

Greenhalgh T, Hinder S, Stramer K, Bratan T and Russell J. Adoption, non-adoption, and abandonment of a personal elec-tronic health record: case study of HealthSpace. British Medical Journal 2010;341:c5814.

Barber N, Cornford T and Klecun E. Qualitative evaluation of an electronic prescribing and administration system. Quality and Safety in Health Care 2007;16:271–8. http://dx.doi.org/10.1136/qshc.2006.019505. PMid:17693675; PMCid:PMC2464937.

Berg M. Patient care information systems and health care work: a sociotechnical approach. International Journal of Medical Informatics 1999; 55:87–101. http://dx.doi.org/10.1016/S1386-5056(99)00011-8.

Cornford T, Doukidis G and Forster D. Experience out-come with a structure, process and framework for evaluat-ing an information system. Omega, International Journal of Management Science 1994;22:491–504. http://dx.doi. org/10.1016/0305-0483(94)90030-2.

Cresswell K, Worth A and Sheikh A. Comparative case study investigating sociotechnical processes of change in the con-text of a national electronic health record implementation. Health Informatics Journal 2012;18:251–70. http://dx.doi. org/10.1177/1460458212445399. PMid:23257056.

Crowe C, Cresswell K, Robertson R, Huby G, Avery A and Sheikh A. The case study approach. BMC Medical Research Methodology 2011;11:100. http://dx.doi.org/10.1186/1471-2288-11-100. PMid:21707982; PMCid:PMC3141799.

Cresswell KM, Bates DW and Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. Journal of the American Medical Informatics Association 2013;20:e9–13.

Pawson R and Tilley N. Realistic Evaluation. London: Sage Publications, 1997.

Wacker JG. A definition of theory: research guidelines for dif-ferent theory-building research methods in operations manage-ment. Journal of Operations Management 1998;16:361–85. http://dx.doi.org/10.1016/S0272-6963(98)00019-9.

Greenhalgh T, Potts H, Wong G, Bark P and Swinglehurst D. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q 2009;87:729–88. http://dx.doi.org/10.1111/j.1468-0009.2009.00578.x. PMid:20021585; PMCid:PMC2888022.

DOI: http://dx.doi.org/10.14236/jhi.v21i2.54


  • There are currently no refbacks.

This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT