Exploiting the information revolution: call for independent evaluation of the latest English national experiment

Philip Scott


Background The English National Health Service (NHS) has a long history of national experiments with information technology; some successful, others less so. The NHS England Five Year Forward View aspires to ‘Exploit the information revolution’ through the transformational work of the National Information Board (NIB). NIB has published a ‘Framework for Action’ that promotes citizen empowerment, information availability, transparency, public trust, innovation, informatics skills and societal value.

Issue The framework sets out many laudable and common sense ambitions, but is light on evidence to support its aspirations, or plans for its evaluation. Considerable resource was invested in evaluation studies in the later stages of the National Programme for IT in England, but the analyses do not seem to have been included. Most of the cited evidence is from an unpublished report by management consultants rather than independent peer-reviewed work.

Recommendation National experiments of this importance should be evidence based and properly evaluated so that each iteration of ‘information revolution’ produces an evidence base to inform subsequent generations of care provision models and technology innovation. Evaluation should be planned from the very start rather than added in as an afterthought. Like any good business plan there should be defined critical success factors for health and social care, and a declaration of how they might be measured. Unintended consequences should be qualitatively explored. Evaluation should also consider critical-interpretive social perspectives to understand the human factors in technology deployment and should seek a theoretically informed insight into the mechanisms of change.

Conclusions The NHS Five Year Forward View and the NIB framework set out challenging ambitions based on transformative use of information technology and collaborative partnerships with commissioners and providers. The framework stands up very well against the Hayes principles, but the learning opportunities associated with this programme should not be missed.



evidence-based practice; health policy; medical informatics; programme evaluation

Full Text:



NHS England. NHS Five Year Forward View, 2014. Available from: http://www.england.nhs.uk/ourwork/futurenhs/.

National Information Board. Personalised Health and Care 2020: Using Data and Technology to Transform Outcomes for Patients and Citizens. A Framework for Action, 2014. Available from: https://www.gov.uk/government/publications/personalised-health-and-care-2020.

NHS England. The Forward View into action: Planning for 2015/16, 2014. Available from: http://www.england.nhs.uk/wpcontent/uploads/2014/12/forward-view-plning.pdf.

Information Management Group. Implementing the Infrastructure for Information Management and Technology in the NHS: A Handbook for IM&T Specialists. Leeds: NHS Executive, 1994.

Burns F. Information for Health, 1998. Available from: http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4014469.pdf.

UK Department of Health. Delivering 21st Century IT: Support for the NHS National Strategic Programme, 2002. Available from: http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4067112.pdf.

Department of Health. Liberating the NHS: An information revolution. A consultation on proposals, 2011. Available from: http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_120598.pdf.

Department of Health. The power of information: giving people control of the health and care information they need, 2012. Available from: http://webarchive.nationalarchives.gov.uk/20130802094648/https://www.gov.uk/government/publications/giving-people-control-of-the-health-and-care-informationthey-need.

University of Birmingham. NHS Connecting for Health Evaluation Programme – Reports, 2014 [cited 2015 January 5]. Available from: http://www.birmingham.ac.uk/research/activity/mds/projects/HaPS/PHEB/CFHEP/reports/reports.

McKinsey. Modelling the potential of digitally enabled processes, transparency and participation in the NHS. In paper.

National Voices. Prioritising person-centred care – the evidence, 2014 [cited 2015 January 5]. Available from: http://www.nationalvoices.org.uk/evidence.

Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Medicine 2011;8(1):e1000387. http://dx.doi.org/10.1371/journal.pmed.1000387. PMid:21267058; PMCid:PMC3022523.

Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Journal of the American Medical Association 2005;293(10):1223–38. http://dx.doi.org/10.1001/jama.293.10.1223. PMid:15755945.

Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Annals of Internal Medicine 2013;159(10):677–87. http://dx.doi.org/10.7326/0003-4819-159-10-201311190-00006. PMid:24247673.

Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506–12. http://dx.doi.org/10.1542/peds.2005-1287. PMid:16322178.

Ammenwerth E and Shaw NT. Bad health informatics can kill-is evaluation the answer? Methods of Information in Medicine 2005;44(1):1–3. PMid:15778787.

Cresswell KM, Bates DW and Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. Journal of the American Medical Informatics Association 2013;20(e1):e9–13. http://dx.doi.org/10.1136/amiajnl-2013-001684. PMid:23599226; PMCid:PMC3715363.

Scott PJ, Brown A, Friedman C, Wyatt J, Georgiou A and Eisenstein E. Improving the Science of Health Informatics by using Validated Instruments and Outcome Measures. Workshop presented at Medical Informatics Europe; Istanbul, Turkey; 2014.

Rigby M, Ammenwerth E, Beuscart-Zephir MC, Brender J, Hypponen H, Melia S et al. Evidence based health informatics: 10 years of efforts to promote the principle. Joint contribution of IMIA WG EVAL and EFMI WG EVAL. Yearbook of Medical Informatics 2013;8(1):34–46. PMid:23974546.

de Lusignan S and Krause P. The Hayes principles: learning from the national pilot of information technology and core generalisable theory in informatics. Informatics in Primary Care 2010;18(2):73–7. PMid:21078229.

Nuffield Trust. A decade of austerity? The funding pressures facing the NHS from 2010/11 to 2021/22, 2012. Available from: http://www.nuffieldtrust.org.uk/publications/decade-austerity-funding-pressures-facing-nhs.

Gaikwad R and Warren J. The role of home-based information and communications technology interventions in chronic disease management: a systematic literature review. Health Informatics Journal 2009;15(2):122–46.

Dobrev A, Jones T, Stroetmann V, Stroetmann K, Vatter Y and Peng K. Interoperable eHealth is worth it: Securing benefits from Electronic Health Records and ePrescribing, 2010. Available from: https://ec.europa.eu/digital-agenda/en/news/interoperableehealth-worth-it-securing-benefits-ehrs-and-eprescribing.

Bower P, Cartwright M, Hirani SP, Barlow J, Hendy J, Knapp M et al. A comprehensive evaluation of the impact of telemonitoring in patients with long-term conditions and social care needs: protocol for the whole systems demonstrator cluster randomised trial. BMC Health Services Research 2011;11:184. http://dx.doi.org/10.1186/1472-6963-11-184. PMid:21819569; PMCid:PMC3169462.

Steventon A, Bardsley M, Billings J, Dixon J, Doll H, Hirani S et al. Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. British Medical Journal 2012;344:e3874. http://dx.doi.org/10.1136/bmj.e3874. PMid:22723612; PMCid:PMC3381047.

Hendy J, Chrysanthaki T, Barlow J, Knapp M, Rogers A, Sanders C et al. An organisational analysis of the implementation of telecare and telehealth: the whole systems demonstrator. BMC Health Services Research 2012;12:403. http://dx.doi.org/10.1186/1472-6963-12-403. PMid:23153014; PMCid:PMC3532839.

Henderson C, Knapp M, Fernandez JL, Beecham J, Hirani SP, Beynon M et al. Cost-effectiveness of telecare for people with social care needs: the whole systems demonstrator cluster randomised trial. Age and Ageing 2014;43(6):794–800. http://dx.doi.org/10.1093/ageing/afu067. PMid:24950690; PMCid:PMC4204660.

Henderson C, Knapp M, Fernandez JL, Beecham J, Hirani SP, Cartwright M et al. Cost effectiveness of telehealth for patients with long term conditions (whole systems demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trial. British Medical Journal 2013;346:f1035. http://dx.doi.org/10.1136/bmj.f1035. PMid:23520339.

MacNeill V, Sanders C, Fitzpatrick R, Hendy J, Barlow J, Knapp M et al. Experiences of front-line health professionals in the delivery of telehealth: a qualitative study. The British Journal of General Practice 2014;64(624):e401–7. http://dx.doi.org/10.3399/bjgp14X680485. PMID:24982492; PMCID:PMC4073725.

Greenhalgh T. Whole System Demonstrator trial: policy, politics, and publication ethics. British Medical Journal 2012;345:e5280. http://dx.doi.org/10.1136/bmj.e5280. PMID: 22868961.

RAND Corporation. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy, 2013. Available from: http://www.rand.org/content/dam/rand/pubs/research_reports/RR400/RR439/RAND_RR439.pdf. PMid:17877873.

Drummond D. 9 Reasons Physicians Hate EMR – The 2013 RAND Study, 2013 [cited 2015 January 5]. Available from: http://www.thehappymd.com/blog/bid/351428/9-Reasons-Phy-sicians-Hate-EMR-The-2013-RAND-Study.

Rashid M, Abeysundra L, Mohd-Isa A, Khan Y and Sismeiro C. Two years and 196 million pounds later: where is Choose and Book? Informatics in Primary Care 2007;15(2):111–9.

Greenhalgh T, Stones R and Swinglehurst D. Choose and Book: a sociological analysis of ‘resistance’ to an expert system. Social Science & Medicine 2014;104:210–9. Epub 2014/03/04. http://dx.doi.org/10.1016/j.socscimed.2013.12.014.

Beale T. The health record: why is it so hard? Haux R and Kulikowski C (Eds). IMIA Yearbook of Medical Informatics 2005. Stuttgart: Schattauer; 2005, pp. 301–4.

Copeland D and McKenney J. Airline reservations systems: lessons from history. MIS Quarterly 1988;12(3):353–70. http://dx.doi.org/10.2307/249202.

Scott S and Zachariadis M. Origins and development of SWIFT, 1973–2009. Business History 2012;54(3):462–82. http://dx.doi.org/10.1080/00076791.2011.638502.

Wanless D. Securing Our Future Health: Taking a Long-Term View, 2002. Available from: http://webarchive.nationalarchives.gov.uk/20130129110402/http://www.hm-treasury.gov.uk/consult_wanless_final.htm.

NHS Connecting for Health. NHS IM&T Investment Survey, 2009. Available from: http://webarchive.nationalarchives.gov.uk/20130502102046/http://www.connectingforhealth.nhs.uk/resources/imtstaff/2009summaryresults.pdf.

NHS England. The Integrated Digital Care Technology Fund, 2014 [cited 2015 January 12]. Available from: http://www.england.nhs.uk/ourwork/tsd/sst/tech-fund/.

Benson T. Why general practitioners use computers and hospital doctors do not--Part 1: incentives. British Medical Journal 2002;325(7372):1086–9. PubMed PMID: 12424171. http://dx.doi.org/10.1136/bmj.325.7372.1090. http://dx.doi.org/10.1136/bmj.325.7372.1086. PMid:12424171; PMCid:PMC131190.

Greenhalgh T and Russell J. Why do evaluations of eHealth programs fail? An alternative set of guiding principles. PLoS Medicine 2010;7(11):e1000360. http://dx.doi.org/10.1371/journal.pmed.1000360. PMid:21072245; PMCid:PMC2970573.

De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C et al. Theory of change: a theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials 2014;15.

Hayes G. The NHS information technology (IT) and social care review 2009: a synopsis. Informatics in Primary Care 2010;18(2):81–8. PMid:21078230.

DOI: http://dx.doi.org/10.14236/jhi.v22i1.139


  • There are currently no refbacks.

This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT