Establishing data-intensive learning health systems: an interdisciplinary exploration of the planned introduction of hospital electronic prescribing and medicines administration systems in Scotland

Kathrin M. Cresswell

Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, UK

Pam Smith

Nursing Studies, School of Health in Social Science, The University of Edinburgh, UK

Charles Swainson

Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, UK

Angela Timoney

Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow, UK, and NHS Lothian, Edinburgh, UK

Aziz Sheikh

Usher Institute of Population Health Sciences and Informatics, The University of Edinburgh, UK

Author address for correspondence

Kathrin Cresswell

Usher Institute of Population Health Sciences

and Informatics, The University of Edinburgh

Edinburgh, UK

E-mail: kathrin.beyer@ed.ac.uk

Cite this article: Cresswell KM, Smith P, Swainson C, Timoney A, Sheikh A. Establishing data-intensive learning health systems: an interdisciplinary exploration of the planned introduction of hospital electronic prescribing and medicines administration systems in Scotland. J Innov Health Inform. 2016;23(3):572–579.

Copyright © 2016 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/


ABSTRACT

Background Creating learning health systems, characterised by the use and repeated reuse of demographic, process and clinical data to improve the safety, quality and efficiency of care, is a key aim in realising the potential benefits and efficiency savings associated with the implementation of health information technology.

Objectives We sought to investigate stakeholder perspectives on and experiences of the implementation of hospital electronic prescribing and medicines administration (HEPMA) systems in Scotland and use these to inform political decisions on approaches to promoting the use and reuse of digitised prescribing and medication administration data in order to improve care processes and outcomes.

Methods We identified and recruited key national stakeholders involved in implementing and/or using HEPMA data from generic and specialty systems. These included representatives from healthcare settings (i.e. doctors, pharmacists and nurses), managers of existing national databases, policy makers, healthcare analytics companies, system suppliers and patient representatives. We conducted multi-disciplinary focus group discussions, audio-recorded these, transcribed data verbatim and thematically analysed the transcripts with the help of NVivo10. In analysing the data, we drew on theoretical and previous empirical work on information infrastructures.

Results We identified the following key themes: 1) micro-factors – usability of systems and motivating users to input data; 2) meso-factors – developing technical and organisational infrastructures to facilitate the aggregation of data; and 3) macro-factors – facilitating interoperability and data reuse at larger scales to ensure that data are effectively generated and used.

Conclusions This work is relevant not only to countries in the early stages of data strategy development but also to countries aiming to aggregate data at national levels. An overall shared vision of a learning health system at individual, organisational and national levels can help to catalyse such data-intensive transformational efforts.

Keywords: big data, data strategy, health information technology


INTRODUCTION

“We seek the development of a learning health system that is designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in health care”.

Harvey V. Fineberg, President, Institute of Medicine1

Healthcare is becoming progressively more complex, with an ever increasing array of treatment options, older and frailer patient populations and increasing costs.2,3 Creating learning health systems is now widely seen as essential to address these challenges through strategically collecting electronic data and repeatedly using these to improve clinical and organisational processes.4

Hospital electronic prescribing and medicines administration (HEPMA) systems, which allow electronic management of prescribing and administration activities as well as analysis of data collected,5 have of late received considerable policy interest in Scotland and the United Kingdom (UK). This is in the main being fuelled by the desire to improve prescribing safety in hospitals.6 What is, however, often less well appreciated is that proactive reuse of HEPMA-generated data can also bring considerable benefits. Whilst this is to an extent true of all health information technology (HIT), the planned implementation of HEPMA systems at scale across National Health Service (NHS) Scotland and the fact these systems have the potential to transcend professional and organisational boundaries provided us with an important opportunity and lens through which to investigate how best to achieve data-informed service delivery and decision making in the NHS.79

Creating learning health systems through effective use and reuse of electronic data is extremely challenging. Underlying reasons are both technical and social in nature.10,11 Social aspects may, amongst others, include ongoing problems with system usability and integration with clinical workflows, which can contribute to poor quality data being entered, which fails to reflect actual clinical problems and/or procedures. Other factors may include social structures and relevant associated actors such as policy makers, system suppliers, professional groups, organisations, legal entities and patient groups. These can have important effects through, for instance, setting standards and providing guidance, making and influencing strategic decisions, and through direct implementation-related activities. Wider technical issues may include a lack of system interoperability and associated inhibited meaningful exchange and potential for aggregation of data. An integrated overall data strategy may help to mitigate these risks, ensuring that social and technical systems are set up from the start to realise effective use and reuse of data to improve quality and efficiency.

Many international examples exist of how such large-scale data strategies have been developed in an ad hoc manner, often rather reactively addressing problems as they emerge as opposed to strategically planning for the efficient reuse of data from the start.12,13 Particularly, little is known about the needs of different stakeholders in relation to effective use and reuse of health-related data, although the value of multi-stakeholder perspectives has been repeatedly advocated in informing policy and some valuable insights have been generated in international settings.1416 However, these may not be applicable to the Scottish context. Consequently, there is a danger of misalignment between needs and healthcare strategy. In addition, there is continuing global debate surrounding the degree of central guidance needed and warranted to support such initiatives.17 NHS Scotland is in a unique position in this respect as the country is relatively small (allowing a degree of centralised guidance that is simply not feasible in larger countries; see Box 1) and some specialty areas have already made significant progress in collecting and reusing data at scale.

Given the opportunity offered by the planned national implementation of HEPMA systems across NHS Scotland, we sought to investigate stakeholder perspectives and experiences surrounding the continued implementation of HEPMA systems and how these can inform political strategy surrounding the use and reuse of the digitised data generated through HEPMA. HEPMA systems were chosen as a timely exemplar to explore issues related to wider health information infrastructures and associated strategy to create learning health systems.18,19


METHODS

In order to stimulate discussion and exchange of experiences, we used a focus group design with two facilitators moderating the discussions.20 The focus group is a qualitative research technique that allows participants from different backgrounds to actively explore and discuss a specific topic. It is well suited to areas where interactions between different stakeholders can bring new insights. The event took place in September 2015 at The University of Edinburgh, UK.

Consent and institutional review board approval

The work received Institutional Ethical Review Board approval from The University of Edinburgh, UK, in November 2014. Informed written consent was obtained from participants prior to the discussions. In order to protect anonymity, we have anonymised names and places mentioned.

Recruitment of participants

We purposefully recruited a range of senior figures from different sectors across Scotland, selected because of their position and strategic role in achieving data-intensive healthcare.21 In doing so, we sought to identify stakeholders who were involved in the wider socio-political deliberations on HEPMA implementation. This approach to sampling is based on our previous work, where we have mapped networks of stakeholders involved in implementing and adopting complex HIT in the NHS.22 We began by drawing up a list of sectors that we wanted to be present. These included system suppliers; NHS early adopter organisations with medical, pharmacy and nursing representatives; patient organisations; NHS services associated with HEPMA; and policy makers. As our focus included discussion of exemplar approaches surrounding ChemoCare®, antibiotic stewardship and pain control (see below for rationale), we deliberately recruited prospective participants from these areas. This initial list included 28 invitees. As our aim was to gain a comprehensive insight into different viewpoints associated with HEPMA, this strategy was viewed as most appropriate to explore how systems were and could be developed, implemented and optimised.

We were also open to suggestions from participants in relation to further recommendations of additional stakeholders and identified another eight individuals based on these.23 If invitees were unable to attend, they were asked to recommend alternative participants.

Data collection and handling

The topic guide was developed jointly by the authors and incorporated feedback from prospective participants (Box 2). Discussion topics were selected with the intention to explore existing international experiences and from advanced areas in Scotland itself with the aim to translate different stakeholder needs into a feasible political strategy. In analysing stakeholder needs, we drew on our previous work surrounding social and technological contexts in implementation and adoption of complex HIT.22

As the focus was on developing strategy, topics addressed existing experiences relating to the implementation/adoption of ChemoCare® in Scotland (which is very advanced in terms of drawing on healthcare data collected in different settings).24 This first part was facilitated by AT, a senior secondary care pharmacist and academic. The second part of the workshop explored potential implications for policy and was facilitated by a senior figure in the Scottish Government. This facilitator was not involved in setting up the research or in analysing/interpreting the findings. In order to provide focus, we used examples of antibiotic stewardship and pain control, areas that are conceptually more advanced in terms of data analytics than prescribing. The facilitators paid special attention to ensure a range of viewpoints were heard and all participants were encouraged to contribute.25 The researchers were present during the discussions taking field notes summarising the social dynamics and impressions during the workshop,26 but they did not actively participate.

Discussions were digitally audio-recorded and participants were provided with an information sheet and a consent form. Audio recordings were then transcribed verbatim by a professional transcriber and transcripts were uploaded into NVivo10 software for analysis.27

Data analysis

Data were initially analysed combining inductive and deductive techniques by KC, an experienced qualitative researcher. The emerging themes were then discussed and refined by the wider research team. Codes were developed based on the four topic areas, extracting common threads of discussion and paying particular attention to diverging views/perspectives and those that were held by the majority of participants. Simultaneously, the researchers developed emerging themes that were not immediately visible in the initial planned structure of discussions.28 These related to perspectives that were perceived as important by attending individuals and therefore presented recurrent episodes during the discussions. Specific analytical considerations included comparing differing viewpoints relating to backgrounds of individuals, areas of overlap and learning from experiences.

In order to interpret the results, we drew on the work surrounding information infrastructures.29 Information infrastructures consist of distributed HIT systems that cross organisational boundaries. Interpretative approaches can help to examine evolving social and technological aspects over time. HEPMA systems are used by a variety of specialties and data are intended to be aggregated across organisations. The framework therefore provided a useful lens for considerations surrounding a national data strategy of implementing HEPMA systems.

Throughout data collection and handling, we sought to minimise bias. We have ensured credibility by drawing on the focus group as a well-established qualitative research method promoting frank and interactive exchange of experiences amongst a varied sample of participants from different backgrounds. We also sought to reduce researcher bias through respondent checking, reflexive field notes and involvement of the extended research team in the analysis. Furthermore, we made a conscious effort to be as transparent and detailed as possible in our reporting of the methods and findings of the work. This resulted in some modifications, particularly relating to emerging recommendations surrounding policy.


RESULTS

Overall, we invited 36 individuals, of whom 21 attended on the day (Box 3). The others declined due to other prior commitments or did not reply to email contact. We summarise key emerging themes in Box 4. A range of stakeholder needs and resulting implications for policy emerged from this work. Throughout, it became apparent that diverse stakeholder groups had different priorities. For instance, whilst healthcare professionals were mainly concerned with system usability and appropriate resourcing, system suppliers and policymakers were primarily interested in exploring issues relating to data reuse at scale.

Micro-factors – usability of systems and motivating users

Amongst healthcare professionals, it was noted that the systems needed to be usable in order to derive maximum benefits. Important characteristics here included an ability to be flexible enough to fit in with complex user workflows and, most importantly, an intuitive user interface that would not slow down healthcare professionals’ work. However, there were some concerns amongst healthcare professionals that HEPMA systems would not deliver in this respect.

I think we need really good intuitive systems so they’re clinician friendly or patient friendly and we have an awful lot of systems that are totally unintuitive to somebody coming to them cold, whereas if you go and open up your Mac just about anybody could go and do it so it’s very intuitive around how you get things to automatically happen but you have quite a lot of clunky systems, I’m just hoping HEPMA is a nice user friendly system. (Participant 14, Senior NHS/Academic Pharmacist)

Motivating users was often conceptualised as convincing them that the data they collected were robust and useful for service and clinical improvement, even if data input took them slightly longer on electronic systems compared to paper. Discussion of this clinical relevance was often perceived to be missing at introductory sessions.

I think whenever a new HEPMA system is introduced in the hospital you get a good sort of presentation and instruction and workshop and the technicalities of how to use it, but there’s never embedded in that someone showing this is why these kind of data are clinically important and important to the care of your patient… (Participant 11, Academic Pharmacist)

Feedback on clinical performance to improve individual practice was seen as particularly beneficial to clinicians. Doing so was viewed as a means of closing the improvement cycle from collecting clinical data, over analysis and reuse, to direct patient care at the individual user level.

Doctors, nurses, I’m a nurse we all want to know how well are we doing in the range, I don’t want to be at the bottom I’d like to be at the top, I’m somewhere in the middle what can I do to improve my practice? (Participant 20, System Supplier)

Meso-factors – developing infrastructures to facilitate the aggregation of data

Suppliers and healthcare professionals highlighted that sufficient resources needed to be made available to fund appropriate technological infrastructures. They were in many cases concerned that the investments made at higher managerial levels were not sufficient to support HEPMA systems.

[HEPMA] is very infrastructure dependent. That’s my major kind of anxiety that there’s enough investment, there’s enough investment in the [Scottish Health Boards] to support it. (Participant 12, Antimicrobial Prescribing Group)

Infrastructures included building a degree of digital maturity as a platform on which HEPMA systems could efficiently run. Short-term considerations comprised the development of common data models, data storage and warehousing, security considerations, wireless networks, and appropriate hardware. Medium- to longer-term infrastructural considerations related to the acquisition of software and devices able to be optimised to facilitate ‘closing the loop’ between prescribing, administration and dispensing of medicines; as well as systems facilitating advanced decision support and data analytics. The necessary investment associated with these infrastructural developments was seen as significant.

…it’s a spend-to-save initiative and the spend has to include the equipment within the hospital to make HEPMA a worthwhile investment. (Participant 7, Oncology Consultant)

Other essential resources included developing capacity and capability for change management to implement systems and maintain business continuity, as well as data analysts to help support analysis of generated data. It was felt that these up-front resources were often underestimated in Health Boards and in overall HEPMA business cases. Experiences from ChemoCare® also supported this.

I think one of the big learning for us with ChemoCare® was you make an estimate about the resource requirements… you’ll need people at the coalface to give you that data and that takes time and you need to design the reports. (Participant 3, Cancer Care Pharmacist)

Macro-factors – facilitating interoperability and data reuse at larger scales

Participants highlighted the importance of creating a local/national repository of data in order to allow collection and meaningful centralised analysis. However, this required some degree of interoperability between different systems, in order to allow data mapping across technologies and healthcare settings. Experiences from collecting data on Systemic Anticancer Therapy (SACT) in NHS England supported this.

… to collect data across Scotland will require some data mapping…we’re seeking some of the learning from NHS England who developed a SACT dataset some years ago and asked Trusts in England to provide this data and that was all going to get fed into this sort of big sort of single national database… I think they’re finding there’s some challenges around that, I think there are three different suppliers of software that are used, plus 50% of the Trusts are still paper-based so while all organisations are mandated to collect data, they don’t have a complete dataset in England. (Participant 3, Cancer Care Pharmacist)

Open technological standards to allow this level of interoperability were seen as necessary and HEPMA was advocated as an exemplar system contributing to this wider information consolidation both at local and national levels.

Developing local and national systems in synergy was viewed as important to facilitate local ownership of data. Participants from healthcare organisations suggested that data needed to be useful at an organisational level in order to be effectively aggregated and reused at a national level. Considering local, regional and national levels of aggregating data together was therefore seen as vital.

… at a national level that you can put the intelligence around that efficiently and effectively to deploy both national policy change and intervention as well as allowing people to feel like they own that data locally… absolutely first and foremost it’s about operational capability to deliver safe medicines every time, but then you’ve got a layering of that local intelligence and then you’ve got the national or the regional intelligence… (Participant 14, Senior NHS/Academic Pharmacist)


DISCUSSION

The identification of different stakeholder experiences and needs is important to catalyse the development of an integrated data strategy to support the effective use and reuse of HEPMA data. Adding to the existing international implementation-related literature, we have identified a number of micro-, meso- and macro-factors that are important in facilitating this in the Scottish context. In particular, our work has highlighted that appropriate user interfaces and sufficient resources are needed to implement systems, and that appropriate infrastructures need to be in place to support collecting data for reuse. Such infrastructural considerations are, however, often neglected in international discourses surrounding data strategies, particularly those surrounding capacity development.1416 Our work further indicates that throughout data strategy planning, it is important to aim for the creation of a learning health system at individual healthcare worker and organisational levels, as this can facilitate data collection and analysis.30 In doing so, there is a need to ensure that users and organisations are motivated and supported to collect accurate data, e.g. in order to improve clinical and organisational performance. Although motivating clinical users has been the subject of many implementation-related efforts, devising effective ways to feedback clinically relevant data to individual users is still a subject of much debate.31 Examining motivations of organisations in reporting data for national aggregation beyond financial incentives is an other issue that has received very little attention and would present a useful next step in exploring how to promote the adoption of national data strategies.

This work may be seen as a starting point, as it has begun investigating how policy decisions in relation to developing strategy to create learning health systems in NHS Scotland may be conceptualised. As such, we have provided contemporaneous insights into senior stakeholder viewpoints associated with the key components of a national strategy, illustrated the main concerns from a variety of perspectives, and provided implications of this work for policy development and future evaluation activities.32 These findings now need to be incorporated into ongoing UK-wide initiatives such as the Farr Institute of Health Informatics Research,33 which is aiming to promote research through data linkage. However, although we facilitated interactive discussions involving a broad range of stakeholders, this work was necessarily limited in scope as only a small number of senior stakeholders were participating (otherwise, the interactive nature of discussions would have been compromised). The transferability of this work to other UK and international contexts may therefore need to be assessed by feeding back emerging results to a wider range of stakeholders and refining policy accordingly. In addition, the close involvement of policy makers may have influenced the results.

A number of recommendations for national strategies to facilitate the creation of learning health systems through HEPMA systems emerge from this work, which we have summarised in Box 5. Although these are likely to be more relevant to countries beginning the journey of system procurement (such as Scotland), they may also be applicable to countries considering a more centralised data strategy in order to effectively aggregate data from locally implemented systems.

Establishing robust information infrastructures is a key for any national healthcare strategy. With the aim of aggregating data on larger and larger scales, such infrastructures may be comparable to other large information networks carrying electronic information like the internet and multi-national corporate systems.29,34 There is therefore a wealth of experience with similar technologies from the information systems literature that healthcare strategy can draw on, particularly in relation to developing risk mitigation strategies. For instance, it is important to recognise that such systems are evolving over time and that they have both technological and social elements.29 In our work, these related to technological systems (hardware and software to collect data) as well as users and organisations (the social element that involves collection and analysis of data). As a result, any strategy to implement associated technologies and social processes needs to be flexible enough to cope with the evolving nature of such systems and their evolving use over time.34 Similarly, technologies need to be adaptable enough to allow for changes in strategy (e.g. in data to be collected) and social contexts of use (e.g. changing professional structures). A number of implications for the design of systems drawing on the literature surrounding information infrastructures have already been outlined and may be applicable to healthcare. These include the need to ensure usability, if possible draw on existing infrastructures, expand use of systems with persuasive tactics, make technological systems easy to use to promote usability and modularise the system in order to promote flexibility.29

International experiences may also facilitate the refinement of a national (Scotland- or UK-wide) healthcare strategy. For instance, ongoing work surrounding digital maturity in England could help other countries to benchmark and serve as a starting point to develop relevant guidance.35 Similarly, experiences from large health systems in North American settings may help to develop governance structures (such as, for example, surrounding data stewardship and potential partnerships with industry), and promote learning from experiences (e.g. surrounding dealing with potential confounders and timeliness of data, or in helping to establish specialist roles that emerge from new systems such as HEPMA Pharmacists).36 Drawing on experiences from other industries, such as banking and retail, may also be fruitful in terms of future work. There will further be a need to ensure continued national high-level leadership and drive through, for instance, the appointment of Chief Data Scientists,37 and the continued use of financial incentives to promote collection of electronic health data.38


CONCLUSIONS

Achieving the vision of learning health systems can be greatly facilitated by drawing on existing national and international experiences. The example of developing HEPMA strategy within NHS Scotland has allowed us to identify key considerations that should be taken into account when planning for national strategies to aggregate healthcare data. These include defining minimum datasets, creating incentives, and promoting a learning health system through data feedback loops at different levels. At present, a truly learning health system is still a vision in most countries, but this work may be used as a starting point to promote international efforts to aggregate data at national scales.

Acknowledgements

We gratefully acknowledge the input from all participants. We also gratefully acknowledge the input of Rosemary Porteous (RP), who transcribed the discussions.

Competing interests

All authors declare that they have no competing interests.

Funding

This article is based on independent research funded by the Edinburgh & Lothians Health Foundation (ELHF, ref 53-107). The views expressed are those of the author(s) and not necessarily those of the NHS or the ELHF. KC is supported by a Chief Scientist Office of the Scottish Government post-doctoral fellowship and AS is supported by the Farr Institute. AS and KC are also grant holders on a Programme Grant funded by the National Institute for Health Research (RP-PG-1209-10099).


REFERENCES

1. Institute of Medicine (US). Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC:Institute of Medicine (US), 2011. Available from: http://www.nap.edu/catalog/12912/digital-infrastructurefor-the-learning-health-system-the-foundation-for. Accessed 12 November 2015. http://dx.doi.org/10.1377/hlthaff.19.3.191.

2. Anderson GF and Hussey PS. Population aging: a comparison among industrialized countries. Health Affairs 2000;19:191–203.

3. Van Baal PH, Polder JJ, De Wit GA, Hoogenveen RT, Feenstra TL et al. Lifetime medical costs of obesity: prevention no cure for increasing health expenditure. PLoS Medicine 2008;5:e29.

4. Krumholz HM. Big data and new knowledge in medicine: the thinking, training, and tools needed for a learning health system. Health Affairs 2014;33(7):1163–70. http://dx.doi.org/10.1377/hlthaff.2014.0053 PMid:25006142.

5. Schneeweiss S. Learning from big health care data. New England Journal of Medicine 2014;370:2161–2163. http://dx.doi.org/10.1056/NEJMp1401111. PMid:24897079

6. Cresswell K, Coleman J, Slee A, Williams R and Sheikh A. Investigating and learning lessons from early experiences of implementing ePrescribing systems into NHS hospitals: a questionnaire study. PLoS One 2013;8(1):e53369.

7. McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, O’Reilly D et al. The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials. Journal of the American Medical Informatics Association 2012;19(1):22–30. http://dx.doi.org/10.1136/amiajnl-2011-000304. PMid:21852412 PMCid:PMC3240758.

8. Beuscart-Zéphir MC, Pelayo S, Anceaux F, Meaux JJ, Degroisse M and Degoulet P. Impact of CPOE on doctor–nurse cooperation for the medication ordering and administration process. International Journal of Medical Informatics 2005;74(7): 629–41. http://dx.doi.org/10.1016/j.ijmedinf.2005.01.004. PMid:16043087.

9. Franklin BD, O’Grady K, Donyai P, Jacklin A and Barber N. The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before-and-after study. Quality and Safety in Health Care 2007;16(4):279–84. http://dx.doi.org/10.1136/qshc.2006.019497. PMid:17693676 PMCid:PMC2464943.

10. Trist E. The Evolution of Socio-Technical Systems. Occasional Paper, 1981.

11. Bijker WE. Of Bicycles, Bakelites, and Bulbs: Toward a Theory of Sociotechnical Change. Cambridge: MIT press, 1997.

12. Groves P, Kayyali B, Knott D and Van Kuiken S. The ‘Big Data’ Revolution in Healthcare. Seattle: McKinsey Quarterly, 2013.

13. NHS England. Good data strategy. Available from: http://www.england.nhs.uk/resources/resources-for-ccgs/good-data-strat. Accessed 12 November 2015.

14. Salzberg CA, Jang Y, Rozenblum R, Zimlichman E, Tamblyn R and Bates DW. Policy initiatives for health information technology: a qualitative study of US expectations and Canada’s experience. International Journal of Medical Informatics 2012;81(10):713–22. http://dx.doi.org/10.1016/j.ijmedinf.2012.07.007. PMid:22902272.

15. Riege A and Lindsay N. Knowledge management in the public sector: stakeholder partnerships in the public policy development. Journal of Knowledge Management 2006;10(3):24–39. http://dx.doi.org/10.1108/13673270610670830.

16. Aarons GA, Wells RS, Zagursky K, Fettes DL and Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. American Journal of Public Health 2009;99(11):2087–95. http://dx.doi.org/10.2105/AJPH.2009.161711. PMid:19762654 PMCid:PMC2759812.

17. Williams C, Mostashari F, Mertz K, Hogin E and Atwal P. From the Office of the National Coordinator: the strategy for advancing the exchange of health information. Health Affairs 2012;31(3):527–36. http://dx.doi.org/10.1377/hlthaff.2011.1314. PMid:22392663.

18. Eden KB, Totten AM, Kassakian SZ, Gorman PN, McDonagh MS, Devine B et al. Barriers and facilitators to exchanging health information: a systematic review. International Journal of Medical Informatics 2016;88:44–51. http://dx.doi.org/10.1016/j.ijmedinf.2016.01.004. PMid:26878761.

19. Shapiro JS, Crowley D, Hoxhaj S, Langabeer J II, Panik B, Taylor TB, et al. Health Information Exchange in Emergency Medicine. Annals of Emergency Medicine 2016;67(2):216–26. http://dx.doi.org/10.1016/j.annemergmed.2015.06.018. PMid:26233924.

20. Morgan DL. The Focus Group Guidebook (Vol. 1). California:Sage Publications, 1997.

21. Patton MQ. Qualitative Research. New Jersey:John Wiley & Sons Ltd, 2005.

22. Cresswell KM, Worth A and Sheikh A. Actor–Network Theory and its role in understanding the implementation of information technology developments in healthcare. BMC Medical Informatics and Decision Making 2010;10(1):67. http://dx.doi.org/10.1186/1472-6947-10-67. PMid:21040575 PMCid:PMC2988706.

23. Biernacki P and Waldorf D. Snowball sampling: problems and techniques of chain referral sampling. Sociological Methods & Research 1981;10:141–163.

24. CIS Oncology. Available from: http://www.cis-healthcare.com/latest/chemocare_goes_live_across_nhs_scotland. Accessed 14 October 2015.

25. Kitzinger J. Qualitative research. Introducing focus groups. British Medical Journal 1995;311:299. http://dx.doi.org/10.1136/bmj.311.7000.299.

26. Emerson RM, Fretz RI and Shaw LL. Writing Ethnographic Fieldnotes. Chicago:University of Chicago Press, 2011.

27. QSR International. NVivo 10 for Windows. Available from: http://www.qsrinternational.com/products_nvivo.aspx. Accessed 12 November 2015.

28. Fereday J and Muir-Cochrane E. Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development. International Journal of Qualitative Methods 2008;5:80–92.

29. Hanseth O and Lyytinen K. Design theory for dynamic complexity in information infrastructures: the case of building internet. Journal of Information Technology 2010;25:1–19. http://dx.doi.org/10.1057/jit.2009.19.

30. Kohn LT, Corrigan JM and Donaldson MS (Ed). To err is human: building a Safer Health System (Vol. 6). Washington, DC:National Academies Press, 2000.

31. Meeker D, Knight TK, Friedberg MW, Linder JA, Goldstein NJ, Fox CR et al. Nudging guideline-concordant antibiotic prescribing: a randomized clinical trial. JAMA Internal Medicine 2014;174(3):425–31. http://dx.doi.org/10.1001/jamainternmed. 2013.14191. PMid:24474434 PMCid:PMC4648560.

32. Catwell L, Sheikh A. Evaluating eHealth interventions: the need for continuous systemic evaluation. PLoS Medicine 2009;6:830. http://dx.doi.org/10.1371/journal.pmed.1000126. PMid:19688038 PMCid:PMC2719100.

33. The Farr Institute. The Farr Institute of Health Informatics Research. Available from: http://www.farrinstitute.org. Accessed 12 October 2015.

34. Ciborra CU and Hanseth O. From tool to Gestell: Agendas for managing the information infrastructure. Information Technology & People 1998;11:305–327. http://dx.doi.org/10.1108/09593849810246129.

35. NHS England. Digital Maturity Index. Available from: http://www.england.nhs.uk/digitaltechnology/info-revolution/maturityindex. Accessed 12 October 2015.

36. McGinnis M, Olsen L and Goodby AW. Clinical Data as the Basic Staple of Health Learning: Creating and Protecting a Public Good: Workshop Summary. Washington, DC:National Academies Press, 2010.

37. The White House Names Dr. DJ Patil as the First U.S. Chief Data Scientist. Available from: https://www.whitehouse.gov/blog/2015/02/18/white-house-names-dr-dj-patil-first-uschief-data-scientist. Accessed 12 October 2015.

38. Electronic Health Record Incentive Program and Health IT Certification Program Final Rule. https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2015-Factsheets-items/2015-10-06.html. Accessed 12 October 2015.

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT