SMASH! The Salford medication safety dashboard

Richard Williams

NIHR Greater Manchester Patient Safety Translational Research Centre (PSTRC), University of Manchester, Manchester, UK and MRC Health eResearch Centre, Division of Informatics, Imaging and Data Science, University of Manchester, Manchester, UK

Richard Keers

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and Division of Pharmacy and Optometry, Centre for Pharmacoepidemiology and Drug Safety, School of Health Sciences, Manchester Academic Health Sciences Centre (MAHSC), University of Manchester, Manchester, UK

Wouter T. Gude

Academic Medical Center, University of Amsterdam, Amsterdam, Netherlands

Mark Jeffries

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and Division of Pharmacy and Optometry, Centre for Pharmacoepidemiology and Drug Safety, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, UK

Colin Davies

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and MRC Health eResearch Centre, Division of Informatics, Imaging and Data Science, University of Manchester, Manchester, UK

Benjamin Brown

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and MRC Health eResearch Centre, Division of Informatics, Imaging and Data Science, University of Manchester, Manchester, UK

Evangelos Kontopantelis

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and NIHR School for Primary Care Research, University of Manchester, Manchester, UK

Anthony J. Avery

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and School of Medicine, University of Nottingham, Nottingham, UK

Darren M. Ashcroft

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and Division of Pharmacy and Optometry, Centre for Pharmacoepidemiology and Drug Safety, School of Health Sciences, Manchester Academic Health Sciences Centre, University of Manchester, Manchester, UK

Niels Peek

NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester, UK and MRC Health eResearch Centre, Division of Informatics, Imaging and Data Science, University of Manchester, Manchester, UK

Author address for correspondence:

Richard Williams

NIHR Greater Manchester Patient Safety Translational Research Centre

G303a, Jean McFarlane Building

University of Manchester

Manchester M13 9PL, UK

Email: richard.williams@manchester.ac.uk

Cite this article: Williams R, Keers R, Gude WT, Jeffries M, Davies C, Brown B, Kontopantelis E, Avery AJ, Ashcroft DM, Peek N. SMASH! The Salford medication safety dashboard. J Innov Health Inform. 2018;25(3):183–193.

Copyright © 2018 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/


ABSTRACT

Background Patient safety is vital to well-functioning health systems. A key component is safe prescribing, particularly in primary care where most medications are prescribed. Previous research has demonstrated that the number of patients exposed to potentially hazardous prescribing can be reduced by interrogating the electronic health record (EHR) database of general practices and providing feedback to general practitioners (GPs) in a pharmacist-led intervention. We aimed to develop and roll out an online dashboard application that delivers this audit and feedback intervention in a continuous fashion.

Method Based on initial system requirements, we designed the dashboard’s user interface over three iterations with six GPs, seven pharmacists and a member of the public. Prescribing safety indicators from previous work were implemented in the dashboard. Pharmacists were trained to use the intervention and deliver it to general practices.

Results A web-based electronic dashboard was developed and linked to shared care records in Salford, UK. The completed dashboard was deployed in all but one (n = 43) general practices in the region. By November 2017, 36 pharmacists had been trained in delivering the intervention to practices. There were 135 registered users of the dashboard, with an average of 91 user sessions a week.

Conclusion We have developed and successfully rolled out of a complex, pharmacist-led dashboard intervention in Salford, UK. System usage statistics indicate broad and sustained uptake of the intervention. The use of systems that provide regularly updated audit information may be an important contributor towards medication safety in primary care.

Keywords: patient safety, drug prescriptions, electronic audit and feedback, dashboard


INTRODUCTION

Patient safety has become an integral part of quality management in healthcare systems worldwide. While patient safety research has traditionally focused on secondary care,13 primary care, as the cornerstone of modern healthcare systems,4 is increasingly recognised as an area where major improvements in patient safety can be achieved,5,6 especially due to the large numbers of medications that are prescribed on a daily basis.7 It has been shown that one in 20 prescriptions in primary care contain errors, and one in 550 contain potentially life threatening errors.7 One in 25 hospital admissions are the result of prescribing errors in primary care,8 and adverse drug reactions, of which most are avoidable, cost the NHS an estimated £500 million per year.9

The potential for Information Technology (IT) systems to improve safety within healthcare is large and well documented.10,11 For prescribing safety, there are examples where IT systems have had positive12 and negative13,14 effects. IT-based interventions for improving prescribing safety fall broadly into two categories: clinical decision support (CDS) and electronic audit and feedback (eA&F). CDS systems, such as pop-up alerts, attempt to influence behaviour at the point of care and while some studies have shown benefits,15 others have shown that clinicians can suffer from ‘alert fatigue’ where poorly targeted alerts lead to their routine dismissal.16,17 eA&F systems, such as dashboards, provide feedback away from the point of care, usually at a population level, to allow for clinicians to review, and potentially change, their practice retrospectively.18 A systematic review of eA&F systems found a wide degree of heterogeneity in both the identified studies and their effects.19 Another literature review, specifically focusing on dashboards, had similar findings and called for more research to be undertaken to help influence the design of such systems.20 Despite the widespread usage of such dashboards, there exists little evidence as to what factors contribute to their success or failure.21

A common limitation of both types of intervention is that they often only indicate problems, without necessarily providing solutions. Even when specific actions are recommended, clinicians do not necessarily have the time or skills to act appropriately.22 The University of Nottingham, therefore, developed the pharmacist-led information technology intervention for reducing medication errors (PINCER) in primary care. The PINCER intervention is based on computer-generated feedback that identifies patients for whom potentially hazardous prescribing practices are present, but crucially adds educational outreach visits by trained pharmacists23 to general practices where they work with the local staff to resolve any confirmed hazardous prescribing incidents and to prevent their reoccurrence. The intervention was shown to be more effective at reducing numbers of at-risk patients than computer-generated feedback alone24; proving the pharmacist visit plays a crucial role in effectively solving prescribing errors. It was also shown to be likely (59% chance) cost-effective in reducing prescription errors.24,25

There are indications that the reduction in risk due to PINCER is only temporary because it does not always reduce the incident erroneous prescribing behaviour.24 This is in part because the PINCER feedback mechanism relies on snapshots of data extracted from the electronic health record (EHR) database, while feedback is known to be more effective when it is provided more than once.26 Therefore, we aimed to build upon PINCER in order to create a continuous feedback loop for cycles of quality improvement. Our objectives were to develop an application that identifies patients exposed to potentially hazardous prescribing to end users and is updated on a daily basis, and roll out the system across Salford, UK, where our previous research has shown the prevalence of potentially hazardous prescribing is greater than 5%.27


METHODS

System requirements

Based on an initial scope definition created by a senior clinical pharmacy researcher (Darren M. Ashcroft) and the principal investigator of the PINCER study (Anthony J. Avery), it was decided that a front-end dashboard application and the associated back end was required that would:

1) receive, validate and process data extracts of patient records from general practitioner (GP) systems on a daily basis, via an existing shared care record infrastructure – the Salford Integrated Record (SIR);

2) execute queries to the GP system data, based on pre-defined indicators, to identify patients at risk of adverse medication events;

3) present lists of at-risk patients to GPs and pharmacists, in a secure environment, restricting the visibility of patient identifying information to those clinicians that are responsible for that patient’s care;

4) provide the results in a timely, user friendly and actionable manner.

The indicators to be used in the dashboard were selected from the set of 56 prescribing safety indicators for GPs identified by Spencer et al.28 Indicators were selected based on their severity and the practicality of extracting the relevant information from clinical records. They aim to prevent: gastrointestinal bleeding; asthma exacerbations; acute kidney injury; liver damage and neutropenia; hypo and hyperthyroidism and thrombotic risks. The list of indicators in the dashboard is available in Table 1 in the Supplementary Material. Computable representations of the indicators were required which would involve creating clinical code sets29 for each component, and determining whether a patient was prescribed to take a medication at a given point in time. This last requirement is not simple as although prescription events in UK primary care are always recorded electronically, there is usually no coded record when a medication is stopped, so instead the termination of the medication must be inferred by the lack of a repeat prescription.

It was also decided that the system’s feedback regarding each indicator would consist of a numerator (also known as the affected or ‘at-risk’ patients), and a denominator (also known as the eligible patients). The ‘affected patients’ are those who have breached the conditions of the indicator and are therefore those patients at risk from potentially hazardous prescribing who need corrective action. The ‘eligible patients’ are those who meet a particular subset of the conditions of the indicator and are the population of patients against which the affected patients can be measured. The terms numerator and denominator are used as the proportion of eligible patients who are also affected patients can be used to compare between practices of different sizes and demographics.

Engineering methods

The initial design of the dashboard’s user interface was created following an iterative process which was informed by short interviews with key stakeholders involving six GPs, seven clinical commissioning group (CCG) pharmacists and one member of our patient and public involvement group. Prototype dashboard designs were reviewed by the stakeholders during interview and feedback sought. This feedback was incorporated into the design for the next dashboard iteration.30

The software architecture of the dashboard, the user interface and all the associated back-end processes were designed by the lead author Richard Williams. The software development was undertaken by Richard Williams and Colin Davies. Where possible open-source technologies were used to ensure that the system could be made publically available in the future if required. Operating systems and server architecture were designed in order to meet the requirements of the secure server hosting within the Salford Royal Foundation Trust (SRFT) data centre.

System usage monitoring

A final requirement was that the system should track usage, and record all interactions with the dashboard down to individual mouse clicks, hovers and non-sensitive key strokes, that is, not passwords. A large volume of usage tracking data will therefore be collected enabling us to report on the frequency and duration with which the dashboard is used, who its primary users are (e.g. pharmacists or GPs), and how this varies between practices and over time. It will also allow us to assess which feedback modalities provided by the dashboard (table, benchmark charts, trend charts and patient lists) are typically accessed by users, and under which circumstances. Finally, we can analyse which areas of medication safety (i.e. which indicators) users tend to focus on when they access the dashboard.

Implementation and roll-out

Any general practice in Salford was eligible for receiving the intervention provided they had access to SIR. Research team members met with Salford CCG leads and GP quality leads to identify practices for recruitment. Once identified, the CCG pharmacy team (supported by the research team) provided practice managers/senior partners with an information sheet about the intervention for distribution amongst practice staff and an invitation to take part.

If practices decided to take part, their staff received online access to the dashboard. No physical installation at the practice level is necessary. In addition, trained NHS clinical pharmacists will visit the practices regularly for at least 12 weeks, access the dashboard, and advise on changes in medication prescribing/monitoring in collaboration with practice staff.

Pharmacist time was resourced by Salford CCG and by the NIHR Greater Manchester Patient Safety Translational Research Centre (http://www.patientsafety.manchester.ac.uk/). Pharmacists were recruited and trained to help deliver the intervention. The training was based on the original PINCER trial pharmacist training23 and involved: introducing the prescribing safety indicators included in the dashboard and the risk they pose to patients; the importance of academic detailing and causation analysis/systems thinking; describing the SMASH dashboard and clinical pharmacist intervention and developing the understanding of how to respond to potentially hazardous prescribing in practice. Each pharmacist was assigned to one or more practices with whom they arranged an initial meeting during which they talked to the practice about the indicators and showed them how to use the dashboard. Pharmacists and GPs were encouraged to focus on resolving cases of potentially hazardous prescribing identified by the dashboard during an initial 12-week ‘intensive’ period. From the date of the initial meeting with each general practice, we followed up that practice for 12 months to discover: whether the numbers of at risk patients reduce; whether any reduction is sustained; how people use the system and how their use affects patient outcomes.


RESULTS

Architecture

The system is deployed to two servers (see Figure 1), running Windows Server 2012, in the secure data centre of the SRFT. One server runs the main patient database using SQL Server 2012, and the other runs SMASH – the web application that the users access. SMASH is rendered on the client side with AngularJs,31 and is served with data via a RESTful API running on NodeJS32 and Express.33 It is available on the secure NHS N3 network meaning only people on that network can access the dashboard. Firewalls ensure that only recruited practices can view the dashboard. Users can access the site remotely using a virtual private network (VPN) connection to their practice.

Each day at midnight, the system receives data from primary care EHR data sources. Currently, this is SIR, a data warehouse containing all primary care data from Salford, though the system allows any suitable primary care data source to be incorporated. Once the data is updated, another batch runs to execute SQL stored procedures against the patient data to generate lists of at-risk patients for the front-end web application. These report data are copied to MongoDB34 on the web application server. The entire process takes about 2 hours.

Indicator development

We have developed computable representations of each indicator. Clinical code sets29 were created for each component of the indicator. For example, the indicator that looks for asthmatic patients who are prescribed a non-selective beta blocker required two main code sets: one code set to identify asthmatic patients and one to identify beta blocker prescriptions. In fact, a third set of codes for the event ‘asthma resolved’ was required so that patients whose asthma is no longer active would not be detected by the indicator.

To determine when a patient was prescribed to take a medication, we used two approaches. The first method was to count any patient with a prescription in the previous n months, where in our case, and for the current PINCER indicators, n = 3 months, but with longer periods for medications, such as combined hormonal contraceptives (n = 6 months), which often have longer prescriptions. The second approach was to reuse an algorithm that we had previously developed that considers the date of the prescription, the amount prescribed and the rate of medication usage (‘take 2 a day’) to determine when the patient has likely stopped the medication without the occurrence of another prescription.35

Figure 1 Architecture diagram of SMASH. The EHR data are received and processed on the database server, report files are transferred to the web server and presented to the end user via a web interface

System functionality

For users assigned to a single practice, the first screen after logging in presents a summary table for their practice on today’s date. For users with multiple practices, the user must first select a practice before viewing the summary. A second tab presents detailed information for each indicator (Figure 2 – all data in this and future screenshots is fictitious). The number of affected and eligible patients, the proportion of eligible patients affected and the current CCG average can be viewed here. A comparison date, defaulting to 30 days ago, allows the system to show the user: how many affected patients have been resolved; how many remain affected and how many new cases there have been since this comparison date. The user can: change the date of the report and the comparison date; sort the table by each column; select certain indicators to always appear on top irrespective of the sorting and drill down into lists of patients by clicking on any number within the table. A third tab displays this information in charts. One example visually compares the user’s practice with the average across the CCG (Figure 3). Another example, not displayed here, shows how the practice’s performance changes over time.

Selecting one of the hyperlinked numbers within the table in Figure 2 takes the user to a screen with a list of patients affected by the selected indicator on the selected date (Figure 4). For each patient, we display: their NHS number, which indicators they are breaching, and for how long they have been affected. The clinician can look up the patient in the practice’s EHR to determine an appropriate course of action. The selected indicator and date of interest can be changed.

From the patient list (Figure 4), the user can select a tab named ‘Information’ to display a summary of the evidence supporting the selected indicator (Figure 5). This includes: the risk to patients; links to the academic literature supplying the evidence; the consequences of inaction and possible actions that could be taken.

Finally, some users are given the role of ‘CCG user’ which allows them to see summary performance measures across all practices within the CCG. They can view the proportion of at-risk patients in each practice for a single indicator, or for all indicators, in table or chart form (Figure 6).

Figure 2 System screenshot: indicator list for a practice. It shows number and proportion of patients affected, compares performance with the past, and allows the user to view lists of patients by clicking the hyperlinked numbers. CKD = chronic kidney disease; NSAID = nonsteroidal anti-inflammatory drug; GastProt = gastrointestinal protective medication; Warf = warfarin; NOAC = novel oral anticoagulant; LABA = long-acting beta-adrenoceptor agonist; ICS = inhaled corticosteroid

Figure 3 System screenshot: chart showing the user’s practice’s proportion of affected patients per indicator compared with the CCG. GiB = gastrointestinal bleed; PU = peptic ulcer; GP = gastrointestinal protective medication; ASP = aspirin; CLOP = clopidogrel; CHC = combined hormonal contraceptive; LOOP = loop diuretic; U&E = urea and electrolytes; HF = heart failure; BMI = body mass index; AMIOD = amiodarone; OEST = oestrogen; MET = metformin; Mtx = methotrexate

Pharmacist and practice recruitment

By October 2017, 36 pharmacists, three pharmacy technicians and two CCG managers were trained to use the dashboard and how to introduce it to general practices. Although pharmacists and general practices were encouraged to focus on an initial 12-week ‘intensive’ period, in many cases, pharmacists continued to monitor and use the dashboard after this time.

We recruited 43 out of 44 general practices within Salford. The missing practice wanted to be involved but does not contribute to SIR and so was ineligible for the intervention. The first practice recruited completed the 12-month follow-up on 17 April 2017, and the last practice recruited will complete in September 2018. Effects on prescribing safety and adverse events will be evaluated thereafter in an interrupted time series analysis.

Usage

As of 17 November 2017, there are 135 registered users (51 pharmacists, 48 GPs and 36 other practice staff such as nurses and technicians) in the dashboard, excluding developers and research team members, with 28 users (all pharmacists) logging into the system in the last month (since 18 October 2017). In 2017, there have been an average of 91 user sessions per week (SD = 39), with an average duration of 17 minutes.

Preliminary findings

On 31 January 2017, we conducted an interim analysis comparing recruited practices with at least 1 month’s usage, with those practices not yet recruited or with less than 1 month’s usage. The number of at-risk patients in recruited practices (n = 32) had fallen from 1444 on 1 January 2016 to 882 on 31 January 2017. This is a mean reduction of 17.6 patients per practice and is significant (p = 0.0002) when compared with the mean reduction of 2.1 patients in practices not yet recruited (n = 12).


DISCUSSION

We have developed SMASH, a dashboard for displaying patients ‘at-risk’ of a serious adverse event, such as acute kidney injury or gastrointestinal bleeds, to pharmacists and GPs, and deployed it as part of a pharmacist-led intervention within Salford, UK. The system has seen high rates of adoption, with usage in all but one practice, and many users logging in each week.

Figure 4 System screenshot: a list of patients currently flagged by SMASH for the ‘asthma and beta blocker’ indicator. It shows the NHS number, which indicators the patient is breaching, and when they first flagged for this indicator

Improving medication safety has become a major aim in all clinical settings. The potential for IT systems to contribute to this aim is substantial, but data on the effectiveness of IT interventions remains controversial. A systematic review of the effects of IT interventions on medication safety in primary care36 found that pharmacist-led IT interventions were successful in reducing medication errors. They suggest that the detection of unsafe medication by pharmacists together with feedback to and discussion with physicians may be an additional safeguard in the complex process of prescribing. Previously, a pragmatic, cluster randomised trial showed that the pharmacist-led PINCER intervention, on which SMASH is based, is an effective method for reducing medication errors in general practice. Recently, a cluster-randomized, stepped-wedge trial in 33 general practices in Tayside, Scotland, showed that a complex intervention combining professional education, informatics and financial incentives reduced the rate of high-risk prescribing of antiplatelet medications and NSAIDs.37

As noted by Ammenwerth et al.,38 any installation of health IT for medication safety must be based on an overarching medication safety strategy that is underwritten by the institution where the installation takes place. In line with this recommendation, we have worked in close collaboration with the local CCG in the development and roll-out of the SMASH dashboard, and they were instrumental in the roll-out becoming a success by resourcing pharmacist time and developing the strategy to recruit general practices and pharmacists.

The dashboard updates each night providing the users with up-to-date information whenever they choose to access it. Previous studies have shown that there is evidence that this immediate access to information may improve patient outcomes.20,39 SMASH conforms to most of the 15 suggestions for practice feedback interventions laid out by Brehaut et al.,39 such as: providing feedback in multiple ways (tables and graphs); providing individual rather than general data and recommending actions that are under the user’s control.

We have developed computable representations of evidence-based prescribing safety indicators, but it is challenging to determine whether a patient is currently prescribed a medication due to the absence of drug termination codes. The simple approach that looks for any prescriptions in the last 3 months is overly sensitive and flags up patients who have since stopped their medication. It also has the problem that there may be nothing actionable the GP or pharmacist can do to remove the patient from the dashboard – they must instead wait until their most recent medication is before the window of inspection. This has the potential to discourage usage of the dashboard and could also lead to extra work as users repeatedly review a patient’s record. The algorithmic approach is more promising, but requires additional pre-processing work to map drug codes to ingredients and strengths.

Figure 5 System screenshot: detailed information and evidence about the asthma and beta blocker indicator

Figure 6 System screenshot: chart showing the proportion of eligible patients who are affected for each practice within the CCG

One limitation is that we have built a system external to the GPs standard EHR systems and so users are required to view patient lists within SMASH, before looking the patients up in a separate system. A more integrated approach would be easier for the end user, though given the high levels of usage that we have observed, it is a problem that the users are perhaps willing to overlook. Future versions of GP systems should consider incorporating safety indicators.

Over the coming years, we will: improve the existing dashboard based on feedback from the trial; deploy more indicators; roll out the system across Greater Manchester and working with industry partners and explore ways of allowing patients to interact with the system. The role of patients in their safe care is important40 and the ability for patients to discover when they are ‘flagged up’ by safety systems such as this will start to change the interactions between the patient and provider, and opens up several interesting avenues for future research.

Acknowledgements

This research was funded by the National Institute for Health Research Greater Manchester Patient Safety Translational Research Centre (NIHR Greater Manchester PSTRC). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. MRC Health eResearch Centre grant MR/K006665/1 supported the time and facilities of EK. Connected Health Cities is a Northern Health Science Alliance led programme funded by the Department of Health and delivered by a consortium of academic and NHS organisations across the north of England. The work uses data provided by patients and collected by the NHS as part of their care and support. The views expressed are those of the author(s) and not necessarily those of the NHSA, NHS or the Department of Health.


REFERENCES

1. Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D and Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Quality & Safety [Internet] 2013;22(1):11–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22849965. Accessed November 2017.

2. Parand A, Dopson S, Renz A and Vincent C. The role of hospital managers in quality and patient safety: a systematic review. BMJ Open [Internet] 2014;4(9):e005055. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25192876. Accessed November 2017.

3. Weaver SJ, Lubomksi LH, Wilson RF, Pfoh ER, Martinez KA and Dy SM. Promoting a culture of safety as a patient safety strategy. Annals of Internal Medicine [Internet] 2013;158(5_Part_2):369. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23460092. Accessed November 2017.

4. Starfield B, Shi L and Macinko J. Contribution of primary care to health systems and health. Milbank Quarterly [Internet] 2005;83(3):457–502. http://doi.wiley.com/10.1111/j.1468-0009.2005.00409.x. Accessed November 2017.

5. Spencer R and Campbell SM. Tools for primary care patient safety: a narrative review. BMC Family Practice [Internet] 2014;15:166. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25346425. Accessed November 2017.

6. Stokes J, Panagioti M, Alam R, Checkland K, Cheraghi-Sohi S and Bower P. Effectiveness of case management for ‘at risk’ patients in primary care: a systematic review and meta-analysis. PLoS One [Internet] 2015;10(7):e0132340. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26186598. Accessed November 2017.

7. Avery AJ, Ghaleb M, Barber N, Dean Franklin B, Armstrong SJ, Serumaga B, et al. The prevalence and nature of prescribing and monitoring errors in English general practice: a retrospective case note review. British Journal of General Practice [Internet] 2013;63(613):e543–53. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23972195. Accessed September 2017.

8. Winterstein AG, Sauer BC, Hepler CD and Poole C. Preventable drug-related hospital admissions. Annals of Pharmacotherapy [Internet] 2002;36(7–8):1238–48. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12086559. Accessed November 2017.

9. Pirmohamed M, James S, Meakin S, Green C, Scott AK, Walley TJ, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. BMJ [Internet] 2004;329(7456):15–9. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15231615. Accessed September 2017.

10. Bates DW and Gawande AA. Improving safety with information technology. The New England Journal of Medicine [Internet] 2003;348(25):2526–34. Available from: papers3://publication/uuid/5605FBB5-3753-444A-9CEF-50D49AC8C85D.

11. Buntin MB, Burke MF, Hoaglin MC and Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs [Internet] 2011;30(3):464–71. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21383365. Accessed November 2017.

12. Waitman LR, Phillips IE, McCoy AB, Danciu I, Robert M, Halpenny MS, et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. The Joint Commission Journal on Quality and Patient Safety [Internet] 2011;37(7):326–32. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21819031. Accessed November 2017.

13. Ash JS, Berg M and Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. Journal of the American Medical Informatics Association [Internet] 2004;11(2):104–12. Available from: https://academic.oup.com/jamia/article-lookup/doi/10.1197/jamia.M1471. Accessed November 2017.

14. Koppel R, Metlay JP, Cohen A, Abaluck B, Localia AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA [Internet] 2005;293(10):1197. Available from: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.293.10.1197. Accessed November 2017.

15. Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior? Journal of the American Medical Informatics Association 2009;16(4):531–8.

16. Hsieh TC, Kuperman GJ, Jaggi T, Hojnowski-Diaz P, Fiskio J, Williams DH, et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. Journal of the American Medical Informatics Association [Internet] 2004;11(6):482–91. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15298998. Accessed October 2017.

17. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB and Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Archives of Internal Medicine [Internet] 2003 ;163(21):2625. Available from: http://www.ncbi.nlm.nih.gov/pubmed/14638563. Accessed October 2017.

18. Brehaut JC and Eva KW. Building theories of knowledge translation interventions: use the entire menu of constructs. Implementation Science [Internet] 2012;114. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23173596. Accessed November 2017.

19. Tuti T, Nzinga J, Njoroge M, Brown B, Peek N, English M, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implementation Science [Internet] 2017;12(1):61. Available from: http://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0590-z.

20. Dowding D, Randell R, Gardner P, Fitzpatrick G, Dykes P, Favela J, et al. Dashboards for improving patient care: review of the literature. International Journal of Medical Informatics [Internet] 2015;84(2):87–100. http://dx.doi.org/10.1016/j.ijmedinf.2014.10.001.

21. Khalil H, Bell B, Chambers H, Sheikh A and Aj A. Professional, structural and organisational interventions in primary care for reducing medication errors. Cochrane Database of Systematic Reviews 2017;10.

22. Ivers N, Barnsley J, Upshur R, Tu K, Shah B, Grimshaw J, et al. My approach to this job is. One person at a time: perceived discordance between population-level quality targets and patient-centred care. Canadian Family Physician [Internet] 2014;60(3):258–66. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24627384. Accessed November 2017.

23. Sadler S, Rodgers S, Howard R, Morris CJ and Avery AJ. Training pharmacists to deliver a complex information technology intervention (PINCER) using the principles of educational outreach and root cause analysis. International Journal of Pharmacy Practice [Internet] 2014;22(1):47–58. Available from: http://eprints.nottingham.ac.uk/2511/1/Sadler_Tarining.pdf. Accessed November 2017.

24. Avery AJ, Rodgers S, Cantrill JA, Armstrong S, Cresswell K, Eden M, et al. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-eff ectiveness analysis. Lancet [Internet] 2012;379(9823):1310–9. http://dx.doi.org/10.1016/S0140-6736(11)61817-5.

25. Elliott RA, Putman KD, Franklin M, Annemans L, Verhaeghe N, Eden M, et al. Cost effectiveness of a pharmacist-led information technology intervention for reducing rates of clinically important errors in medicines management in general practices (PINCER). Pharmacoeconomics [Internet] 2014;32(6):573–90. Available from: http://link.springer.com/10.1007/s40273-014-0148-8. Accessed October 2017.

26. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. In: Ivers N (Ed.), Cochrane Database of Systematic Reviews [Internet] Chichester, UK: John Wiley & Sons, Ltd; 2012. http://doi.wiley.com/10.1002/14651858.CD000259.pub3. Accessed November 2017.

27. Akbarov A, Kontopantelis E, Sperrin M, Stocks SJ, Williams R, Rodgers S, et al. Primary care medication safety surveillance with integrated primary and secondary care electronic health records: a cross-sectional study. Drug Safety [Internet] 2015;38(7):671–82. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4486763&tool=pmcentrez&rendertype=abstract.

28. Spencer R, Bell B, Avery AJ, Gookey G and Campbell SM. Identification of an updated set of prescribing-safety indicators for GPs. British Journal of General Practice [Internet] 2014; 64(621):e181–90. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24686882. Accessed December 2017.

29. Williams R, Kontopantelis E, Buchan I and Peek N. Clinical code set engineering for reusing EHR data for research: a review. Journal of Biomedical Informatics [Internet] 2017;70:1–13. Available from: http://www.sciencedirect.com/science/article/pii/S1532046417300801. Accessed May 2017.

30. Keers RN, Williams R, Davies C, Peek N and Ashcroft DM. Improving medication safety in primary care: developing a stakeholder-centred electronic prescribing safety indicator dashboard. Pharmacoepidemiology and Drug Safety 2015.

31. Google. AngularJS javascript MVW framework [Internet] 2015. Available from: https://angularjs.org/. Accessed 19 December 2017.

32. Node.js [Internet]. Available from: https://nodejs.org/en/. Accessed 19 December 2017.

33. Express. Express – Node.js web application framework [Internet] 2013. Available from: https://expressjs.com/. Accessed 19 December 2017.

34. MongoDB. MongoDB [Internet] 2017. Available from: https://www.mongodb.com/. Accessed 19 December 2017.

35. Williams R, Brown B, Peek N and Buchan I. Making medication data meaningful: illustrated with hypertension. Studies in Health Technology and Informatics [Internet] 2016;228:247–51. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27577381. Accessed October 2016.

36. Lainer M, Mann E and Sonnichsen A. Information technology interventions to improve medication safety in primary care: a systematic review. International Journal for Quality in Health Care [Internet] 2013;25(5):590–8. Available from: https://oup.silverchair-cdn.com/oup/backfile/Content_public/Journal/intqhc/25/5/10.1093/intqhc/mzt043/2/mzt043.pdf?Expires=1487091891&Signature=LFxJM7~C7dqo9clP2aeSxC6yr5NVyrErX4kPFGXcK62mKMBCSbE9Z6sE1jdKjUz0sAtC4X~YJn9qgf~A9utGUGeveGnc0W1SkayIu5Al3BFW~QQ.

37. Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C and Guthrie B. Safer prescribing — a trial of education, informatics, and financial incentives. The New England Journal of Medicine [Internet] 2016;374(11):1053–64. Available from: http://www.nejm.org/doi/10.1056/NEJMsa1508955. Accessed December 2017.

38. Ammenwerth E, Aly AF, Bürkle T, Christ P, Dormann H, Friesdorf W, et al. Memorandum on the use of information technology to improve medication safety. Methods of Information in Medicine 2014;53(5):336–43.

39. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Annals of Internal Medicine [Internet] 2016;164(6):435–41. Available from: http://annals.org/article.aspx?doi=10.7326/M15-2248. Accessed December 2017.

40. Vincent CA, Coulter A. Patient safety: what about the patient? Quality and Safety in Health Care [Internet] 2002;11:76–80. Available from: http://qualitysafety.bmj.com/content/qhc/11/1/76.full.pdf. Accessed 19 October 2017.


Supplementary Material

Table 1 Descriptions of the 13 indicators deployed in the current dashboard and the clinical aim of each

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT