The adoption of an electronic health record did not improve A1c values in Type 2 diabetes

Harry B. Burke, MD, PhD

F. Edward Hebert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA

Dorothy A. Becher, MPH

F. Edward Hebert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA

Albert Hoang, PhD, DSc

F. Edward Hebert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA

Ronald W. Gimbel, PhD

Department of Public Health Sciences, Clemson University, Clemson, USA

Author address for correspondence

Harry B. Burke, MD, PhD,

Associate Professor of Medicine,

F. Edward Hebert School of Medicine,

Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda,

MD 20814, USA. harry.burke@usuhs.edu

Cite this article: Burke HB, Becher DA, Hoang A, Gimbel RW, The adoption of an electronic health record did not improve A1c values in Type 2 diabetes. J Innov Health Inform. 2016;23(1):433–438.

Copyright © 2016 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/


ABSTRACT

Background A major justification for the clinical adoption of electronic health records (EHRs) was the expectation that it would improve the quality of medical care. No longitudinal study has tested this assumption.

Objective We used hemoglobin A1c, a recognized clinical quality measure directly related to diabetes outcomes, to assess the effect of EHR use on clinical quality.

Methods We performed a five-and-one-half-year multicentre longitudinal retrospective study of the A1c values of 537 type 2 diabetic patients. The same patients had to have been seen on at least three occasions: once approximately six months prior to EHR adoption (before-EHR), once approximately six months after EHR adoption (after-EHR) and once approximately five years after EHR adoption (five-years), for a total of 1,611 notes.

Results The overall mean confidence interval (CI) A1c values for the before-EHR, after-EHR and five-years were 7.07 (6.91 – 7.23), 7.33 (7.14 – 7.52) and 7.19 (7.06 – 7.32), respectively. There was a small but significant increase in A1c values between before-EHR and after-EHR, p = .04; there were no other significant differences. There was a significant decrease in notes missing at least one A1c value, from 42% before-EHR to 16% five-years (p < .001).

Conclusion We found that based on patient’s A1c values, EHRs did not improve the clinical quality of diabetic care in six months and five years after EHR adoption. To our knowledge, this is the first longitudinal study to directly assess the relationship between the use of an EHR and clinical quality.

Keywords: diabetes, electronic health record (EHR), medical care, quality


BACKGROUND

A major justification for the clinical adoption of electronic health records (EHRs) was the promise that it would improve the quality of medical care.1,2 Cross-sectional studies of the impact of EHRs on quality have shown inconsistent results.312 No randomized clinical trial or longitudinal study using a direct measure of clinical quality has been conducted to test this assumption. We performed a longitudinal retrospective study using hemoglobin A1c, a recognized clinical quality measure directly related to diabetes outcomes,13 to assess clinical quality. We hypothesized that the use of an EHR for five years would lower the A1c values of type 2 diabetic patients.


METHODS

We performed a five-and-one-half-year multicentre longitudinal retrospective study. An independent contractor blinded to the study selected the notes of 537 type 2 diabetic outpatients who were treated by physicians at five U.S. Military Health System (MHS) outpatient primary care clinics. The MHS’s commercial EHR is similar to most EHRs and the MHS population is similar to the general population.1417 The patients had to have been seen on at least three occasions: once approximately six months prior to EHR adoption (before-EHR), once approximately six months after EHR adoption (after-EHR) and once approximately five years after EHR adoption (five-years), for a total of 1611 notes.

The means and confidence intervals(CIs) for the A1c values at each time point were calculated. Patients were also stratified into one of two groups based on whether their charts contained a missing A1c value. The ‘complete’ group consisted of the patients where all three notes contained an A1c value. The ‘partial’ group consisted of the patients where at least one of their notes contained an A1c value, but all three notes did not contain A1c values. Patients missing all three A1c values were not included in the study. In addition, patients were stratified into tertiles based on their A1c values and the tertile’s mean (CI) A1c value for each time point was determined. The mean A1c values were compared across time intervals by Student’s t-test, significance p < .05, two-sided (SAS 9.7, SAS Institute Inc., Cary, North Carolina).


RESULTS

The overall mean (CI) A1c values for the before-EHR, after-EHR and five-years were 7.07 (6.91 – 7.23), 7.33 (7.14 – 7.52) and 7.19 (7.06 – 7.32), respectively, labelled All in Figure 1.

There was a small but significant increase in A1c values between before-EHR and after-EHR, p = .04, but there were no significant differences between before-EHR and five-years and between after-EHR and five-years.

The effect of missing data on A1c values is shown in Figure 1. The All group contains all the patients with A1c values, the complete group contains the patients with an A1c value for each of the three time points, and the partial group contains the patients with an A1c value for either one or two time points but not all three time points. For the complete group, the before-EHR, after-EHR and five-year mean (CI) A1c values for the complete group were 7.16 (6.84 – 7.38), 7.25 (7.02 – 7.48) and 7.25 (7.03 – 7.47), respectively. For the partial group, they were 6.96 (6.75 – 7.17), 7.43 (7.10 – 7.76) and 7.15 (7.00 – 7.30), respectively. The only significant difference was in the partial group, between the before-EHR and after-EHR, p = .02. There were no other significant differences between partial groups and there were no significant differences between complete groups. Twenty-nine patients (5%) were missing all three A1c values. The notes missing an A1c value (including the patients missing all three values) decreased from 42% before-EHR and after-EHR to 16% five-years (p < .001).

The A1c tertiles are shown in Figure 2. The lowest tertile’s before-EHR, after-EHR and 5-year mean A1c values were 5.93 (5.88 – 5.99), 6.07 (6.00 – 6.13) and 5.98 (5.92 – 6.03), respectively. There was a small but significant increase in A1c values between before-EHR and after-EHR, p = 0.001, and there was a small but significant decrease in A1c values between after-EHR and five-years, p = 0.04, but there was no significant difference between before-EHR and five-years. The middle tertile’s before-EHR, after-EHR and five-year mean A1c values were 6.77 (6.72 – 6.82), 6.92 (6.88 – 6.97) and 6.82 (6.78 – 6.86), respectively. There was a small but significant increase in A1c values between before-EHR and after-EHR, p < 0.001, and there was a small but significant decrease in A1c values between after-EHR and five-years, p = 0.001, but there was no significant difference between before-EHR and five-years. The highest tertile’s before- EHR, after-EHR and five-year mean A1c values were 8.55 (8.26 – 8.84), 9.09 (8.71 – 9.49) and 8.9 (8.4 – 9.4), respectively. There was a significant increase in A1c values between before-EHR and after-EHR, p = 0.03, and there was a significant decrease in A1c values between after-EHR and five-years, p = 0.049, but there was no significant difference between before-EHR and five-years.

We expected to observe a significant decrease in A1c values primarily in the highest tertile, over five years of EHR use. We followed the same diabetic patients for five years after the introduction of the EHR, but we did not find a significant decrease in A1c values. In other words, based on their A1c values, the EHR use did not improve the clinical quality of diabetic care in six months and five years after EHR adoption, either across all patients or in the highest A1c tertile. Missing A1c values did not affect these results. We did find that the EHR significantly reduced the number of missing A1c values.

The increase and then decrease in mean A1c values observed at the before and after time points are consistent with complexity theory, which states that during the process of moving from one fitness peak to another, an organization experiences a decrease in performance that can last as long as a year.18 The finding that missing A1c values did not affect the A1c results suggests that the existence or absence of an A1c value is not a measure of clinical quality. The observation that EHRs reduced the frequency of missing A1c values can be explained by either increased testing, resulting in increased test results available for documentation, or the increased documentation of test results. Since there was usually only a slight increase, usually 3% – 4%, in A1c testing attributable to EHR use,19 the reduction in missing A1c values was probably due to the increased documentation engendered by the automated population of test results.

Figure 1 Mean A1c values by time point and missing data

Figure 2 Mean (CI) and A1c values by tertile

The observed inability of EHRs to improve clinical quality is congruent with physician perceptions. A 2014 national survey of 967 practicing physicians found that only 35% (CI = 3.2%) believed that their EHR had significantly or somewhat improved the quality of their patient care and 65% believed that either it had not changed or it had made worse the quality of their patient care.20

No randomized clinical trial has assessed the ability of EHRs to improve medical care and it is unlikely that one will be conducted since it would require half of the physicians in the study to return to handwritten notes. The relationship between EHRs and clinical quality has been investigated in cross-sectional studies,310 in studies that used claims data5, in studies that contained process measures3,5,6,810 and in studies that assessed the ability of alerts and reminders to improve outcomes.11,12 The results have been mixed. No study has demonstrated a benefit across all its measures. Some studies have shown a partial benefit,6,7,10,11 while others have not demonstrated that EHRs have a significant clinical impact.35,8,9,12 In a recent examination of the impact of the patient centred medical home (PCMH) on ten Healthcare Effectiveness Data and Information Set measures over one year, investigators found that the odds of receiving recommended care in the PCMH group were 7% higher for the physicians who had wrote their notes but only 6% higher for those physicians who used an EHR.20 There have been calls for longitudinal studies,10 but, to our knowledge, none has been performed using a direct measure of clinical quality over an appropriate time period.

The American Recovery and Reinvestment Act of 200921,22 has increased physician adoption of EHRs; the Centers for Disease Control and Prevention reported that 71.8% of office-based physicians were using an EHR in 2012.23 But the electronic medical record systems may not, by themselves, be able to improve the quality of care. A recent RAND report stated that, ‘the current state of EHR technology significantly worsened professional satisfaction in multiple ways. Poor EHR usability, time-consuming data entry, interference with face-to-face patient care, inefficient and less fulfilling work content, inability to exchange health information between EHR products, and degradation of clinical documentation were prominent sources of professional dissatisfaction’.24 Finally, shortcomings in the design and implementation of health information technology systems have caused physicians to complain that current EHRs do not deliver sufficient clinical value to compensate for their difficulty and expense.20 A recent review concluded that we should rethink the definition of meaningful use, reduce EHR difficulty and improve their clinical utility.25 In other words, EHRs may be necessary but not sufficient for increasing the quality of medical care. Clinical decision support systems, which are federally mandated as part of the meaningful use, have potential to add significant clinical value to EHRs.26

A study limitation is that it only assessed one diabetic quality measure, namely, A1c. But this measure has been used extensively to measure clinical quality3,5,8,10,11,27 and to assess EHR use.13


CONCLUSIONS

In summary, we assessed the clinical justification for the adoption of the EHR, namely, that it would improve the quality of medical care. In this five-and-one-half-year multicentre, longitudinal retrospective study did not find an improvement in clinical quality, as measured by an improvement in A1c values in six months and five years after the adoption of an EHR. We believe that further functionality must be added to EHRs, for example, clinical decision support systems, if they are to improve the quality of care. To our knowledge, this is the first longitudinal study to directly assess the relationship between the use of an EHR and clinical quality.

Contributorship

All the authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; they assisted in drafting the article or revising it critically for important intellectual content; and they provided final approval of the version to be published.

Funders

Extramural funding was provided by the U.S. Army Medical Research and Materiel Command under cooperative agreement W81XWH-08-2-0056. The funding source had no involvement in the collection, analysis and interpretation of the data, nor in the writing of the report or decision to submit the article for publication.

Prior presentations

None

Conflict of interest

The authors do not have any conflicts of interest. (See the Conflict of interest form).

Disclaimer

If the manuscript is accepted for publication, we are required by policy to include the following disclaimer: The views expressed in this manuscript are solely those of the authors and not necessarily reflect the opinion or position of the Uniformed Services University of the Health Sciences, the U.S. Army Medical Research and Materiel Command or the U.S. Department of Defense.

This study was approved by the Uniformed Services University of the Health Sciences Institutional Review Board.


REFERENCES

1. Institute of Medicine. Key Capabilities of an Electronic Health Record System. Washington, DC: National Academies Press, 2003.

2. Bates DW, Ebell M and Gotlieb E et al. A proposal for electronic medical records in U.S. primary care. Journal of the American Medical Informatics Association 2003;10(1):1–10. http://dx.doi.org/10.1197/jamia.M1097. PMid:12509352; PMCid:PMC150354.

3. Linder JA, Ma J, Bates DW, Middleton B and Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Archives of Internal Medicine 2007;167:1400–1405. http://dx.doi.org/10.1001/archinte.167.13.1400. PMid:17620534.

4. Keyhani S, Hebert PL, Ross JS, Federman A, Zhu CW and Siu AL. Electronic health record components and the quality of care. Medical Care 2008;46:1267–1272. http://dx.doi.org/10.1097/MLR.0b013e31817e18ae. PMid:19300317.

5. Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ and Bates DW et al. The relationship between electronic health record use and quality of care over time. Journal of the American Medical Informatics Association 2009;16:457–464. http://dx.doi.org/10.1197/jamia.M3128. PMid:19390094; PMCid:PMC2705247.

6. Friedberg MW, Coltin KL, Safran DG, Dresser M, Zaslavsky AM and Schneider EC et al. Associations between structural capabilities of primary care practices and performance on selected quality measures. Annals of Internal Medicine 2009;151:456–463. http://dx.doi.org/10.7326/0003-4819-151-7-200910060-00006. PMid:19805769.

7. Walsh MN, Yancy CW, Albert NM, Curtis AB, Stough WG and Gheorghiade M et al. Electronic health records and quality of care for heart failure. Journal of the American Heart Association 2010;159:635–642. http://dx.doi.org/10.1016/j.ahj.2010.01.006. PMid:20362723.

8. Poon EG, Wright A, Simon SR, Jenter CA, Kaushal R and Volk LA et al. Relationship between use of electronic health record features and health care quality: results of a statewide survey. Medical Care 2010;48:203–209. http://dx.doi.org/10.1097/MLR.0b013e3181c16203. PMid:20125047.

9. Romano MJ and Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine 2011;171:897–903. http://dx.doi.org/10.1001/archinternmed.2010.527. PMid:21263077; PMCid:PMC4016790.

10. Kern LM, Barrón Y, Dhopeshwarkar RV, Edwards A, Kaushal R and HITEC Investigators. Electronic health records and ambulatory quality of care. Journal of General Internal Medicine 2013;28:496–503. http://dx.doi.org/10.1007/s11606-012-2237-8. PMid:23054927; PMCid:PMC3599037.

11. Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D and Sperling M et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. Journal of the American Heart Association 2005;12:431–437. http://dx.doi.org/10.1197/jamia.M1788. PMid:15802479; PMCid:PMC1174888.

12. Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M and Bugbee D et al. A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. Journal of the American Heart Association 2008;15:424-429. http://dx.doi.org/10.1197/jamia.M2602. PMid:18436905; PMCid:PMC2442256.

13. Agency for Healthcare Research and Quality. Diabetes mellitus: average HbA1c value for diabetic patients in the clinical information system. Available from http://www.qualitymeasures.ahrq.gov/content.aspx?id=27617&search=Hemoglobin+A1c+measurement. Accessed on 19 June 2014.

14. Jackson JL, Strong J, Cheng EY and Meyer G. Patients, diagnoses, and procedures in a military internal medicine clinic: comparison with civilian practices. Military Medicine 1999;164(3):194–197. PMid:10091492.

15. Jackson JL, O’Malley PG and Kroenke K. A psychometric comparison of military and civilian medical practices. Military Medicine 1999;164(2):112–115. PMid:10050567.

16. Jackson JL, Cheng EY, Jones DL and Meyer G. Comparison of discharge diagnoses and inpatient procedures between military and civilian health care systems. Military Medicine 1999;164(10):701–704. PMid:10544622.

17. Gimbel RW, Pangaro L and Barbour G. America’s “undiscovered” laboratory for health services research. Medical Care 2010;48(8):751–756. http://dx.doi.org/10.1097/MLR.0b013e3181e35be8. PMid:20613659.

18. Egede LE, Mueller M, Echols CL and Gebregziabher M. Longitudinal differences in glycemic control by race/ethnicity among veterans with type 2 diabetes. Medical Care 2010;48(6):527–533. http://dx.doi.org/10.1097/MLR.0b013e3181d558dc. PMid:20473215.

19. Wong ND, Patao C, Wong K, Malik S, Franklin SS and Iloeje U. Trends in control of cardiovascular risk factors among US adults with type 2 diabetes from 1999 to 2010: comparison by prevalent cardiovascular disease status. Diabetes and Vascular Disease Research 2013;10(6):505–13. http://dx.doi.org/10.1177/1479164113496828. PMid:23975724; PMCid:PMC4227398.

20. Verdon DR. EHRs: the real story. Why a national outcry from physicians will shake the health information technology sector. Medical Economics 2014;91(3):18–27. PMid:25211940.

21. American Recovery and Reinvestment Act of 2009, Pub L, 111–5, 123 Stat 115.

22. Steinbrook R. Health care and the American recovery and reinvestment act. The New England Journal of Medicine 2009;360:1057–1060. http://dx.doi.org/10.1056/NEJMp0900665. PMid:19224738.

23. Hsiao C-J and Hing E. Use and characteristics of electronic health record systems among office-based physician practices: United States, 2001–2012. Centers for Disease Control and Prevention, National Center for Health Statistics Data Brief 2012; (111).

24. Friedberg MW, Chen PG, Van Busum KR, et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. Rand Corporation, 2013. (p. xvi). Available from http://www.rand.org/content/dam/rand/pubs/research_reports/RR400/RR439/RAND_RR439.pdf. Accessed on 20 November 2014.

25. Kellerman AL and Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Affairs 2013;32(1):63–68. http://dx.doi.org/10.1377/hlthaff.2012.0693. PMid:23297272.

26. Blumenthal D. Launching HITECH. The New England Journal of Medicine 2010;362:382–385. http://dx.doi.org/10.1056/NEJMp0912825. PMid:20042745.

27. Jha AK. Meaningful use of electronic health records. The Journal of the American Medical Association 2010;304:1709–1710. http://dx.doi.org/10.1001/jama.2010.1497. PMid:20959581.

Please see the associated commentary in this issue: de Lusignan S, Williams J, Munro N. A lack of a comparator group makes it hard to be sure whether computerised medical record system implementation achieved a better or worse outcome. J Innov Health Inform. 2016;23(1):485-487.

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT