Connecting medical records: an evaluation of benefits and challenges for primary care practices

Deborah R. Compeau

Department of Management, Information Systems, and Entrepreneurship, Carson College of Business, Washington State University, Pullman, WA, USA

Amanda L. Terry

Department of Family Medicine, Centre for Studies in Family Medicine, Schulich School of Medicine and Dentistry, The University of Western Ontario, London, ON, Canada, and Department of Epidemiology and Biostatistics, Western Centre for Public Health and Family Medicine, Schulich Interfaculty Program in Public Health, Schulich School of Medicine and Dentistry, The University of Western Ontario, London, ON, Canada

Author address for correspondence:

Deborah Compeau

Department of Management, Information Systems, and Entrepreneurship

Carson College of Business

Washington State University

Pullman, WA 99163, USA

Email: deborah.compeau@wsu.edu

Cite this article: Compeau D.R, Terry A.L. Connecting medical records: an evaluation of benefits and challenges for primary care practices. J Innov Health Inform. 2017;24(2):204–215.

Copyright © 2017 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/


ABSTRACT

Background Implementation of systems to support health information sharing has lagged other areas of healthcare IT, yet offers a strong possibility for benefit. Clinical acceptance is a key limiting factor in health IT adoption.

Objectives To assess the benefits and challenges experienced by clinicians using a custom-developed health information exchange system, and to show how perceptions of benefits and challenges influence perceptions of productivity and care-related outcomes.

Methods We used a mixed methods design with two phases. First, we conducted interviews with stakeholders who were familiar with the health information exchange system to inform the development of a measure of benefits and challenges of the use of this system. Second, using this measure, we conducted a survey of current and former users of the health information exchange system using a modified Dillman method. Data were analyzed using structural equation modelling, implemented in partial least squares.

Results 105 current and former users completed the survey. The results showed information quality, ease of completing tasks and clinical process improvement as key benefits that reduced workload and improved patient care. Challenges related to system reliability, quality of reports, and service quality increased workload and decreased impact on care, though the effect of the challenges was smaller than that of the benefits.

Conclusion Even very limited health information exchange capabilities can improve outcomes for primary care users. Improving perceptions of benefits may be even more important than removing challenges to use, though it is likely that a threshold of quality must be achieved for this to be true.

Keywords: computerized medical record, information system, primary care


INTRODUCTION

Health information technologies (HIT) offer tremendous potential to improve the quality and cost effectiveness of healthcare. However, projects to implement HIT around the world have met with mixed results.14 Central to all the explanations for HIT implementation success and failure is clinical adoption.5,6 Various models have been examined including TAM,7 UTAUT810 and an integrated model11 based on both TAM and Cenfetelli’s12 dual factors of adoption. In a related research stream, Lau and his colleagues13,14 and van der Meijden and his collegues15 have advanced a benefit evaluation framework that also addresses adoption issues.

This study examines the clinician’s perceptions of HIT (a common strategy in adoption research), but focuses on their perspectives regarding benefits arising from HIT adoption. In this way, this study bridges the two main themes in IS research on HIT identified by Agarwal et al.5: adoption and evaluation. It examines how a health information exchange technology was understood and valued by primary care practitioners. This research demonstrates both the potential benefits from a provider perspective and the limiting factors that must be addressed in ongoing system development.

The specific goal of the research was to understand the benefits and challenges of ‘PhysicianConnect’ (a pseudonym). PhysicianConnect is a technology designed to share data about patients between hospitals and primary care physicians. Sharing of data through IT remains a particular challenge in e-health. A recent survey16 showed that while primary care physicians have excellent adoption of electronic medical records (EMR) (i.e. internal systems), only about half routinely share information with other providers in an electronic format, even in countries where EMR adoption rates are 98%.

Two characteristics of PhysicianConnect are of particular importance to note. First, only three types of reports are sent through PhysicianConnect: results of in-hospital laboratory tests, in-hospital diagnostic imaging and discharge summaries for patients. Non-hospital-based laboratory tests or diagnostic images are not included nor are detailed patient reports for admitted patients. Second, the flow of data is one way; no data from the primary care EMR is shared with the hospital information system.

PhysicianConnect was developed in a bottom-up fashion. It was initiated by a physician and a hospital CIO, who saw an opportunity to connect their systems. It was designed using very simple technologies as a custom add-on to existing systems. It did not attempt to identify all of the possible data that a physician might want from a hospital information system, nor did it attempt to include other organizations from which primary care physicians received data, such as private laboratories. Regardless, it demonstrated the potential to connect systems, even in a rudimentary way, in order to avoid processing paper documents. After about two years, the hospital sought additional funding to expand the program to all physicians in its catchment area and to do some ‘recalibrating and adjusting’ of the system based on experience in working with it [interview with PhysicianConnect technologist]. Funding was provided by a government agency with the aim of expanding it beyond the initial hospital to two other hospital groups that existed within the region that had been developing a similar interface. The project ultimately expanded in four phases so that by the time of this research, the hospitals had been using it for at least 2 years, with individual physicians having access for a few months to 2 or more years.


METHODS

The study was conducted in two phases, using a mixed-methods design.17 During the qualitative phase, semi-structured interviews were conducted with two primary care physicians who had adopted the technology, two of their administrative staffs and one hospital IT manager who supported PhysicianConnect. A group interview was also conducted with three staff members at the government agency that funded and supported the project. Respondents were asked to explain when/how they had become PhysicianConnect adopters and how they had conducted various tasks related to the receipt of reports before and after the implementation and to identify the various benefits and challenges that they perceived related to the use of PhysicianConnect. Interviews were tape recorded and transcribed. Data were coded to identify salient benefits and challenges.

In the quantitative phase of the study, the benefits and challenges identified in phase one were used to develop a survey that was sent to all registered physician users of PhysicianConnect. The questionnaire included 12 items regarding benefits and 13 items focused on the challenges, all measured on a five-point scale. In total, 295 surveys were sent using an adapted Dillman method.18 Figure 1 shows the steps in the survey administration and the responses received at each stage.

Based on the results of the interviews and a review of prior literature, a model for assessing the benefits of PhysicianConnect was constructed (Figure 2). The dual factor model represents the benefits (enablers) and challenges (inhibitors) as second-order latent constructs.19 Benefits are composed of three first-order factors: information quality, process improvement and ease of completing tasks. Challenges are composed of four first-order factors: reliability, report quality, number of reports and service quality. Dual-factor models are built on the premise that inhibitors influence behaviour differently than enablers and must, therefore, be examined separately. While increasing or decreasing an enabler is expected to promote or inhibit more use, removing an inhibitor does not necessarily promote more usage but simply does not discourage it.12,20 This model was assessed in the second phase of the study. Four hypotheses were tested, linking the benefits and challenges to each of the clinical outcomes.

The model was tested using SmartPLS,21 following the Wetzels et al.22 approach for second-order factors. Box 1 provides a more detailed explanation of the partial least squares (PLS) method. The analysis proceeded in three stages: assessment of first-order measurement model, second-order measurement model and structural model. The seven first-order factors, modelled as reflective constructs, were assessed by examining the loadings and cross loadings, and some items were removed to improve measurement quality. The three reflective second-order factors were assessed based on their loadings and cross loadings, internal consistency reliability and discriminant validity. The structural model was assessed using a bootstrapping method with 500 samples. An additional model was run with the second-order factors modelled as formative. The structural model results are similar. The main difference is that only one first-order factor was a significant predictor of the second-order factors, an outcome of the high degree of multi-collinearity between the first order factors.


RESULTS

Interview findings

Participants in the interviews identified both positive and negative aspects of PhysicianConnect as well as numerous outcomes of using it. Appendix 1 summarizes the positive and negative attributes of the system identified along with the key outcomes experienced. For each item, a representative quote from the interviews is shown, along with an explanation of where the item fits within the research model.

Survey results

Of 295 surveys, 26 were returned as undeliverable leaving the potential sample at 269. Of the 118 respondents (44% response rate), three responses had questionable data quality (greater than 20% missingness) and were dropped, leaving 115 responses. Ninety-nine were current users of PhysicianConnect. Six respondents received information via PhysicianConnect but had discontinued their usage, while ten had never used it. No other data were collected from these ten respondents. Our final sample, then, was 105 current and former users. The majority of responses came from users of the three most commonly used EMRs. Ninety-nine surveys were completed by physicians, followed by one nurse, one nurse practitioner and four administrators. They represented each of the three hospital systems who were providing data. They had been using the system for between one month and five years, with an average usage of 19 months. A comparison of early versus late responders showed no significant differences on the variables of interest.23

Figure 1 Survey administration and sample

Figure 2 Research model and hypotheses

Appendix 2 shows descriptive statistics for the survey items. Overall, the responses suggested positive outcomes from the use of PhysicianConnect. All of the means were above the scale midpoint. The most positive effects were for speed of obtaining patient results, reduction in filing errors and administrative staff workload. Beliefs about the benefits of PhysicianConnect were also generally positive. The most commonly cited benefit from using PhysicianConnect was ease of access to information. Improved information sharing, greater flexibility in where work can be done, staff ease of response to patient inquires, improved organization of information, improved ability to search for patient information and better follow-up with patients were also widely reported as benefits. Improved legibility of information was reported by some, but not all respondents. Three benefits were not as widely reported (average was less than 3): ­ease of creating referrals, updating the cumulative patient profile (CPP) and creating alerts.

The respondents report fewer challenges from using PhysicianConnect than they did benefits, and fewer than expected based on the phase one interviews, though most reported having experienced or observed each challenge. Duplicate reports were most often reported as a significant problem. Two other challenges were reported frequently: lack of support from the EMR vendor and not being able to tell whether the data are flowing. Three challenges were reported as not being significant concerns. The majority of respondents indicated having had no problems with date formats from PhysicianConnect not being compatible with the EMR. Nearly 50% of users reported that they had not experienced or observed an issue with getting too many reports from PhysicianConnect, nor had they been unable to update their EMR because of PhysicianConnect.

Survey – measurement model

First-order factors

For the three first-order factors of benefits, all but two loadings were above 0.7. The loading for ‘improved ability to search for patient information’ was 0.68, which is very close to the 0.7 threshold, so this item was maintained. The loading for ‘improved legibility compared to paper’ was 0.52. While the item seems to reflect information quality, it clearly did not fit with the other items. One possibility is that most respondents were already receiving reports by fax and so were not seeing PhysicianConnect as a replacement for handwriting but rather as a replacement of fax. Regardless, the item was the most variable of all of the benefits items, and since it did not load strongly on the construct, it was dropped.

For the four first-order factors of challenges, the measurement model was mixed. The service quality and reliability constructs both had loadings above 0.7 for each item. The number of reports construct had one very high (0.87) and one very low loading (0.23). Since removing the latter would leave a construct with a single item, and since both the number of reports and report quality issues related to the functionality of the system, the construct was merged with the report quality issues. The construct reflecting report quality had no loadings over 0.7. Examination of the items and reviews of the interview data suggested one possible explanation for this outcome. Several of the items in the report quality item reflected issues with a single EMR product: ‘inability to upgrade’, ‘incompatible date formats’ and ‘reports received in a way that does not enable cut and paste’. Since they were not experienced universally by respondents, they were dropped from the analysis.

The items for impact on workload are all loaded > 0.7. The loadings for the care outcomes construct were weaker, with one loading over 0.7, one loading (errors compared to reports that were scanned) of 0.56 and three items with loadings between 0.65 and 0.7. The higher loading items were retained consistent with prior decisions and the lower one was dropped.

With these modifications, the measurement model was reassessed. All internal consistency reliabilities were above 0.70 and each item loaded more strongly on its own construct than on the related constructs, indicating adequate discriminant validity. To proceed to the next phase of the assessment, the factor scores were saved for the first-order dimensions.

Table 1 Loadings and cross loadings

Second-order factors

The second-order factors were also modelled as reflective, since they were expected to be highly correlated. The three items reflecting benefits all had loadings (Table 1) greater than 0.7. Two of the three items reflecting challenges had loadings greater than 0.7 with reliability loading at 0.65. Since reliability is an established dimension of the benefits evaluation framework, it was retained despite the lower loading.

The cross loadings for the items were also examined showing that each item loaded more strongly on its intended construct than on any of the others. To assess discriminant validity at the construct level, the square root of the average variance extracted (AVE) was compared with the inter-construct correlations. In all cases, the analysis showed that there was more variance in common between the constructs and their measures than between constructs, supporting discriminant validity. Each construct also had an internal consistency reliability above 0.70 (Table 2).

Survey – structural model

As shown in Figure 3, all four hypotheses were supported. The benefits of using PhysicianConnect significantly and positively influenced the workload and care outcomes experienced by physicians. Stronger perceptions of information quality, process improvement and ease of completing tasks resulted in a perception that nursing, administrative and even physician workload were improved. They also influenced perceptions that care was improved by reducing errors and by improving the speed of referrals and receipt of patient results.

Challenges with using PhysicianConnect also significantly influenced the outcomes, negatively as expected. Greater challenges with reliability, report quality and service quality promoted perceptions of worsening workload and worsening care.

The analysis also showed that benefits had a stronger positive influence on the outcomes than challenges had a negative one (p<0.001). Improving (perception of) benefits had a stronger impact than reducing (the perception of) challenges. This differential impact is consistent with the dual-factor conceptualization of enablers and inhibitors.

Table 2 Correlations between constructs

Figure 3 Structural model results


DISCUSSION

Findings

This study suggests that benefits were realized from the use of PhysicianConnect, particularly in relation to quality of care and productivity. Even a very limited form of connectivity (one-way communication of test results in a simple text format) resulted in perceived net benefits to clinical productivity and quality of care. This finding supports the view that integration of health records is a source of clinical value, and further demonstrates that even limited integration can provide some benefits.

Some what surprisingly, these benefits were realized in spite of challenges in both system and service qualities. Viewed from the perspective of traditional IS evaluation models,24 this should not be successful. Chatterjee et al.25 report that system and service qualities were essential to the realization of benefits in the context of mobile health technologies. They contend that if clinicians cannot depend on a system to function reliably or do not receive adequate support when there are problems, they will reject the system. A conversation with one of the staffs who had been managing the PhysicianConnect program echoed this view. Yet, despite acknowledged problems with PhysicianConnect, clinicians in this study were very positive towards it. They did not report a lot of problems with it (the average scores in the survey on the challenges related to system reliability and service quality are not as high as one might have expected given the acknowledged challenges from phase one). More importantly, the impact of the challenges construct on productivity and workload was lower than the impact of benefits.

It appears as though, in this case, the strong benefits related to information quality outweighed the limitations in system and service quality, and resulted in clinical acceptance of the system and ultimately the desired benefits. It appears as though the system and service were ‘good enough’ for adopters to realize the benefits associated with the access to information. This reflects quadrant II in Cenfetelli’s12 dual-factors model, and the results support his prediction that this quadrant can yield positive results (for adoption in his case, but benefits as observed here).

This finding is important because it shows that a new technology can be successful in a healthcare environment even if it falls short of the ideal on some quality aspects. Providing a workable system that gives clinicians tangible benefits in terms of health outcomes may be sufficient to encourage initial adoption, and thus may create the opportunity for continuing technology development to improve the system and extend its use. Thus, while the benefits evaluation framework that has been used in much of the literature24 presents system quality and service quality as linear predictors of use and satisfaction, our results suggest that they may represent a threshold factor. Whether this is unique to the PhysicianConnect context or whether this finding would generalize more broadly is unknown, and requires further investigation.

Limitations

A key limitation is the cross-sectional design with a single source of data (user perceptions). Still, examining perceptual data does provide some basis to estimate the potential benefits, and other studies26 have focused more on the quantification of benefits from HIT. As Goh et al.27 note, further research is needed on the complex processes of clinical adoption and acceptance that result in these benefits. This study extends prior literature to demonstrate how beliefs about enablers and inhibitors associated with the system influence perceptions of the benefits.


CONCLUSION

This study demonstrates the potential effectiveness of health information exchange, even in a limited fashion, and contributes to our understanding of benefits evaluation by linking the benefits evaluation model14,24,28 to the dual factors model of IT acceptance.11,12 In doing so, this study of PhysicianConnect shows the potential for successful HIT implementation and benefits realization, even in the face of system limitations.


REFERENCES

1. Abraham C and Junglas I. From cacophony to harmony: a case study about the IS implementation process as an opportunity for organizational transformation at Sentara Healthcare. Journal of Strategic Information Systems 2011;20(2):177–97. Available at: https://doi.org/10.1016/j.jsis.2011.03.005.

2. Jensen TB. Design principles for achieving integrated healthcare information systems. Health Informatics Journal 2013;19(1): 29–45. Available at: https://doi.org/10.1177/1460458212448890.

3. Cho S, Mathiassen L and Nilsson A. Contextual dynamics during health information systems implementation: an event-based actor-network approach. European Journal of Information Systems 2008;17:614–30. Available at: https://doi.org/10.1057/ejis.2008.49.

4. Lapointe L and Rivard S. A triple take on information system implementation. Organization Science 2007;18(1):89–107. https://doi.org/10.1287/orsc.1060.0225.

5. Agarwal R, Gao G, DesRoches C and Jha AK. Research commentary–The digital transformation of healthcare: current status and the road ahead. Information Systems Research 2010;21(4):796–809. Available at: https://doi.org/10.1287/isre.1100.0327.

6. Boonstra A and Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Services Research 2010;10(1):231. Available at: https://doi.org/10.1186/1472-6963-10-231.

7. Holden RJ and Karsh BT. The technology acceptance model: its past and its future in health care. Journal of Biomedical Informatics 2010;43(1):159–72. Available at: https://doi.org/10.1016/j.jbi.2009.07.002 PMid:19615467 PMCid:PMC2814963.

8. Kijsanayotin B, Pannarunothai S and Speedie SM. Factors influencing health information technology adoption in Thailand’s community health centers: applying the UTAUT model. International Journal of Medical Informatics 2009;78(6):404–416. Available at: https://doi.org/10.1016/j.ijmedinf.2008.12.005.

9. Li J, Talaei-Khoei A, Seale H, Ray P and Macintyre CR. Health care provider adoption of eHealth: systematic literature review. Interactive Journal of Medical Research 2013;2(1):e7. Available at: https://doi.org/10.2196/ijmr.2468.

10. Venkatesh V, Sykes T and Zhang X. “Just what the doctor ordered”: a revised UTAUT for EMR system adoption and use by doctors. Proceedings of the 44th Hawaii International Conference on System Sciences, Vol Koloa, Kauai, IEEE Computer Society 2011;1–10. Available at: https://doi.org/10.1057/palgrave.ejis.3000717.

11. Bhattacherjee A and Hikmet N. Physicians’ resistance toward healthcare information technology: a theoretical model and empirical test. European Journal of Information Systems 2007;16(6):725–37. doi: 10.1057/palgrave.ejis.3000717.

12. Cenfetelli R. Inhibitors and enablers as dual factor concepts in technology usage. Journal of the Association for Information Systems 2004;5(11):472–492.

13. Lau F, Hagens S and Muttitt S. Framework for health information systems in Canada. Healthcare Quarterly 2007;10(1):112–18.

14. Lau F, Price M and Keshavjee K. From benefits evaluation to clinical adoption: making sense of health information system success in Canada. Healthcare Quarterly 2011;14(1):39–45. Available at: https://doi.org/10.12927/hcq.2011.22157. PMid:21301238.

15. van der Meijden MJ, Tange HJ, Troost J and Hasman A. Determinants of success of inpatient clinical information systems: a literature review. Journal of the American Medical Informatics Association 2003;10(3):235–43. Available at: https://doi.org/10.1197/jamia.M1094. PMid:12626373 PMCid:PMC342046.

16. Schoen C and Osborn R. The commonwealth fund 2012 international health policy survey of primary care physicians. 2012 International Symposium on Health Care Policy 2012:33.

17. Cresswell JW and Plano VL. Designing and Conducting Mixed Methods Research, 2nd edition. Thousand Oaks, CA: Sage, 2011.

18. Dillman DA. Mail and Internet Surveys: The Tailored Design Method, 2nd edition. New York: John Wiley, 2000.

19. Law KS, Wong CS and Mobley WH. Towards a taxonomy of multidimensional constructs. Academy of Management Review 1998;23(4):741–55. Available at: https://doi.org/10.5465/AMR.1998.1255636. Available at: https://doi.org/10.2307/259060.

20. Bhattacherjee A and Hikmet N. Physicians’ resistance toward healthcare information technology: a theoretical model and empirical test. European Journal of Information Systems 2007;16(6):725–37. Available at: https://doi.org/10.1057/palgrave.ejis.3000717.

21. Ringle CM, Wende S and Will A. SmartPLS. 2005. SmartPLS 2.0.M3. Hamburg: SmartPLS. Retrieved from Available at: http://www.smartpls.com. Accessed 3 February 2015.

22. Wetzels M, Odekerken-Schroder G and van Oppen C. Using PLS path modeling for assessing hierarchical construct models: guidelines and empirical illustration. Management Information Systems Quarterly 2009;33(1):177–95.

23. Armstrong JS and Overton TS. Estimating nonresponse bias in mail surveys. Journal of Marketing Research 1977;14(3):396–402. Available at: https://doi.org/10.2307/3150783.

24. Delone W and McLean E. The DeLone and McLean model of information systems success: a ten-year update. Journal of Management and Information Systems 2003;19(4):9–30.

25. Chatterjee S, Chakraborty S, Sarker S, Sarker S and Lau FY. Examining the success factors for mobile work in healthcare: a deductive study. Decision Support System 2009;46(3):620–633. Available at: https://doi.org/10.1016/j.dss.2008.11.003.

26. Devaraj S and Kohli R. Performance impacts of information technology: is actual usage the missing link? Management Science 2003;49(3):273–289. Available at: https://doi.org/10.1287/mnsc.49.3.273.12736.

27. Goh J, Gao G and Agarwal R. Evolving work routines: adaptive routinization of information technology in healthcare. Information Systems Research 2011;22(3):1–51. Available at: https://doi.org/10.1287/isre.1110.0365.

28. Lau F, Hagens S and Muttitt S. A proposed benefits evaluation framework for health information systems in Canada. Healthcare Quarterly 2007;10(1):112–118.


APPENDIX 1: ASSESSMENTS OF PHYSICIAN CONNECT IN PHASE 1 INTERVIEWS


APPENDIX 2: MEASUREMENT ITEMS

PhysicianConnect impact (net benefits)

How does having the PhysicianConnect interface for receiving patient results in your practice affect each of the following outcomes?

PhysicianConnect benefits

To what extent have you personally observed or experienced each of the following potential benefits from having the PhysicianConnect interface for receiving patient results in your practice?

PhysicianConnect challenges

To what extent have you personally observed or experienced each of the following potential challenges from having the PhysicianConnect interface for receiving patient results in your practice?

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT