The impact of interoperability ofelectronic health records on ambulatory physician practices: a discrete-eventsimulation study

Yuan Zhou

Graduate Student, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Jessica S Ancker

Assistant Professor, Departments of Pediatrics and Public Health, Weill Cornell Medical College, New York, NY 10065, USA

Mandar Upadhye

Graduate Student, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Nicolette M McGeorge

Graduate Student, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Theresa K Guarrera

Graduate Student, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Sudeep Hegde

Graduate Student, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Peter W Crane

Assistant Professor, Department of Emergency Medicine, University of Rochester, Rochester, NY 14642, USA

Rollin J Fairbanks

Director, National Center for Human Factors in Healthcare, MedStar Institute for Innovation, MedStar Health, Washington, DC 20008, USA

Associate Professor, Department of Emergency Medicine, Georgetown University, Washington, DC 20007, USA

Adjunct Associate Professor, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Ann M Bisantz

Professor, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Rainu Kaushal

Executive Director, Center for Healthcare Informatics and Policy, Weill Cornell Medical College, NewYork, NY 10065, USA

Li Lin

Professor, Department of Industrial and Systems Engineering, University at Buffalo, Buffalo, NY 14260, USA

Health Information Technology Evaluation Collaborative Investigators

Cite this article: Zhou Y, Ancker JS, Upadhye M, McGeorge NM, Guarrera TK, Hegde S, Crane PW, Fairbanks RJ, Bisantz AM, Kaushal R, Lin L. The impact of interoperability of electronic health records on ambulatory physician practices: a discrete-event simulation study. Inform Prim Care 2013;21(1):21–29.

Author address for correspondence:

Yuan Zhou

Department of Industrial and Systems Engineering

University at Buffalo

339b Bell Hall

Buffalo, NY 14260, USA

Email: yuanzhou@buffalo.edu


ABSTRACT

Background The effect of health information technology (HIT) on efficiency and workload among clinical and nonclinical staff has been debated, with conflicting evidence about whether electronic health records (EHRs) increase or decrease effort. None of this paper to date, however, examines the effect of interoperability quantitatively using discrete event simulation techniques.

Objective To estimate the impact of EHR systems with various levels of interoperability on day-to-day tasks and operations of ambulatory physician offices.

Methods Interviews and observations were used to collect workflow data from 12 adult primary and specialty practices. A discrete event simulation model was constructed to represent patient flows and clinical and administrative tasks of physicians and staff members.

Results High levels of EHR interoperability were associated with reduced time spent by providers on four tasks: preparing lab reports, requesting lab orders, prescribing medications, and writing referrals. The implementation of an EHR was associated with less time spent by administrators but more time spent by physicians, compared with time spent at paper-based practices. In addition, the presence of EHRs and of interoperability did not significantly affect the time usage of registered nurses or the total visit time and waiting time of patients.

Conclusion This paper suggests that the impact of using HIT on clinical and non-clinical staff work efficiency varies, however, overall it appears to improve time efficiency more for administrators than for physicians and nurses.

Keywords: electronic health record (EHR), interoperability, computer simulation, physician practice

What this paper adds:

• Currently, physician practices use EHR systems at various interoperability levels which directly affect their workflows and time efficiencies.

• With higher EHR interoperability levels, the time spent on each interoperable task can be significantly reduced.

• The administrators tend to be more time efficient with the EHR-based system than the paper-based system, and their time efficiency could be further improved by using the EHR system at higher levels interoperability.

• Some barriers are still present in physician use of EHRs which decreases physician time efficiency.


INTRODUCTION

Attention to the use of health information technology (HIT) has increased significantly in the United States since The American Recovery and Reinvestment Act (ARRA) of 2009 earmarked an investment of $36 billion to help modernize HIT systems. Approximately $19 billion of this amount was allotted for incentivizing the meaningful use of electronic health records (EHRs).1 The percentage of office-based physicians using EHR systems rose to an estimated 57% in 2011.2 When EHR systems are interoperable, it becomes possible for health care providers and other stakeholders to share useful information electronically within and between enterprises to care for their patients. Therefore, interoperability is considered a key component of meaningful use.3 Despite potential benefits offered by the use of interoperable HIT systems, barriers to adoption are substantial. 48 One of the main barriers is the complexity and heterogeneity of clinical workflows that may not be fully supported by HIT systems.4 The implementation of EHR systems can introduce radical changes to both clinical and administrative workflows which could have undesirable impacts on end-user satisfaction, time efficiency, quality of care, and patient safety.5 Because of these effects on workflow, there have been significant concerns about the net effect of EHRs on efficiency and workload of the various stakeholders in the health care system.

This paper focuses on small independent primary and specialty care physician practices, which comprise about one-third of U.S. physicians practice.912 To further explore the impact to workflow by EHRs and extend the work to cover forms of interoperability that are becoming more widely available, we modeled workflow effects with a discrete-event simulation (DES) model, which relies upon highly realistic models of workflow, and then uses the model to qualify effects of interest such as time spent on tasks. We constructed our workflow models through an in-depth quantitative study13 that produced a detailed description of interoperability levels achieved by HIT systems.14


METHODS

Twelve ambulatory practices in the Rochester, NY, area participated in this paper from February 2010 to June 2011, including seven adult primary care practices and five adult specialty care practices. We categorized all practices into one of four stages: (1) fully paper-based; (2) largely paper-based: the majority of tasks were conducted using paper and only a few tasks were done with the EHR system; (3) largely EHR-based: the majority of tasks were conducted using the EHR system and limited tasks were done using paper; and (4) fully EHR-based. The Institutional Review Boards of the University of Rochester and the University at Buffalo approved this paper.

Data collection

Data were collected in two stages by three faculty members and four graduate students from University of Rochester Medical Center and University at Buffalo. First, semistructured interviews and direct observations were conducted to gather general information about practice operations (i.e., daily tasks performed by staff, number of patients per day, and so on). Field notes were taken during interviews to capture the flow of tasks and confirmed by the interviewees. Second, participants were invited to self-report estimated minimum, maximum, and modal times for patient arrival rates and process times through in-person interviews or telephone follow-up interviews.

Development of an interoperability model

A framework for classifying levels of interoperability was developed based on how health information exchange affects the tasks of primary and specialty care providers.1518 This framework produced a seven-level decision ladder (Figure 1) allowing us to classify the degree of IT involvement in the execution of individual clinical and administrative tasks. The levels range from no use of IT to share information (Level 0) to fully automated and semantically interoperable systems to complete a task (Level 6).

Workflow analysis

The micro-level tasks collected during the first stage of data collection were aggregated into ‛standard tasks’ that are typically performed in every practice. For instance, all practices had a standardized task of ‘prepare lab reports,’ but each used somewhat different micro-level steps to accomplish it. Some practices received lab reports by mail, paper fax, or electronic fax, while others queried an electronic health information exchange portal supported by the local regional health information organization (RHIO) or received electronic data directly into the EHR system. These standardized tasks were used as the basic elements of the simulation model. Furthermore, each standardized task was classified as ‛requires’ or ‘does not require’ health information exchange with external organizations by either paper or electronic means (Table 1). In this paper, we focused on the four interoperable tasks: ‘prepare lab reports,’ ‘request lab orders,’ ‘prescribe medications,’ and ‘write referrals.’ ‘Billing’ was not considered because the information technology use is fairly mature and efficient for this purpose and some practices employ billers who were out of the scope of this paper.

Figure 1 Decision ladder of proposed interoperability level.14

Data processing for simulation input

DES models are ‛run’ using specialized software packages resulting in measures including wait times or queue lengths for each process stage; resource utilization; and time for entities to move through the system. DES can be used to estimate the impact of changes in patient flow, to examine resource needs, or to investigate the complex relationships among different model variables.1922 The simulation model required input distributions for (1) arrival times of patients and (2) process times of tasks. For patient arrival times, a Poisson distribution with constant arrival rates was assumed. We used empirical distributions on the basis of the time data collected in the second stage. In cases where data was extremely sparse (i.e., only a single data point presented for a task), triangular or uniform distributions were used to estimate the duration that was obtained directly from the collected data.

Table 1 List of all the practice daily tasks

Model development, verification, and validation

The simulation model was constructed to model patient flow and practice operation using ARENA, a DES package [Rockwell Automation, 2000]. This modeling approach requires entities, resources, and processes as inputs. In this simulation, patients were considered entities which were created upon their arrival to the practice, moved around in the practice and disappeared when they left the practice. Administrators, nurses, and physicians were considered to be the available resources. For simplicity in modeling typical small practices, we assigned one person for each role. In addition, to capture the full workload, tasks of indirect patient care were also included in the simulation model such as ‘prepare lab reports,’ ‘phone calls,’ and ‘prepare patient charts.’

For model verification and validation, three approaches were taken. First, the logic and components of the model were evaluated by health care professionals (i.e., clinical staff) during the first stage of data collection to ensure that the model represented the system realistically. Second, after the input distributions were determined and the initial simulation model was constructed, the results of several pilot runs were compared with the collected data to ensure consistency and accurate estimation of system performance. Finally, direct observations were conducted for the check-in and check-out processes, and the associated activity times were recorded by researchers. Two-sample t-test and Levene’s test were used to assess the differences of sample means and variances obtained from self-reported data and observational data. No significant differences were found, indicating that the self-reported data were fairly reliable.

Outcome measures

Our qualitative findings confirmed that ongoing concerns for evaluating EHR implementation in various health care settings are resource utilization and workflow efficiency, and suggested three types of metrics for measuring these variables.

1. The first, average duration of the interoperable tasks, was simply an indicator of the amount of time spent on a task at each level of interoperability.

2. The second measure was the relative changes in time associated with the tasks. As we collected data on the major tasks of administrators, nurses, and physicians, we also noted multi-tasking activities, although not often, and additional and rare events that are difficult to observe. These were not considered in the model directly without prolonged observation of complete daily work because of their lack of patterns. In addition, billing was not included in this model. Therefore, we used a relative measure of utilization. That is, all the times were normalized against the highest value in the same experiment to represent the relative variation compared with the ‘busiest’ case. This relative measure allowed conditions to be compared against one another, although not against an absolute benchmark.

3. The third measure was patient flow time, including the total time a patient spent at the practice and the waiting time during the visit.

Experimental design

The simulation model was used to conduct a Monte-Carlo experiment in which different parameters and model configurations served as independent variables of interest, and repeated simulation runs were used to measure the outcome variables. In this experiment, the interoperability of the EHR was considered the independent factor of interest, and the seven levels of interoperability were treated as the levels of the factor (L0–L6). To assess the main effects of the EHR interoperability, the experiment was conducted for all meaningful scenarios under each implementation stage. Table 2 lists all the configurations evaluated within each implementation stage. The first 16 configurations represented a full factorial design for the interoperability and task combinations possible within a fully EHR-based system: ‘prepare lab reports’ (at four possible interoperability levels), ‘request lab orders’ (two levels), ‘write referrals’ (two levels), and ‘prescribe medications’ (one level). Configurations 17, 18, and 19 represent the largely EHR-based, largely paper-based, and fully paper-based systems, respectively. The number of simulation runs per configuration was determined at a significance level of 0.05 with a 4% relative error per output performance. Analysis of simulation outputs was conducted by general linear models (GLMs) in SPSS [SPSS Statistics, version 19.0, 2010]. Analyses of variance (ANOVA) were used to explore the significance of individual performance models, the main effects of the input factors on each performance, and the differences among configurations. Tukey’s HSD tests were conducted within models statistically significant at 0.05 to identify significant differences among the configurations.

Table 2 Configurations of the experimental design


RESULTS

Among tasks that required information exchange, time spent decreased as the levels of interoperability increased. ANOVA showed significant differences in durations for all four of the information exchange tasks (p < 0.001). Figure 2 shows the average durations and standard deviations of the four interoperable tasks at all permissible interoperability levels. Figure 2A demonstrates the durations of ‘prepare lab reports’ at L4 and L5 (49.8 ± 1.8 and 45.1 ± 1.3 s, respectively) were shorter than that of L0, L2, and L3 (240.0 ± 5.2, 239.9 ± 4.9, and 180.1 ± 4.7 s, respectively). Figure 2B shows that the shortest time, on average, to request a lab order was 90.5 ± 5.5 s when the order was completed and sent directly from the EHR system (L4). This represented 55% and 80% time savings compared with the electronic fax method (L2, 201.7 ± 17.2 s) and the traditional paper method (L0, 449.5 ± 38.4 s), respectively. Figure 2C shows that the average task duration dropped more than 30% when paper prescriptions (L0, 119.7 ± 6.6 s) were replaced with e-prescribing (L5, 80.1 ± 3.8 s). Writing and sending referrals (Figure 2D) consumed approximately 447.9 ± 65.86 s per referral when the paper method was used (L0), which could be shortened by ~46% when some basic electronic technology was used, such as e-mails (L2, 240.5 ± 18.2 s). Up to 66% of time was saved when automatic connections were available between EHR systems (L3, 151.2 ± 24.6 s).

To explore which of the 19 experiment configurations were significantly different from the others; GLMs were constructed individually for each performance measure of interest (relative time usage of administrators, nurses and physicians, and the total time and waiting time of patient visits). By ANOVA, only the relative time usage of administrators (p < 0.001) and the relative time usage of physicians (p < 0.001) were significant. Within the administrators’ model, Tukey’s HSD test shown (1) the full paper-based system (configuration 19, 1.00 ± 0.02) and the largely paper-based system (configuration 18, 0.98 ± 0.03) were significantly less efficient than the other configurations (diagonal crosshatching, Figure 3A). (2) Configurations that included preparing lab reports with high interoperability levels L4 or L5 (grey, Figure 3A) were significantly more efficient than other configurations. The effect on physician time was the opposite: the two configurations that were significantly different within physician time usage were the full paper-based system (configuration 19, 0.85 ± 0.08) and the largely paper-based system (configuration 18, 0.86 ± 0.08; diagonal crosshatching, Figure 3B); both showed less time usage on day-to-day activities than any of the electronic configurations.

Finally, ANOVA was used to examine the main effect of EHR interoperability for the two statistically significant models. We found significant differences in two main effects: (1) increasing interoperability associated with increasing efficiency in ‘preparing lab reports’ by administrators, (p < 0.001) (Figure 4A) and (2) increasing interoperability levels from L0 to L2/L4 associated with decreasing efficiency in ‘requesting lab orders’ by physicians (p < 0.001) (Figure 4B).


DISCUSSION

Principal findings

Across the tasks that required some sort of information exchange, increasing levels of EHR interoperability were associated with increasing amounts of time saved. For example, preparing a lab report with a stand-alone EHR system (L2) was just as time-consuming as doing it on paper (L0), whereas higher levels of interoperability such as results delivery made report preparation increasingly efficient.

Figure 2 Average durations of interoperable tasks with standard deviation.

Figure 3 Relative time usage of all experiment configurations.

Figure 4 Main effect plots of EHR interoperability.

Our methods also permitted a comparison of EHRs versus paper. Effects of technology in general and interoperability specifically on administrative efficiency were particularly pronounced. Overall, EHRs tended to be more time efficient than paper for administrators. For instance, preparing lab reports on paper (L0) was a significant use of time by administrators. Administrators could save about 25% (L0 versus L3) of time in preparing lab reports when electronic fax was substituted for mail or paper fax, and up to 80% of time (L0 versus L5) when the report could be automatically populated into the correct file with the desired format. About 10% of overall administrators’ time could be saved when some basic EHR functions were implemented to facilitate the task (L2 and L3), and efficiency could improve by as much as 15% with higher levels of interoperability (L4 and L5). However, the EHR use and interoperability did not affect nurses, patient flow time, and patient waiting time significantly. In addition, there was no evidence from simulation results that EHR-based systems were more efficient for physicians than paper ones; in fact, paper-based systems appeared overall more efficient for physicians.

Implications of the findings

The primary conclusion is that interoperability improves time efficiency for those tasks that requires information exchange, and that increasing levels of interoperability provide increasing efficiency benefits for these tasks. The benefits accrue primarily to administrators rather than to clinical staff. An interesting secondary finding is that our data also suggest that EHRs alone increased time requirements for physicians compared with paper alone. Some barriers to physician’s EHR use could be lack of computer skills, lack of EHR interoperability, and the impact of a computer on physician–patient interaction.8 Our findings add further evidence that successful EHR implementation requires addressing clinical workflow, and additionally suggests that multiple levels of interoperability could affect workflow at the practice level.

Comparison with the literature

Previous studies in exploring the impact of EHRs on operational efficiency have shown different findings. Some health care systems have reported positive net effects on provider productivity or efficiencies from reduced workload of administrative or support staff,23,24 while some other qualitative studies reported that EHR implementation often has an undesirable impact on time efficiency and disruption to workflow. 46 Meanwhile, recent time-motion studies consistently found no statistically significant differences in both primary care and specialty physicians’ time utilization before and after EHR implementation.2527 Most quantitative studies have examined the effect of EHRs and associated technologies, but have not yet examined the effect of interoperability. The quantitative results from this paper add to the growing body of evidence about EHRs, interoperability, and the corresponding operational efficiency. Our findings of increased efficiency for nonclinical tasks are consistent with other studies showing administrative efficiencies, such as an academic outpatient network that demonstrated a positive return on investment.28

Limitations of the method

This paper used self-reported times, which may not be as accurate as times collected through the time-motion method. However, it is not clear which direction the bias would be expected to be, and our validation in this particular case indicated a good match between reported data and real observations. In addition, past research has indicated that using self-reported data for task times is reliable.2931 Another limitation is that we estimated only relative time, representing the relative time use among different scenarios compared with the busiest case, rather than the absolute time use. However, because the focus of this paper was to assess the impact the different EHR interoperability levels on practice operations, we considered relative time usage to be a potentially valuable indicator that would demonstrate the changes associated with interoperability.

Call for future research

The implementation of EHRs and various interoperability levels have been changing providers’ workload; further research is needed to investigate the practice workflow change in a more granular level, such as task switching pattern, concurrent tasks, and so on. DES models can be used to build upon providers’ perspectives to better understand the impact on EHR interoperability perceived by providers.


CONCLUSION

The effects of the interoperability of EHRs are likely to vary for different provider types and staff types in the ambulatory setting. Overall, interoperability appears to save time for administrators, require more time for physicians, and leave nurses unaffected. Higher levels of EHR interoperability improve the efficiency of performing tasks which require health information exchange. The discrete event simulation employed in this paper has the ability to estimate these effects at a fine granular level and would be highly relevant to evaluating the effects of health information technology in other healthcare settings.


Acknowledgements

This paper was supported by funding from the New York State Department of Health (NYS contract number C023699) and was conducted as part of the Health Information Technology Evaluation Collaborative (HITEC), a consortium of research institutions conducting evaluation research on New York State health information technology and health information exchange projects. The authors acknowledge the contribution of the HITEC investigators for their feedback on the study design.


REFERENCES

1.Oracle Corporation. Interoperability: A Key to Meaningful Use. Oracle 2010. http://www.oracle.com/us/industries/healthcare/interoperability-wp-188782.pdf.

2. Hsiao CJ, Hing E, Socey TC and Cai B. Electronic health record systems and intent to apply for meaningful use incentives among office-based physician practices: United States, 2001– 2011. NCHS Data Brief 2011;(79):1–8.

3. Dobrev A, Stroetmann KA, Stroetmann VN et al. The conceptual framework of interoperable electronic health record and ePrescribing systems. EHR Impact 2008. http://www.ehr-impact.eu/downloads/documents/EHRI_D1_2_Conceptual_framework_ v1_0.pdf

4. Lee J, Cain C, Young S, Chockley N and Burstin H. The adoption gap: health information technology in small physician practices. Health Affairs (Millwood) 2005;24(5):1364–6. http://dx.doi.org/10.1377/hlthaff.24.5.1364. PMid:16162585.

5. Zheng K, Haftel HM, Hirschl RB, O’Reilly M and Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. Journal of the American Medical Informatics Association 2010;17(4):454–61. http://dx.doi.org/10.1136/jamia.2010.004440. PMid:20595314; PMCid:PMC2995654.

6. Miller RH and Sim I. Physicians’ use of electronic medical records: barriers and solutions. Health Affairs (Millwood) 2004;23(2):116–26. http://dx.doi.org/10.1377/hlthaff.23.2.116.

7. Poissant L, Pereira J, Tamblyn R and Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. Journal of the American Medical Informatics Association 2005;12(5):505–16. http://dx.doi.org/10.1197/jamia.M1700. PMid:15905487; PMCid:PMC1205599

8. Yan H, Gardner R and Baier R. Beyond the focus group: understanding physicians’ barriers to electronic medical records. Joint Commission Journal on Quality and Patient Safety 2012;38(4):184–91. PMid:22533131.

9. Audet AM, Doty MM, Peugh J, Shamasdin J, Zapert K and Schoenbaum S. Information technologies: when will they make it into physicians’ black bags? Medscape General Medicine 2004;6(4):2. PMid:15775829; PMCid:PMC1480565.

10. Liebhaber A and Grossman JM. Physicians moving to mid-sized, single-specialty practices. Track Report 2007(18):1.

11. Lohr S. How to make electronic medical records a reality. The New York Times 2009. http://www.nytimes.com/2009/03/01/business/01unbox.html

12. Miller RH, West C, Brown TM, Sim I and Ganchoff C. The value of electronic health records in solo or small group practices. Health Affairs (Millwood) 2005;24(5):1127–37. http://dx.doi. org/10.1377/hlthaff.24.5.1127. PMid:16162555.

13. McGeorge NM, Hegde S, Guarrera TK, Zhou Y, Lin L, Crane PW et al. Studying the impact of interoperable electronic health records on workflow in ambulatory care. International Journal of Industrial Ergonomics 2013 (in preparation).

14. Guarrera TK, McGeorge NM, Ancker JS, Hegde S, Zhou Y, Lin L et al. Characterizing the effect of interoperability on healthcare work: a novel framework. Theoretical Issues in Ergonomics 2012 (in preparation).

15. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW and Middleton B. The value of health care information exchange and interoperability. Health Affairs (Millwood) 2005;(Suppl): W5-10–W5-18.

16. Garde S, Knaup P, Hovenga EJS and Heard S. Towards semantic interoperability for electronic health records-domain knowledge governance for openEHR archetypes. Methods of Information in Medicine 2007;46(3):332–43.

17. Lewis GA, Morris E, Simanta S, Wrage L (Eds). Why standards are not enough to guarantee end-to-end interoperability. Seventh International Conference on Composition Based Software Systems. ICCBSS 2008 (Feb 25–29, 2008). 164–73.

18. Brailer DJ. Interoperability: the key to the future health care system. Health Affairs (Millwood) 2005;(Suppl):W5-19–W5-21.

19. Jun J, Jacobson S and Swisher J. Application of discrete-event simulation in health care clinics: a survey. The Journal of the Operational Research Society 1999;50(2):109–23.

20. Côté MJ. Patient flow and resource utilization in an outpatient clinic. Socio-Economic Planning Sciences 1999;33(3):231–45. http://dx.doi.org/10.1016/S0038-0121(99)00007-5.

21. Law AM and Kelton WD. Simulation modeling and analysis. New York: McGraw-Hill, 1991.

22. Kelton WD, Sadowski RP and Sadowski DA. Simulation with ARENA. Boston, MA: McGraw-Hill, 2002.

23. Lorenzi N, Kouroubali A, Detmer D and Bloomrosen M. How to successfully select and implement electronic health records (EHR) in small ambulatory practice settings. BMC Medical Informatics and Decision Making 2009;9(1):15. http://dx.doi.org/10.1186/14726947-9-15. PMid:19236705; PMCid:PMC2662829.

24. Samaan ZM, Klein MD, Mansour ME and DeWitt TG. The impact of the electronic health record on an academic pediatric primary care center. The Journal of Ambulatory Care Management 2009;32(3):180–7. http://dx.doi.org/10.1097/JAC.0b013e3181ac9667. PMid:19542807.

25. Overhage JM, Perkins S, Tierney WM and McDonald CJ. Controlled trial of direct physician order entry effects on physicians’ time utilization in ambulatory primary care internal medicine practices. Journal of the American Medical Informatics Association 2001;8(4):361–71. http://dx.doi.org/10.1136/jamia.2001.0080361

26. Pizziferri L, Kittler AF, Volk LA, Honour MM, Gupta S, Wang S et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. Journal of Biomedical Informatics 2005;38(3):176–88. http://dx.doi.org/10.1016/j.jbi.2004.11.009. PMid:15896691.

27. Lo HG, Newmark LP, Yoon C, Volk LA, Carlson VL, Kittler AF et al. Electronic health records in specialty care: a time-motion study. Journal of the American Medical Informatics Association 2007;14(5):609–15. http://dx.doi.org/10.1197/jamia.M2318. PMid:17600102 PMCid:PMC1975804

28. Grieger DL, Cohen SH and Krusch DA. A pilot study to document the return on investment for implementing an ambulatory electronic health record at an academic medical center. Journal of the American College of Surgeons 2007;205(1):89–96. http://dx.doi.org/10.1016/j.jamcollsurg.2007.02.074. PMid:17617337.

29. Swisher JR, Jacobson SH, Jun JB and Balci O. Modeling and analyzing a physician clinic environment using discrete-event (visual) simulation. Computers & Operations Research 2001; 28(2):105–25. http://dx.doi.org/10.1016/S0305-0548(99)00093-3.

30. Lee Y and Elcan A. Simulation modeling for process reengineering in the telecommunications industry. Interfaces 1996;26(3):1–9. http://dx.doi.org/10.1287/inte.26.3.1.

31. Kumar K. Technology for supporting supply chain management: introduction. Communications of the ACM 2001;44(6):58–61. http://dx.doi.org/10.1145/376134.376165.

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT