Implementation of data management and effect on chronic disease coding in a primary care organisation: A parallel cohort observational study

Michelle Greiver

Department of Family and Community Medicine, University of Toronto, Toronto, Canada, North York General Hospital, Toronto, Canada, and North York Family Health Team, Toronto, Canada

Kimberly Wintemute

Department of Family and Community Medicine, University of Toronto, Toronto, Canada, North York General Hospital, Toronto, Canada, and North York Family Health Team, Toronto, Canada

Babak Aliarzadeh

Department of Family and Community Medicine, University of Toronto, Toronto, Canada

Ken Martin

Department of Family Medicine, Queen’s University, Kingston, Canada

Shahriar Khan

Department of Family Medicine, Queen’s University, Kingston, Canada

Dave Jackson

Department of Family Medicine, University of Calgary, Calgary, Canada

Jannet Leggett

The College of Family Physicians Canada, Mississauga, Canada

Anita Lambert-Lanning

The College of Family Physicians Canada, Mississauga, Canada

Maggie Siu

University of Western Ontario, London, Canada

Author address for correspondence

Michelle Greiver

North York Family Health Team

Toronto, Canada

Email: mgreiver@rogers.com

Cite this article: Greiver M, Wintemute K, Aliarzadeh B, Martin K, Khan S, Jackson D, Leggett J, Lambert-Lanning A, Siu M. Implementation of data management and effect on chronic disease coding in a primary care organisation: a parallel cohort observational study. J Innov Health Inform. 2016;23(3):580–590.

Copyright © 2016 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/


ABSTRACT

Background Consistent and standardized coding for chronic conditions is associated with better care; however, coding may currently be limited in electronic medical records (EMRs) used in Canadian primary care.

Objectives To implement data management activities in a community-based primary care organisation and to evaluate the effects on coding for chronic conditions.

Methods Fifty-nine family physicians in Toronto, Ontario, belonging to a single primary care organisation, participated in the study. The organisation implemented a central analytical data repository containing their EMR data extracted, cleaned, standardized and returned by the Canadian Primary Care Sentinel Surveillance Network (CPCSSN), a large validated primary care EMR-based database. They used reporting software provided by CPCSSN to identify selected chronic conditions and standardized codes were then added back to the EMR. We studied four chronic conditions (diabetes, hypertension, chronic obstructive pulmonary disease and dementia). We compared changes in coding over six months for physicians in the organisation with changes for 315 primary care physicians participating in CPCSSN across Canada.

Results Chronic disease coding within the organisation increased significantly more than in other primary care sites. The adjusted difference in the increase of coding was 7.7% (95% confidence interval 7.1%–8.2%, p < 0.01). The use of standard codes, consisting of the most common diagnostic codes for each condition in the CPCSSN database, increased by 8.9% more (95% CI 8.3%–9.5%, p < 0.01).

Conclusions Data management activities were associated with an increase in standardized coding for chronic conditions. Exploring requirements to scale and spread this approach in Canadian primary care organisations may be worthwhile.

Keywords: chronic diseases, clinical audits, electronic medical records, meaningful use, primary health care


INTRODUCTION

Chronic diseases are largely managed in the primary health care setting.13 The adoption of electronic medical records (EMRs) in primary care has been associated with expectations that these applications would support quality improvement efforts for chronic conditions.4,5

However, the implementation of EMRs has not been consistently associated with better care.610 Improvements in care require management of data to support quality improvement activities.11,12 Gaps exist in terms of the EMR software used to manage data, the availability of data management personnel in primary care and the quality of data in the EMRs.

EMR applications were built to help record care for individual patients rather than for analyzing data to manage quality for practice populations.12,13 These applications often have rudimentary reporting, data export and analytic capabilities.14 In addition, running large queries can tax servers, slowing them down and interfering with daily clinical activities.15

In primary care, EMR-based measurement has often relied on the efforts of individual physicians in querying their own applications. Physicians may be often be too busy with daily patient care and may not have time to undertake these activities.16,17 Primary care teams may be able to reallocate some of the work of measuring and reporting care and outcomes to nonclinical team members such as data managers.1820

The quality of data in EMRs continues to present challenges.2124 Diagnostic coding may be missing.25 Free text may be used instead of structured data and data may be entered in inconsistent fields.2634 EMRs often require structured or coded data to enable automated recalls, point of care reminders, practice population quality improvement activities and computerized decision support.4,11,35,36

A recent analysis of 11.5 million primary care electronic records in the U.S. found significantly better quality of care for patients when a coded diagnosis of diabetes was present in the problem list.25 Lack of standardization and coding in EMRs is associated with challenges in benchmarking and comparisons, which are important activities for primary care clinical quality improvement efforts.37,38 As a result, there have been calls to improve data and implement consistent coding for chronic conditions in primary care.39

Changes in the organisation of primary care in Ontario, Canada

Primary care in Ontario, Canada, has recently evolved through the formation of interdisciplinary family health teams (FHTs)40 and the adoption of EMRs. Currently, almost 3000 family physicians are working in 240 FHTs and are serving 3 million patients or 25% of Ontario’s population.41,42 Eighty-five percent of Ontario’s family physicians report using an EMR.43

Evidence to date on improvement in FHT performance is limited,44,45 and until recently, FHTs had not been required to systematically report quality of care.

As of 2013, the Excellent Care for All Act 2010 in Ontario46 mandated the development and public reporting of quality improvement plans by FHTs. The Ministry of Health and Long Term Care of Ontario recently funded quality improvement and decision support specialist (QIDSS) positions to provide analytic services to FHTs.47

Our objectives were to describe the adoption of data management activities in a primary care organisation in Ontario, Canada, and to evaluate effects on coding to support the formation of registries of specific chronic conditions.


METHODS

Setting

This project was conducted at North York Family Health Team (NYFHT). At the time of the study, the FHT included 67 community-based family physicians and 43 allied health professionals practicing out of 17 clinical locations and providing care for over 65,000 patients (http://nyfht.com).

EMR systems had been adopted by physician groups prior to the foundation of the organisation. Physicians in the FHT used two of the most common EMR applications in Ontario: Nightingale onDemand® and TELUS Practice Solutions Suite®. These software applications accounted for 45% of the Ontario market share as of 31 October 2014.48 Data were distributed across multiple servers with no communication across servers. A major challenge for the organisation was therefore the management of data residing in different server silos and different EMR systems, leading to substantial difficulties in merging data into a single reporting system, conducting analyses and generating amalgamated reports across the FHT.

Planning the intervention

We based this project on the data infrastructure provided by the Canadian Primary Care Sentinel Surveillance Network (CPCSSN). CPCSSN is Canada’s largest multi-disease EMR surveillance system.26 It includes 11 primary care practice based research networks in seven provinces and one territory across Canada. Consenting family physicians and other primary care providers participating in CPCSSN contribute EMR data; anonymized data extracted from different EMR applications are further deidentified in several stages of processing and are sent via secure electronic file transfer protocols to a CPCSSN regional data repository where they are cleaned, standardized and then aggregated into a single national database.26,49 Case definition algorithms in the CPCSSN database have been validated using chart audits for eight chronic conditions (diabetes, hypertension, chronic obstructive pulmonary disease (COPD), depression, osteoarthritis, dementia, parkinsonism and epilepsy).50

The Data Presentation Tool reporting software

The CPCSSN Data Presentation Tool (DPT) was developed by one of the CPCSSN data managers (DJ). The DPT allowed intuitive visualization and reporting using CPCSSN data. It used Boolean terms to query both original data as extracted by CPCSSN from EMRs and cleaned, standardized data. Figure 1 presents a screenshot of the DPT. DPT can export the results of queries for further manipulation, analysis or importing to statistical software.

For this project, data originating from NYFHT were merged and cleaned as part of the usual CPCSSN processes. These data were then transferred back to the organisation along with the DPT software as shown in Figure 2.

Patient and provider re-identification processes

In order to implement data and quality improvement activities, both patients and physicians needed to be identified. This required the reidentification of the returned CPCSSN data. During the quarterly data extraction for the CPCSSN, a data linkage file was generated and remained on site at NYFHT. This file contained an identifier generated by the patient’s EMR and a linked, randomly generated number for CPCSSN and enabled reidentification.

Development of data governance, privacy and security procedures

The demonstration project required a team-based approach for data governance and privacy. The NYFHT Information Management–Information Technology Committee oversaw data governance. A privacy impact assessment and threat risk assessment were conducted by independent, external reviewers to ensure that sufficient physical, organisational and technological safeguards were operationalized.

Planning the study of the intervention

Data standardization for selected chronic conditions

Following team discussions, the Information Management–Information Technology Committee oversaw the implementation of registers of chronic conditions through standardized coding for chronic diseases. International Classification of Diseases, version 9 (ICD9) codes were used; physicians were familiar with this coding method as it was used for provincial billing. The codes were diabetes-ICD9 250, chronic obstructive pulmonary disease-496, hypertension-401 and dementia-290. These codes were used in the validated CPCSSN case definitions.51 The FHT chose the codes most commonly used across CPCSSN for each condition as their preferred codes. Following a feasibility project,52 the DPT was used to generate lists of patients that met CPCSSN case definitions and that did not have the approved code in their problem list. Each patient was associated with a physician defined as being most responsible for their primary care in the EMR. The lists were then faxed to physicians for verification. Once verified lists were returned, data entry clerks entered coded data into the EMRs.52

Figure 1 Screenshot of DPT

Figure 2 Overview of data flow for the project

We used a parallel cohort design to study the effectiveness of the intervention. We compared changes in standardized coding for four chronic conditions. The primary outcome was the proportion of coded chronic conditions we studied present in the problem list. We compared the change in the proportion of coded entries over a six month period (31 March 2013 to 30 September 2013) between physicians exposed to the intervention and a parallel cohort of unexposed physicians.

Data sources

We used CPCSSN data extracted from the EMRs as of 31 March 2013 and 30 September 2013. We included data for all patients who had at least one encounter with their practice in the past 24 months as of 31 March 2013 and who did not opt out of participation in CPCSSN. Less than 0.01% of patients decline participation.53 We included data for all physicians participating in both 31 March 2013 and 30 September 2013 who had data available for both extractions. Data extraction procedures have been described previously.26

We used the following data elements from the EMR: dates of each encounter in the past two years prior to 31 March 2013 to estimate utilization in primary care, presence or absence of selected chronic conditions using validated algorithms for case definitions,50 patient age as of 31 March 2013, patient gender, and size of practice for each physician.54 Physicians filled out a survey for CPCSSN; we used the following data from the survey: physician age, gender, urban/suburban/rural practice location and number of years of EMR use. Characteristics of patients and providers are shown in Table 1.

Statistical analysis

For each chronic disease of interest, we used number of cases with validated CPCSSN definitions as the denominator and the number of codes present in problem lists for the conditions as the numerator for both cohorts. CPCSSN case definitions include multiple chart elements, such as problem list codes, free text terms for a condition, billing codes and laboratory values.51

Table 1 Physician and patient characteristics (as of 31 March 2013)

We compared the change in the proportion of coded diagnoses for physicians exposed to the processes and a parallel cohort of CPCSSN physicians not exposed, over a six-month period (from 31 March 2013 to 30 September 2013). We compared the use of any code included in a case definition associated with a particular disease, as shown in Table 2; we also compared the use of a preferred code, which was the code most commonly found in the CPCSSN database for each condition. We calculated composite scores by summing codes for all conditions studied as the numerator and summing the number of case definitions for each condition as the denominator. For example, if a patient had both dementia and diabetes, this was counted as two.

Multi-variate logistic regression analysis was used to compare the two cohorts. We used generalized estimating equations to adjust for the clustering structure of the data in regression models. When comparing the two groups, we adjusted for patient and physician age and gender, number of comorbidities, number of encounters and average number of patients per provider using multi-variate logistic regression analysis.

The analyses were performed with SAS version 9.4 (Cary, North Carolina, USA). We used the SQUIRE guidelines for reporting health care quality improvement research.55

CPCSSN has received ethics approval from the Research Ethics Boards of each host universities for all participating networks and from the Health Canada Research Ethics Board. All participating CPCSSN sentinel primary care providers have provided written informed consent for the collection and analysis of their EMR data. The North York General Hospital’s Research Ethics Board reviewed and approved this project.


RESULTS

Fifty-nine out of the 67 physicians in NYFHT participated in this project. Eight physicians had not implemented EMRs, or did not use key aspects of the EMR such as medication prescribing; they did not participate in CPCSSN and were not included. 315 physicians at CPCSSN sites across Canada were included in the parallel cohort. Table 2 provides information on coding for chronic conditions at baseline.

An overview of study flow at NYFHT is shown in Figure 3. Between 83% and 90% of physicians verified and returned lists for each condition. Fifty-one percent of conditions queried were indicated as being positive by physicians returning data. 10,473 health conditions already had preferred codes and 2323 new codes were added, for an increase in standardized coding of 22%.

Table 3 provides information on coding proportions and changes for each condition in the six months of interest. Table 4 presents changes in composite scores for the use of any code and for the use of preferred codes. While overall coding increased in both cohorts, the increase at NYFHT was significantly greater. Coding was more common at baseline for diabetes or hypertension than for COPD or dementia in both cohorts. Conditions that were less frequently coded at baseline at NYFHT had larger increases in coding: the increase in preferred codes was 20.2% for COPD and 22.6% for dementia. This did not occur for the parallel cohort, as there was a decrease of 0.8% for COPD and a small increase of 1.3% for dementia.

Table 2 Baseline coding on 31 March 2013

Figure 3 Overview of study flow


DISCUSSION

Data management activities were implemented in the primary care organisation we studied. This implementation was associated with significantly greater increases in coding for chronic conditions studied compared to other Canadian practices.

There is limited evidence on which interventions are most effective in improving data quality.56,57 Repeated assessments, feedback and training may be effective.57 However, this represents a significant time investment for practitioners; the extra work could compete with the already extensive requirements associated with providing clinical care,16 which may limit acceptance. Using an automated EMR alert based on clinical criteria and prompting the clinician to add a condition to the problem list if it is missing has been found to be effective.58 However, programming this in commercial EMRs routinely used may be challenging. There are about 20 EMR applications used across Canada and each one would require individual programming and support for this process. Regulations could be used to mandate EMR-based data improvement activities.

During the pilot for this project, we found that a simple approach minimizing physician workload was not costly and was acceptable to the physicians involved.52 We used their expertise only to verify cases, with no additional training; this did not interfere with clinical encounters. Most of the work was delegated to other members of the primary care team or to data clerks as appropriate. Acceptance in this project was high, with 83%–90% of physicians returning their lists of patients.

The organisation we studied was interested in data quality and had already made efforts to implement some coding for chronic conditions prior to the project.52 Data clerks had entered the diagnostic code for diabetes in the previous year. This explains the high rate of baseline coding for that condition (81.7% in NYFHT) compared to other CPCSSN physicians (63.6%). Coding for diabetes increased less than for the other conditions possibly because of ceiling effects due to prior efforts; however, the increase was still greater than for the comparator group. No consistent efforts had been made for the other three conditions; baseline coding prevalence was similar to other physicians across Canada.

The lack of adoption of terminology standards and the prevalence of uncoded, ‘local’ or idiosyncratically coded data presents challenges in terms of electronic communication and interoperability.59,60 We used the most commonly entered codes in a national database for each condition studied as an initial step towards more consistent terminology. We demonstrated that coding conforming to an external norm could be implemented in a complex and distributed primary care organisation in Canada.

Maintaining data quality will require ongoing efforts. In order to improve sustainability, the data clerks documented the processes used for this project. A handbook is provided at http://drgreiver.com/NYFHTSummerStudentProgramHandbook.pdf. NYFHT has also developed a manual for standardized data entry, which is available to all members of the team and has been shared with other teams. Data quality activities using the same approach are ongoing at NYFHT and have been expanded to include additional conditions. Scalability should also be considered; small primary care teams may not have the resources to implement this approach. However, in Ontario, primary care analysts (QIDSS) have been embedded in FHTs. The DPT and associated processes have already been provided to additional analysts, each supporting multiple FHTs in Ontario, as well as to primary care networks in Alberta. To assist with governance and processes, we have provided templates and tools developed as part of this project for privacy, and data entry and analytics to the Association of Family Health Teams of Ontario.61 The Association has been tasked with assisting the provincial implementation of analytics in FHTs through the QIDSS program.47 Additional resources for support, continuing development and broader implementation of DPT in primary care are being actively pursued by CPCSSN.

Table 3 Changes in coding over six months for each condition

Table 4 Changes in composite percentage of conditions coded

Similar approaches could be used elsewhere. In the U.S., significant funding has been devoted to improving data in EMRs. For example, a problem list needs to be used for 80% or more of patients in order to meet meaningful use goals.62 Adaptations of these processes could be used to rapidly improve the completeness and coding of data in problem lists.

Limitations

This study was a convenience sample for both cohorts. However, physicians participating in CPCSSN were reasonably similar to others in Canada.63,64 An observational cohort study was used; this is subject to both measured and unmeasured confounders. We used statistical adjustments for factors we measured. We could not measure factors possibly affecting coding, such as dictation; however, we compared the change in coding over time rather than providing a cross-sectional comparison. Data reflects only patients seen for care over time; it is possible that more frequent visits could lead to improved recognition of a chronic condition and increases in associated coding. However, physicians at NYFHT had less frequent patient visits than the national cohort. The specificity of CPCSSN case definitions varied, with some false positive cases. It is also possible that clinicians may not recognize that a condition was present for some patients on their verification list.

In conclusion, data management activities were implemented by the primary care organisation; this was associated with an increase in standardized coding for four chronic conditions. A similar environment currently exists in other primary care organisations in Ontario. Planning for resources and activities that would allow the adoption of data management within primary care organisations in order to support data and quality improvement may be worthwhile.

Where this study fits in

EMR data quality in primary care including standardized coding for chronic conditions is currently limited.

We implemented data management activities in a large primary care organisation in Ontario, Canada. This included the return of merged, cleaned EMR data and reporting software to the organisation’s data manager.

The team used these resources to code and standardise designations for chronic conditions; coding improved to a greater degree than in comparable practices across Canada.

Improving data quality in primary care using this approach appears to be feasible.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MG drafted the initial version of the article. MG, KM, DJ, BA, MM, JL and ALL contributed substantially to conception and design. BA and SK contributed substantially to the analysis of data. All authors contributed to the interpretation of data and drafting of the article. All authors revised the article critically for important intellectual content and gave final approval of the version to be published.

Acknowledgement

This project was funded by Canada Health Infoway. The views expressed herein do not necessarily represent the views of Canada Health Infoway. Funding for this publication was provided by the Public Health Agency of Canada. The views expressed herein do not necessarily represent the views of the Public Health Agency of Canada. Dr. Greiver holds an investigator award from the Department of Family and Community Medicine, University of Toronto.


REFERENCES

1. Green LA, Fryer GE, Yawn BP, Lanier D and Dovey SM. The ecology of medical care revisited. New England Journal of Medicine 2001;344(26):2021–5. http://dx.doi.org/10.1056/NEJM200106283442611. PMid:11430334.

2. Manuel DG, Maaten S, Thiruchelvam D, Jaakkimainen L and Upshur R. Primary Care in the Health Care System. Jaakkimainen L UR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE and Wang L (Ed). Toronto: Institute for Clinical Evaluative Sciences, 2006.

3. Schultz SETJ, Guttmann A and Jaakkimainen L. Characteristics of Primary Care Practice. Jaakkimainen LUR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE and Wang L (Ed). Toronto: Institute for Clinical Evaluative Sciences, 2006.

4. Baron RJ. Quality improvement with an electronic health record: achievable, but not automatic. Annals of Internal Medicine 2007;147(8):549–52. http://dx.doi.org/10.7326/0003-4819-147-8-200710160-00007. PMid:17938393.

5. Romanow R. Building on Values: the Future of Health Care in Canada. Ottawa, 2002. PMid:11861605.

6. Greiver M. Implementation of Electronic Medical Records and Preventive Services: A Mixed Methods Study. Toronto: University of Toronto, 2011.

7. Greiver M, Barnsley J, Glazier RH, Moineddin R and Harvey BJ. Implementation of electronic medical records: effect on the provision of preventive services in a pay-for-performance environment. Canadian Family Physician 2011 Oct;57(10):e381–9. PMid:21998246 PMCid:PMC3192104.

8. Linder JA, Ma J, Bates DW, Middleton B and Stafford RS. Electronic Health Record Use and the Quality of Ambulatory Care in the United States. Archives of Internal Medicine 2007;167(13):1400–5.

9. Romano MJ and Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine 2011;171(10): 897–903. http://dx.doi.org/10.1001/archinternmed.2010.527. PMid:21263077 PMCid:PMC4016790.

10. Khangura S, Grimshaw J and Moher D. CIHR Funded Knowledge to Action Research Group, Evidence Summary: Electronic Health Records (EHRs). Ottawa: Ottawa Hospital Research Institute, 2014. PMCid:PMC4016960.

11. Baron RC, Melillo S, Rimer BK, Coates RJ, Kerner J, Habarta N et al. Intervention to increase recommendation and delivery of screening for breast, cervical, and colorectal cancers by healthcare providers a systematic review of provider reminders. American Journal of Preventive Medicine 2010;38(1): 110–7. http://dx.doi.org/10.1016/j.amepre.2009.09.031. PMid:20117566.

12. Crosson JC, Ohman-Strickland PA, Cohen DJ, Clark EC and Crabtree BF. Typical Electronic Health Record Use in Primary Care Practices and the Quality of Diabetes Care. Annals of Family Medicine 2012;10(3):221–7.

13. Greiver M, Keshavjee K, Jackson D, Forst B, Martin K and Aliarzadeh B. Sentinel feedback: path to meaningful use of EMRs. Canadian Family Physician. 2012;58(10):1168.

14. Fernandopulle R and Patel N. How The Electronic Health Record Did Not Measure up to the Demands of Our Medical Home Practice. Health Affairs. 2010;29(4):622–8.

15. Williamson T, Natarajan N, Barber D, Jackson D and Greiver M. Caring for the whole practice: the future of primary care. Canadian Family Physician 2013;59(7):800.

16. Ostbye T, Yarnall KS, Krause KM, Pollak KI, Gradison M and Michener JL. Is there time for management of patients with chronic diseases in primary care? Annals of Family Medicine 2005;3(3):209–14. http://dx.doi.org/10.1370/afm.310. PMid:15928223 PMCid:PMC1466884.

17. Yarnall KS, Pollak KI, Ostbye T, Krause KM and Michener JL. Primary care: is there enough time for prevention? American Journal of Public Health 2003;93(4):635–41. http://dx.doi.org/10.2105/AJPH.93.4.635. PMid:12660210 PMCid:PMC1447803.

18. Perlin JB, Kolodner RM and Roswell RH. The Veterans Health Administration: quality, value, accountability, and information as transforming strategies for patient-centered care. American Journal of Managed Care 2004;10(11 Pt 2):828–36. PMid:15609736.

19. Scott JT, Rundall TG, Vogt TM and Hsu J. Kaiser Permanente’s experience of implementing an electronic medical record: a qualitative study. British Medical Journal 2005;331(7528):1313–6.

20. McGlynn EA, Schneider EC and Kerr EA. Reimagining Quality Measurement. New England Journal of Medicine 2014;371(23):2150–3. http://dx.doi.org/10.1056/NEJMp1407883. PMid:25470693.

21. Greiver M, Keshavjee K, Martin K and Aliarzadeh B. Who are your patients with diabetes?: EMR case definitions in the Canadian primary care setting. Canadian Family Physician. 2012;58(7):804.

22. Greiver M, Barnsley J, Glazier RH, Harvey BJ and Moineddin R. Measuring data reliability for preventive services in electronic medical records. BMC Health Services Research 2012;12:116.

23. Price M, Singer A and Kim J. Adopting electronic medical records: are they just electronic paper records? Canadian Family Physician 2013;59(7):e322–e9.

24. Roth CP, Lim YW, Pevnick JM, Asch SM and McGlynn EA. The challenge of measuring quality of care from the electronic health record. American Journal of Medical Quality 2009;24(5):385–94. http://dx.doi.org/10.1177/1062860609336627. PMid:19482968.

25. Holt TA, Gunnarsson CL, Cload PA and Ross SD. Identification of undiagnosed diabetes and quality of diabetes care in the United States: cross-sectional study of 11.5 million primary care electronic records. Canadian Medical Association Open Access Journal 2014;2(4):E248–E55.

26. Birtwhistle R, Keshavjee K, Lambert-Lanning A, Godwin M, Greiver M, Manca D et al. Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. Journal of the American Board of Family Medicine 2009;22(4):412–22. http://dx.doi.org/10.3122/jabfm.2009.04.090081. PMid:19587256.

27. Jordan K, Porcheret M and Croft P. Quality of morbidity coding in general practice computerized medical records: a systematic review. Family Practice 2004;21(4):396–412.

28. Whitelaw FG, Nevin SL, Milne RM, Taylor RJ, Taylor MW and Watt AH. Completeness and accuracy of morbidity and repeat prescribing records held on general practice computers in Scotland. British Journal of General Practice 1996;46(404): 181–6. PMid:8731627 PMCid:PMC1239581.

29. Selak V, Wells S, Whittaker R and Stewart A. Smoking status recording in GP electronic records: the unrealised potential. Informatics in Primary Care. 2006;14(4):235–41; discussion 42-5. http://dx.doi.org/10.14236/jhi.v14i4.635.

30. Mant J, Murphy M, Rose P and Vessey M. The accuracy of general practitioner records of smoking and alcohol use: comparison with patient questionnaires. Journal of Public Health. 2000 Jun;22(2):198–201. http://dx.doi.org/10.1093/pubmed/22.2.198. PMid:10912559.

31. Hollowell J. The General Practice Research Database: quality of morbidity data. Population Trends 1997 Spring(87):36–40. PMid:9134574.

32. Mukhi S and Keshavjee K. Developing a primary care electronic medical record chronic disease surveillance system in Canada: data quality and lessons learned. Canadian Association of Health Services and Policy Research, Calgary, Canada, 14 May 2009.

33. Birtwhistle R, Keshavjee K, Martin K and Lambert-Lanning A. Improving data quality in EMRs for chronic disease management. Family Medicine Forum, Calgary, Canada, 28 October 2009.

34. CIHI CPCSSN Data Analysis Report, Cycle 2A. Toronto: Canadian Institutes for Health Information, 2010.

35. Green CJ, Fortin P, Maclure M, Macgregor A and Robinson S. Information system support as a critical success factor for chronic disease management: necessary but not sufficient. International Journal of Medical Informatics 2006;75(12): 818–28. http://dx.doi.org/10.1016/j.ijmedinf.2006.05.042. PMid:16920013.

36. Wang JK, Shabot MM, Duncan RG, Polaschek JX and Jones DT. A clinical rules taxonomy for the implementation of a computerized physician order entry (CPOE) system, Proceedings of the AMIA Symposium, 2002:860–3. PMid:12463947 PMCid:PMC2244545.

37. Pearce CM, de Lusignan S, Phillips C, Hall S and Travaglia J. The computerized medical record as a tool for clinical governance in Australian primary care. JMIR Research Protocols 2013;2(2):e26. http://dx.doi.org/10.2196/ijmr.2700.

38. CIHI. Chronic disease management in primary health care: a demonstration of EMR data for qualty and health system monitoring. Ottawa: Canadian Institute for Health Information, 2014. Available from: https://secure.cihi.ca/free_products/Burden-of-Chronic-Diseases_PHC_2014_AiB_EN-web.pdf.

39. de Lusignan S, Sadek K, McDonald H, Horsfield P, Sadek NH, Tahir A et al. Call for consistent coding in diabetes mellitus using the Royal College of General Practitioners and NHS pragmatic classification of diabetes. Informatics in Primary Care 2012;20(2):103–13. PMid:23710775.

40. Rosser WW, Colwill JM, Kasperski J and Wilson L. Patient-Centered Medical Homes in Ontario. New England Journal of Medicine 2010;362(3):e7. http://dx.doi.org/10.1056/NEJMp0911519. PMid:20054034.

41. Government of Ontario. Progress Report 2014: Health Care. Toronto: Government of Ontario, 2014. Available from: https://www.ontario.ca/government/progress-report-2014-health-care.

42. Government of Ontario. Ministry of Health and Long-Term Care Overview: Results Based Plan Briefing Book 2013–2014. Toronto: Government of Ontario, 2014 Available from: http://health.gov.on.ca/en/common/ministry/publications/plans/rbplan13/.

43. National Physician Survey. National Physician Survey, 2007. Provincial Results by FP/GP or Other Specialist: Ontario. Mississauga: National Physician Survey, 2014. Available from: http://nationalphysiciansurvey.ca/wp-content/uploads/2014/11/2014-ON-EN.pdf.

44. Report of the Auditor General of Ontario, 2011. Toronto, 2011.

45. Glazier RH, Kopp A, Schultz SE, Kiran T and Henry DA. All the right intentions but few of the desired results: lessons on access to primary care from Ontario’s patient enrolment models. Health Care Quarterly 2012;15(3):17–21. http://dx.doi.org/10.12927/hcq.2013.23041 PMid:22986561.

46. Canadian Legal Information Institute. Excellent Care for All Act, 2010, SO 2010, c14. Toronto: Government of Ontario, 2010 . Available from: https://www.canlii.org/en/on/laws/stat/so-2010-c-14/latest/so-2010-c-14.html.

47. Recommendations on the optimal configuration of the Quality Improvement and Decision Support Specialist (QIDSS) role. Toronto: Association of Family Health Teams of Ontario; 2013 [cited 2014 November 26]; Available from: http://www.afhto.ca/wp-content/uploads/QIDSS-recommendations-2013-01-31.pdf.

48. Ontario Medical Association. Funded EMR offerings as at November 30th, 2014. Toronto: Ontario Medical Association, 2014. Available from: https://www.ontariomd.ca/idc/groups/public/documents/omd_file_content_item/omd012788.pdf.

49. Birtwhistle RV. Canadian Primary Care Sentinel Surveillance Network: A developing resource for family medicine and public health. Canadian Family Physician 2011;57(10):1219–20.

50. Williamson T, Birtwhistle R, Green M, Khan S, Wong S, Natarajan N et al. Validating the CPCSSN Algorithms for Eight Chronic Conditions. Family Medicine Forum, Vancouver, Canada, 2013.

51. Williamson T, Green ME, Birtwhistle R, Khan S, Garies S, Wong ST et al. Validating the 8 CPCSSN Case Definitions for Chronic Disease Surveillance in a Primary Care Database of Electronic Health Records. Annals of Family Medicine 2014;12(4):367–72.

52. Greiver M, Barnsley J, Aliarzadeh B, Krueger P, Moineddin R, Butt DA et al. Using a data entry clerk to improve data quality in primary care electronic medical records: a pilot study. Journal of Innovation in Health Informatics 2011;19(4):241–50. http://dx.doi.org/10.14236/jhi.v19i4.819.

53. Wong ST, Manca D, Barber D, Morkem R, Khan S, Kotecha J et al. The diagnosis of depression and its treatment in Canadian primary care practices: an epidemiological study. Canadian Medical Association Open Access Journal 2014;2(4):E337–E42.

54. Greiver M, Williamson T, Bennett TL, Drummond N, Savage C, Aliarzadeh B et al. Developing a method to estimate practice denominators for a national Canadian electronic medical record database. Family Practice 2013;30(3):347–54. http://dx.doi.org/10.1093/fampra/cms083 PMid:23307818.

55. Davidoff F, Batalden P, Stevens D, Ogrinc G and Mooney S. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Quality and Safety in Health Care 2008;17 Suppl 1:i3–9. http://dx.doi.org/10.1136/qshc.2008.029066 PMid:18836063 PMCid:PMC2773518.

56. Brouwer H, Bindels P and Weert H. Data quality improvement in general practice. Family Practice 2006;23(5):529–36.

57. Porcheret M, Hughes R, Evans D, Jordan K, Whitehurst T, Ogden H et al. Data quality of general practice electronic health records: the impact of a program of assessments, feedback, and training. Journal of the American Medical Informatics Association 2004 Jan-Feb;11(1):78–86. http://dx.doi.org/10.1197/jamia.M1362. PMid:14527973 PMCid:PMC305461.

58. Wright A, Pang J, Feblowitz JC, Maloney FL, Wilcox AR, McLoughlin KS et al. Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial. Journal of the American Medical Informatics Association 2012;19(4):555–61.

59. Ethier J-Fo, Dameron O, Curcin V, McGilchrist MM, Verheij RA, Arvanitis TN et al. A unified structural/terminological interoperability framework based on LexEVS: application to TRANSFoRm. Journal of the American Medical Informatics Association 2013;20(5):986–94.

60. Rector AL. Clinical terminology: why is it so hard? Methods of Information in Medicine 1999;38(4–5):239–52.

61. Association of Family Health Teams of Ontario. Making it easier for teams to access data needed to improve care: 2014 Bright Lights Award for Using Data to Improve Care. Toronto: Association of Family Health Teams of Ontario, 2014. Available from: http://www.afhto.ca/wp-content/uploads/BL-2014-Theme-6-North-York-FHT.pdf.

62. Blumenthal D and Tavenner M. The “meaningful use” regulation for electronic health records. New England Journal of Medicine 2010;363(6):501–4. http://dx.doi.org/10.1056/NEJMp1006114. PMid:20647183.

63. Queen’s University. Primary Health Care Intelligence: 2013 Progress Report of the Canadian Primary Care Sentinel Surveillance Network (CPCSSN). Kingston, Ontario: Queen’s University, 2013.

64. College of Family Physicians of Canada and Canadian Medical Association. National Physician Survey, 2010 ed. Ottawa: College of Family Physicians of Canada and Canadian Medical Association, 2010. Available from: http://nationalphysiciansurvey.ca/surveys/2010-survey/2010-results.

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT