Methods to Describe Referral Patterns in a Canadian Primary Care Electronic Medical Record Database: Modelling Multilevel Count Data

Bridget L Ryan, Joshua Shadd, Heather Maddocks, Moira Stewart, Amardeep Thind, Amanda L Terry


Background:  A referral from a family physician (FP) to a specialist is an inflection point in the patient journey, with potential implications for clinical outcomes and health policy. Primary care electronic medical record (EMR) databases offer opportunities to examine referral patterns. Until recently, software techniques were not available to model these kinds of multi-level count data. 

Objective:  To establish methodology for determining referral rates from FPs to medical specialists using the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) EMR database. 

Method: Retrospective cohort study, mixed effects and multi-level negative binomial regression modelling with 87,258 eligible patients between 2007 and 2012. Mean referrals compared by patient sex, age, chronic conditions, FP visits, and urban/rural practice location.  Proportion of variance in referral rates attributable to the patient and practice levels. 

Results:  On average, males had 0.26, and females 0.31 referrals in a 12-month period.  Referrals were significantly higher for females, increased with age, FP visits, and number of chronic conditions (p<.0001). Overall, 14% of the variance in referrals could be attributed to the practice level, and 86% to patient level characteristics. 

Conclusions:  Both patient and practice characteristics influenced referral patterns. The methodologic insights gained from this study have relevance to future studies on many research questions that utilize count data, both within primary care and broader health services research. The utility of the CPCSSN database will continue to increase in tandem with data quality improvements, providing a valuable resource to study Canadian referral patterns over time.


"Referral and Consultation"[Mesh]; "Primary Health Care"[Mesh]; "Electronic Health Records"[Mesh]; "Practice Patterns, Physicians'"[Mesh]

Full Text:



Forrest CB, Majeed A, Weiner JP, Carroll K and Bindman AB. Comparison of specialty referral rates in the United Kingdom and the United States: retrospective cohort analysis. BMJ 2002;325:370–71. Available at: PMid:12183310; PMCid:PMC117891.

Forrest CB, Nutting P, Werner JJ, Starfield B, von SS and Rohde C. Managed health plan effects on the specialty referral process: results from the Ambulatory Sentinel Practice Network referral study. Medical Care 2003;41(2):242–53. Available at: Available at: PMid:12555052.

Chan BTB and Austin PC. Patient, physician, and community factors affecting referrals to specialists in Ontario, Canada. Medical Care 2003;41:500–11. Available at: Available at: PMid:12665714.

Franks P, Williams GC, Zwanziger J, Mooney C and Sorbero M. Why do physicians vary so widely in their referral rates? Journal of General Internal Medicine 2000;15(3):163–68. Available at: PMid:10718896; PMCid:PMC1495354.

O’Donnell CA. Variation in GP referral rates: what can we learn from the literature? Family Practice 2000;7(6):462–71. Available at:

Shadd JD, Ryan BL, Maddocks H and Thind A. Patterns of referral in a Canadian primary care electronic health record database: retrospective cross-sectional analysis. Informatics in Primary Care 2011;19:217–23. Available at:

Sullivan CO, Omar RZ, Ambler G and Majeed A. Case-mix and variation in specialist referrals in general practice. British Journal of General Practice 2005;55(516):529–33. PMid:16004738; PMCid:PMC1472770.

Owen SV and Froman RD. Why carve up your continuous data? Research in Nursing & Health 2005;28(6):496–503. Available at: PMid:16287057.

Crotteau JS, Ritchie MW and Varner JM. A mixed-effects heterogeneous negative binomial model for postfire conifer regeneration in Northeastern California, USA. Forest Science 2014;60(2):275–87. Available at:

Sellers KF and Shmueli G. Data dispersion: now you see it… now you don’t. Communications in Statistics – Theory and Methods 2013;42:3134–47. Available at:

Maddocks H, Ryan BL, Shadd J, Chevendra V and Terry A. Identifying new referrals from FPs using EMRs. Sentinel Eye Series. Canadian Family Physician 2014;60:949. PMid:25316751; PMCid:PMC4196822.

StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP, 2013.

Williamson T, Birtwhistle R, Khan S, Garies S, Wong S, Natarajan N, et al. Validating the 8 CPCSSN case definitions for chronic disease surveillance in a primary care database of electronic health records. Annals of Family Medicine 2014;12(4):367–72. Available at: PMid:25024246; PMCid:PMC4096475.

Statistics Canada. Postal Code Conversion File. Ottawa, Canada: Statistics Canada, Data Liberation Initiative, 2006.

Paccagnella O. Sample size and accuracy of estimates in multilevel models: new simulation results. European Journal of Research Methods for the Behavioral and Social Sciences 2011;7(3):111–20. Available at:

Moineddin R, Matheson FI and Glazier RH. A simulation study of sample size for multilevel logistic regression models. BMC Medical Research Methodology 2007;7:34. Available at:

Bryan ML and Jenkins SP. Multilevel modelling of country effects: a cautionary tale. European Sociological Review 2016;32(1):3–22. Available at:

Cameron AC and Miller DL. A practitioner’s guide to cluster-robust inference. The Journal of Human Resources 2015;50(2):317–370. Available at:

Queenan JA, Williamson TW, Khan S, Drummond N, Garies S, Morkem R, et al. Representativeness of patients and providers in the Canadian Primary Care Sentinel Surveillance Network: a cross-sectional study. CMAJ Open 2016;4(1):E28–32. Available at: PMid:27331051; PMCid:PMC4866925.

Spriensma AS, Hajos TRS, de Boer MR, Heymans MW and Twisk JWR. A new approach to analyse longitudinal epidemiological data with an excess of zeros. BMC Medical Research Methodology 2016;13(21):1–7



  • There are currently no refbacks.

This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT