Creating and using real-world evidence to answer questions about clinical effectiveness

Simon de Lusignan, Laura Crawford, Neil Munro


New forms of evidence are needed to complement evidence generated from randomised controlled trials (RCTs). Real-World Evidence (RWE) is a potential new form of evidence, but remains undefined.

This paper sets to fill that gap by defining RWE as the output from a rigorous research process which: (1) includes a clear a priori statement of a hypothesis to be tested or research question to be answered; (2) defines the data sources that will be used and critically appraises their strengths and weaknesses; and (3) applies appropriate methods, including advanced analytics. These elements should be set out in advance of the study commencing, ideally in a published protocol.

The strengths of RWE studies are that they are more inclusive than RCTs and can enable an evidence base to be developed around real-world effectiveness and to start to address the complications of managing other real-world problems such as multimorbidity. Computerised medical record systems and big data provide a rich source of data for RWE studies.

However, guidance is needed to help assess the rigour of RWE studies so that the strength of recommendations based on their output can be determined. Additionally, RWE advanced analytics methods need better categorisation and validation.

We predict that the core role of RCTs will shift towards assessing safety and achieving regulatory compliance. RWE studies, notwithstanding their limitations, may become established as the best vehicle to assess efficacy.



computerized, data collection, epidemiological methods, evidence-based practice, health services research, medical record systems, real-world analytics, research design, statistics as topic

Full Text:



Sanson-Fisher RW, Bonevski B, Green LW and D’Este C. Limitations of the randomized controlled trial in evaluating population-based health interventions. American Journal of Preventive Medicine 2007;33(2):155–61. PMid:17673104.

Williams BA. Perils of evidence-based medicine. Perspectives in Biology and Medicine 2010;53(1):106–20.

Schattner P, Barker F and de Lusignan S. Minimally disruptive medicine is needed for patients with multimorbidity: time to develop computerised medical recordsystems to meet this requirement. Journal of Innovation in Health Informatics 2015;22(1):250–4.

Porter M and Lee T. The strategy that will fix health care. Harvard Business Review 2013;91(10):50–70.

de Lusignan S and van Weel C. The use of routinely collected computer data for research in primary care: opportunities and challenges. Family Practice 2006;23(2):253–63. PMid:16368704.

de Lusignan S, Metsemakers JF, Houwink P, Gunnarsdottir V and van der Lei J. Routinely collected general practice data: goldmines for research? A report of the European Federation for Medical Informatics Primary Care Informatics Working Group from MIE2006, Maastricht, The Netherlands. Informatics in Primary Care 2006;14(3):203–9.

Harris RP, Sheridan SL and Lewis CL. Real-world evidence about potential psychosocial harms of lung cancer screening – Reply. JAMA Internal Medicine 2014;174(8):1416–7.

Lexchin J, Bero LA, Djulbegovic B and Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. British Medical Journal 2003;326(7400):1167. PMid:12775614; PMCid:PMC156458.

Lundh A, Sismondo S, Lexchin J, Busuioc OA and Bero L. Industry sponsorship and research outcome. The Cochrane Database of Systematic Reviews. 2012;12:MR000033.

Naci H, Dias S and Ades AE. Industry sponsorship bias in research findings: a network meta-analysis of LDL cholesterol reduction in randomised trials of statins. British Medical Journal 2014;349:g5741.

van Lent M, Overbeke J and Out HJ. Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals. Public Library of Science 2014;9(8):e104846.

McGhan WF, Al M, Doshi JA, Kamae I, Marx SE and Rindress D. The ISPOR Good Practices for Quality Improvement of Cost-Effectiveness Research Task Force Report. Value Health 2009;12(8):1086–99.

de Lusignan S, Sadek N, Mulnier H, Tahir A, Russell-Jones D and Khunti K. Miscoding, misclassification and misdiagnosis of diabetes in primary care. Diabetic Medicine 2012;29(2):181–9.

de Lusignan S, Sun B, Pearce C, Farmer C, Steven P and Jones S. Coding errors in an analysis of the impact of pay-for-performance on the care for long-term cardiovascular disease: a case study. Informatics in Primary Care 2014;21(2):92–101.

Liaw ST, Taggart J, Yu H and de Lusignan S. Data extraction from electronic health records – existing tools may be unreliable and potentially unsafe. Australian Family Physician 2013;42(11):820–3. PMid:24217107.

Liaw ST, Taggart J, Yu H, de Lusignan S, Kuziemsky C and Hayen A. Integrating electronic health record information to support integrated care: practical application of ontologies to improve the accuracy of diabetes disease registers. Journal of Biomedical Informatics 2014;52:364–72.

de Lusignan S, Liaw ST, Michalakidis G and Jones S. Defining datasets and creating data dictionaries for quality improvement and research in chronic disease using routinely collected data: an ontology-driven approach. Informatics in Primary Care 2011;19(3):127–34.

Schroll JB, Maund E and Gøtzsche PC. Challenges in coding adverse events in clinical trials: a systematic review. Choonara I (Ed). Public Library of Science 2012;7(7):e41174.

Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L and Joffe A et al. Quality of reporting in systematic reviews of adverse events: systematic review. British Medical Journal 2014;348:f7668.

Berger ML, Dreyer N, Anderson F, Towse A, Sedrakyan A and Normand SL. Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health. 2012;15(2):217–30.

Jones AG, Shields BM, Hyde CJ, Henley WE, Hattersley AT. Identifying good responders to glucose lowering therapy in type 2 diabetes: implications for stratified medicine. PLoS One. 2014;9(10):e111235.

de Lusignan S, Chan T, Theadom A and Dhoul N. The roles of policy and professionalism in the protection of processed clinical data: a literature review. International Journal of Medical Informatics 2007;76(4):261–8. PMid:16406791.

de Lusignan S. Effective pseudonymisation and explicit statements of public interest to ensure the benefits of sharing health data for research, quality improvement and health service management outweigh the risks. Informatics in Primary Care 2014;21(2):61–3.

New JP, Bakerly ND, Leather D, Woodcock A. Obtaining real-world evidence: the Salford Lung Study. Thorax 2014;69(12):1152–4.



  • There are currently no refbacks.

This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT