Informatics 96mar2

Journal of Informatics in Primary Care 1996 (March):2-7


Papers


An evaluation of CAPSULE, a computer system giving advice to general practitioners about drug prescribing

Robert Walton, Bury Knowle Health Centre, Headington, Oxford

This paper was presented at the Annual Conference of the Primary Health Care Specialist Group, Cambridge, September 1995, and was the winner of the John Perry Prize


Most general practitioners in the United Kingdom use computers to store information on the patient's past illnesses and drug therapy, and to write prescriptions for drugs. This paper describes the development and evaluation of a computerised decision support system for prescribing: CAPSULE (Computer Assisted Prescribing Using Logic Engineering), designed to use all relevant patient data recorded on the doctor's computer system for giving advice about treatment decisions. The system has been evaluated to determine whether advice from a computer can improve the quality and cost-effectiveness of doctors' prescribing.

The study has shown that prescribing with computer advice was significantly quicker, more economical, and more rational than without advice. Computerising guidelines for prescribing is possible with current technology and could improve the quality and reduce the cost of drug therapy in general practice.


1 Introduction

About half of all consultations in primary care result in the prescription of a drug. The consequences for the patient of poor prescribing practice are difficult to quantify. However, between 3% and 5% of hospital beds in the United Kingdom are occupied by patients admitted because of adverse drug reactions[1]. The costs and consequences of poor prescribing have been highlighted recently by an Audit Commission report. £3,600 million - 10% of all National Health Service expenditure - was spent on prescribing drugs by general practitioners in England and Wales in 1992/93. Prescribing varies considerably between general practitioners, both in the nature of the drugs prescribed and the length of the course used[2]. There may be a conflict between the wish of governments to restrain rapidly increasing prescribing budgets and the concern of doctors to give the best treatment to their patients which may involve using newer, more expensive drugs. Against these considerations of economics and medical effectiveness of drugs, it is essential to respect the autonomy of the patient in making decisions about treatment[3].

1.1 Computers and Guidelines for Clinical Practice

Computer advice has been shown to be effective in helping doctors to adhere to guidelines on clinical care[4]. It may also be an effective way of changing doctors' behaviour[5] but it has not previously been shown to influence the drugs that a doctor prescribes.

More than 90% of general practices have computers and over 60% of general practitioners already use the computer to write a prescription for a drug[6]. Legal obstacles to the use of guidelines for medical practice may be fewer than appear at first sight. The subject has been well reviewed by Hurwitz[7]. Many of the legal arguments about printed guidelines could also apply to clinical guidelines embodied in computer systems. In the United Kingdom it may soon become legally defensible to hold the entire patient record electronically[8], which may mean that more information is stored in the future that could be used for decision support. The accuracy of data stored on general practitioners' computers is surprisingly high[9].

Limited decision support is provided by existing general practitioner computer systems as a list of drugs with dosages from which a treatment can be selected. Some systems give warnings about contraindications and drug interactions but none as yet suggest therapy for particular illnesses, based on guideline recommendations, tailored to the needs of the individual patient.

1.2 The role of computerised decision support in drug therapy

Computers may help the doctor to select the most rational treatment for a patient by:

  • respecting patients' previously expressed preferences
  • giving advice based on the patient's pre-existing illness and past drug treatment
  • suggesting cheaper, more effective drugs where appropriate

1.2.1 More rational treatment

It is important to consider interactions between the new drug and drugs that the patient is already taking, together with contraindications to prescribing that may arise from pre-existing illnesses. In some cases the drug regimen can be simplified when pre-existing illnesses are treatable by the new drug. An example would be a person taking hydrochlorothiazide, amiloride, and nifedipine for high blood pressure and low potassium caused by diuretics, who then develops heart failure: substituting enalapril for nifedipine would treat the heart failure and hypertension. The amiloride, which was used to prevent the fall in potassium caused by the hydrochlorothiazide, could be stopped because enalapril also prevents loss of potassium by the kidney. The patient would then need to take only two drugs instead of three, with potential benefits of improved health, convenience, reduced risk of side effects, and reduced cost.

Certain drugs may be of specific benefit to individuals but this may only be evident by empirical trial. An example would be the use of ipratropium in adults with asthma, which is not usually used as first-line therapy, but seems to be effective in some people. Guidelines produced by the British Thoracic Society for the treatment of asthma suggest a therapeutic trial to determine the most effective therapy[10]. The computer may be useful in recording these details and taking them into account when providing advice.

Finally, the patient may prefer not to take a drug for a reason which is not "scientific" but nevertheless perfectly valid. For example, if a relative or neighbour had taken a drug and suffered unwanted effects, the patient may prefer to take a different treatment or none at all. Those reasons may not affect the clinical efficacy of the drug but may be important in improving patient satisfaction and compliance with the drug regimen.

The procedure used by the computer to provide the advice, if it is to be as good as or better than a competent doctor, must synthesise as much of this information as possible and produce a list of reasonable choices. A decision support system does not need to recommend the perfect treatment: optimising the use of available information to produce a few safe and effective options from which the doctor can choose is sufficient.

1.2.2 Reducing costs

Computers would be useful in helping doctors select cheaper, equally effective drugs. Drug prices constantly change, and keeping up to date with information that doctors regard as largely administrative is given low priority. However, finding a cheaper drug is not a straightforward task - it is not sufficient simply to recommend a less expensive drug of the same type: the decision support system must have information on the disease that is being treated, its severity, and the relative effectiveness of drugs of different classes for different stages of the disease.

It is important to remember, however, that the final choice of therapy will be made by the patient, with advice from the doctor. Many additional social and psychological factors will influence the decision to take medication and determine the choice of drug. Recent work on computer-assisted diagnosis has suggested that the combination of doctor and machine is better than either alone[11], and the conclusion may also hold true for computer-assisted therapy.


2 The CAPSULE Study

The CAPSULE prescribing decision support system provides advice on the selection of drugs. It contains a knowledge base of 780 facts covering everyday prescribing problems. The system uses a symbolic decision procedure[12] developed in the DILEMMA project, and builds on the "logic engineering" approach to expert system construction previously used in the Oxford System of Medicine[13] and the Bordeaux Oncology Support System[14]. Logic engineering differs from other methods of software engineering in that all the knowledge used by the system to make decisions is explicitly represented in a "knowledge base". Other systems may have rules that are implicitly encoded in programs, for example, those relating to combining probabilities. One of the advantages of this approach is that the way in which the systems makes decisions can be modified easily by the author of the knowledge base, who need not have a detailed knowledge of computer programming.

2.1 The CAPSULE Program

The program was designed to provide advice on drug therapy for forty prescribing problems taken directly from everyday medical practice. The computer screens that the user sees were carefully designed to conduct a controlled experiment to determine the best way of presenting the advice to the doctor. A standard Microsoft Windows™ interface was used.

The computer program works in three different ways. The alphabetical mode reproduced the computer system that the general practitioner normally uses without interaction and contraindication checking. This mode is used for the control cases in the experiment. The limited decision support mode gives the doctor suggestions for treatment of the patient as a list of up to six drugs that might be useful. The doctor is presented only with appropriate drugs determined by the decision support system but if he or she wishes to prescribe something different the complete list of drugs is easily available. The full decision support mode gives the same suggestions as the limited decision support mode with a brief explanation of the reasons for the advice. Again, if the general practitioner does not wish to prescribe a drug from the computer suggestions it is easy to select from the alphabetical list instead.

2.2 Constructing the Test Cases

Test cases were intended to represent illnesses frequently treated by general practitioners. They were selected from consecutive sets of notes from four consecutive surgeries carried out by four different doctors at Bury Knowle Health Centre, Oxford, in May 1994. The computer record of the consultation made by the general practitioner during the consultation was also inspected. Forty-three sets of notes were examined and the final four consultations recorded in each set of notes were selected. Where more than one problem was presented to the general practitioner during a consultation, the problems were separated and individual test cases were constructed. In all, 92 problems were presented to the doctors and 67 (72%) received drug therapy. The first forty of these prescribing problems were selected for the experiment. Four of the cases were selected to familiarise the doctor with the system. These practice cases were selected so that any information gained from seeing the computer's suggestions would be unlikely to be of any use in the experiment.

2.3 Choosing the Optimum Treatment

The "gold standard" for the treatment of each case was derived from the consensus opinion of a panel of two pharmacologists, two general practitioners, and a pharmacist. The group met for four hours in May 1994 and discussed the general principles which should govern the prescribing advice given by the computer. A list containing the best treatments in rank order for the first twenty cases was produced at the meeting, and suggested treatments for the remaining twenty cases were circulated to the experts for written comment. All experts provided further comments which were then used to create the final list of treatments which the computer would suggest for the cases. The information from the expert panel was used to create the knowledge base which the computer used to make its decisions.


3 Evaluation Methods

In the study, the doctor was presented with 36 short case synopses on paper and used the computer to prescribe treatment. The cases had been randomised into three sets of twelve cases (Table 1). The software presented the sets of cases to the doctors in three blocks arranged in random order. The first block was presented with the alphabetical list, the second with limited decision support, and the third with full decision support including reasons. There was a "run-in" period with four sample cases illustrating the different modes of operation of the system to familiarise doctors with the system. Doctors were asked to use the computer to write a prescription for the patient. Tasks included selecting the drug, formulation, frequency, and duration of the course. The general practitioners were asked to prescribe exactly what they normally would, unless they felt that the computer's suggestion, based on expert consensus, was an improvement on their normal practice. Doctors were asked to prescribe as many or as few drugs as are needed for each case. In some cases the experts felt that no drug treatment was needed.

Doctors were told that they were taking part in an experiment on the effects of presenting information to doctors in different ways. After they had used the system to prescribe for the 36 cases, a semi-structured interview was conducted with each doctor, to elicit their comments and suggestions for improvements.

3.1 Experimental Design

Because some of the problems presented in the cases are similar, the general practitioners may gradually learn the "correct" answer to some of the cases. It is likely that this "carry-over" effect was present both from one block to another and within each block. It seems likely that this learning effect will be strongest with the full decision support mode including reasons. Simply prescribing for the case with the alphabetical list may influence subsequent performance on similar cases because the doctor has been made to think about the problem. Having rehearsed the arguments once, the doctor may come to a better decision more quickly. However, this effect is likely to be smaller than that seen when the computer suggestions are shown with the reasons. The learning effect of seeing the cases with the limited decision support is likely to be intermediate between the other modes. There may also be a learning effect within a block of cases presented with a particular mode of decision support. Therefore, within the block design it was ensured that each set was presented to three doctors in one order, and to another three in reverse order (Table 1).

Table 1: Overview of the experimental design


Subjects                   Order of administration of the cases                

                 First block            Second block           Third block       

                Set        Order        Set        Order        Set        Order    

Doctor 1         1           a           2           a           3           a      

Doctor 2         1           b           3           b           2           b      

Doctor 3         2           a           1           a           3           b      

Doctor 4         2           b           3           a           1           a      

Doctor 5         3           a           1           b           2           a      

Doctor 6         3           b           2           b           1           b      



3.2 Participating Doctors

The study was conducted in November 1994 with six general practitioners in the Oxford Region, chosen for their diverse ages and backgrounds. The doctors (five men and one woman) were aged between 27 and 72 years, and had been in practice for between 6 and 48 years. All were principals, except the eldest who retired two years ago but continues to work in his practice as an assistant. Four of the doctors used the EMIS computer system for their clinical work, one used VAMP, and one used Meditel. Four doctors came from practices involved in teaching students. Two doctors came from practices involved in postgraduate training for general practice.

The doctors had varying degrees of computer literacy. One did not use the computer during consultations and never used it for prescribing. He did not own a computer and had not used a word processor or spreadsheet. Another had used the computer for prescribing during consultations for five years, but was not familiar with word processors or spreadsheets and had not used Microsoft Windows™. Only three had used Windows™ before the experiment although four were familiar with word processors. One doctor was very computer-literate, being familiar with databases and statistics packages.

3.3 Scoring the Cases

The rational prescribing score was developed specifically for the study and is a measure of how closely the choice of the general practitioner agreed with the recommendation of the expert panel. All scoring was conducted blind according to a system previously determined by the expert panel. A simple computer program was constructed to present the cases to the scorer in a random order. The fields containing information about the general practitioner were not displayed. Similarly the mode of operation of the CAPSULE software when the general practitioner performed the task was also concealed from the scorer. The drugs prescribed by the general practitioner were displayed with the dosage and frequency, grouped by case number, so that all instances of one case were scored at the same time.

The costs of the drugs were calculated and entered at the same time as the scoring, as were the other variables, such as whether the drug was generic or proprietary, and whether there was a cheaper, equally effective substitute. The British National Formulary (BNF) March 1994 was used for reference. The scoring was repeated ten days later by the same person with reference to the same criteria determined by the expert panel. The scoring procedure was found to be reliable using the method of Bland and Altman[15].


4 Results

4.1 Semi-structured Interview

Five doctors said that they found the system useful or very useful, and all but one would be likely or very likely to use it in a consultation. The remaining doctor felt that the system would be more useful if the reasons for the choice were more interesting. He suggested that the system should display more information, such as a sentence from the BNF to explain the clinical reasons for the recommendations. Five doctors found the system easy to use or very easy to use. Although all disagreed with some of the advice, all found the computer's suggestions helpful.

4.2 Time Taken to Prescribe

The mean time taken to prescribe was 111 seconds (SD 50 sec) from the alphabetical list, 60 seconds (SD 40 sec) with computer advice, and 54 seconds (SD 19 sec) when reasons for the advice were offered (p=0.0094). All general practitioners dealt with the cases much more quickly using the decision support system (Figure 1).

Figure 1: Time taken for each general practitioner to prescribe for the 36 cases in the three different modes of operation of the software
Sorry - figure not available!

4.3 Costs of the drugs prescribed

All general practitioners were more likely to give a cheaper, equally effective drug when they had computer advice. General practitioners ignored a cheaper option in 31% (SD 14%) of cases using the alphabetical list, 20% (SD 8%) using advice alone, and 12% (SD 7%) when shown the reasons (p=0.0098). For example, general practitioner number six never ignored a cheaper substitute when given full decision support whereas he did so in 28% of the cases when prescribing using the alphabetical list. General practitioner number three ignored a cheaper substitute in 55% of the cases with the alphabetical list compared to only 17% of cases with the full decision support (Figure 2).

Figure 2: Percentage of cases in which the general practitioner ignored a cheaper, equally effective drug in the three different modes of operation of the software
Sorry - figure not available!

The average cost of a prescription for the test set of cases was £4.98 using the alphabetical list, £4.52 with the limited decision support, and £3.90 with the full decision support. In this small sample, these differences did not reach statistical significance (Figure 3).

Figure 3: Costs of prescriptions given by each general practitioner in the three different modes of operation of the software
Sorry - figure not available!

4.4 Rational Prescribing Score

All doctors scored lowest using the alphabetical list without computer advice (Figure 4). All except general practitioner number one scored highest with the full decision support with the reasons for the computer suggestions. There was a highly statistically significant difference in the scores of the general practitioners using the different modes of decision support; doctors using an alphabetical list had a mean score of 33.0 units (SD 2.5 units). Using suggestions generated by the computer, the score was 35.6 units (SD 1.6 units). When the reasons behind the suggestions were available, the score rose to 27.1 units (SD 1.5 units) (p=0.0057 Friedman's Two-way Anova). There was some variation between the mean scores for the doctors, with general practitioner number three on average scoring 32.7 units (SD 5.1 units), and general practitioner number six scoring 36.5 units (SD 3.8 units). The general practitioner with the lowest score on the alphabetical list (number three) improved most with decision support.

Figure 4: Rational Prescribing Scores for each general practitioner for each set of cases using the software in the three different modes
Sorry - figure not available!


5 Conclusion

We have successfully applied logic engineering to the task of prescribing drugs in general practice. The results of the preliminary CAPSULE study indicate that computer advice provided by the decision support system can substantially improve the prescribing practice of general practitioners. Doctors found the system helpful, quick, and easy to use, and thought it would be useful in their practice. The CAPSULE system is currently undergoing further evaluation with a larger, randomly-selected sample of general practitioners, funded by the Anglia & Oxford Regional Health Authority.


Acknowledgments

The DILEMMA project is supported under the Commission of the European Communities Health Telematics Programme. I thank Prof M Vessey and Dr Helen Doll for advice on study design, and the prescribing experts and general practitioners who gave their time freely to participate in the study. I am grateful to Prof John Fox, Claude Gierl, Richard Thompson and Paul Fergusson for their help in constructing the software.


References

  1. Jones I, Griffiths S. A prescription for improvement. Towards more rational prescribing in general practice. Audit Commission, HMSO, London, 1994
  2. Ryder SD, O'Reilly S, et al. Measuring prescribing: the shortcomings of the item. BMJ 1994; 308:827
  3. Barber N. What constitutes good prescribing? BMJ 1995; 310:923
  4. Johnston MEK, Langton KB et al. Effects of a computer-based clinical decision support system on clinician performance and patient outcome. Ann Int Med 1994; 120:135
  5. Grimshaw JM, Russell IT. Effects of clinical guidelines on medical practice - a systematic review of rigorous evaluations. Lancet 1993; 342:1317
  6. Social Surveys (Gallup Poll) Ltd. Computerisation in GP Practices 1993 Survey. Department of Health, NHS Management Executive, 1993
  7. Hurwitz B. Clinical guidelines: proliferation and medicolegal significance. Quality in Health Care 1994; 3:37
  8. Sheldon M. Personal communication
  9. Johnson N, Mant D, et al. Use of computerised general practice data for population surveillance: comparative study of influenza data. BMJ 1991; 302:763
  10. British Thoracic Society. Guidelines on the management of asthma. Thorax 1993; 48(Suppl):S1
  11. Berner ES, Ed D, et al. Performance of four computer-based diagnostic systems. N Eng J Med 1994; 330:1792
  12. Huang J, Fox J, et al. Symbolic decision support in medical care. Artificial Intelligence in Medicine 1993; 5:415
  13. Fox J, Glowinski A, et al. Logic engineering for knowledge engineering: design and implementation of the Oxford System of Medicine. Artificial Intelligence in Medicine 1990; 2:323
  14. Renaud-Salis JL, Taylor P. The Bordeaux Oncology Support System: Program documentation. Deliverable of the LEMMA project. EEC programme for Advanced Informatics in Medicine 1990
  15. Bland JM, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 335:307

Refbacks

  • There are currently no refbacks.


This is an open access journal, which means that all content is freely available without charge to the user or their institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal starting from Volume 21 without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open accessFor permission regarding papers published in previous volumes, please contact us.

Privacy statement: The names and email addresses entered in this journal site will be used exclusively for the stated purposes of this journal and will not be made available for any other purpose or to any other party.

Online ISSN 2058-4563 - Print ISSN 2058-4555. Published by BCS, The Chartered Institute for IT