WHO Teacher's Guide to Good Prescribing

Publication date: 2001

As a contribution to problem-based pharmacotherapy teaching, in 1994 WHO published the Guide to Good Prescribing. Widely acclaimed as an innovative and practical tool for use in undergraduate medical teaching, the manual has been adopted as a standard text in universities throughout the world. It has also been adapted for training primary health care paramedical prescribers in some countries. The Teacher’s Guide to Good Prescribing is a companion volume for those who wish to use this problem-based method. The Teacher’s Guide is in three parts. The first and most extensive part explains the educational approach underlying the Guide to Good Prescribing and how to use this in pharmacotherapy training. Topics covered include the importance of setting learning objectives, constructing good patient examples and equipping students with skills to critically review medical literature. Part 2 advises on student, teacher and course assessment, and on measuring the impact of training. The emphasis is on how to ensure that both the student assessment methods and the teaching method focus on transfer of practical prescribing skills, rather than on knowledge of drugs alone. The final section looks at ways to mobilize support for problem-based pharmacotherapy teaching, and presents a strategy for sensitizing colleagues and others to the benefits of the method. The text is supplemented throughout by examples of patient cases and innovative teaching ideas from developed and developing countries. The Teacher’s Guide to Good Prescribing has been developed by an international group of health professionals and academics, working with the WHO Department of Essential Drugs and Medicines Policy. WHO/EDM/PAR/2001.2 Distribution: General Original: English Essential Drugs and Medicines Policy, World Health Organization, 1211 Geneva 27, Switzerland World Health Organization Essential Drugs and Medicines Policy Geneva GUIDE TO GOOD PRESCRIBING LANGUAGE EDITIONS English, French, Spanish and Russian versions are available from WHO The manual has also been issued or is in preparation by other organizations and publishers in the following languages: Arabic, Bengali, Chinese, German, Indonesian, Italian, Japanese, Korean, Mongolian, Polish, Portuguese and Slovakian (contact the Department of Essential Drugs and Medicines Policy) The publication is on the Internet in English and French at: http://www.who.int/medicines/library/par/ggprescribing/begin.htm Teacher’s Guide to Good Prescribing WHO/EDM/PAR/2001.2 DISTRIBUTION: GENERAL ORIGINAL: ENGLISH World Health Organization Department of Essential Drugs and Medicines Policy Geneva, Switzerland Authors Hans V. Hogerzeil1 (Editor) Karen I. Barnes2 Rob H. Henning3 Yunus E. Kocabasoglu3 Helene Möller4 Anthony J. Smith5 Rob S. Summers6 Theo P.G.M. de Vries7 with contributions from Hannelie Meyer, Sule Oktay, Budiono Santoso and Sri Suryawati 1 WHO Department of Essential Drugs and Medicines Policy, Geneva, Switzerland 2 WHO Collaborating Centre for Drug Policy, Information and Safety Monitoring, Department of Clinical Pharmacology, Medical School, University of Cape Town, Cape Town, South Africa 3 WHO Collaborating Centre for Pharmacotherapy Teaching and Training, Department of Pharmacology and Clinical Pharmacology, Medical Faculty, Groningen University, Groningen, The Netherlands 4 WHO, South African Drug Action Programme, Pretoria, South Africa 5 WHO Collaborating Centre for Pharmacotherapy Teaching and Training, Discipline of Clinical Pharmacology, Medical School, Newcastle, Australia 6 WHO Collaborating Centre for Pharmacy Curriculum Development and Rational Use of Drugs, School of Pharmacy, Medical University of Southern Africa, Pretoria, South Africa 7 Department of Pharmacology, Medical Faculty, University of Amsterdam, Amsterdam, The Netherlands Acknowledgements The support of the following persons in reviewing earlier drafts of this book is gratefully acknowledged: F. Danish (Kabul, Afghanistan), A. Haeri (Tehran, Islamic Republic of Iran), A. Helali (Algiers, Algeria), K.K. Kafle (Kathmandu, Nepal), R.O. Laing (Boston, USA), J.C. Lombard (Sovenga, South Africa) and I. Moodley (Johannesburg, South Africa). © World Health Organization 2001 This document is not a formal publication of the World Health Organization (WHO), and all rights are reserved by the Organization. The document may, however, be freely reviewed, abstracted, reproduced and translated, in part or in whole, but not for sale nor for use in conjunction with commercial purposes. The views expressed in documents by named authors are solely the responsibility of those authors. Printed in France Annex 4 Contents iii Introduction 1 Part 1: How to teach pharmacotherapy with the Guide to Good Prescribing 5 Chapter 1: The role of the teacher 7 Chapter 2: How to write learning objectives 11 Chapter 3: How to construct good patient examples 15 Chapter 4: Teaching notes for the Guide to Good Prescribing 21 Chapter 5: Developing critical appraisal skills 35 Chapter 6: Application in primary care settings 45 Part 2: How to assess the students, the teachers and the course 49 Chapter 7: How to assess the students 51 Chapter 8: How to assess the teachers 67 Chapter 9: How to measure the impact of the training 69 Part 3: How to mobilize support 75 Chapter 10: How to mobilize support for problem-based pharmacotherapy teaching 77 Annexes 1: Examples of patient cases used for training paramedical workers in South Africa 83 2: Example of a prescribing score sheet used for paramedical workers in South Africa 91 3: Examples of patient cases used in a research study in Yemen 92 4: Example of a scoring form used in a research study in Yemen 96 Authors by chapter 98 1 Introduction Introduction 1 De Vries TPGM, Henning RH, Hogerzeil HV, Bapna JS, Bero L, Kafle KK, Mabadeje AFB, Santoso B, Smith AJ. Impact of a short course in pharmacotherapy for undergraduate medical students. Lancet 1995;46:1454–7. 2 De Vries TPGM, Henning RH, Hogerzeil HV, Fresle DF. Guide to good prescribing. Geneva: World Health Organization; 1994. WHO/DAP/ 94.11. Irrational prescribing is a “disease” which is difficult to treat. Prevention is possiblehowever. For this reason the WHO Department of Essential Drugs and Medicines Policyaims to improve the teaching of pharmacotherapy to medical students. The Teacher’s Guide to Good Prescribing is a companion volume to the Guide to Good Prescribing. Its target audience is university teachers who wish to use the Guide for teaching undergraduate medical students. Its main objectives are: to explain the educational approach underlying the Guide; to explain how to teach pharmacotherapy with the Guide; to give practical advice on how to assess the students, the teachers and the course; and to assist in mobilizing support for problem-based pharmacotherapy teaching. The WHO Guide to Good Prescribing Surveys in Canada, the USA and Europe have concluded that structured training in pharma- cotherapy is relatively uncommon. In many medical schools pharmacotherapy teaching is characterized by the transfer of knowledge about drugs, rather than by the skill to treat patients. However, in the last decade a number of educational programmes have been developed to improve the teaching of pharmacotherapy. Based on positive experiences in Groningen (The Netherlands), followed by a large international study involving seven medical schools in developed and developing countries,1 WHO has developed a manual for undergraduate medical students on the principles of rational prescribing, the Guide to Good Prescribing.2 This manual presents the students with the following normative model for pharmacotherapeutic reasoning. First, students are taught to develop a standard treatment for common disorders, resulting in a set of first-choice drugs, called P(ersonal)-drugs. In the course of developing their P-drugs, students are taught to consult existing national and international treatment guidelines, formularies, textbooks and other sources of drug information. Then they are shown how to apply this set of P-drugs to specific patient problems, using a six-step problem-solving routine: (1) define the patient’s problem; (2) specify the therapeutic objective; (3) verify the suitability of your P-drug and choose a treatment for this individual patient; (4) write a prescription; (5) inform and instruct the patient; and (6) monitor and/or stop the treatment. The rationale behind this approach is that medical students develop, at some time in the course of their studies or early in their career, a set of drugs which they will use regularly from then on. However, this choice of drugs is often made on irrational grounds, e.g. by copying the prescribing behaviour of clinical teachers or peers without considering Teacher’s Guide to Good Prescribing 2 alternatives or knowing how to choose between them. The Guide to Good Prescribing not only helps students to select P-drugs in a rational way, but also to consult, understand and use existing treatment guidelines. For example, it teaches the students how to verify, for each individual patient, whether their P-treatment is also the most appropriate choice in this individual case; and, if necessary, how to adapt the drug, the dosage form, the dosage schedule, or the duration of treatment. Further on in their careers doctors are subject to many other influences on their prescribing, including scientific publications, commercial information and patient pressures. The Guide makes students aware of these influences and helps them to make optimal use of the information available to them to update their P-drugs in a rational way. The Guide to Good Prescribing has been widely acclaimed as an innovative and very practical teaching tool. Although published by WHO it can be freely abstracted, translated and reproduced, in part or in whole, but not for sale or for commercial purposes. It can also be downloaded from the following website: http://www.med.rug.nl/pharma/who-cc/ggp/ homepage.htm/. In practice, this means that there is no need for students to procure original copies, as the Guide can easily be reproduced as part of student hand-outs. By 2000 it had been translated into 18 languages. Problem-based pharmacotherapy teaching The six-step routine offers a logical structure to guide students through the process of pharmacotherapy, and using the Guide for self-study is probably beneficial in itself. However, medical students need to be trained in additional skills necessary to apply the method successfully in pharmacotherapy. The training of cognitive skills requires special teaching methods, and the recommended teaching method is problem-based learning in small groups. In addition, specific educational methods are required to teach communication skills, such as using simulated patients and bedside teaching. The main message of this Teacher’s Guide is that problem-based pharmacotherapy teaching is possible within the structure of a traditional (non problem-based) curriculum. This manual contains practical information on how to implement it. The link with clinical training Problem-based pharmacotherapy teaching alone, either as an introductory course before students enter the wards or as an ongoing part of integrated clinical teaching, is not enough to “vaccinate” medical students against the pressures towards irrational prescribing that they will face in their professional career. A first risk period, and probably the one with the strongest influence, is the time of clinical attachments and junior clerkships. After some years of mainly theoretical studies most students are eager to become clinically active and are very sensitive to the role model of their clinical teachers. Unfortunately, irrational prescribing is widely reported from teaching hospitals,3 while clinical teaching of undergraduate students is often insufficiently planned and supervised, � Problem-based pharmacotherapy teaching is possible within a traditional curriculum 3 Hogerzeil HV. Promoting rational prescribing—an international perspective. British Journal of Clinical Pharmacology 1995;39:1–6. 3 and usually delegated to junior staff. Any irrational prescribing behaviour in the wards will therefore almost automatically be copied by the juniors. Even on those occasions when the students have the chance to observe the example of clinical professors and consultants, their prescriptions may reflect the treatment of a rare disease or difficult complications and do not necessarily represent the type of patient problems the students are likely to meet in the first years of their professional life. Therefore, pharmacotherapy teaching should not only be problem-based but should also be based on clear objectives (Chapter 2). What type of doctor should the undergraduate curriculum produce? Which types of diseases and complaints should the young graduates be able to recognize and treat? Which drugs should they be able to use effectively and confidently? And which skills are needed to choose the right treatment and adequately inform the patient, and to read information on new drugs with a critical eye and get the maximum benefit from it? It is essential that these skills, once taught in the undergraduate phase, be reinforced during the clinical attachments. The set-up and academic content of these attachments should reflect this. Chapter 5 summarizes some innovative teaching ideas from medical schools in developed and developing countries. Since 1994, when the Guide to Good Prescribing was published, the book has also been used as the basis for teaching paramedical prescribers, such as medical assistants and nurses, within in-service training programmes. This application of the Guide was therefore innovative in two aspects: paramedical students, and in-service training. This aspect is discussed in Chapter 6. While the teaching method aims to transfer the practical skill of prescribing rather than just the knowledge on drugs, the method of assessing the students must also reflect this objective. Here continuous assessment methods should be considered, as well as open- book examinations and objective structured clinical examinations. Part 2 contains practical information on how to organize such examinations and ends with a chapter on how to mobilize support for problem-based pharmacotherapy teaching (Chapter 10). This Teacher’s Guide has been developed by a group of authors working under the responsibility of the WHO Department of Essential Drugs and Medicines Policy. Comments on the text and the examples in this manual, as well as reports on its use are actively solicited and should be sent to the Director of the WHO Department of Essential Drugs and Medicines Policy, 1211 Geneva 27, Switzerland (fax + 41-22-7914167). Introduction PART 1 How to teach pharmacotherapy with the Guide to Good Prescribing CHAPTER 1 The role of the teacher Chapter 1. The role of the teacher 7 Be glad they make mistakes, there is no better opportunity to teach them. Modified from M. Gaus, dog trainer If you are reading this book you are probably interested in improving the teaching ofpharmacotherapy. You may have read the WHO Guide to Good Prescribing, and you maybe thinking about implementing parts of it. Two concepts lie at the heart of the Guide to Good Prescribing and this accompanying Teacher’s Guide to Good Prescribing: the six-step logical method to teach prescribing, and the problem-solving learning method. The six-step logical method was first developed and used in Groningen (The Netherlands) and has proved very useful in many other settings, including the teaching of paramedical workers. Teachers have usually adopted the essentials of the method very quickly, as most of them are already familiar with clinical work and pharmacology. However, the main risk for teachers is to keep focusing on the transfer of knowledge rather than on the skills of selecting and prescribing the right treatment. The students should be taught how to prescribe, not what. As one teacher said: “The drug they select is their responsibility, but the way they do it is mine”. The second concept underlying the Guide to Good Prescribing is problem-based learning, a concept with which most teachers are much less acquainted. This situation is probably related to the fact that few medical schools around the world use problem-based curricula. It is therefore assumed that you are not experienced in problem-based teaching. You may, however, have various ideas about the concept, ranging from unrestricted support to vigorous disagreement. Research over the years has shown that students who have been trained by problem-based learning methods gain about the same level of knowledge, but perform better on skills and attitude compared to students from traditional curricula. Moreover, students enjoy problem-based learning much more, and so do many teachers. It is a common misunderstanding to think that problem-based learning can only be used in the setting of a full-blown problem-based curriculum, where students work in small groups supported by costly logistic and technical facilities. However, problem-based learning is what it says: driven by the quest for the solution to clinical problems (by the students), and not by learning various subjects by heart from chapters of textbooks or hand-outs prepared by the teachers. Initially teachers may prefer working with smaller groups, as small group processes are easier to control; but it can also be applied to large audiences. Although the students are the main players in the learning process, problem-based learning is controlled by the teacher. The boundaries of learning are set by the teachers, and are based on the objectives of the teaching programme. Such objectives define which knowledge Teacher’s Guide to Good Prescribing 8 the students should acquire, which clinical problems they should be able to solve and which skills they should master, before they can qualify as a doctor. Fortunately, all students are already experienced problem-solvers although the process is largely subconscious. Consider the following example. Suppose that last night the roof of your house was damaged by a storm. Soon the rains will come. By then the roof must be fixed. Stop here, and think. By now, you have already thought of a number of solutions, although you do not have a degree in architecture or engineering. What is your knowledge about roofs anyway? Yet you will be confident that you can solve the problem, just as most people will. Why? Because you trust your problem-solving skills. The actual solution may vary from person to person. One may repair the roof by her/himself, the second may have a friend or a relative who can help, a third just hires somebody to repair the roof, the fourth moves to another house. All are correct! The way to get to the solution is universal: define the problem, make an inventory of potential solutions, choose the best option, and implement it. The nature of the problem is irrelevant. It can be a car which does not start, a patient with a cold, a patient with lymphoma. The role of the teacher is to trigger the problem-solving routine in the students, and to employ this natural human behaviour to guide them in solving patient problems. This process may sound easy, but for many teachers it requires a major shift in the way they deal with students. It requires a completely different approach, communication and attitude. The centre of the learning universe is no longer the “knowledgeable” teacher, but the students. Students and teachers become equal partners in the process, although they have different roles. The teacher stops lecturing and starts asking questions. Practice in conducting problem-based learning is the main thing you will need. Inform your students that you are not all that experienced, and that you would welcome suggestions on improvement. Plan your classes well and share those plans with the students. Start your fourth or fifth session by briefly reviewing the earlier ones. The moment you get any spontaneous response, you know you are on the right track. If you are inexperienced at problem-based learning, it is advisable to gather one or two colleagues or friends who are going through the same process. It may help to share your experiences, to swap classes, etc. Be fair with your students, and never be rude. Students often lack detailed knowledge of the disease you are discussing and of the properties of the relevant drugs. In the early stage this is a golden opportunity for them to learn how to use literature sources. Care should be taken not to ‘punish’ the students for lacking knowledge. At first some students may find your advice to look for the literature offensive or very threatening. Keep in mind that people learn from failures, especially if they discover the failures for themselves. A more-or-less correct answer from a fellow student is often much more convincing and motivating than your expert opinion. Therefore, allow the students to divide the work between a number of them (a solution which may horrify some teachers in traditional curricula) and arrange for a session in which they report back to the group. Encourage those students who propose wild or stupid ideas. Share with the group your observation on the roles that different students have. Often, comparing their roles with animals, sports heroes or politicians arouses much hilarity and gets you closer to the group. Never exclude yourself from any assessment within the group, and be prepared to be compared � The teacher stops lecturing and starts asking questions 9 to an animal yourself! If students are hesitant to express themselves, ask them to write things down, anonymously if necessary. The final challenge for a problem-based learning programme, and for the teachers, is to give the group or the individual students the opportunity to treat real patients under supervision, for example as part of clerkships. Practice makes perfect! Chapter 8 presents some suggestions on how to assess your role as a teacher, including an example of a student observation sheet. BOX 1. PRACTICAL HINTS FOR PROBLEM-BASED TEACHING � Get the students to sit in a circle or around a large table. Sit among them or in a corner of the classroom, but not in front of them. � Define one objective for the teaching session, and inform the students about it. If you find it difficult to formulate clearly, prepare it in advance and write it on the blackboard. If necessary, attach a time frame. For example, the learning objective “to understand the steps you have to take to solve a patient problem” is quite different from “to choose the correct drug-treatment for a patient with asthma”. � Start the session by defining one single problem, or extracting one problem from a compli- cated case. Make sure everyone understands that this is the problem. If the problem is likely to be too complicated, simplify it. � Don’t interfere with the group during the first ten minutes of the session. Use this time to define the roles the group members play. When you feel that an intervention is needed, do not intervene immediately but continue to observe the group for several minutes (look at your watch: most moderators are tempted to act too soon). Then reconsider what you had wanted to say to the group, and change it if needed. � If the group is chaotic or does not get going: start all over again after redefining the roles of its members (chair, reporter). Make clear what is expected from a student who fulfils a certain role, rather than just using him or her as a writer on the blackboard. � Avoid eye contact with the student who is talking or trying to get attention. Often students are looking for your reaction while their colleague is talking. Only use eye contact or body language to direct them to the speaker. � Think of interventions only as influencing the group process. There is quite a difference between an intervention on the group process level (with questions such as “Why are you not capable of solving the problem?”, “Why are some of you so quiet?”), compared to one at the content level (such as giving information on a certain drug, or questions such as “What would be the correct drug?”, or “What would be the next step?”). � When intervening, address the group in general rather than a single individual. � Groups work at a pace of their own. A slow pace need not be due to a lack of knowledge, but is often due to the group process. Address the problem of slow pace directly, with emphasis on the process and not on the content. It does not help to speed up a group by content- directed interventions. � Ask quiet students in the group to summarize, in order to find out whether they just did not prepare for the session or whether their silence is caused by the group process. Ask them what difficulties may be preventing them from participating; their reasons may also apply to other ‘quiet’ students. � Each and every time you are planning to intervene on the content of the discussion (rather than the process), think of all of the above points. Count to ten, and then don’t intervene! Chapter 1. The role of the teacher CHAPTER 2 How to write learning objectives Why do we need learning objectives? For most of us who began medical training many years ago, the idea of writing outspecific objectives for courses or for individual teaching sessions may seem, at first,an unnecessary and rather bureaucratic exercise. However, if we think back to our own undergraduate days we will often remember the ill-prepared, rambling discourses that passed for lectures—frequently given by the most illustrious members of staff—and the examinations which seemed to bear little relation to anything we had been taught or learned on our own. Education was handed down from on high, and any suggestion that students or staff might evaluate the curriculum and its delivery would have been dismissed as unnecessary. Over the past two decades the influence of educationalists has made a slow but certain impact on medical and other health science teaching, and we now recognize the need for more structure in our programmes, and better advance planning. Formulating learning objectives is part of this planning process. A teaching objective, be it for a single lecture or for a whole course, is a statement of where you want the student to be after your teaching. In other words: what will the student be able to do after the teaching that s/he could not do before? Examples of learning objectives For a course: At the end of this course, students will be able to apply the principles of the Guide to Good Prescribing to solve prescribing problems in cardiovascular diseases. For a single teaching session: At the end of this session, students will have learned how to select between different drugs on the basis of comparative efficacy, safety, cost and suitability, and will have practised this technique on a new problem not previously studied. Learning objectives are a clear statement of what is expected of the teaching programme, but are also fundamental to the design of student assessments (“examinations”). Any examination is a sampling process. Not everything that has been taught and learned can be examined. If the teaching programme has clear objectives it is much easier to select items that are representative and to know to what depth to examine them. Many educationalists would say that creating your examinations should precede (not follow) the development of the teaching programme. Once you have decided what is important enough to be examined, you are better placed to decide how to teach it. In constructing an examination it is important to keep referring to the learning objectives of the course, in order to ensure that the examination is relevant, representative and fair. Teacher’s Guide to Good Prescribing 11 Teacher’s Guide to Good Prescribing 12 This approach also works to the advantage of students who will have a clear view of what is expected of them. Experience suggests that the inescapable stress of examinations is reduced if students have had clear objectives and know that the examination will be based on these objectives. Fear is often based on uncertainty! Objectives are also important for evaluating a session or a course. If we know where we want to be after the course, we can judge whether we got there—or in what way we fell short of realising the objectives. Students are always vocal critics, and a session with them evaluating a course at its conclusion may be very revealing. If the review session is focused on discussing the course objectives it will be much more critical and will avoid many of the trivial issues that could easily distract from the review of key concepts. Example: Evaluating the teaching programme in Newcastle, Australia In Newcastle the undergraduate medical course is based on 47 programme objectives—basically statements of where and what the students are expected to be at the point of graduation. The assessment system is based on these objectives. Each year the examination effectively asks to what extent the students have achieved the objectives, appropriate to their level of seniority in the school. In the early 1990s, it was decided to embark on an “outcome evaluation” of the programme, to study how good the graduates were once they had left the medical school. However, this aspect proved very difficult to measure. It was eventually decided to measure the performance of the graduates against the original learning objectives. These objectives gave a firm statement of what was expected of the graduates. They were the best starting point for the evaluation of the programme. How to write learning objectives If objectives are to be central to progamme construction, examination and evaluation, they should be framed in such a way that they can be assessed. For example, “At the end of this session students should appreciate the principles of good prescribing for pain” may sound good as an objective—until you try to write an examination question. How do you test “appreciation”? A more concrete objective might read “At the end of this session, students will be able to write an appropriate and correct prescription for a patient with post-operative pain”. Now the goal of the teaching session, and the way in which it may be examined, are clear. In an evaluation it would be fairly easy to judge whether the teaching session had achieved its purpose. The six steps in the prescribing routine on which the Guide to Good Prescribing is based can easily be embodied in a teaching course as a set of objectives. Taken as a progressive series they might read: “At the end of this programme the student, when given a clinical scenario, will be able to: 1. Identify a patient’s problem 2. Specify the therapeutic objective 3. Select a drug on the basis of comparative efficacy, safety, cost and suitability 4. Write a correct prescription 5. Counsel the patient on appropriate use of the medicine 6. Make appropriate arrangements for follow-up.” 13 Each of these is a concrete statement, and the way in which the programme might be examined is also clear. Points 1–3 refer to problem-solving and cognitive activity, and can be examined in that mode. Writing a correct prescription is a practical skill, which can be tested by asking students to write one or more prescriptions as part of their examination. Points 5–6 involve behavioural skills and are best tested as part of an observed interaction between the student and a real or simulated patient. An Objective Structured Clinical Examination (OSCE) is a useful structure for this part of the examination (see Chapter 7). Learning objectives for the individual sessions become more obvious as soon as the objectives for the course as a whole have been written. Such session objectives depend on the amount of time and the number of sessions available to you as a teacher, and the way you want to structure the programme. For example, if you have been allocated six sessions to introduce the principles of the Guide to Good Prescribing you would need to think very carefully about the emphasis you would put on each of the overall programme objectives in the time you have at your disposal. If you work in a problem-based medical school you might not need to put as much time into identifying and defining the patient’s problem, but perhaps more into specifying the therapeutic goal. If the six sessions are also to serve as a mini-revision of some therapeutic areas (e.g. if a course is set in the final year of a medical course) you may need more emphasis on the drug selection process, and the criteria for discriminating between different drugs and drug groups. If the course is set up with a major curricular emphasis on behavioural science and learning interpersonal skills, you might concentrate on the behavioural aspects of the prescribing process. However the course is structured, its objectives should specify what each session will include, and where you expect the students to be in knowledge, understanding and practical skills at the end of the session. Conclusion Writing teaching objectives takes time and thought, but once they are written and agreed upon they provide the essential foundation for constructing and maintaining a teaching programme, for assessing the students and for evaluating the course itself. Objectives “anchor” the course, and the time spent in putting them together is never wasted. Some more information on learning objectives for pharmacotherapy teaching is given in Chapter 5. Chapter 2. How to write learning objectives Teacher’s Guide to Good Prescribing 14 15 CHAPTER 3 How to construct good patient examples Chapter 3. How to construct good patient examples The best patient examples are to be found in outpatient clinics or hospital beds—that is, in the real world of the clinical encounter. Applying the principles of theGuide to Good Prescribing at the bedside, with a real prescription as an outcome, is clearly the best way to simulate the process that is undertaken by doctors at least 200,000 times in a lifetime of medical practice. However, reality dictates that students must learn the prescribing process by using patient problems that are constructed by the teaching staff. These problems should be specially designed to lead the discussion down particular channels (avoiding tempting side-issues which could take up time and would distract from the main aim). In the rest of this chapter it is assumed that: • a patient case will be worked through in a tutorial setting with an experienced tutor • the objectives of the session are clear • the tutor has a series of issues to be discussed and a clear concept of where the tutorial is to end • the six steps of the problem-solving routine of the Guide to Good Prescribing are part of the tutorial. Steps in constructing a clinical scenario Step A. Make sure the focus is on therapy and not diagnosis As you write the mini case history, make sure that either the diagnosis is stated, or sufficient clues are provided for it to be absolutely explicit. There is nothing doctors and medical students like more than a good discussion about a difficult diagnosis. Give your students not quite enough diagnostic information, and they will attack the scenario like piranhas and tear apart the diagnostic material, call for more information, claim the problem is insoluble in its present form, and totally ignore the therapeutic issues you wanted to focus on. Example of a clinical scenario with bad diagnostic information “… A man of 45 has recurrent blood pressure readings around 160/95. He has no symptoms and wants to know whether treatment should be initiated …” At an international conference on clinical pharmacology, the process of the Guide to Good Prescribing was presented by means of a demonstration of a teaching session, with senior pharmacologists as “students”. One of the student groups spent 50 of its 60 allocated minutes discussing the adequacy of the diagnostic information of this patient with benign hypertension. The skilled facilitator of the group could not get to a discussion of treatment, even though the group had been asked to arrive at a prescribing decision. Instead, the discussion went into the accuracy of measurement, the number of readings required to be sure hypertension had been Teacher’s Guide to Good Prescribing 16 established, the extent of investigations needed, and the nature of his family history! In the remaining ten minutes of the session they barely scratched the surface of many therapeutic issues—drugs versus non-drug treatment, choice between drug groups—which should have occupied most of the time. The case would have been much more useful had it been written as “A man of 45, with established benign hypertension and no evidence of underlying primary cause or of target organ damage, requires treatment. How would you proceed?” This scenario would have pre-empted the diagnostic discussion and would have led quite easily to the treatment issues and to the six points of the Guide to Good Prescribing. Do not be afraid to state the diagnosis. Both cases below, though rather brief, have sufficient clinical detail to give reality to the presentation. Both, explicitly or implicitly, lead to a rapid diagnostic decision (malaria and pulmonary embolus) and the scene is set for spending the rest of the time on a discussion of therapy. Two examples of clinical scenarios with good diagnostic information A woman of 36 has had intermittent fever for 6 days, with rigors, headache and anorexia. A thick film of her blood shows many red cells invaded with P. falciparum. A man of 62 has undergone hip surgery, complicated by deep venous thrombosis on the operated side. On the ninth post-operative day he experiences a very sudden severe pleuritic pain in the right chest. He has had a small haemoptysis and has a tachycardia of 120 beats per minute. Step B. Modify your patient case to bring about special teaching points in drug choice The difficulty of a clinical scenario for students can be increased or decreased by simple changes in the clinical story. Example of a simple scenario A man of 52 has developed inflammatory arthritis in several small joints in both hands. Paracetamol has not helped. He is intolerant of oral aspirin. What symptomatic relief can you provide? This scenario has blocks in place by making paracetamol ineffective and by making the patient intolerant of aspirin. The discussion would probably revolve around the choice (on the basis of efficacy, safety, cost and suitability) of a non-steroidal anti-inflammatory drug. This could yield a very productive session. Now alter the scenario in a very simple way by adding the text in italics: Example of a small change which complicates the scenario A man of 52 has developed inflammatory arthritis in several small joints in both hands. Paracetamol has not helped, and he is intolerant of oral aspirin. He receives long-term warfarin following heart valve surgery two years ago. What symptomatic relief can you provide? You could also add “He has had recurrent problems with peptic ulcer disease, and is taking ranitidine 150 mg twice daily”. With these additions you have altered a fairly simple problem 17 into a much more complex issue of either potential interaction with warfarin or the risk of aggravating peptic ulcer disease. Let us look at another example. Example A 62-year old man presents with classical angina pectoris. You wish to prescribe a prophylactic for him as well as provide immediate treatment for the attacks. Now add one word to the scenario and the discussion changes completely: Example A 62-year old asthmatic man presents with classical angina pectoris. You wish to prescribe a prophylactic for him as well as provide immediate treatment for the attacks. In the second scenario the selection of the most appropriate drug is totally changed by the patient’s asthma. By modifying other aspects of your case, other considerations come into force. The same patient with or without renal failure (a simple statement of serum creatinine in the patient example) will have different requirements for dosing with digoxin, gentamycin or lithium. Altering the age of the patient from adult to child brings in dose calculation by age or by weight. Another major variable which can be introduced into a problem is pregnancy (see next example). Example A 23-year old woman has had three witnessed grand mal convulsions. No lesion is demonstrated after full investigation. There is a strong family history of epilepsy. This problem is straightforward, but may be complicated by oral contraceptives if she is taking them, which may lead the group to a prescription for phenytoin. Now add pregnancy to the case: Example A 23-year old pregnant woman has had three witnessed grand mal convulsions. No lesion is demonstrated after full investigation. There is a strong family history of epilepsy. Immediately the problem is complicated by risks of teratogeneticity of the varying available anti-convulsants. In the Guide to Good Prescribing a set of P-drugs is generated for a particular condition suitable for the straightforward management of the uncomplicated patient—the treatment of first choice. Within a training course, a first set of patient scenarios should probably illustrate this fact and make the point that many patients can normally be treated with first-choice drugs. Chapter 3. How to construct good patient examples However, specific patient factors (age, organ functions, pregnancy, other diseases, other drugs) may dictate a shift away from the P-drugs for the condition. This can be illustrated by the more complicated examples mentioned above. Such scenarios may lead the discussion to drugs normally accorded a lower priority (“second choice”) for the condition. In constructing the patient problems you can choose where the emphases will fall, enabling you to illustrate the difference between P-drugs and patient drugs—those suitable for a more complicated individual. In summary, very small changes to your patient’s age, pre-existing disease, other drug therapy and reproductive state can bring profound differences in the flow of a tutorial, and in the therapeutic decisions taken in the end. By such simple means the tutorial can be adapted to the learning objectives of the session. Step C. Modify a clinical scenario to emphasize problems in patient understanding or compliance All the illustrations above deal with the safety or efficacy of prescribing drugs in special circumstances. The principles are medical or pharmacological. However, the Guide to Good Prescribing goes further than just writing a prescription: it also deals with the interaction between prescriber and patient. At this point it is possible to modify the clinical scenario to illustrate some of the more difficult aspects of guaranteeing patient understanding or ensuring adherence to treatment. If you make your patient developmentally disabled the students must think of ways of ensuring regular drug taking. If you make your patient deaf the students must find ways of communicating (this situation can be particularly stressful for the student if you use role- play for this part of the tutorial). If you make your patient old and forgetful, the consideration of the likelihood of adequate compliance becomes far more important than the earlier consideration of drug choice. There is an unlimited number of possible combinations. Step D. Rehearse your clinical scenario Once the clinical scenarios come together, rehearse in your mind the way in which the tutorial will run. As you have constructed a patient problem, you will need at some point to test the suitability of the student group’s P-drug for the patient in your clinical scenario. This assumes the group has already developed a P-treatment or a list of P-drugs for this condition. Alternatively, if the group has not (yet) developed a P-treatment for this condition, the patient problem could first be used as the reason to generate such a P- drug. In that case verify during the discussions that the patient scenario is actually representative for the “normal” case for this condition. Jot down the major discussion points that you think your patient problem will raise. Do you need any resources specifically for this problem (for example, articles about drugs in pregnancy or renal disease, or drug interactions)? How does this patient problem fit within the overall prescribing course? What are the key points that you want to bring about through this clinical scenario? In short, what are your learning objectives for this scenario? Finally, check the problem against the understanding of a critical colleague. Is it credible, true to clinical reality, illustrative of important points for student learning, and logically placed in the development of your curriculum? 18 Teacher’s Guide to Good Prescribing Summary: practical points for constructing a patient scenario • Make sure the focus is on therapy and not on diagnosis • Start with a simple case where the P-drug can be used • Modify the scenario to bring about special teaching points in drug choice • Modify the scenario to emphasize problems in patient understanding or compliance • Always rehearse your clinical scenario Chapter 3. How to construct good patient examples 19 Teacher’s Guide to Good Prescribing 20 21 CHAPTER 4 Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing The objective of this chapter is to summarize some of the experiences of teacherswho have used the model of the Guide to Good Prescribing in both traditional andproblem-based learning curricula. It provides you with some basic ideas and examples to stimulate you to develop your own teaching programme. Most of the examples in this chapter originate from problem-based training courses and are therefore presented in the form of a patient problem. However, they can be adapted to the setting of more traditional curricula. Teaching notes for Chapter 1: The process of rational treatment Chapter 1 is intended to provide the reader with a brief overview of the method. Although only five pages long, it introduces key concepts that are dealt with extensively later in the book. These key messages are that: • the process of treating a patient follows a systematic methodology • rational treatment is based on a thorough understanding of the patho-physiology of the disease • treatment options can be divided into the following types: 1. advice/information 2. non-drug treatment 3. drug treatment 4. referral • there are two stages in treating a patient: 1. choosing a P-treatment and P-drug(s) for common conditions and complaints 2. the six-step routine in treating a patient. The chapter is often used as introductory reading material to a course in pharmacotherapy. In such a case, care should be taken that students extract the key concepts of this chapter. Many students may respond to the question “What was the subject of the chapter?” with “The treatment of dry cough”. As a general introductory classroom session you may present the following assignment: Suggested student assignment Many of you have visited a doctor. List chronologically and along general lines the process that was going on in the doctor’s head during the consultation. Try to subdivide broader issues, such as examination, into logical parts (physical examination, laboratory examination). Teacher’s Guide to Good Prescribing 22 To increase awareness of the methodology, one or more homework assignments may accompany the reading of the chapter. A few examples of such homework assignments are given below. In a subsequent classroom session the homework assignments may be used as starter questions for a group discussion. Suggested student assignments 1. Identify three key concepts in the process of rational treatment. 2. Summarize the treatment of the patient with dry cough in a flow chart, identifying the different steps and actions taken. 3. After reading Chapter 1, construct an example of rational treatment, choosing one diagnosis from a short list of current topics for the students. The easiest way is to start with a patient with a single diagnosis. 4. Which steps would change if the patient was a 5-year old girl with the same symptoms of dry cough? Teaching notes for Part 2: Selecting your P(ersonal)-drugs (Chapters 2 to 5) General difficulties with this section At first, most students are easily confused by the P-drug concept. One of the most common mistakes is that they think that P(ersonal)-drugs are drugs personal to the patient, rather than personal to the doctor. As a consequence, when they start treating individual patients in a later stage of the programme, they want to start again with the process of choosing P-drugs for that patient, rather than just checking the suitability of the P-drug they had already selected for this condition. Another problem is that the P-drug concept may not get across: a P-drug is not the name of a group of drugs or of a single drug. It is a drug that is ready for action: a drug treatment of first choice with its strength, dosage form, duration of treatment and necessary warnings and information to the patient. It may be good to stress the concept of P- treatment here. If this concept is not made clear, students may again start choosing a P-drug at the moment of treating patients. The phrase “ready for action” is known to be very helpful in clarifying the P-drug concept. Students often lack detailed knowledge of the condition or disease they are discussing, and of the properties of the related drugs and drug groups. At an introductory stage this is not a problem but rather an opportunity for the students to learn to access the literature. It is good to anticipate this lack of knowledge and to help the students to seek answers to the problem. A typical strategy in problem-based learning is to identify a number of gaps in the knowledge of the group and to have pairs of students find the answer and report back to the group. Usually they cannot do this within one setting. The tasks may then be given as an assignment for homework. The more conventional solution is for the teacher to identify certain parts of books or articles where the students will find the answer, and give them a certain limited time to do it. However, the students will not actively learn to find the relevant information themselves by this method. It is therefore not recommended. � A P-drug is a drug ready for action 23 Step i. Define the diagnosis The essential part of this step is to (re)define the diagnosis for therapeutic purposes. It is very common for the next steps in the process to be frustrated because students are unaware of the meaning of this first step. In addition, defining the diagnosis is often hampered by students lacking knowledge of the specific condition. Both problems should be recognized at an early stage and immediate measures must be taken. For example, the students could choose a diagnosis they are familiar with, rather than being forced to work with the one in the book; or the session could be postponed to allow them more time for catch-up reading. Another approach may be to define the diagnosis as part of the problem. For example, the assignment may read “Choose a set of P-drugs for moderate essential hypertension (diastolic pressure 105–125 mm Hg) without signs of end-organ damage”. During the classroom session or in subsequent assignments the subject may be broadened as follows: “Choose a set of P-drugs for severe hypertension due to renal artery stenosis and for mild hypertension due to pre-eclampsia”. It is a good idea to start with the moderate variant, as the drug groups included in the P-drug list will most likely be considered in the mild and severe variants as well. Suggested student assignment Consider the indication asthma for which you want to choose P-drugs. How many different types of asthma would you consider for therapeutic purposes? Also include categories for patients from high-risk groups, if appropriate. Comment: Asthma is a good example because the students should end up with at least five different diagnoses, depending on their way of classification: mild/moderate/severe, asthma in children, with/without infection, hyperreactive/allergic, status asthmaticus, acute attacks, exercise-induced. Step ii. Specify the therapeutic objective Students (and many teachers!) tend to specify the objective of treatment in too vague terms such as: relief of the symptom, or reduction of a parameter. However, the amount of effect and the time to achieve it should be specified as much as possible. For example, in case of moderate essential hypertension the therapeutic objective “get the blood pressure back to normal” is too vague. Instead, it may read “to prevent end-organ damage and normalize life-expectancy by reducing (within one month) and maintaining the diastolic blood pressure under 90 mm Hg”. This example also shows that there are at least two endpoints: blood pressure and end-organ damage! During the process the term ‘end-organ damage’ will have to be defined, which often requires self-study by the students. Suggested student assignments The indication is bacterial pneumonia (temperature: 39.6 °C, moderate dyspnea). Specify your therapeutic objectives for choosing one or more P-drugs if this indication concerns: • children under 5 years of age • adults of 20–40 years • adults older than 70 years. Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 24 Step iii. Make an inventory of effective groups of drugs Making the inventory of effective groups of drugs is usually not a problem. Do not delete ineffective drug-groups by yourself (although you may be tempted to save time), but let the students go through a process of considering and weighing a drug-group that is for some reason of no use in therapy. This experience will contribute to their understanding of the process. Step iv. Choose an effective group according to criteria This step is a difficult one for students to take. At this point in the discussion some students are usually confused by several questions: which criteria should be used, how does one assign weights to various pieces of evidence, and where can the necessary information be found? Weighting the various pieces of evidence is at the very heart of choosing a P-drug. Many teachers choose to teach this aspect by discussing an analogy with a subject close to the student’s mind. Examples are: choose a destination for your next holiday, choose a car, or a dinner to cook. Such subjects guarantee a lively discussion and are ideal for students who are not very familiar with group work. However, if the students are more experienced with problem-based learning you may choose a less familiar topic. This approach has the advantage that the students are less able to solve the problem quickly by relying on their problem-solving skills without struggling with the choosing process. A possible assignment could then be to choose a household product on the basis of three different advertisements. Another method to weight different pharmacological group options is the multi-attributive utility analysis (MAUA). In brief, the available drugs or drug-groups are listed and scored according to four criteria: efficacy, safety, suitability and cost. The average score for each option is calculated. The scores determine the ranking order of the alternatives. The importance of specific weighting criteria may be enhanced by attaching weighting factors to them. For example, for the treatment of an acute severe illness, such as a heart attack, efficacy bears more weight than safety, suitability or cost. The weighting criteria could then be set at 0.4 for safety and 0.2 each for the three other criteria, or even 0.7 for efficacy and 0.1 for the others. The weights should always add up to 1. There are also other ways of ranking the groups. One alternative would be to create a table of the various drugs and the four criteria, and give positive and negative points or plusses and minuses to the various criteria for each drug. Another way is to list for each drug the positive aspects (e.g. effects) and negative ones (e.g. side-effects) without attributing any value to them. Suggested student assignment Determine the weighting factors for the following indications for efficacy, safety, suitability and cost: • Moderate hypertension in a person aged 30–45 • Moderate hypertension in a person aged 30–45, with signs of end-organ damage • Moderate hypertension in a person aged 70 • Moderate hypertension in a pregnant woman Explain the differences, or why no differences are present. 25 MAUA visualizes the choosing process. Students become aware of the use of different criteria to weight. Gaps in their knowledge are easily identified by themselves or by the tutor. Usually MAUA stimulates lively group discussion. It has, however, a number of pitfalls: • Students who cannot think schematically do not grasp the method instantly. It is advisable that the process of MAUA is learned first by using the everyday examples of choosing bicycles, cars or food, or using very easy indications (e.g. iron deficiency anaemia). Do not introduce weighting factors until the choosing process has been completed. In the first round it is perhaps easier to use + and - scores only. • Students do not understand the criteria. This situation occurs frequently, especially with ‘suitability’. Although this criterion is discussed on page 24 of Guide to Good Prescribing, it still may not be clear. Most students feel that the convenience of a dosage form is to be scored under suitability, but many argue that an inconvenient dosage form leads to less efficacy and should be scored there. You may take two approaches to this problem: the typical problem-based learning (PBL) approach would be to let the students themselves define which aspects are to be scored under which criterion. Alternatively, you may define yourself what is to be scored where (see next point). • Students are not capable of identifying meaningful scores for the different criteria. Therefore, most of the groups tend to use non-numeric scoring (±, +, -) rather than numeric scoring (0–10), which hampers the application of weighting factors. In this case, students should find the answer to the question of how to convert non-numeric scores into numbers. • However, the heart of the matter is that students have not developed a perception of the difference between a score of ‘6’ or ‘7’. Indeed, this lack represents the major weakness in using the MAUA (would you know how to score efficacy of aspirin for moderate tension headache? 4, 5, 6, 7, less, more?). Using the MAUA over the years has provided some solutions, e.g. to score efficacy, you imagine 100 patients and estimate the number of patients in which you would reach your therapeutic objective given that 100% of the patients comply with the therapy. If literature shows that a certain drug is effective in 58% of the patients, the score on efficacy may be 6. Similarly, to score suitability, estimate the number of patients who will not discontinue the therapy due to interactions, contraindications or problems with convenience of dosage form or schedule. To score cost, the price of the total treatment should be calculated, rather than the costs of a single dosage form. It is often advisable to use predefined ranges of costs per year or cost per treatment period and attach scores to them (e.g. <50 US$/year = 10; 50–100 US$/year = 9; 100–200 US$/year = 8; etc.). • Although these solutions are helpful in scoring efficacy, suitability and cost, attaching scores to safety remains a problem. Part of this is caused by the inaccessibility of data on side-effects. Some student groups, often those with mathematically gifted members armed with spreadsheet computer programmes, have come up with models to calculate scores. Another solution used by groups is to employ relative scores: the score of one option is set to 5, and the other options are scored relative to this number. Whatever scoring was used, the important thing is that two aspects of safety are reflected in the score: serious side-effects (often rare and endangering the patient’s well-being) and common side-effects (which are inconvenient to the patient and induce non-compliance). Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 26 • Students lack time to dig into literature. This difficulty can be dealt with by splitting the overall tasks into smaller tasks and sharing them amongst individuals or pairs. This approach may be carried out by giving every student a different indication to work on. Compile the responses afterwards. At the cost of investing in one indication the student will get many in return. If the students employ a uniform scoring system, they may just concentrate on a single criterion and/or a limited number of drugs. Remember, the important thing is to get the method across, not the knowledge about all drugs. • Alternatively, the lack of time is a mere reflection of what students will face in the reality of everyday practice. Their complaint about lack of time may be a good opportunity to discuss time-efficient strategies to solve problems. Many students in PBL curricula have access to large libraries and literature databases on computer such as MEDLINE. However, often students, especially in their first years, have not developed a specific search routine for pharmacological literature. It may be very rewarding to make them discuss strategies for finding such literature in their local circumstances, before their actual quest for information starts. Further, such discussion may serve to acquaint them with the concept of ‘Quality of Evidence’ (see below). • Students lack resources to solve the problem. Lack of resources per se does not impose a problem in training students in the concepts of MAUA and choosing P-drugs, although students may find the procedures useless, because they in the end do not know whether they ‘are right’. In the typical PBL setting this result is, of course, the ultimate goal of the training: providing students with the opportunity to discover ways of finding their own answers to problems, which they feel absolutely confident about. Hence, if the problem of not knowing whether the answer is correct persists among students, a special session may be needed for the students to deal with their difficulties. Cost and suitability In choosing drug groups it is often difficult to give marks for the columns cost and suitability as individual drugs within the group may differ considerably. Therefore, at that stage it is advisable to emphasize the other criteria and deal with cost and suitability only in general terms. Cost and suitability will really come into play when choosing individual drugs. Step v. Choose a P-drug In this step the discussion on the relative efficacy of drugs within a certain therapeutic group will have to take place in relation to other aspects such as safety, cost and suitability. In that sense, the discussion of choosing between therapeutic groups is repeated. The discussion about relative efficacy and safety is probably the first one to take place. In comparing different drugs the concept of quality of evidence also needs to be discussed. Quality of evidence Some sources of information on the efficacy or side-effects of drugs are superior to others. For example, a randomized placebo-controlled double-blind trial outweighs the personal opinion of a single individual. The strength of the evidence for a drug action therefore depends on its source. A very good example of this approach is the level of evidence used by the Scottish Intercollegiate Guidelines Network (SIGN).4 For all treatment 4 www.show.scot.nhs.uk/sign/home.htm 27 recommendations developed and published by SIGN the strength of the supporting evidence is indicated with an A, B or C. It is important that students become acquainted with this issue during the choosing process. Table 1. Levels of medical evidence to support treatment guidelines used by the Scottish Intercollegiate Guidelines Network (SIGN) Level Type of evidence Grade of recommendation Ia Meta-analysis of randomized controlled trials (A) Ib At least one randomized controlled trial IIa At least one well-designed controlled study without randomization (B) IIb At least one other type of well-designed quasi-experimental study III Well-designed non-experimental descriptive studies, such as comparative studies, correlation studies and case-control studies IV Expert committee reports or opinion and/or clinical experience of (C) respected authorities (in the absence of level I-III evidence) Source: Scottish Intercollegiate Guidelines Network Suggested student assignment (assuming that the group has chosen a certain drug as a P-drug for a certain indication) “Briefly review the scientific literature and other information sources that you have used to assess this drug’s efficacy. Rank these different sources according to their level of evidence.” Cost comparisons between drugs For the cost of the various treatment alternatives it is important to stress that the cost per tablet or per unit dose is not always a good measure (although it is usually the information that is available). In any case the unit dose should be recalculated into the cost of one day of treatment. It is even better to calculate the total cost of treatment, to compensate for differences in the length of treatment. The discussion can be expanded further by including other costs, such as the cost of laboratory examinations or X-rays, the cost of hospitalization, extra costs of injections, or even the cost of treatment of potential side- effects. Suggested student assignments Simple student assignments can illustrate the above-mentioned points. For example, anthelmintics have different prices but the length of treatment varies (e.g. piperazine and mebendazole for ascaris); amoxycillin usually has the same price as ampicillin, but is taken three times rather than four times daily and is therefore cheaper per day; doxycycline capsules are more expensive than tetracycline capsules, but are taken only once daily. It is also very useful to construct an assessment showing that syrups and sachets are much more expensive than tablets or capsules. Similar assignments can be given for other criteria, such as the safety of various drugs. Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 28 Value for money Now comes the difficult comparison between efficacy/safety on the one hand, and cost on the other. Which drug offers the best value for money? The choice between drugs is easy when one is more effective and/or safe, and also cheaper than the other. The problem starts when a drug is a little bit better/safer and much more expensive than the other. Is the small advantage of efficacy/safety worth the large extra costs? A drug ready for action A final important point to get across is that a P-drug is a drug ready for action. This definition implies that it is not only an active substance defined by its generic name, but with a recommended route of administration (e.g. orally, parenterally, by inhalation), strength, dosage schedule, duration of treatment, and information and warnings to the patient. Research has shown that students consider choosing the duration of treatment as the most difficult issue regarding their choice of P-drugs. Care should be taken that they define it and discuss the issue. Final chapter of Part 2 The final chapter of Part 2 attempts to broaden the perspective from drug therapy to the total treatment plan. Often it is necessary to highlight the four treatment options that are implicitly considered for every indication: advice and information, non-drug treatment, drug treatment and referral. This point can easily be illustrated by using the example of malaria. Suggested student assignments Which therapeutic options are available to prevent malaria? Which therapeutic options are available to treat malaria (P. vivax)? Which therapeutic options are available to treat cerebral malaria (P. falciparum)? Suggested student assignments Individual homework assignments (“choose your P-drug for the following indication: ….”) followed by group discussion, lectures or computer assisted education at the next session. Small group assignments to choose a P-drug for a certain condition; the groups can later defend or exchange their P-drug sets for different indications. 29 Teaching notes for Part 3: Treating your patients (Chapters 6 to 11) Part 3 of the Guide to Good Prescribing deals with the actual treatment of patients, uniting all three skills required in pharmacotherapy. Cognitive skills are necessary to apply general pharmacological principles to the patient case. Communication skills are needed to inform and instruct the patient. And, in some cases, practical skills are needed to administer drugs (e.g. by injection or inhalation). Although students will encounter many problems at this stage, there are only three solutions: practice, practice and practice. Therefore, the process of prescribing should follow a rigid structure. Moreover, it is very important that a set of P-drugs for the indication of the patient cases to be discussed is already prepared or at hand. A proper set of P-drugs will cover drug therapy in about 80% of all cases without any adjustment being made. The other 20% of patients will need adjustments in dosage, dosage form or even consideration of drugs not included in the P-drug set. The main problem during this part of the training is that students do not take their P-drug list as the starting point for deciding on the treatment of the patient. Instead, they start the process of choosing a P-drug all over again, or incorporate additional drugs from outside their P-drug list. If this situation occurs immediate remedial action is necessary. It can initially be a general intervention in the group, such as: “What do you feel about the way you handled this patient case?” up to more directed ones such as: “What is the place of your P-drug set in this consultation?”. Usually, redoing a P-drug choice for a patient case is very time- consuming for a group. Another way of addressing this problem is to discuss whether the students feel they could afford this amount of time in actual medical practice. You could then put the question “What would be a less time-consuming way to choose your drug therapy in medical practice?”. Another way to promote the insight that P-drugs are the starting point is illustrated in the Guide to Good Prescribing with the patients with sore throat (page 34). Yet another possibility is to discuss an example of a specialized clinic (e.g. hypertension, diabetes, asthma), where the need for standardization is obvious. Another common reason why students start choosing P-drugs again while treating an individual patient is their lack of confidence in the correctness of the P-drug selection. This uncertainty is usually based on a lack of knowledge or time when the P-drug set was compiled. If the students show signs of uncertainty about their drug choice it is strongly advisable to introduce special sessions to boost their confidence in prescribing the right P-drug to their patients. The strongest proof of your confidence in their abilities is to give them—under supervision—actual responsibility for the drug therapy of real patients. Remember, if students possess a well-constructed P-drug set, they will define a correct drug therapy in about 80% of the patients just by applying their first P-drug choice! Many pharmacotherapy teachers, even in long-standing problem-based curricula, intuitively oppose the idea of exposing real patients to the therapeutic abilities of their students. Although the logistical problems of such an exercise in daily teaching practice is often given as a justification, the true opposition might stem from a plain disbelief in students being able to treat patients. Such disbelief is probably rooted in one’s own medical training. Most medical teachers were trained in medical systems that gave full therapeutic responsibility only at a very late stage of medical practice, usually after graduation or Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 30 even later. Moreover, additional training in clinical pharmacology or clinical pharmacy further tends to enhance their view on therapeutics as an art or a specialization in itself. Thus, their superior knowledge and skills in pharmacotherapeutics may actually hamper their teaching! If it is possible to create a setting in which a group of students share the responsibility for actual drug therapy, you will be astonished by their enthusiasm, group interaction, steep learning curve, and also by their therapeutic insight and their vigorous efforts to improve it. If it is truly impossible to realize a setting in which students are responsible for treating real patients, other possibilities remain (see below). Suggested ways to present and discuss (simulated) patients Written patient cases can be used to teach specific aspects of the therapeutic consultation, especially cognitive skills. Simulation patients can be recruited through local branches of the Red Cross or other organizations which are used to role-play victims at simulated disaster training. Students may role-play a patient and instruct each other. If you can manage the logistics, it is preferable that the student playing the patient is from another year or group. A special type of consultation that always brings much fun into a training session is a consultation of a patient by telephone. If such consultations are common in the student’s future medical practice you should assist them in practising it. This method is usually easier to arrange than real consultations or simulated patients. Special devices for the other students to allow them to overhear the consultation are not necessary. You can also use your own real patients for this purpose; they usually enjoy it. Step 1. Define the patient’s problem Problems similar to those encountered in Part 2, step i, may arise here as well (see page 20). The most common problems are unawareness of the meaning of this step, and lack of knowledge about the diagnosis. The reader is referred to that section for further information on how to deal with this. In addition, students may lack vital information on the specific patient case, e.g. the patient’s social environment, history, physical and laboratory examinations, intoxications and drug allergies. This part of the training focuses on the therapeutic aspects of the consultation. Care must be taken that students do not go into the process of diagnosis again. Apart from instructing them not to do so, a few simple measures may prevent it, especially with paper patient simulations: • Provide a patient with a clear diagnosis • Add a statement to each case that things not listed are not present (a female patient is pregnant only when indicated, the patient smokes only when indicated, etc.) • Ask students to develop a therapeutic plan, rather than “advise this patient” • Assign one group member as a “watch-dog” for the above • Provide students with an empty prescription form • Limit their time to obtain additional information and have someone around to provide the answers. 31 Step 2. Specify the therapeutic objective This step is almost identical to specifying the therapeutic objective when choosing P- drugs (Part 2, step ii, page 23). Again it is important to spend sufficient time on specifying the therapeutic objective as precisely as possible. You will have to construct specific assignments to accomplish this goal, for example with a specialized clinic of patients. Another alternative is to use medical records of real patients to make students find the correct diagnosis and therapeutic objective through a game of questions and answers. Bedside teaching may also be very helpful here. Suggested student assignments Define your therapeutic objective for the treatment of pain in: 1. a 67-year old female with history of rectal carcinoma, and intractable pain due to metastasis in lumbar spine; no reaction to paracetamol/codeine suppositories 2. a 30-year old office clerk with tension headache for 3 months, who smokes 30 cigarettes a day 3. a 20-year old student with tension headache for 3 months, with an important examination coming up in one week’s time 4. a 41-year old farmer, kicked by his horse. Pain in the chest while breathing and coughing, due to 2 fractured ribs Step 3. Verify the suitability of your P-drug Many students, and especially those trained in traditional curricula, will have passed examinations on pharmacodynamics and pharmacokinetics. Yet most students are unable to apply what they have learned in practice. This problem is not their fault, as the pharmacology training has probably not been geared to this objective. Special assignments are therefore needed to train the students in this specific cognitive skill, with emphasis on problems dealing with fundamental pharmacological principles, such as a decreased or increased clearance of a drug, intoxication or tolerance. In Annex 1 to the Guide to Good Prescribing an attempt has been made to bridge the gap between pharmacological concepts and their implications for daily practice, by introducing the concept of “the window and the curve”. Often it is very helpful for students to get used to visualizing the drug therapy of a patient in terms of therapeutic window and plasma concentration curve. It is therefore advisable that the students read Annex 1 to the Guide before they start treating patient cases. In addition, the first few patient cases should be relatively easy, probably without any complicating factor. In subsequent cases, additional difficulties can be introduced one at a time. You may also want to construct and present a series of patient cases that aim to teach students the most important pharmacological principles. For example, if you construct a patient case that deals with overdosing you could specifically formulate the problem as “Which circumstances could have caused the toxic effects?”, rather than ask “Revise drug therapy of this patient” or “Manage this patient”. Students do not perceive all aspects of treating patients as equally difficult. For example, multi-centre research has shown that most students find the choice of dosage and duration of treatment much more difficult than choosing the drug and dosage form. Groups therefore tend to “escape” from dosage and duration by focusing their discussion on the choice of Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 32 drug and dosage form. This difficulty can be avoided by asking that the answer be given in the form of a written prescription. Usually a blackboard or flipcharts in the classroom provide space for a number of students to write their prescriptions, which facilitates the comparison of different solutions. A systematic structure in evaluating these prescriptions ensures that all aspects are covered. When using patient cases on paper, a discussion on a particular subject may go on forever because one student (or the group) tries to convince the other of the superiority of a particular aspect of drug therapy. This situation usually occurs when both sides have different interpretations of the patient case without conclusive evidence. For example, a 5-year old boy needs treatment for his first acute attack of asthma. You provided the diagnosis: medium to severe asthma attack, mainly due to hyper-reactivity. The students quickly agree on the effectiveness and safety of their P-drug, in this case terbutaline. The conflict may start when they discuss the effectiveness and suitability of the dosage form (which was an inhaler). One group argues that a 5-year old boy having his first asthma attack is frightened and cannot learn how to use the inhaler appropriately. Therefore, they choose subcutaneous injection. The other side argues that the boy will be capable of using the inhaler, as inhalers can be used by children of 4 years of age or older. Injections will scare the child and aggravate his condition. Moreover, “most children know to use it by the age of 4, my sister has a child who could use it at the age of 3, his mother or father is present and will calm him down.” Such arguments will not solve the issue. The group may then turn to the facilitator for a final answer. You will probably not know the best answer yourself, which gives you a perfect opportunity to show the students that there are often no absolute answers in pharmacotherapy, that they have to deal with a certain margin of insecurity, and that, as treating physicians, they will have to make the final choice. IMPORTANT Although most teachers are aware that they should not provide the answer themselves, few can resist the temptation to comment and to provide their personal solution directly in the classroom, or privately after class. In many cases this intervention will ruin the group process and will prevent the students from becoming aware of their own responsibility as prescribers. So, before expressing a personal opinion: first COUNT TO TEN, and then DON’T DO IT! Step 4. Write a prescription Usually this step does not cause any problems. The Guide to Good Prescribing provides general guidelines to common legal aspects of prescribing drugs, such as writing the prescription itself, how to deal with repeat prescriptions and opiate prescriptions. Be sure that your students have access to the correct (country-specific) information on how to write a prescription. Step 5. Give information, instructions and warnings This step usually does not pose specific problems, except that a short course in pharmacotherapy may not provide enough opportunity to the students actually to practise this step. One solution is to arrange for a carousel of simulated patients or real patients (see OSCE examination). Selecting and structuring the information for the patient is a 33 cognitive skill, which can easily be practised with paper patient cases. In group discussions, take care to identify any irrelevant information. Suggested student assignment A good exercise is to construct a patient case and then ask: “Which three priority issues concerning the drug you prescribed would you communicate with the patient?” Some specific dosage forms may pose problems. Many students will never have seen an inhaler, a suppository or a vaginal applicator and very few will actually know how to use them. Yet they will have to instruct patients about the practical aspects of such dosage forms! Annex 3 to the Guide to Good Prescribing provides check-lists and visual aids for different dosage forms. It is strongly advised that some teaching sessions are organized in which the students have to instruct patients (or each other) about the use of various dosage forms, using dummy examples. Step 6. Monitor and stop the treatment The main problem with this step is that students often fail to see the relationship between it and step 2 (define the therapeutic objective). This failure results in insufficient monitoring of the effect of the treatment. Specific assignments can be constructed to remedy this situation. Often you can use a patient case from an earlier part of the programme. Suggested student assignment Students need much practical exercise to master all six steps of the model of therapeutic reasoning. The further they have developed their skills, the more useful it becomes to include real-life patients in the training. A simple method without involving a real patient is ‘backward reasoning’. You visit the wards with the students and randomly pick a medical record. You tell the students the current medication and ask them to find the diagnosis; they may pose additional questions regarding therapeutics (but no other questions). Once they have solved this problem ask them if they would propose amendments to the drug therapy. Chapter 4. Teaching notes for the Guide to Good Prescribing Teacher’s Guide to Good Prescribing 34 35 CHAPTER 5 Developing critical appraisal skills Chapter 5. Developing critical appraisal skills Problem-based teaching with the Guide to Good Prescribing is part of a much widerteaching philosophy, which is based on the following four principles: Pharmacotherapy teaching should: 1. be based on the essential drugs concept 2. be based on clear objectives 3. develop critical appraisal skills 4. be linked to clinical teaching First, each of these components is reviewed. Then a number of innovative teaching ideas are given which have been developed in different medical schools around the world. Pharmacotherapy teaching should be based on the essential drugs concept The essential drugs concept is that a range of carefully selected essential drugs leads to better health care, to better availability of drugs and to lower costs. The practical implications can be summarized as follows. There are several implications for the private sector. Rational selection criteria are used for allowing drugs on the market. The national essential drugs list, or other limited lists, are used as the basis for positive or negative lists for reimbursement by insurance systems. Selective support, such as tax benefits, can be given to the national pharmaceutical industry for the development and production of essential drugs. For the public sector the implications are linked to the development and use of standard treatment guidelines, leading to lists of essential drugs for the supply system, for prescribing by public sector health workers, and for audit and supervision. For training institutions and teaching hospitals the treatment guidelines and essential drugs lists should also be the basis for training students, for prescribing by clinical teachers, and for student examinations. For medical schools and teaching hospitals this concept implies that pharmacotherapy teaching should be linked to the development and use of standard treatment guidelines for common disorders, and to the hospital formulary. The main purpose of this link is that the initial teaching of the principles of rational prescribing is later confirmed by what the students observe as actual practice during their clinical training in the teaching hospital. The main message for the students is therefore two-fold. First, the pharmacology training programme should focus on those drugs which the national authority and/or the drug and Teacher’s Guide to Good Prescribing 36 therapeutics committee of the teaching hospital have selected as first-line drugs. This focus helps to support the message that essential drugs are not second-class drugs for rural areas in poor developing countries only, but that they are the most cost-effective and safe drugs for a given condition; and that the concept of essential drugs applies to teaching hospital settings as well. Secondly, the students should be taught to review critically the selection of essential drugs in the hospital formulary (and in any other reference material, such as the national list of essential drugs and the WHO Model List of Essential Drugs) and then make their own selection of P-drugs. This list of personal drugs is, in practice, much shorter than the national essential drugs list or the hospital formulary. This part of the training should emphasize the importance of the evidence-based selection of essential drugs, and should also equip the students with life-long skills in reviewing the potential (dis)advantages of new drugs in relation to existing treatments, using reliable sources of information and critical appraisal skills. The drugs in the personal formulary reflect the evidence-based selection of the treatment of first choice for common medical conditions that the students are likely to face in their professional life. Personal formularies are therefore developed in exactly the same way as national and institutional essential drugs lists. A list of P-drugs is a personal essential drugs list. Pharmacotherapy teaching should be based on clear objectives The development of clear learning objectives has already been discussed in Chapter 2. However, a few additional remarks can be made here. The learning objectives for the pharmacotherapy teaching programme should be based on the future needs of the students. In other words, the selection and presentation of the topics should reflect the daily practice that most students will face after graduation. In many countries the students will first work as general doctors in primary care settings. The learning objectives of the medical training should therefore be related to the skills needed for a general prescriber in a primary care setting. This need has an implication on the selection of diseases and drugs to be emphasized during the curriculum. Secondly, learning objectives should also be developed for the clinical attachments. For example, in many teaching hospitals a large proportion of clinical cases in the outpatient department and wards are actually referred cases for specialist treatment. Although interesting and challenging from a scientific point of view, these are not necessarily the best cases for didactic purposes. For example, leukaemia may seem a common disorder in a haematological department, and students may spend a lot of time studying the disease and its treatment. But, on average, in 30 years of general practice, leukaemia will only be seen by general practitioners in about 2–3 cases! In basic medical education much more emphasis should therefore be placed on the diagnosis and correct treatment of common disorders such as acute childhood diseases, respiratory diseases such as the common cold, dermatological disorders, and musculoskeletal disorders —and diseases which are not necessarily very common among the patients seen in a teaching hospital. A possible solution would be to strengthen the links with the community medicine department, and arrange for much more training in a primary care setting. In � A list of P-drugs is a personal essential drugs list 37 Amsterdam, students in the final stages of their medical study actually prescribe drugs, under supervision (see below). Prescribing by medical students in Amsterdam In the Medical School of the University of Amsterdam, The Netherlands, pharmacotherapy is an integral component of teaching throughout the curriculum. In the first two years, students work on the principles of pharmacotherapy, using the six-step model of the Guide to Good Prescribing. In the 3rd and 4th years they learn to write prescriptions on the basis of paper cases and patient examples. During the clinical attachments in the 5th and 6th years they actually select the treatment and write the prescription under the supervision of teaching staff. Pharmacotherapy teaching should develop critical appraisal skills Prescribing practices by doctors and other health workers are shaped by the practice of their teachers and colleagues, by their interpretation of medical literature and new data, and by the way they extrapolate all of this for the treatment of patients. Critical appraisal skills are crucial if students are to become rational prescribers. Consequently, students should be equipped with skills to critically review not only medical literature, but also promotional material and the prescribing patterns of fellow clinicians. Only through this critical review can they reach valid and useful conclusions, and incorporate the findings into their practice. It is also important that students are able to communicate this assessment coherently, concisely and confidently to their peers and seniors. In a field that, in its broader sense, could encompass most of epidemiology and biostatistics, it is important to define the limits of your expectations by setting appropriate objectives. It may also be useful to provide students with a list of concepts and terms that they are expected to familiarize themselves with. For example, students can be assigned to select a drug treatment based on reliable comparative efficacy data reported in different terms (e.g. relative risk, relative risk reduction, attributable risk reduction, odds ratio and number needed to treat). Using these terms in a practical exercise can help students consolidate their understanding. Medical literature The medical literature is one of the most commonly used sources of information for students and for practising health care professionals trying to keep up to date. Clinical trials are the gold standard for assessing therapeutic interventions, especially for (new) drugs. Students who are beginning to form reading habits will benefit from early exposure to the concepts of adequate study design, appropriate sample selection, and use of statistical inference. Since meta-analyses and systematic reviews are published more frequently and are convenient summaries, students should also acquire the skill to evaluate these publications, with a clear understanding of their advantages and disadvantages. Medical literature must be critically appraised to confirm the validity of results and conclusions. Further analysis is required to show if the results from a clinical trial, where Chapter 5. Developing critical appraisal skills � The extent to which beliefs are based on evidence is very much less than believers suppose. Bertrand Russell, 1928 Teacher’s Guide to Good Prescribing 38 BOX 2. FACILITATOR’S CHECK-LIST FOR EVALUATING PUBLISHED CLINICAL TRIALS The essential questions � Were treatments randomly allocated? � Were all the patients accounted for? � Were outcomes assessed “blind”? The detailed questions Method—the single most important section of any research publication. Read it first! � Are the aims clearly stated? Does the objective clearly identify the study sample and method of measuring response? � Was the sample size justified? � Are the measurements likely to be valid and reliable? � Could the choice of subjects (inclusion and exclusion criteria) influence the treatment effect? Was the study sample representative of the patient population to which the study results were intended to be generalized? � Was the description of treatment and its administration clear enough to be reproduced by the reader? � What is the control treatment? Comparator drug is preferably the best current therapy in optimal dose, with the same indication and formulation as study drug. � Are the statistical methods described and appropriate (different data need different statisti- cal tests)? � Could lack of blinding introduce bias? � Are the outcomes clinically relevant? Are the outcome variables appropriate for measuring efficacy and safety of the study drug? Hard endpoints are better than surrogate endpoints. Conduct � How was randomization carried out? � Did adverse events occur during the study? � Was funding obtained from an unbiased source? Could the sponsor have biased the results? � Was patient compliance measured? Analysis � Were the treatment groups comparable at baseline? � Were results analysed by intention to treat? � Was statistical significance assessed (confidence intervals are more informative than P- values)? � Were the data adequately described (results best presented in absolute, not relative values)? � Do the numbers add up? � Were adverse events analysed? Interpretation � What do the main findings mean? Statistically significant is not the same as clinically sig- nificant. Was the study large enough, and continued for long enough, to make the results credible? Were study conclusions consistent with the results and did they relate to the study’s objective? � How are null findings interpreted? � Are important effects overlooked? � How do the results compare with previous reports? � What implications does the study have for your practice? Are the likely treatment benefits worth the potential harm and costs? 39 patients are carefully selected and followed up in detail, can be extrapolated to other patients and different settings. Students need to learn to come to their own conclusions after careful, critical review, so that unjustified or exaggerated claims may be challenged. It usually comes as a surprise to students to learn that many papers published in peer- reviewed medical journals have potentially serious methodological flaws, and should certainly not be used to inform practice. Numerous check-lists have been published to guide systematic evaluation of the medical literature.5,6,7,8 Box 2 is an example of a check-list for appraisal of clinical trials, which can be used as a starting point for teaching. When students are provided with a check-list they may mechanically “tick off” the list without understanding how to interpret the significance of a missing tick, and without developing real critical appraisal skills. For example, a small case series of adverse events should be regarded as a signal, and not discarded because it is not a controlled study. Sample size is often justified on the basis of numbers required to show a difference in efficacy, yet the authors draw conclusions regarding safety (which would require larger numbers). It may be preferable to get a group of students to first develop their own check-list for appraisal of, for example, a randomized controlled clinical trial for a new drug. This will allow them to develop a thorough understanding of why given features are necessary for a valid conclusion to be drawn. Articles can be selected for appraisal by the students to specifically illustrate the importance of criteria missed in the students’ check-list. Examples could include a study basing its conclusions on unvalidated surrogate markers, or a paper concluding a causal link without thoroughly assessing causality, or one not applying an intention to treat analysis despite a large unexplained loss to follow up. Critical appraisal tutorials in Cape Town At the University of Cape Town (South Africa) two students present their joint critical appraisal of a relevant article from a reputable, peer-reviewed journal in five minutes at the beginning of each pharmacology tutorial. Articles are selected to be relevant to the tutorial and to illustrate the importance of certain criteria for validity. Students summarize the study design and its strengths and weaknesses, after which the paper is opened to the rest of the group for comment and discussion (usually another 5–10 minutes). This method aims at ensuring that students not only acquire critical appraisal skills, but can also communicate these coherently. A working understanding of biostatistics is crucial for adequately describing the study populations and for drawing conclusions about populations based on the occurrence of events in a small group (statistical inference). It is beyond the scope of this manual to review the appropriateness of the statistical analysis. Students should be encouraged to revise their own notes and texts on biostatistics. Chapter 5. Developing critical appraisal skills 5 Crombie IK. The pocket guide to critical appraisal: a handbook for health care professionals. London: BMJ Publishing Group; 1996. p49. 6 Elenbaas RM, Elenbaas JK, Cuddy P. Evaluating the medical literature. Part III: Results and discussion. Annals of Emergency Medicine 1993;12:679–686. 7 Guyatt GH, Sackett DL, Cook DJ. User’s guide to the medical literature IIA. Journal of the American Medical Association 1993; 270(21):2598–2601. 8 Greenberg P. The interpretation of clinical trials. Australian Prescriber 1997; 20(3):61–64. Teacher’s Guide to Good Prescribing 40 Promotional material Promotional literature is of low scientific validity and should not influence the practice of students or prescribers. Some even recommend that practitioners should not see drug representatives or detailers at all. If they are to be seen, students need to be equipped with special skills to ensure that only useful, unbiased evidence is obtained from this potentially biased source. The WHO-sponsored video Prescribing Rationally, not Fashionably9 is a recommended introduction to this important subject. Students need a clear understanding of levels of evidence and should be encouraged to request peer-reviewed publications (which they now have the skills to evaluate) rather than accept glossy monographs and pens! Critical exposure of undergraduates to promotional material, which enables students to differentiate “facts from fiction” may reduce the pharmaceutical industry’s impact on prescribing patterns. Reviewing promotional materials in Yogyakarta In Yogyakarta (Indonesia) a very simple but effective student assignment has been developed, which teaches the students a critical attitude towards drug advertisements.10 It takes only one afternoon session of 2–3 hours and needs very few materials; it is preferably done in small groups. First, a discussion is started in which the students are invited to list the characteristics of a good drug advertisement. This process takes about half an hour. Then small groups of students are given four to six photocopies of advertisements from common national and international medical journals. Their assignment is to review to what extent these advertisements are in line with the characteristics they have just developed. The striking lack of compliance is usually a shock to the students. A randomized controlled study has shown that this single session results in a lasting skill to spot deficiencies in advertisements. Budiono Santoso and Sri Suryawati Promotional material does not simply include advertisements and material distributed by drug representatives but is more subtly included in conference packages and continuing education programmes. There are several ways in which statistical material can be presented in a positive way. For example, studies may be presented with carefully selected patients who are most likely to benefit and least likely to experience adverse effects. Surrogate markers are frequently used, even when there is no evidence that this marker is reliable, reproducible, sensitive, specific, a true predictor of disease, and rapidly reflects the response to therapy. Relative risk reduction is often presented, rather than absolute risk reduction. Although the pharmaceutical industry can be a source of information about new drugs, their value in informing prescribing practice has to be questioned, given their necessarily biased perspective. Every drug, especially a newly launched drug, should therefore be assessed on the evidence basis of efficacy, safety, suitability and cost. All promotional material should be critically reviewed against these criteria. 9 For further details contact: The Audio Visual Studio, Family Medicine Programme, 70 Jolimont Street, Jolimont, Victoria, Australia 3002. 10 WHO Essential Drugs Monitor 1997;23:23. � If you prescribe drugs, the pharmaceutical industry is interested in you and is investing staggering sums of money trying to influence you. T. Greenhalgh, 1997 41 Bedside teaching of critical appraisal Clinical cases are an intrinsic part of the training of most health care professionals. The bedside is not only ideal for learning about appropriate treatment and patient education and monitoring, but also for enabling students to critically evaluate the prior management of a patient. Confidence and skill gained from being able to critically review reputable journal articles can be applied to carefully evaluating the current and past management of each clinical case. Students will be able to consider the efficacy, safety, suitability and cost of the treatment for this particular patient. Students often express astonishment when they realize that the management prescribed by a hallowed clinician could be improved. This frequently illustrates reasons for irrational prescribing such as insufficient consultation time, uncritically following prescribing habits of role models and not involving the patient in management decisions. It provides effective warning against students repeating these mistakes. However, criticism of a colleague’s management should never occur in front of a patient. To avoid causing friction it is important for students to be aware that more information may be available about this patient or treatment now than was available when the treatment decision was made. There may also be factors that influenced the consultant’s decision of which the students are unaware. Ideally the prescribers should be included in the discussion in a non- confrontational way. This will either allow them to justify their management, or to prevent them from making the same “mistake” in the future. Students benefit enormously from these encounters. Students will develop their communication skills further when they present their critical assessment of the management of a clinical case. Students will also hopefully be “vaccinated” against unthinkingly repeating prescriptions of colleagues before considering the necessity and appropriateness of each drug. Pharmacotherapy teaching should be linked to clinical training Unfortunately, in many medical schools an excellent training in pharmacotherapy is not reconfirmed during the clinical attachments. When the students enter the wards they are often confronted with a plethora of different treatments, prescribed under brand name by a large number of professors, consultants and registrars, with much gusto but little explanation or justification. In this bewildering environment, the senior prescribers serve as a role model for the students, who often do not know better than to follow their example. This situation was the very reason why the Guide to Good Prescribing was developed. Its main goal was to help students in choosing their personal treatment in a rational way, rather than by just copying the prescribing behaviour of senior staff. The best way to ensure that the teaching philosophy of the Guide to Good Prescribing is supported and adopted by the clinical staff is, of course, to involve them in the development of the learning objectives and the training programme. One way of bringing them into the Chapter 5. Developing critical appraisal skills � We are misled by the ease with which our minds fall into the ruts of one or two experiences. Sir William Osler Teacher’s Guide to Good Prescribing 42 process is to involve the clinical departments in the selection of the disorders and drug treatments for the teaching programme in basic pharmacology and pharmacotherapy. Participation may also be encouraged through the drugs and therapeutics committee of the teaching hospital. Another way is to start a discussion with the clinical teachers on the necessary prescribing skills which the students should acquire during their clinical attachments; and to review the methods of examination (see Chapter 7). Problem-based pharmacotherapy teaching in small groups has often resulted in a more inquisitive and critical attitude by the students in the wards. An open atmosphere during the small group sessions, in which critical questions are actively invited (and never discouraged or ignored) and a constant search for the evidence base of treatment choices, are probably the most important contributory factors. Students can be further “vaccinated” against the potential infection by patterns of irrational prescribing they will undoubtedly encounter later, by using a set of critical questions developed by Herxheimer (see Box 3). Students can ask their clinical teachers this list of questions when they review drug treatment during ward rounds. BOX 3. QUESTIONS FOR STUDENTS TO ASK THEIR CLINICAL TEACHERS WHEN REVIEWING DRUG TREATMENT IN THE WARDS Name: For each drug, what is the generic name? Class: To which class does the drug belong? Aim: What is the aim of giving the drug? Which disorder is to be corrected, or which symptom is to be relieved? Observations: Which observations can be made to judge whether this aim has been achieved? Route and dosage: By which route, in which dose, at which intervals should the drug be given, and why? Alternatives: Which other treatments might have been chosen instead? Is this drug the best choice (efficacy, safety, suitability and cost)? Duration: How long should the treatment continue, and on which grounds is the decision made to stop or change the treatment? Elimination: How is the drug eliminated? Will the patient’s illness change this? Unwanted effects: Which unwanted effects may occur? Are they acceptable? What is their frequency? Interactions: Are there any other drugs that should be avoided while the patient is receiving this drug? If yes, which are they, and why should they be avoided? Patient ideas: What does the patient believe about the drug? What has (s)he been told about it, and what has (s)he remembered? Does (s)he need additional information? Adapted from: Herxheimer A. Towards parity for therapeutics in clinical teaching. Lancet 1976(ii):1186-87 43 Other examples of innovative student assignments Many medical schools around the world have developed interesting and simple student assignments which all support the problem-based approach. Some of these are summarized below. Developing a personal formulary in The Netherlands Both in Groningen and in Amsterdam medical students are taught and encouraged to develop a personal formulary during the course of their medical study. First the students learn the principles of rational pharmacotherapy, using the Guide to Good Prescribing. Then they are told to develop P-treatments for the most common conditions they encounter during their different clinical attachments. In this way the personal formulary is built up over the years, and is completed by the time they leave medical school. The growing formulary is reviewed from time to time, and used as one way of assessing the student’s performance. The key phrase used here is that P-drugs are “drugs ready for action” which means a list of carefully selected drugs of first choice for the most common conditions, with information on dosage, duration of treatment, contraindications, and key information for the patient (see page 60 of the Guide to Good Prescribing for a sample page of such a personal formulary). Developing a student formulary in Yogyakarta In Yogyakarta (Indonesia) 4th year medical students are asked to make a model student formulary together. The 160 students of the year are divided into groups of four. Each group is assigned a specific therapeutic group or class of drugs. They have to review a set of treatment guidelines, which includes the governmental treatment guidelines for health centres, some major therapeutic reference books and the WHO Model List of Essential Drugs. After discussing the various treatment options each group is expected to formulate their treatment of first choice and write a short section in the student formulary. The work of all groups is combined into one formulary and shared with all students. This method aims to train the students in comparing the efficacy, safety and cost of drugs, and in making an informed choice. It also helps students to change their thinking from being drug-oriented towards being patient- or disease-oriented. This approach is a simplified version of the previous example. Reviewing common over-the-counter preparations in Iraq In one of the medical schools in Iraq students are presented with an exercise which is brilliant in its simplicity. During one of the classes in pharmacotherapy they receive a sheet with a collection of photocopies of relevant parts of the outside package of four to six well-known anti-diarrhoeal preparations commonly used in Baghdad. The copies are made in such a way that for each product the (well-known) brand name or logo is shown, plus the list of ingredients in generic names; usually in small print. Among the products are also the labels of one generic antibiotic syrup (for example, ampicillin or cotrimoxazole) and one generic ORS preparation. The assignment reads as follows: “Little Ali, three years old, has had frequent watery diarrhoea for about two days, without fever. Which of these products would you recommend, and why? Also indicate why you would not recommend the other products.” Chapter 5. Developing critical appraisal skills This assignment, especially when performed in small groups, can lead to a very intensive discussion about why such well-known preparations should (not) be used. The session can also lead to a discussion on patient demand and how to resist it; on why patients like to use such treatment; on the influence of senior members of the family, such as the grandmother; on the evidence for efficacy of the preparations; on drug promotion; and on the need for consumer education. The preparation for the exercise is very simple, and the exercise can easily be repeated with well-known over-the-counter cough and cold preparations, vitamins and tonics, and analgesic preparations. Involving students in drug use surveys The WHO indicators for rational drug use11 cover the most important types of irrational prescribing, such as polypharmacy, use of injections and antibiotics, non-adherence to institutional formularies, and unnecessary use of brand names. Valuable information can already be obtained from samples of as little as 30 prescriptions from one department, clinic or pharmacy. Patient indicators, such as the average consultation time and patient knowledge of the drugs they have just received, can be obtained by observing or interviewing as few as ten patients. Such small surveys can easily be carried out by undergraduate students, as student assignments or as part of a larger study, for example a drug use intervention study. These surveys can also be done in rural clinics, in collaboration with the department of community medicine. There are signs that an early involvement of medical students in community medicine and work in rural areas results in a larger proportion of students who later choose to work in these areas. It is also likely that a review of the results of such surveys will lead to a more critical attitude by the students towards prescribing. 11 WHO. How to investigate drug use in health facilities. Geneva: World Health Organization; 1993. WHO/DAP/93.1. Teacher’s Guide to Good Prescribing 44 Introduction In many countries problems with prescribing are not limited to medical doctors. Hugedemands for primary health care have resulted in role substitution, with professionalsother than medical practitioners doing most of the diagnosis and treatment. In many developing countries, and especially in rural areas, the nurse is given this responsibility. Primary care nurse prescribers are then authorized to initiate the first level management and to refer patients to higher levels of care. However, structured training on pharmacotherapy is rarely included in nurses training, apart from a short course in the basic principles of pharmacology. The most common way for nurses to learn to prescribe is to copy their colleagues or visiting doctors, to look up some recommended dosage in whatever old therapy sheet or treatment guidelines happens to be available in the clinic, and just prescribe. Prescribing problems are therefore common among both primary care doctors and nurses. BOX 4. DO YOU HAVE A PRESCRIBING PROBLEM IN YOUR COUNTRY? An easy way to find out whether you have a prescribing problem in your country is to do a simple drug use survey, using the standardized WHO indicators for investigating drug use in health facilities.11 Such indicators are based on samples of less than 100 prescriptions per health facility. A small survey in one department or clinic can be done in about two hours. It is not possible to define the exact line between rational and irrational drug use. As a general rule of thumb the average number of drugs per prescription should be below two, less than 30% of prescriptions should include one or more antibiotics, and less than 10% of prescriptions should include one or more injections. If any of these basic indicators give a higher value, there is likely to be a prescribing problem and further investigations are justified. Experience in Indonesia and South Africa has shown that it is possible to give in-service training on rational prescribing to paramedical health workers, based on the six-step model of the Guide to Good Prescribing. In both cases such training took place within an ongoing programme of collaboration between the university and district or provincial health authorities. CHAPTER 6 Application in primary care settings Chapter 6. Application in primary care settings � Rural nurses usually learn to prescribe by copying their nurse colleagues or visiting doctors 11 WHO. How to investigate drug use in health facilities. Geneva: World Health Organization; 1993. WHO/DAP/93.1. 45 Training paramedical workers with this method is so new that there is not yet sufficient experience to develop clear guidelines. Instead, the next section only describes briefly what has been done, while leaving it to the reader to decide how their own training activities should be designed. Problem-based approach for on-site training of paramedical workers in health centres in Indonesia In Indonesia most patients visiting health centres are treated by paramedical workers, normally nurses or midwives. However, these workers are not legally allowed to prescribe drugs, nor are they trained to do so. This has created the problem that formal in-service training on prescribing could not be organized by the district health office. Therefore, an informal training strategy had to be developed. In the Sleman district in Eastern Java an on-site prescribing training programme for health centres was started, using in small groups the problem-based approach as outlined in the Guide to Good Prescribing. The training was interactive, involving 8–12 participants in health facilities, facilitated by the health centre doctor. This limited experience has stimulated the Ministry of Health, in collaboration with the Centre for Clinical Pharmacology and Drug Policy at the Gadjah Mada University in Yogyakarta, to develop and field test a formal training programme for paramedical prescribers at the health centre level. The training materials focus on five common diseases: acute respiratory tract infection, acute diarrhoea, muscle and joint pains, hypertension and dyspepsia. The treatment guidelines for these problems have been developed following the framework of the Guide to Good Prescribing. Each guideline covers the patient’s problem, the therapeutic goal, the recommended treatment (with or without drugs), the prescription, patient information, and follow-up. In addition, the guidelines also include, for each health problem, the common treatment practices that are not recommended. Session guides have also been developed, as well as trainer’s guidelines for small group teaching. Each training session lasts about 1–2 hours and is undertaken every two weeks in the health facilities. Small group training utilizes a problem-based and problem-solving approach. Budiono Santoso and Sri Suryawati WHO Collaborating Centre for Research and Training in Rational Drug Use, Yogyakarta Training programme for primary health care nurses in South Africa In South Africa nearly all patients presenting at rural clinics are diagnosed and treated by nurses who are often not trained for this task. There are referral possibilities to general doctors at the district hospital level, but such hospitals may be as much as 100 kilometres away in some areas. Prescribing problems are common and serious (see below). Irrational prescribing in rural South Africa In a large survey in rural South Africa, as many as 51% of patients who visited a clinic for minor ailments received one or more antibiotics; in one clinic 92% of them received an antibiotic. In another provincial survey, 52.3% of patients attending outpatient services at district hospital level received at least one injection. In the Northern Province of South Africa a training programme was developed by the Medical University of South Africa (MEDUNSA) for in-service training of primary health care nurses. The programme has also been used for in-service training of medical practitioners. 46 Teacher’s Guide to Good Prescribing Outline of the Effective Prescribing Workshop The principles of rational prescribing outlined in the Guide to Good Prescribing were set as the main learning objectives for the course. Sessions on these principles use the problem- based approach. One extra session, “How to make use of objective sources of drug information” was added. BOX 5. OBJECTIVES OF THE TRAINING COURSE FOR PARAMEDICAL PRESCRIBERS “At the end of the course, when given a clinical scenario, participants are able to: 1. identify a patient’s problem 2. specify a therapeutic objective 3. choose between drug therapy and/or non-drug therapy 4. select the most appropriate treatment, applying the criteria of efficacy, safety, suitability and cost 5. write a correct prescription 6. educate patients on the appropriate use of the medicines, and 7. monitor the treatment, which includes follow-up visits.” Training materials A trainer’s guide and participant’s working sheets are critical to the success of this programme. The session notes will guide the trainers through each session. The intention is that each participant achieves the objectives listed for each session. Should the trainer consider it important, participants may be asked to revisit sections or to read further, regardless of whether or not the guide recommends the action. An ideal trainer’s guide should include the following materials: 1. A session guide for the trainers, which summarizes the main activities during the sessions 2. A detailed description of the activities that will take place during each session 3. The complete section from the participant’s guide, which includes the session objectives. Ensure that the page numbers in the trainer’s version of the participant’s guide correspond to the actual material given to the participants 4. A page for notes; this page is for the trainer to write down additional points or reminders to tell participants during the workshop. The trainer may also note problems encountered during training, or any matter for discussion during the trainers’ meetings. The problem-based approach implies that most teaching is done using practical examples. Several simple patient cases, especially relevant for rural health care in developing countries, are given in Annex 1. They can be used for training and for student assessments. One example of a training manual has been developed by the Universities of Durban and Cape Town, South Africa.12 Chapter 6. Application in primary care settings 12 University of Durban-Westville, University of Cape Town. Rational drug prescribing training course training manual. Durban: Health Systems Trust; 1997. 47 Teacher’s Guide to Good Prescribing 48 Assessment of the students through a simplified OSCE An Objective Structured Clinical Examination (OSCE, see Chapter 7) was used at the end of the course. This test functioned as a self-assessment tool for the participants. It also provided a final reinforcement of the teaching message. The test was done in a very simple way without the need for any special support structures, such as separate cabins, one-way mirrors and the like. Students were seated in groups of about five, around a table. In the first OSCE, one student received the case and acted as the prescriber (whose performance was to be scored in this round), interacting with a second student who played the patient. Simple, but very common, patient cases, such as those presented in Annex 1, can be used for this purpose. One other student acted as timekeeper, and the remaining students as scorers of the prescriber, using a standardized scoring sheet (reproduced in Annex 2). After the first case, everybody around the table changed roles. This process continued until all students had been assessed once. Measuring the impact of the training workshops The impact of the training programme was measured in a very systematic way and showed very positive results (see below). Impact of a training programme in Northern Province, South Africa About 250 nurses and other paramedical prescribers were trained in rational prescribing during a three-and-a-half day course using the six-step model of the Guide to Good Prescribing. By means of a staggered intake into the programme a prospective controlled study could be done on the actual prescribing indicators before and after the training. The results indicated statistically significant better prescribing in upper respiratory tract infection (URTI), one of the subjects taught at the course. There was an increase in non-drug treatment, generic prescribing and compliance with the 1996 South African Primary Health Care Standard Treatment Guidelines, and a decrease in the average number of drugs per prescription, the use of antibiotics and the use of injections. Similar improvements were noted in the treatment of diarrhoea and vomiting (DV). A very interesting aspect of the study was that the treatment of DV had not been specifically discussed at the course. In addition to improved prescribing in URTI (retention effect) the improvement in treatment of DV indicates that the participants had also learned to apply the principles of effective prescribing to other conditions in their daily practice (transfer effect). The trainees had acquired the skill to think critically and prescribe more rationally, and not just improved knowledge of the treatment of URTI. Hannelie Meyer. Dissertation for MSc (Med), School of Pharmacy, MEDUNSA, South Africa, 1998 Teacher’s Guide to Good Prescribing 48 49 PART 2 How to assess the students, the teachers and the course Teacher’s Guide to Good Prescribing 50 51 CHAPTER 7 How to assess the students Chapter 7. How to assess the students Educational objectives and evaluation Education aims at changing the behaviour of the learner. In a medical school the intended changes in the behaviour of the students constitute its educational objectives. The educational objectives define what students should be able to do at the end of a learning period that they could not do before. Planning an educational programme begins with defining these objectives. Defining educational objectives has become almost fashionable over the last two decades. However, what counts is not their formal definition but whether they correctly describe the relevant knowledge and skills students must have after the training. Good educational objectives fulfil four criteria. Firstly, they relate to essential components of professional competence and are relevant to the future health needs of society. Secondly, they are internally consistent, focused and clear. Thirdly, they are feasible within the time allowed and with the resources available. And finally, they indicate an acceptable level of performance which is measurable with qualitative or quantitative values. Each objective therefore consists of a specific task description and the criteria to measure whether or not the objective has been achieved. Education by objectives cannot exist without such a measurement, and this measurement should be adapted to the type of objective. At least four different types of objectives have been described. Behavioural objective. The endpoint is a human action. Example: By the end of this workshop, 80% of the students will choose a set of P-drugs for iron-deficiency anaemia. Performance objective. A behaviour is expected to occur within a specific time frame at an expected proficiency level. Example: Within a year at least 80% of students will be able to select a set of P-drugs for any diagnosis and will pass a final Objective Structured Clinical Examination (OSCE). Example: At the end of this semester, 90% of students will be able to write a prescription in a standard format and pass a basic proficiency test in legible prescribing. Process objective. The endpoint is the way in which something occurs. Example: We will document the teaching methods used in the new programme, identifying those with the greatest impact on improved drug selection skills. Product objective. The endpoint is a tangible result. Example: By the end of the year 80% of students will have created a personal formulary covering 20 primary indications. Evaluation provides the information necessary to improve the educational programme further. It should therefore be built into all phases of the programme. Examples are: a Teacher’s Guide to Good Prescribing 52 needs assessment to identify the educational objectives and evaluation of human and material resources (in the design stage of the programme); monitoring the educational process (during the programme); and assessing the performance of the students and the teachers (during and after the programme). Student assessment The core component of such a comprehensive evaluation system is the assessment of student performance, the “output” of the educational programme. This assessment requires considerable time and effort because human behaviour is too complex to be measured by a single observation or to be summarized by a single score. It is therefore recommended that a system of dual assessment be arranged. During the training programme, “formative” (or diagnostic) assessment of the students aims at assessing progress in learning, identifying weak areas where more work is needed, and identifying and understanding students who do not perform well. Later on in the programme “summative” assessment methods are used to certify that the student can pass to the next stage of the study. This dual assessment calls for a well-constructed evaluation system with a range of measurement methods. Such a system has many advantages. Firstly, teaching, learning and evaluation become easier to distinguish. By allowing students “learning time” early on in a course and focusing on formative evaluation, learning becomes the primary focus. Later on in the programme, when students are more aware of the expectations and more ready to be assessed, summative (certifying) evaluation becomes more important. Secondly, students can be informed of the evaluation mechanism and can participate actively in the process. When students know how they will be assessed and what to expect, their anxiety will decrease. Students who receive written course objectives and a description of the assessment methods at the beginning of a course often do better than those students who do not. A third benefit is that the dual assessment helps teachers to identify and understand students who are not maturing. If a student can be regarded as not ready to pass to the next performance level rather than as a failure it becomes easier for the teacher not to let the student pass. Summative and formative assessment Summative assessment Summative (certifying) assessment is an absolute measurement which allows a judgement as to whether a student succeeds or fails a certain set of criteria. It is often used for the selection of students at the entry level, for passing from one year to the next, and for certifying the final competence at graduation. The number of students passing a certifying test can also indicate the efficacy of the training programme. It is very important to realize that the type of assessment determines the type of learning � It is a highly questionable practice to label someone as having achieved a goal when you don’t even know what you would take as evidence of achievement. R.F. Mager 53 behaviour. Students will quickly identify a specific learning pattern which is associated with high marks for a test. If the test mainly requires recalling facts, students will become memorizers. If the test requires problem-solving they will focus on acquiring problem-solving skills. Whoever controls the examination controls the curriculum. Formative (diagnostic) assessment Students learn better if they are constantly appraised in a constructive and supportive manner. This assessment is done by periodic tests which measure the students’ progress from the moment they start the programme, providing them with feedback on the level they have reached, making them aware of the parts of the course they have not understood, and informing them about what they still have to learn before they achieve the educational objectives. Clearly stated objectives, understood by both teachers and students, are therefore essential. The importance of formative assessment must also be appreciated by the teachers, as it requires time and effort. Formative assessment can be very effective in guiding the students and prompting them to seek help. It can use a wide variety of methods and be formal and informal. Self-evaluation by students is an important part of the process. Students should preferably have some input into the methods used. Formative assessment should never be used for final certifying examinations. For some tests the students should remain anonymous. Formative assessment can also be used for qualitative and quantitative feedback to the teachers, helping them to judge whether the students have understood the contents of the programme. The evidence suggests that if no measurements are taken, most teachers tend to assume that the objectives have been achieved. The assessment results help the teacher to judge whether any changes need to be made to the training programme. Relative or absolute criteria An absolute criteria test enables a student’s performance to be assessed in relation to a previously specified acceptable level of performance. The aim is to determine whether a person has mastered a particular task, and not to compare one performance with another. For example, if the educational objective requires that all students should be able to master the emergency procedure for managing an acute asthma attack, this skill can only be assessed against a standard, using an absolute criteria test and regardless of the performance of other students. It is theoretically possible and, of course, desirable that all students should pass, demonstrating a high efficacy of the training programme. However, it is also possible that all students fail. A relative criteria (norm-referenced) test is competitive and aims to make a valid discrimination on the basis of relative performance. This test refers to the norm, that is the curve of the results of all students who took the same test. Relative criteria tests are frequently used for examination purposes. In fact, they do not guarantee the competence of the students who succeed. For example, if a group of students is well trained in managing an asthma attack, the relative criteria tests will lead to some of them failing the test although their level of performance was satisfactory in absolute terms. On the other hand, Chapter 7. How to assess the students � Changing the examina- tion system without changing the curriculum had a much more profound impact upon the nature of learning than changing the curriculum without altering the examination. G.E.Miller Teacher’s Guide to Good Prescribing 54 if the group as a whole performs poorly, for example because the appropriate instruction was not given, the relative criteria system may allow poor students to pass if they are above the average of their group. What do you measure? Student performance is usually assessed on the basis of three major aspects of professional tasks: knowledge and intellectual skills, communication skills and practical skills. Human behaviour cannot be divided so strictly, and the only purpose of this artificial classification system is to allow for a better analysis of the learning process. Knowledge and intellectual skills Three separate levels of intellectual skills can be distinguished. The first level is recall of facts and procedures necessary for the efficient performance of a task. For example, recall the pathophysiology of diarrhoea, the definition and consequences of dehydration, the composition and action mechanism of ORS. The next level is interpretation of data: the students can solve routine problems by applying the principles or methods they recall. The highest level is problem-solving: the students can find solutions to unfamiliar problems by analysing the data and applying scientific methods with no precedent to serve as a guide. Communication skills In a narrow sense, communication skills determine the interaction between the care-giver and the patient. They include listening and responding to the patient, giving appropriate information, instructions and warnings, allowing the patient to express concern, noticing non-verbal messages, and reassuring the patient. Again, three levels can be distinguished. The first level is noting an incident, e.g. noticing that an elderly patient is confused while being instructed. The next level is to respond, e.g. by repeating the instructions. The highest level is empathy, responding by internalization, e.g. trying to define why the patient is confused and making it easier for the patient to understand the instructions. Practical skills Practical skills are related to professional actions, such as intubating a patient or giving an intravenous injection. Here the three levels are imitating the task, being able to do it, and doing it routinely with confidence. Qualities of a test Evidence shows that most tests are ambiguous, unclear, disputable or trivial. To minimize such errors teachers need to review their questions critically before a test is used. In a large analysis of 39,000 multiple-choice questions (MCQ) one-quarter of all errors were attributable to the teacher and could have been avoided by a critical review of the questions before use. A test should be directly related to the learning objectives and should assess the ability to perform the tasks expected of the students at that stage of their studies. For the results to be meaningful a test must have certain qualities. 55 1. It should be relevant and valid, exactly and consistently measuring the achievement of the learning objectives 2. It should be objective, with independent examiners in agreement on what constitutes a good answer 3. It should be specific, discriminating between the good and the poor students 4. The number of questions should be proportional to the various subjects in the course 5. The test should be feasible and practical, and not need too many resources to construct, implement and score. The author of a test is not the best judge of its clarity, precision and relevance. Critical review by colleagues is essential. A check-list can be used for this purpose (see box). BOX 6. EXAMPLE OF A CHECK-LIST TO REVIEW TEST QUESTIONS 1. Is the question realistic and practical? 2. Does it stem directly from specific educational objectives? 3. Does it correspond to the teaching content? 4. Does it deal with a matter that is professionally useful and important? 5. Does it require intellectual skills of a professional kind? 6. Is it independent from every other item on the test? 7. Is it specific? 8. Is it clear and comprehensible? 9. Is it brief, focused and complete? 10.Does it avoid giving away the correct answer? For multiple-choice questions: 11.Are the distractors plausible answers rather than obvious distractors? 12.Are the distractors approximately of the same length and also otherwise homogeneous? Common errors There are several common errors in constructing a test. Some of these are listed below. • Test questions only represent a small sample of what could have been asked • Errors in phrasing the question, or (in MCQs) the stem, distractors or the correct answer • Use of ambiguous language • Forcing the student to answer to accommodate the opinion or preference of the examiner • Formulation that makes the correct answer obvious; or wording of one question helps in finding the correct answer for another question • (In MCQs) allowing the student to identify the answer by studying only a few distractors. Chapter 7. How to assess the students Teacher’s Guide to Good Prescribing 56 BOX 7. EXAMPLES OF TRUE-FALSE AND MULTIPLE-CHOICE QUESTIONS Simple true-false question: T F In a 42-year old man with mild hypertension, metoprolol would be appropriate treatment. Multiple true-false question: In a 42-year old man with mild hypertension, would you consider starting treatment with metoprolol? T F metoprolol T F sodium nitroprusside T F indapamide T F nifedipine T F amiloride Simple multiple-choice question: In a 42-year old man with mild hypertension, which one of the following drugs would you consider starting treatment with? a) atenolol b)clonidine c) furosemide d)hydralazine e) verapamil Methods of student assessment Several methods can be used to assess students’ performance. There is no single assessment method that will satisfactorily measure performance as a whole. Hence it is necessary to use more than one type of test. Assessment methods can be divided into direct and indirect methods. Direct methods allow for measuring the student’s performance by direct observation during actual practice or in a simulation. Actual practice can be observed during group dynamics assessments and ward or laboratory observations. Simulated situations can be created through Objective Structured Clinical Examinations (OSCE, see below). Indirect methods measure students’ knowledge and skills, assuming they are representative of their performance in actual practice. Examples are the different types of written tests (long, short or modified essay questions and multiple-choice questions), oral tests (structured and unstructured) and various practical tests. The various test methods are briefly described below. Their advantages and disadvantages are summarized in Table 1. Written tests Multiple-choice Questions Multiple-choice questions (MCQs) are widely used to evaluate students in health sciences. The examinee is to study all distractors and identify those that are right or wrong. They can test a wide range and variety of facts at a time, they are relatively cheap to develop and use, and easy to score. However, it is not so easy to construct unambiguous questions. Some questions provide cues that do not exist in real practice, and misinterpretation is common. Most MCQs only test a simple recall of facts and rarely require the student to interpret data and solve problems. It is often assumed that any question that includes 57 patient data necessarily involves problem-solving. However, patient data are often merely a window dressing for a question which addresses a general condition which could also have been formulated without the patient. In the context of problem-based pharmacotherapy teaching the main disadvantage of MCQs is therefore that they cannot test practical skills (such as writing a correct prescription) or communication skills (such as explaining to the patient the correct use of an inhaler). However, MCQs can be used as part of an assessment provided other methods are used as well. A few practical hints for writing good MCQs are given in Box 8. BOX 8. TIPS ON WRITING GOOD MULTIPLE-CHOICE QUESTIONS � Keep the stem of the question short and clear; use a complete statement and not a single word. � Make each question independent of other questions. � Avoid negative statements; if you must use them underline; never use double-negatives. � Avoid “none of the above” as a distractor. � Avoid using clues that might suggest the correct answer. � Use plausible or logical distractors; each distractor must appear to be related to the question. � Make distractors and correct response fairly similar in content or length. � Arrange numerical values from large to small, or vice versa. Short Essay Questions The cues which make students choose among various multiple-choice answers do not reflect actual medical practice. Patients do not present themselves to their doctor with a choice among four drugs to prescribe! To overcome some of these limitations, Short Essay Questions (SEQ) can be used. SEQs are as reliable and economical as MCQs, but have important advantages. They test the student’s abilities to recall information without providing cues, and they can also test certain higher level problem-solving skills which cannot be tested by MCQs. For example, in clinical problem-solving very few things are absolutely certain. This aspect of “probability” should be underlined in teaching but does not match very well with the true/false choices of a MCQ examination. SEQs reflect more the reality of medical practice and the content of the teaching. Some studies also suggest that SEQs are more effective in discriminating academically marginal examinees. BOX 9. EXAMPLES OF SHORT ESSAY QUESTIONS Restricted response essay question: First-pass metabolism after oral administration decreases bioavailability of certain drugs. Explain the mechanism by which this occurs. Short answer question: A 20-year old patient in a coma has been admitted to the emergency ward. He has symmetrical pinpoint pupils. What is the most likely diagnosis? (answer: morphine intoxication). Chapter 7. How to assess the students Teacher’s Guide to Good Prescribing 58 BOX 10. PRACTICAL TIPS ON WRITING AND SCORING SHORT ESSAY QUESTIONS � Keep the question short and clear, demanding a short, precise answer. � Use explicit terms (identify, compare, define, calculate, arrange in order) rather than diffuse terms (discuss, explain). � Make each question independent of the others. � Use maximum peer feedback possible. � Define in advance the correct answer called for by the wording of the question. � If conceptually different answers are possible, reword the question. � Before the test, define the scoring procedure. � Arrange for marking by two scorers. � Mark the papers anonymously. � When scoring, stick to the agreed correct answers. BOX 11. DIFFERENCE BETWEEN AN MCQ AND SEQ FOR A SIMPLE RECALL-OF-FACTS QUESTION Multiple-choice Question: Which of the following statements defines the ipratropium bromide’s mechanism of action: a) beta antagonist b)muscarine antagonist x c) beta-1 antagonist d)muscarine agonist e) calcium channel antagonist Short Essay Question: Answer Key: Name the following features of ipratropium bromide: Mechanism of action: muscarine antagonist Indication: asthma Three side-effects: dry mouth, blurred vision, urinary retention Route of administration: inhalation Modified Essay Questions In its original form the Modified Essay Question (MEQ) is a paper exercise based on an evolving situation presented by a patient in primary care. Now it usually consists of a case history followed by a set of short open-answer questions, which present an evolving situation. The case might be presented in writing, using a videotaped patient interview or through a role-play. It is the main written method for assessing a student’s problem- solving skills, attitudes, intelligence, ability and personality. MEQs demonstrate a stronger correlation with the in-clinic clinical reasoning than MCQs on the same subject. There is also some evidence that the outcome of MEQs is related to professional performance at postgraduate level, whereas the MCQs are not. 59 BOX 12. EXAMPLE OF A MODIFIED ESSAY QUESTION Mrs Brown, a 38-year old primary school teacher, complains about fatigue and tachycardia. She has been admitted to the general medical unit on which you work, for further investigation. Question 1: What are the three most likely diagnoses? Question 2: List five specific questions which would help you distinguish between these possibilities. A routine blood test reveals microcytic hypochromic anaemia with a haemoglobin level of 9.8 g/dl. Question 3: List two typical signs you would look for when you examine the patient. Question 4: Did this information affect your first diagnosis? If yes, how (explain briefly)? Patient management problems The strength of problem-based learning arises from the fact that it provides transferable and sustainable skills. Although clinical reasoning is increasingly recognized as a crucial component of problem-based learning there is still no valid, reliable, objective and practical assessment instrument available to measure the therapeutic reasoning component of clinical competence. The aim of Patient Management Problems (PMPs) is to measure the problem- solving component of clinical competence by a simulation on paper. PMP starts with a description of a clinical situation. The student is asked to study the details carefully and make appropriate decisions, such as to take or not take a certain line of action. The student is presented with a list of decisions based on the actual practice routine, which are plausibly linked to the initial situation. Each decision results in feedback to the student on the consequences of that decision. The step taken will define the way the examination proceeds. This leads to a new situation where a new decision is needed. Gradually more information is revealed, which might require the student to reorient his/ her subsequent decisions. The measurements focus on assessing the student’s ability to detect and satisfactorily interpret abnormal signs and symptoms and then to reach a reasonable diagnosis and to show satisfactory judgement in the choice of treatment. There is some controversy about PMPs. Poor performance with PMPs mostly reflects a specific lack of knowledge rather than a deficiency in problem-solving. In addition, a common definition of problem-solving ability has not yet been developed. Scoring procedures are difficult to standardize. For the time being, this method is not recommended. Long Essay Questions Long essay questions (LEQs) have a low inter-examiner reliability and poor objectivity. Yet they are much used in undergraduate medical examinations, probably because examiners feel that essays have greater inherent validity than some of the modern techniques, such as Multiple-choice Questions or Short Essay Questions. It is recommended to restrict the use of LEQs to evaluating those types of performance which cannot be measured efficiently by other methods, such as making a summary of a scientific paper, making a synthesis of complex concepts, comparing two phenomena, finding and interpreting relationships, criticizing the relevance of a concept, and formulating a policy or plan of action. Chapter 7. How to assess the students Teacher’s Guide to Good Prescribing 60 BOX 13. EXAMPLE OF A LONG ESSAY QUESTION Describe which diseases you would consider in the differential diagnosis of essential hyperten- sion. Give reasons for each disease and state which diagnostic tests are valuable in the differ- ential diagnosis. BOX 14. TIPS ON WRITING AND SCORING LONG ESSAY QUESTIONS � Choose problems which can be answered satisfactorily in the time allocated. � Limit the problem and describe the task clearly; define the structure of the answer. � Avoid ambiguity, use explicit terms (identify, compare, calculate, arrange in order) rather than indirect terms (discuss, explain). � Avoid questions where conceptually different answers will do for a given question. � Pre-test the questions. � For every question, list the scoring elements which should appear in the answer. � Show all answers to all markers; arrange for dual marking; if that is not possible, the same examiner should score a question for all students. � Make sure that all examiners agree on the scoring procedure and correct answers before the test. � Mark papers anonymously. � Score the answers of all students to a question before scoring the second question. � Do not form a judgement of a candidate on the basis of a single answer; for each student, judge each answer independently and add up the individual scores for the final score. � Be prepared to use each and every score available on your scale; avoid grouping candidates around the mean, just because of a tendency not to give extreme assessments. � Do not be influenced by better or worse answers coming just before the answer you score. Oral Tests Oral examinations are commonly used for certifying evaluation in undergraduate medical examinations, particularly during clinical training. They consist of a dialogue with the examiner who, most often, asks a series of not necessarily interrelated questions to which the student must reply. In their standard form they are closed-book tests which measure whether the students can express their knowledge of isolated facts or groups of facts they ought to remember. This type of examination suffers from a scarcity of examiners who can really make good use of it, with the result that its advantages are rarely used in practice. Previously, oral examinations were unstructured. They therefore had poor inter-examiner reliability and poor objectivity. They suffered from undue influence of irrelevant factors, they permitted favouritism and abuse of personal contact. Oral examinations are improved by a standardized approach. However, they remain costly in terms of professional time and are generally not worth it. 61 Structured Oral Examination The poor inter-examiner reliability and poor objectivity of traditional oral examinations can be prevented partially by a Structured Oral Examination (SOE). A SOE typically consists of various predetermined clinically-oriented scenarios. Each scenario has a pool of five to ten questions, each with a specific marking scheme. Students are presented with the scenario and asked a set number of questions randomly selected from the pool. The answers are assessed by at least two independent examiners. The SOE is a suitable and reliable tool for assessing the clinical knowledge and problem-solving abilities of clinical students. There are significant correlations between the SOE and MCQ and OSCE. Examiners report a higher degree of satisfaction with the examination as their role is more active than in the OSCE setting (see below). BOX 15. PRACTICAL HINTS ON DEVELOPING AND SCORING ORAL EXAMINATIONS � Develop a clear description of a clinical situation. � Prepare a few “starting questions” to commence describing the chosen phenomena. � For each starting question, develop several questions to broaden the inquiry. � For every question set the scoring elements which should appear in the answer and a scoring scale. � Use more than one examiner. � Ensure that all examiners agree on the scoring procedure and correct answers before the test. � Identify and discuss with other examiners the external factors which interfere with rating— including the student’s clothing, general appearance, communication style and skills. � Keep the number of “scoring” questions constant per SOE. � Do not hesitate to ask auxiliary questions to keep the discussion in focus, to understand the mechanism of thinking or to clarify an answer; do not score answers to auxiliary questions. � Do not ask directing questions. � Do not give unplanned cues to a student. � Be prepared to use each and every score available on your scale. � Do not form a judgement of a candidate on the basis of a single answer. Direct observations Any method for evaluating the clinical competence of a student must assess complex behaviours beyond those encountered in the classroom or in simulated environments. Direct observations of the student performing a professional task, for example in an outpatient clinic or ward, would be ideal but are not always possible. Ward evaluations Ward evaluations of clerkship performance by faculty members rarely reflect clinical competence. Such evaluations are often inflated and do not correlate well with other assessments. Most physicians make global ratings of the students, using a single criterion and not distinguishing specific evaluation categories. Moreover, this single criterion (“good physician” or “bad physician”) is defined differently by different specialities and is often Chapter 7. How to assess the students Teacher’s Guide to Good Prescribing 62 used intuitively. This undifferentiated judgement is easily biased and does not identify deficient performance skills. Observation ratings of clerkship performance are often influenced by differences in the complexity of the patient case, different focus and standards of the examiners, and lack of agreement on what constitutes acceptable performance. In addition, the process is rather time-consuming and requires a good administrative system. Most of these problems can be avoided if the purpose of the measurements is well defined, a rational rating system is designed and evaluators are adequately trained. Practical tips are summarized in the box. BOX 16. HOW TO DESIGN A DIRECT OBSERVATION RATING STRUCTURE 1. Make a list of observable types of behaviour which show that the objectives have been reached. 2. Make a list of observable types of behaviour which show that the objectives have not been reached. 3. Determine the essential features of behaviour in both lists. 4. Assign positive and negative weight to the items on both lists. 5. Decide on the acceptable performance score. 6. Develop an observation rating scale (see example). Table 2. Example of rating scale for giving a mother information on her child’s medication Score Description of general attitude Description of specific attitude 5 Student has taken all necessary precautions Gives accurate information and the mother appears relaxed Makes sure she understands the instructions Answers mother’s questions Allows room for the mother to express her anxiety, and responds to that 4 Student has taken all necessary precautions Gives accurate information and has reassured the mother several times Makes sure the mother understands the instructions. Answers mother’s questions 3 Student has made an effort and has Gives information followed it up Makes sure that the mother understands the instructions Does not reply to the mother’s questions 2 Student has made an effort without Gives information following it up Does not reply to the mother’s questions 1 Student seems to be quite unaware of Gives no spontaneous information the problem Does not reply to the mother’s questions Objective Structured Clinical Examination One possibility to standardize the assessment of clinical performance further is the Objective Structured Clinical Examination (OSCE). OSCE was first used in surgery to overcome the disadvantages of the traditional examinations. The traditional evaluation was criticized as 63 BOX 17. HOW TO DESIGN AN OBJECTIVE STRUCTURED CLINICAL EXAMINATION (OSCE) An OSCE examination consists of a series of OSCE stations, and each station has to be designed separately. The following steps are recommended. 1. First identify the type of student performance which will indicate that the learning objec- tives have been reached. Then make an inventory of observable types of skills, activity, attitude or product connected to that performance. 2. Based on this inventory, define clinically relevant problems and assignments which resemble the actual professional tasks and for which the observable skills, activities, attitudes or products can be measured. 3. Decide on the acceptable performance score for each assignment. With a group of faculty, first describe and estimate the score which a minimally competent student should achieve on each tested element of the OSCE, and test the scale on real students before finalizing it. 4. Develop the measurement tool (score list, check-list). 5. Make an inventory of examination materials needed and make them available (for writing a standard patient case and using a simulated patient). Examples are: a chest x-ray needed to test proficiency in reading a chest x-ray; a simulated patient with a patient dossier, prescrip- tion pad and sample inhaler; a simulated patient, case description, stethoscope, reflex ham- mer, pen light and opthalmoscope for testing proficiency in general physical examination). 6. Decide on practical issues. How much time is needed for each question (pre-test the tasks with interns to find out how long they take to complete them)? How many OSCE stations do you need to cover all assignments (try to keep the number of stations low)? How many scorers do you need (ideally two per station, scoring independently, but one is also accept- able)? Will you observe using one-way screen rooms, closed-circuit video or video recordings, or a sitting-in observer (objectivity of scoring is proportional to distance to the examinee)? being arbitrary and lacking reliability, while written examinations could only test cognitive aspects. The OSCE focused on actions rather than on abstract knowledge, and evaluated these actions and skills in a uniform way. An extensive body of research now shows that OSCE is a valid method to assess clinical skills, which are so fundamental to medical practice. OSCEs are increasingly being used to assess students’ clinical competence and to provide feedback to the students. The OSCE consists of a series of “test stations”; each student passes each station. Each station tests different cognitive and practical skills which resemble professional tasks, such as taking the history, examining the patient, interpreting an X-ray, or teaching a patient how to use an inhaler. OSCE stations use standardized patients, who are usually lay people trained to play patient encounters accurately and consistently. Each student performs the required task or interacts with the patient while being observed and assessed in a standardized way, using a check-list and standardized ratings. At the end of the session a faculty member gives the student educational feedback. Such immediate feedback provides students with valuable self-assessment that may stimulate further learning. The whole assessment may include stations with multiple-choice questions (MCQs) and/or short essay questions (SEQs). A major impediment to the wide implementation of OSCE is that it is a labour-intensive and costly form of assessment. However, the costs for physicians, patient trainers, support personnel and data analysis are less if these duties are performed as part of their overall Chapter 7. How to assess the students Teacher’s Guide to Good Prescribing 64 academic responsibilities. Costs for standardized patients can be minimized by using them in both teaching and evaluative sessions. Simulated patients can also score the students’ performance, reducing the need for faculty staff to do so. There are several variations of the original OSCE. The objective structured practical examination (OSPE) is used as an objective instrument for assessing laboratory exercises in pre-clinical sciences. These examinations usually consist of short stations each designed to assess a single discrete skill. Small group OSCE (GOSCE) allows for the assessment of a large number of students’ abilities without the time and expense needed to evaluate students individually. GOSCE can be used for formative and informal summative assessments, as a resource for learning about interpersonal and interprofessional relationships, and as a framework for problem-based short courses. There is also a simplified OSCE technique where five students sitting round a table play rotating roles of prescriber, doctor and scorer (see page 48). Project Execution Test and portfolio The Project Execution Test (PET) is an outcome-based assessment. It is an indirect evaluation technique, which requires the student to carry out an activity in a certain period of time; a teacher evaluates the product. The assessment should be structured to make the PET more consistent and equitable. Some PET examples include preparing a histopathological slide (practical skills), writing a bibliography or compiling a review article (intellectual skills). It is advisable to use a PET when the main component of the task is a complex practical or intellectual skill, and when the product is more important than the student’s manner of working. The portfolio is a cumulative collection of authentic examples of academic work and presentations. It allows summative or formative assessment of project design and execution skills of the student over a long period of time, based on miscellaneous project assignments. For example, such a portfolio might bring together a personal formulary, several patient dossiers, self-written prescriptions, a review article on new treatment modalities of a given indication and a self-developed multi-attribute utility analysis model in selecting P-drugs for hypertension. The PET and portfolio allow for an evaluation of product-type educational objectives, demonstrating the student’s ability to use knowledge and skills to generate a product or to fulfil a task within a deadline. They are costly in staffing terms and need a relationship of confidence to avoid cheating. BOX 18. PRACTICAL TIPS FOR DESIGNING GOOD PETS � Define a specific objective with a tangible product. � Identify competency areas (data collection, critical information-appraisal, recording skills). � Define a criteria-based assessment system for student performance: define achievement criteria for each competency area, a set of descriptors for each criterion, and a scaling mechanism. � Design a skills training programme. � Define the assignment and time frame. 65 Chapter 7. How to assess the students Which type of assessment best reflects pharmacotherapeutic skills? The main advantages and disadvantages of the various assessment methods described above are summarized in Table 3. From this table it can be seen that some methods are more or less unique in testing a specific aspect of pharmacotherapy. For example, for testing the recall of a wide range of facts the MCQ examination is very good. However, the PET is the only method for testing the skill of delivering a product within a certain deadline. And, although several methods can test problem-solving skills, an OSCE is the only way to test communication skills and practical skills in a standardized and objective way. Whoever controls the examination, controls the curriculum. And, it should be added, also controls the way students learn. Teaching pharmacotherapy on the basis of the Guide to Good Prescribing implies teaching the total process of prescribing—that is, comparing alternative treatments, developing P-drugs, prescribing the right treatment, informing the patient, and adapting the treatment if necessary. This skill cannot be tested by simple essay questions or by a multiple-choice examination alone. Changing the teaching implies changing the examination. Which type of pharmacotherapy examination is feasible in your own setting? The answer depends, of course, on the number of students, the number of teaching staff, the building and other circumstances. Changes in the curriculum and in the examination are notoriously difficult to achieve. However, it is recommended to define the best way of testing the students in accordance with the learning objectives first; then do as much as possible within the given limitations. The “ideal plan” can be kept in reserve for the time of the next major curriculum change (which is bound to come once in a while anyway). The assessment method which is most in line with the teaching philosophy of the Guide to Good Prescribing is the OSCE, which is the only method to test communication skills and practical skills in a structured and objective way. The problem is that OSCEs are time- consuming to develop and perform; they also require a special arrangement for the different OSCE stations where the students can be observed. However, don’t think that a series of one-way mirrored rooms and closed-circuit video systems are an absolute requirement for an OSCE—they are not. A number of adjacent rooms, or a large hall divided by movable screens are sufficient to start. And don’t forget that a discussion with your colleagues on how to examine the students should immediately lead you back to discussing the teaching objectives. Such a discussion will generate some resistance, but is an essential element in convincing your colleagues about the merits of changing pharmacotherapy teaching. � Changing the teaching implies changing the examination Teacher’s Guide to Good Prescribing 66 Table 3. Comparison of advantages (+) and disadvantages (-) of various assessment methods Written tests Oral tests Other LEQ SEQ MEQ MCQ OE SOE OSCE PET What do you test? Knowledge—recall of facts + + ++ + + + Organizing complex ideas and concepts ++ + + Problem-solving skills + ++ + + + Communication skills + Practical skills + Delivering a product within a deadline ++ Wide range of facts – + + ++ – – + Close to clinical practice + ++ How good is the test? Objective, valid, reproducible + + + ++ – +/– ++ – Unintended cues – Open to misinterpretation – Useful feedback to student – + + + + ++ How labour-intensive and costly is the test? Time/cost to construct + +/– – – – – Time/cost to perform + + + + – – – – – Time/cost to score – – – ++ Problems with student handwriting – – – + + + + Special (dis)advantages Direct personal contact + + +/– Flexible approach +/– Performance anxiety – – Needs relationship of trust – Key: LEQ: long essay questions; MCQ: multiple-choice questions; MEQ: modified essay questions; OE: oral examination; OSCE: objective structured clinical examination; PET: project execution test; SEQ: short essay questions; SOE: standardized oral examination. 67 CHAPTER 8 How to assess the teachers Chapter 8. How to assess the teachers It is important to make sure that teachers are competent and comfortable with theteaching methods used. If they are not, they should be given the necessary trainingand support. Some sort of evaluation must also be planned. In the case of pharmacotherapy teaching with the Guide to Good Prescribing, supervision and evaluation are essential to ensure that teachers do not put the students into a passive learning mode. There are various ways to assess the proficiency level of teachers. Here are some examples. Self-assessment Self-assessment by teachers is often underrated and under-used. Tutors can very well evaluate their own level of proficiency, for example, with the help of a questionnaire. Evidence suggests that self-assessment can be very effective in developing the reflective skills of the teacher and improve the quality of education delivered. It should therefore be part of any appraisal. Observation and analysis of the learning activities by students Evaluation can include direct observation and measurement of the actual teaching process. Several techniques can be used. Free observation by the students can be followed by a discussion; such observations can be improved by supplying the students with a check- list. There are also indicator-based observation methods. For example, time logs could be kept for the amount of time the teacher is active while students are passive, or the amount of time that all are active. These data can be visualized in a “group process tracking chart” and the teacher’s style could be evaluated by the frequency of different types of activity. The teacher could ask one student in the group to keep the time log, and share the outcome with the group at the end of the session. Evaluation by students An evaluation made by students can provide teachers with the most useful feedback of information on the quality of their teaching. Evaluation can be verbal at the end of the sessions, but it is preferably done in writing, using a brief questionnaire. Such a questionnaire can be based on a series of statements (for example: “Well-chosen examples are used to clarify points”) and a five-item scale (“strongly agree, agree, undecided, disagree, strongly disagree”). See example from Groningen below. This method can be further refined. One could first ask the students for their rating of the session (“Where do you place this statement on the evaluation scale?”) using the above- mentioned scale; and then asking them where it should be to satisfy them (“where should that be to satisfy you?”). The first question measures students’ perception of reality and the second the level of their expectations. The mean of the answers to the first and second questions is calculated separately. The difference between the two means shows the level of the students’ dissatisfaction, with a low value indicating a high level of satisfaction. If Teacher’s Guide to Good Prescribing 68 the statements cover various aspects of the session or course (presentations, examples, visual aids, exercises, role-play) the information can be very useful to identify strong and weak aspects of the session. BOX 19. PRACTICAL HINTS FOR DESIGNING A QUESTIONNAIRE � Formulate some of the statements in a negative sense, in order to avoid “response-set”. � Base the statements on a description of the tasks for the tutor (e.g. guiding students through the learning process, content knowledge input, and commitment to the group’s learning). � Do not use a binary scale (yes-no, true-false) since this provides little information. � Use clear, simple and complete statements; do not use double negatives. � Use one single thought, concept or tutor’s task per statement. � Keep the scale understandable; one uniform scale throughout the questionnaire is desirable. � Keep the number of statements as low as possible; preferably not more than 50. Finally, a clear process to evaluate the training programme gives the teacher a sense of security, and the certainty that good components of the training will be preserved while ineffective parts will be deleted. Unless continuing constructive evaluation is evident in some form, both teaching and learning quickly degenerate into a boring routine. Student observation sheet from the daybook of the Groningen Summer School of 1996 Facilitator: ……………………………….…. Date: ………………………….………. Subject: hypertension / diabetes / peptic ulcer / heart failure Theme: P-drugs / patient drugs / practical aspects of prescribing Objectives and outline of the lesson Brief, to the point, clear? Behaviour Enthusiastic? Friendly, not intimidating? Able to smile, sense of humour? Respect for students? Open to criticism? Group facilitation skills Discusses group dynamics with the group? Does not talk too much? Interventions are appropriate, stimulates discussion when necessary? Avoids ready-made answers / facts? Recognizes strengths and weaknesses and gives feedback? Asks students to summarize regularly? Willing to receive feedback from the group, pays attention? Helps the group maintain its focus? Respects the pace of the group? Summary of the lesson Brief, to the point, clear? Further assignments Brief, to the point, clear? Other remarks? 69 CHAPTER 9 How to measure the impact of the training Chapter 9. How to measure the impact of the training Why do research? If the effectiveness of a new trainingprogramme is to be assessed by theexamination results of the students, a pre/post experiment without a control group is performed. This implies that there is no absolute certainty that the assessment results are a direct result of the teaching programme. A small but systematic research programme can help. Apart from curiosity, which is in itself a good reason, research may generate clear proof that the new way of teaching is better than the previous one. It may also convince colleagues and policy-makers of the advantages of the new method. Apart from the multicountry study published in 19951 a more recent example of such a systematic study has been reported from Yemen.13 How can research be done? It is far beyond the scope of this Teacher’s Guide to discuss all aspects of research in teaching. However, a few practical aspects are mentioned here, with special reference to testing the effect of problem-based pharmacotherapy teaching. They are presented as logical steps in the process. Step 1. Define the aim of the research The first step is to determine the purpose of the research. The reasons could be dissatisfaction with the results of present teaching, or observations of irrational prescribing by young graduates. Dissatisfaction in teaching may lead to several different research aims: to measure the effect of a new teaching method, to compare the effect of a new teaching programme in different study years; to compare two teaching methods (for example lectures vs. small group teaching), or to compare two methods of assessment (e.g. multiple- choice vs. objective structured clinical examination). 1 De Vries TPGM, Henning RH, Hogerzeil HV, Bapna JS, Bero L, Kafle KK, Mabadeje AFB, Santoso B, Smith AJ. Impact of a short course in pharmacotherapy for undergraduate medical students. Lancet 1995;346:1454–7. 13 Hassan NAGM, Abdulla AA, Bakathir HA, Al-Amoodi AA, Aklan AM, De Vries TPGM. The impact of problem-based pharmacotherapy training on the competence of rational prescribing of Yemen undergraduate students. European Journal of Clinical Pharmacology 2000;55(11/12):0873–0876 � Experiment is the only way of verifying educational improvement. D.T. Campbell and J.C. Stanley Step 2. Define the research question(s) Describing a general aim is not enough; it should be translated into a general research question. This general question should be split into specific research questions, and ultimately into measurable (operational) variables. A general research question could be: “Will problem-based pharmacotherapy teaching improve the competence in rational prescribing?” In this approach “problem-based teaching” is the intervention and “competence in rational prescribing” the variable, the effect to be measured. By further defining “competence in rational prescribing” the research question should indicate more clearly what exactly will be measured: “Will problem-based pharmacotherapy teaching improve the competence in prescribing drugs according to the six-step WHO method?” The concept “competence” determines the situation of how the dependent variables will be measured: as a skill, measured under observed or examination circumstances, and not as an observation of real prescribing. The scope of the study can be further narrowed, for example by studying only the topics “asthma” and “diarrhoea” and by testing only 4th year students. Step 3. Maximize internal and external validity The design, study subjects, materials, data processing and statistical analysis should provide potential answers to the research question(s). In order to obtain valid results these procedures should be planned and carried out with great care. If the procedures are flawed, neither the results nor the interpretation of the results can be valid. Because validity of research is such a crucial concept it will be discussed briefly. The internal validity of a study is the extent to which the outcome is the result of the intervention, and not of other factors which have not been controlled. Examples are other interventions that occur at the same time, such as other changes in the curriculum; initial differences between the study group and the control group; or an effect of the measurement (when students in the control group learned from the pre-test). External validity refers to the extent to which the results of the study will apply to other similar situations. Two factors may influence this: subject selection and setting. For example, by selecting only very intelligent or motivated students, or by using very sophisticated and labour-intensive teaching methods, the results may not necessarily be reproducible elsewhere. Maximising internal validity usually creates special circumstances which cannot easily be reproduced; this then reduces the external validity. An acceptable balance between the two has to be found. Step 4. Choose the appropriate study design The study design is determined by the research question. There are many different types of general designs which have their own advantages and disadvantages regarding the internal and external validity and the practical consequences. Some relevant general research designs are discussed briefly below. Pre-experimental design studies are an easy and useful way of obtaining preliminary information about the feasibility of a research project. The materials and methods can be tested this way, but the validity of such a study is questionable. A one-group post-only Teacher’s Guide to Good Prescribing 70 case study (without control group) has a low internal validity (other things may have caused the effect) and low external validity (there is no random selection). The one-group pre-post design (without control group) is an improvement over the previous one, but it still lacks internal validity (test effect, no control group) and external validity (no random selection). True experimental design studies are less easy than the previously mentioned designs. However, the use of control groups and randomized selection addresses most factors that reduce validity. The pre/post design (with control group) controls other influencing factors and subject selection, but not the test effect, e.g. the effect of the pre-test on the post- test.* The same applies to another variation of this type, comparing two different interventions. Step 5. Select the students The selection procedure of participants in the study is important for three reasons. The selection of students defines the internal and external validity of the study; for practical reasons the number should be as low as possible; and there is a minimum number of subjects which is statistically needed to accept or reject the hypothesis. A statistician should be consulted to assist in making a power analysis. In a recent multi-centre controlled study on the effect of teaching the P-drug concept, a total of 584 students from eight different universities were divided into three groups of about 195 students each (two intervention groups and one control group). In the much smaller Yemen study 111 students were selected on a voluntary basis from two universities (out of five) and two health institutes (out of about 60). Step 6. Prepare the materials The materials should be based on the research question. For example, in the Yemen study the operational research question was: “Will problem-based pharmacotherapy teaching improve the competence in prescribing drugs according to the six-step WHO method”. Both for the tests and for the teaching programme patient cases were developed for the two clinical subjects (asthma and diarrhoea) and at four different levels of complexity (uncomplicated case, case with contraindication to the first-line drug, serious case, and a case with side-effects to the first-line drug). All patient cases had the same format and contained relevant information about the patient, e.g. history, examination, results, diagnosis. Some examples are given in Annex 3. A treatment plan form should be developed for the teaching programme. The same form can serve as a scoring form during the test (Annex 4). This form contains the six WHO steps (variables) and, on the right hand side, a coding system for each step. In the Yemen study students were trained to use these forms when solving a patient problem, and were also tested with the same forms. Chapter 9. How to measure the impact of the training * For this reason post-test only study and control groups could be added (Solomon four-group design). This design may cause some practical problems as more subjects (students) and more facilities (teaching rooms, materials, teachers) will be needed. Cross- contamination (information exchange) between different student groups must also be prevented. 71 Step 7. Explore other ways to improve validity If a control group is used, instructing students not to exchange information between the groups should prevent bias. If more groups are used, all teachers should be instructed carefully to teach in the same way. Students are usually very disappointed to end up in the control group; a situation which can be helped by promising them the same training course after the post-test. This approach may also reduce cross-contamination by preventing students from the study group(s) sharing what they have learned with those in the control group. Step 8. Process the data Data processing starts when the test materials have been assembled after all tests. When the competence of students is being measured the test forms are often of a short essay type. In that case the answers have to be scored first. After that the scores have to be processed. Sometimes the scoring occurs directly, for example in OSCE. The scoring procedure for a research project requires the same accuracy as for an examination. For example, it is preferable to use two independent scorers. The pre- and post-tests sheets for the study and control groups should all be mixed and scored anonymously at the same time. The first step in processing the data is to set up a data structure, either on paper or on the computer. Many spreadsheet or database programmes can be used, or their data can easily be imported into statistical programmes. The rows are used for the subjects (students) and their data. Every row (student) has a record number. The columns organize the data in variables. The variables should be defined and given a short name: group (study or control), patient problem and step (e.g. patient problem 1: treatment goal), and score (e.g. 0=no answer, 1=poor answer, 2=arguable answer, 3= acceptable answer, 4=good answer, 9=no answer needed). After completing this data structure, the database or spreadsheet has to be defined according to this structure. After that the scores of the test forms can be put into the computer. Descriptive statistics can be elaborated with a spreadsheet or database, or even on paper by simple calculating. Examples are the mean scores of pre- and post-test of the study and control groups; or the initial mean score and the subsequent increase. Results should preferably be expressed as a mean score with 95% confidence limits. Step 9. Present the results The results should be presented as plain facts, directly derived from the statistical outcome. Most results are best presented in tables and/or figures. In a report or paper the results should also be described briefly in the text but there is no need to repeat in the text any data which are already in a table. The results should not be interpreted here but in the “discussion” section. Step 10. Discuss the results The results should be interpreted and discussed. Even if the original hypotheses are supported the researcher should still consider alternative explanations for the findings. 72 Teacher’s Guide to Good Prescribing And if the hypotheses are not supported the researcher should consider why not: what went wrong during the study, was the research plan chosen correctly, or were the expectations (hypotheses) wrong after all? The conclusions should only be based on the results and be directed at answering the research question(s). The conclusion may lead to practical recommendations. Step 11. Formulate recommendations There are three types of recommendations. The first is the recommendation to perform further research on the basis of the findings of this study. A second type refers to the practical implications of the findings, for example a recommendation to introduce problem- based pharmacotherapy teaching to all students. Thirdly, the experience of the study may also lead to recommendations on the research design or methodology. Chapter 9. How to measure the impact of the training 73 Teacher’s Guide to Good Prescribing 74 75 PART 3 How to mobilize support Teacher’s Guide to Good Prescribing 76 77 CHAPTER 10 How to mobilize support for problem-based pharmacotherapy teaching Changing the behaviour of university teachers is just as difficult as changing thebehaviour of prescribers. There are early changers and late changers (and some whowill never change). This chapter summarizes the arguments in favour of problem- based pharmacotherapy teaching, together with the most common arguments used against it. The final section discusses a strategy that can be used to sensitize and convince colleagues to review and possibly change the way in which they are teaching. Arguments in favour of problem-based pharmacotherapy teaching There are many arguments in favour of problem-based pharmacotherapy teaching. They can be summarized as follows: 1. Good teaching is based on learning objectives The main learning objectives of pharmacotherapy teaching are for the students to obtain the skills to: • Select and prescribe adequately an appropriate treatment for the most common conditions they will meet in their professional life, and • Assess critically any new drugs or non-drug treatments which will become available in the future, and define their place in therapy in relation to existing treatments. 2. Pharmacotherapy is a skill; it is more than knowledge alone Pharmacological knowledge of a large range of drugs is not enough to prescribe rationally. The six steps of the Guide to Good Prescribing (define the problem, specify the therapeutic objective, choose the treatment, write the prescription, inform the patient, monitor/stop the treatment) are a logical model for good prescribing. Each step represents a skill which needs to be taught, practised and examined. Problem-based teaching in small groups is the best way to achieve this goal. 3. Drugs change; knowledge of current drugs is not enough for a life-long professional career A continuous stream of new drugs and treatments appears on the market. Pharmacological knowledge of the drugs currently available is likely to be outdated within the next five years. New drugs are often marketed aggressively. A critical attitude is therefore needed to review such new treatments and give them a place (or not) in the therapeutic armoury. Failure to inculcate such an attitude and skill in the students will result in their being insufficiently prepared to withstand high-pressure salesmanship in the future. Chapter 10. How to mobilize support for problem-based pharmacotherapy teaching 77 4. Students love problem-based learning One argument that can also be used is that students love problem-based learning. The art of writing a prescription, which is such a visible expression of the power of the doctor, has a certain magical attraction for the students; at the same time they may feel very insecure about the matter as it is often badly taught. The real-life situations of most of the patient problems in the Guide to Good Prescribing are very attractive to students. They recognize them immediately as very relevant to their needs and as bringing them a bit closer to their final responsibility as a doctor. For this reason it is important to include some sort of student evaluation of the teaching programme. In most cases the students will be very clear in expressing that they really like the method, and want more of it. 5. Problem-based pharmacotherapy now can lead to a problem-based curriculum later In a few cases problem-based pharmacotherapy has been the first step in a long process of changing from traditional teaching towards an integrated curriculum with problem-based learning (see below). Problem-based pharmacotherapy teaching leads to an integrated curriculum in Groningen In Groningen (The Netherlands) problem-based pharmacotherapy teaching had been opera- tional within an otherwise traditional curriculum for a number of years when, suddenly, the decision was taken that the whole curriculum would be changed into problem-based integrated teaching. At that moment there was a great need for faculty members who had experience in problem-based teaching. These were found in the Department of Clinical Pharmacology and Community Medicine. They had used the teaching method for many years. These faculty mem- bers proved very useful in assisting the curriculum review committee. Moreover, on the basis of their experience they could ensure that problem-based pharmacotherapy teaching became part of nearly all integrated teaching sessions in the new curriculum. Arguments often used against problem-based learning in small groups A number of arguments are often used against problem-based pharmacotherapy teaching in small groups. The most common are listed here, together with a suggested counter- argument. Scientific knowledge about drugs is very important for the future prescriber Basic pharmacology is, of course, essential to understand the pharmacokinetics and dynamics of drugs. Yet this argument is often used to defend the status quo of pharmacology teaching. However, drugs currently in use will gradually be replaced by newer drugs. It is therefore of little use to force the students to memorize a lot of information about current drugs. It is much more useful to teach them where to find such knowledge, and how to use it. Even the usual basic pharmacology, as it is taught in many universities, may not be all that adequate. Annex 1 in the Guide to Good Prescribing presents the core components of Teacher’s Guide to Good Prescribing 78 pharmacology in a different way, which is more oriented towards clinical practice for a general practitioner. The same argument may also be used to retain practical laboratory exercises on the programme, for example to show the effect of certain drugs on laboratory animals or isolated organs. Although instructive, this tends to take a lot of time and resources, while the learning objectives could easily be achieved through a video film demonstration. This would make a lot of extra time available for other teaching methods, such as small-group discussions (see below). We cannot change to a full problem-based curriculum A presentation on problem-based pharmacology teaching tends to end in a discussion on problem-based teaching in general; and at that stage a frequently heard argument is that it would be impossible to change into a full problem-based curriculum. The counter argument here is that there is no need to change the whole curriculum. It is clearly possible to teach pharmacotherapy in a problem-based manner, in small groups, within the scope of an otherwise traditional curriculum. One can change the teaching of pharmacology (usually basic pharmacology in the second or third year, some laboratory practicals and long hours of teaching of drug data in the fourth) into problem-based teaching in small groups. A good example of this approach is given below in the description of problem-based pharmacotherapy teaching in Turkey. There is no time in the curriculum It is well recognized that it is nearly impossible to increase the allocation of hours in a curriculum, which is usually overloaded anyway. However, it is possible to use the hours allocated to pharmacology in a different way. Time previously spent on laboratory practicals (usually in small groups) can easily be converted into small group teaching (saving on equipment, materials, laboratory animals and support staff). In addition, in many curricula a large number of contact hours are spent on going through all therapeutic groups. If agreement is reached that students should be taught how to find the information on drugs, rather than supplying them with it and forcing them to memorize it, this goal can be much better achieved by giving the students assignments and patient problems to deal with. One hour of small-group teaching with home assignments is as instructive as many hours of traditional lecturing. So, even though small-group teaching implies fewer contact hours per student, it is very much worth it. And finally, several universities have managed to incorporate small-group teaching in close collaboration with other departments, such as community medicine, or during internal medicine clerkships. This approach also helps to solve the problem of staff shortages (see the example of Turkey below). There are not enough teachers for small-group teaching Teaching in small groups means, of course, that more teaching hours are necessary. There are several ways of doing this. One way is to reduce the number of hours per student. For example, changing 40 hours of lecturing to 100 students into 10 hours each for groups of 25 students. An alternative way is to involve other staff in teaching, for example junior staff, PhD students and registrars. Yet another way is to recruit some students of the year above, and train them to facilitate small group work sessions. Remember that the role of the facilitator is to structure and facilitate the small group discussion, and to guide the Chapter 10. How to mobilize support for problem-based pharmacotherapy teaching 79 Teacher’s Guide to Good Prescribing 80 Implementing problem-based pharmacotherapy teaching in Turkey Marmara University School of Medicine was founded in 1983. The total number of students is approximately 100 per year, of whom 10% are foreigners. Teaching is in English. The medical education lasts for six years, of which the first three years are preclinical, the fourth and fifth year are clinical clerkships and the sixth year is an internship. Medical education is organized in an integrated way. The lectures are programmed in groups to cover different aspects of a general title (e.g. the respiratory system). Pharmacology starts in the second year. The general pharmacologi- cal concepts, i.e. routes of drug administration, absorption, distribution, and metabolism and excretion, are taught during the second half of the year, all through lectures. Lectures about pharmacological properties of different drugs are given during the third year, integrated with pathology and clinical features of the corresponding diseases. Knowing that clinical pharmacology is a bridging discipline between basic pharmacology and clinical sciences, we were looking for an opportunity to fit clinical pharmacology teaching into the curriculum. The students’ demand for extra pharmacology during the clinical years gave us the support we needed. Starting from 1991, elective pharmacology seminars and lectures were ar

View the publication