Back to Position Papers

Problem-based learning View review What is PBL and how does it work? Problem-Based Learning (PBL) is a method of learning in which students first encounter a problem, followed by a student-centered inquiry process (Neufeld and Barrows, 1974; Schmidt, 1993; Boud and Feletti, 1997; Barrows, 2000). Both content and the process of learning are emphasized in PBL. Many variants of PBL have evolved during the past 30 years and yet its essential elements have remained relatively constant. Axiomatic to PBL is that the problem comes first without advance readings, lectures, or preparation, serving as a stimulus for the need to know. Typically, five to eight students work collaboratively in a group (tutorial), together with one or more faculty facilitators (tutors), to identify and define problems, develop hypotheses to explain the problem(s), and explore preexisting knowledge relevant to the issues. Students determine and explore what they already know and what they need to learn in order to advance their understanding of the problem(s). Key elements of PBL are the formulation of questions that can be explored and answered through systematic, self-directed inquiry and the testing and revision of hypotheses through the application of newly acquired knowledge. Active discussion and analysis of problems, hypotheses, mechanisms, and learning issues among students are essential to this process, enabling students to acquire and apply content knowledge and to learn and practice both individual and group communication skills critical to learning and teaching. Problems (sometimes referred to as cases) are created/selected by the faculty to represent important priority health problems of the region as well as prototypical situations and knowledge domains (Bordage, 1987; Neufeld et al., 1997). The format of the problem simulates professional practice or a real life situation and can involve a real or standardized patient or a paper case. Other forms of media such as video, the Internet, and digitalized computer applications allows for even more variability in format. Problems cannot be solved easily at first glance or with only the initial information presented. The case is constructed so that as students explore what they know it becomes evident to them that they do not have enough information to decide among the hypotheses they have developed, and their need for additional information becomes imperative. The tutor facilitates and guides learning without contributing directly to the solution of the problem or being the primary source of information. The tutorial process is facilitated when the tutor creates an optimal learning environment in which students feel free to be themselves and express their uncertainties about case-related subjects and the group process. The tutor listens carefully, responds, and uses questions to explore and stimulate students’ thinking. In addition, the tutor helps the group set standards for depth and breadth of knowledge, develop reasoning ability, enhance communication skills, adopt professional behavior and attitudes, and develop skills for self and peer assessment. Students are usually randomly assigned to groups. The length of time that groups stay together varies from program to program (usually from eight weeks to a semester) and they meet one or more times per week for two or three hours. A problem requires two or three sessions, depending on the length of the sessions. The pace of the inquiry process is dictated by the amount of time available for the case and the length and complexity of the case. Self-directed study between sessions is a key part of the process. Although the sequence of activities involved in a group coming together and working through cases may vary, it typically involves certain predictable steps. A new group meeting for the first time will begin by getting to know each other and establishing “ground rules” that provide parameters for how the group will work together, as well as establishing institutional, tutor, and individual expectations. A case most often begins with a chief complaint, the reason for a person’s visit with a health professional. Students identify and record on the board key words, facts, and issues and formulate a statement of the problem(s). A list of ideas about what might be going on (hypotheses) is also developed and listed on the board. For each hypothesis generated, students are encouraged to explain their thinking and understanding out loud to the group. Students may work individually for a few minutes before presenting their ideas to the group in order to enhance their individual skills and learn from others in the group. This serves to make visible existing individual and collective knowledge, reasoning, and communication skills. The initial list of hypotheses is usually broad and long because students generate initial hypotheses early in the case, when little information is available. Further inquiry about the patient’s history, physical examination, and laboratory findings helps students evaluate and prioritize their hypotheses. As information in the case is disclosed serially (rather than all at once), learners add to and/or reevaluate/reprioritize their hypotheses in light of new information and further explore their understanding of relevant mechanisms that help explain the problem. In doing so, they continually explore their understanding of relevant mechanisms that help explain the problem. At every step along the way, students are prompted by the tutor and by each other to explore what they know and to come to the edge of their knowledge, where curiosity is transformed into exploration and then inquiry--a central motive for learning in PBL. A major goal of the tutorial process is to generate questions that lead to the acquisition of new knowledge that builds upon and connects with existing knowledge. The formulation of learning issues (content questions that cannot be answered with current knowledge) functions to close discussions when there is insufficient knowledge to resolve uncertainties and to focus group and individual self-directed inquiry between tutorial sessions. Recognition and articulation of preexisting knowledge helps provide a link for newly acquired information and aids in its storage and retrieval. Most commonly, students progress through the problem by the end of the first session have developed a list of learning issues. At this point, students do not continue to read through the case. Instead, they review and prioritize their learning issues and decide how to proceed with their studies, setting expectations of themselves for the time between meetings. At the conclusion of each meeting, it is useful for the group to take time to reflect on how they are working together, how the tutor is doing, and how individual learners are doing. The focus is on what is needed for growth and development in predetermined domains such as knowledge; reasoning; communication; professional attitudes, values, skills, and ethics; group process and self and peer assessment. Frequent and timely self-reflection, peer and tutor feedback about group and individual progress enhance learning and development. Specific recommendations are articulated and incorporated into the group’s work at the next meeting. At the beginning of the next session, students review what happened last time and set an agenda for the meeting. The group reviews its learning issues and the problem and discusses what has been learned in light of the questions that led to their self-directed inquiry. When major learning issues have been studied by each student in the group (rather than divided up among students), the discussion and learning can yield a highly productive exchange of differences and similarities. They discuss, debate, compare and contrast their learning with each other. This process of elaboration of information contributes to learning and retention of knowledge (Norman and Schmidt, 1992; Regehr and Norman, 1996). The newly acquired information is applied to the problem, hypotheses are reevaluated or changed, and associated mechanisms are analyzed and synthesized. A key role for the tutor during this exchange is to help students’ build bridges that connect and extend their existing knowledge with newly acquired knowledge and to facilitate the application of knowledge to the understanding of mechanisms relevant to the patient’s problem(s). This, in turn, leads to more questions, and thus, the process is iterative. After a period of independent study, students return to the case, which continues to unfold progressively. Throughout the process, students explore what they know and what they need to learn and formulate learning issues. At the end of the case, they summarize what they have learned and reflect on their progress as a group and as individuals. What is the rationale for PBL? Evidence indicates that active participation in learning is more satisfying than passive transfer of information from the teacher to the student and that active learning leads to enhanced retention and recall (Bransford, Brown, Cocking, 2000). PBL emphasizes active student-centered learning in which students are challenged to examine, inquire, reflect, make meaning, and understand the sciences basic to medicine as they develop approaches toward the solution of defined problems in a context relevant to their future professional careers (Neufeld & Barrows, 1974). The discussion of clinical or other types of problems in small groups ("elaboration") promotes a connectedness of ideas and concepts (Schmidt, 1983) and fosters cooperation rather than competition among students (Schmidt & Moust, 2000). The exploration of prior knowledge, the formulation of inquiries derived from and defined by the learners’ need to know, and the active construction of meaning through dialogue and reflection promote long-term retention of newly acquired information (Schmidt, 1981; 1983; Norman and Schmidt, 1992; Regehr and Norman, 1996). The exponential expansion of knowledge requires that learners be able to ask well-formulated questions and utilize refined information retrieval and evaluation skills. PBL provides conditions for the development and practice of self-directed learning. PBL graduates' ability to deal with problems and to retrieve information may be an important index for life-long learning (Shin et al., 1993). PBL curricula are often integrated across the sciences basic to medicine, as well as among departments and activities such as clinical skills and doctor-patient-society courses that have traditionally been restricted to particular years of the curriculum (Walton & Matthews, 1989). By utilizing cases and problems designed to match student perception of their future profession and their current knowledge, PBL serves as a powerful stimulus for their intrinsic motivation to learn (Dolmans, 1997). The integration of subjects also permits an emphasis on other important aspects of the training of future competent health professionals, such as communication, teamwork, and professional attitudes, values, and ethics. The combination of early and sustained community and primary care experiences with carefully selected PBL cases relevant to priority community health problems can synergize with the promotion of population and community health education. (Neufeld et al., 1997). How is PBL being implemented? A wide variety of curricula purported to be PBL are practiced (Barrows, 1986; Harden & Davis, 1998). In many schools, the PBL format eventually implemented largely mirrors the willingness of the faculty to shift from the role of "instructor of knowledge" to "facilitator of learning" and to accept "central" rather than "departmental" control of the curriculum. Students may play an important role in discussing the transition from a “traditional” to a PBL curriculum, either negatively by expressing fear of inferior education, or positively by opting for a more self-directed approach to learning (Visser, 1997). Local circumstances (e.g., lack of resources to provide sufficient rooms for small-group meetings or to equip an adequate library) may hamper the implementation of a PBL curriculum. Many schools, however, have developed excellent programmes with minimal resources (Ilorin, Umtata, Ilo-Ilo, to name a few). One successful strategy in transforming a curriculum is to offer as a pilot experiment a single course or semester in a PBL format (Kantrowitz, Kaufman, Mennin, et al., 1987; Mennin and Kalishman, 1998). The likelihood of success for this approach is enhanced when there is little or no competition from the traditional curriculum. It is important that unscheduled hours given to students to pursue learning issues related to tutorial problems not be scheduled for other purposes. Optimal conditions for pilot experiments are difficult to achieve. Another strategy involves a group of dedicated staff creating a PBL parallel track for a limited number of students. Leaving the traditional curriculum in place, interested staff and students can undertake the design and implementation of a PBL curriculum. When fully developed, outcome studies may be undertaken to compare the efficacy and efficiency of both programmes. Several schools have recently demonstrated the feasibility of changing from a traditional to a PBL curriculum (DesMarchais, 2001; van Rossum, 2000; Schwartz, Mennin, Webb, 2001). Most commonly, 3-5 years are needed to reach an agreement among the staff to plan, develop, and begin the implementation of the PBL curriculum. An unavoidable drawback of this approach (and of expansion of a parallel track as described above) is the temporary co-existence of an “old” and a “new” curriculum. Students in the old curriculum may feel neglected by staff who are absorbed with implementation of the new one. No doubt the planning and development of a new school provides an excellent opportunity to commence a PBL curriculum. Even if such a school eventually were changed to a more traditional educational approach, that transformation would probably be easier than changing a traditional program into a PBL one. What are some of the outcomes? Recent reviews have summarized the literature on outcomes (Vernon and Blake, 1993; Albanese and Mitchell, 1993). In addition, several thoughtful discussions of PBL are available in the literature (Norman and Schmidt, 1992; Schmidt, 1993; Regehr and Norman, 1996; Norman and Schmidt, 2000; Nendaz and Tekian, 1999; Barrows, 2000). The comments that follow will draw in part from the above literature, to which the reader should refer for additional details. The majority of the data concerning outcomes of PBL are descriptive and quasi-experimental. In general, studies on PBL are difficult to compare or generalize due to wide variances in conditions, settings, practices at different institutions and small numbers of participants. In addition, variances in assessment methodologies and differences of opinion as to the objectives of medical school curricula make obtaining data as to the effectiveness of a curriculum inherently difficult. There is, however, general agreement that students prefer the challenge, stimulation, and motivation offered by PBL and that development of critical thinking skills is an important component of medical education. Overall, comparisons of students in PBL programmes to those in traditional programmes have yielded the following results: Academic Achievement Students perform a little better or a little worse on standardized examinations of basic science, such as the USMLE Step 1 exam used in the United States and better on progress tests used in The Netherlands. Results vary from school to school based on curriculum and individual programme characteristics. Overall, there is not a significant difference in performance based on pedagogy. Basic science grades derived from performance on objective examinations are not significantly different as a function of teaching/learning methods. There is evidence that long-term recall is enhanced for students in a PBL curriculum. The rate of progress through school (the attrition rate; irregular progress) is better or not different for students in a PBL versus a traditional curriculum. Clinical Achievement In general, students from PBL vs. traditional programmes appear to have superior clinical skills and perform better in the clinical setting. PBL students may score higher on tests of humanistic knowledge, attitudes, and skills. There is a slight trend toward a significantly higher test score on standardized tests during the clinical years. Results of performance on tests of clinical knowledge are similar or slightly higher for students in PBL curricula compared to more traditional curricula. Approaches to Learning There are reported differences in how students study and approach their learning based on whether they are in a PBL or a traditional curriculum. Students in PBL more often indicate that they study for understanding and meaning, whereas students in more traditional curricula indicate that they study more often for rote learning and memorization. PBL students use the library more often, utilize a wider variety of learning resources, and choose those resources in a self-directed way rather than as assignments from teachers. Students in PBL tracks report being more satisfied, less stressed, and more positive about their learning environment when compared to students from traditional tracks. They are more likely to report that their early medical school years were challenging, engaging, and difficult compared to students from traditional programmes, who report their experience as being irrelevant, passive, and boring. Graduates Graduates of McMaster University and the University of New Mexico School of Medicine report being as prepared or more prepared for post-graduate study and practice compared to graduates of traditional programmes. Clinical ratings by post-graduate supervisors found graduates from these PBL programmes to be equal or superior to other students in specified areas and competencies. Graduates from McMaster University were more likely to spend more time in direct patient care and to pay more attention to psychosocial issues (i.e., bill more for psychiatric, pediatric, maternal and infant services). Career Preference There is no evidence that either the PBL or traditional curriculum approach by itself promotes selection of careers in primary care or tertiary care. Teacher Satisfaction There is general agreement among faculty that PBL is a satisfying way to teach and interact with students. Faculty who teach do so because they enjoy being with students. PBL provides more opportunity for teachers to spend quality time with students. Costs and Time Studies show that the costs in terms of faculty time and effort are approximately the same for PBL and traditional programmes. With higher numbers of students, more faculty time is required for PBL programmes, however it may be more related to faculty development, non-tutorial education and support requirements. Using larger groups and modifying some techniques can allow for some of the advantages of PBL while minimizing the effects of somewhat larger group sizes. Teachers in PBL vs. traditional curricula spend more of their time dedicated to teaching with students. Global Outcome There can be no doubt that the advent and dissemination of PBL around the world has contributed greatly to a reexamination of our pedagogical approaches to the health professions. This has occurred during a time characterized by the rapid growth of information and technology, greater demands on health service systems, and a reexamination of how health professional institutions organize and conduct the education of future professionals. All things being equal, PBL is a more enjoyable and stimulating way to learn. Clearly there is a need for more research and a greater application of what we have learned with respect to the education of health professionals. For example, we need to know more about the relationship of the quality of care provided by graduates from different pedagogical programmes. We still know little about the best approaches to teaching and learning. Teachers in health professions schools generally have an expertise in a specific domain of knowledge, but lack training and developed skills in educational theory and its application in their settings. What is the best approach to preparing teachers for PBL? What are some of the lessons learned? One lesson that has emerged repeatedly is the risk of compromising the benefits of PBL when it is blended, hybridized, or otherwise placed in competition or juxtaposition with more traditional approaches to education. Learning in small-groups is much more difficult with a little PBL and a lot of traditional pedagogy. In curricula that combine PBL and more traditional methods, students may view PBL as secondary to more traditional aspects of the curriculum if assessment strategies do not reflect tutorial skills and content in a significant way. In addition, close association between lecture and tutorial material may attenuate students’ self-directed evaluation of new information and provide a false sense of a deeper understanding. There is continuing necessity for well-trained teachers who can conduct small-group PBL sessions skillfully. Training needs to continue and be iterative over time beyond a short single introductory session. Assessment methods for students in PBL programmes need to be consistent with how students learn. Parallel tracks as a means of introducing and testing PBL have been shown to be a successful strategy (Kantrowitz, Kaufman, Mennin et al, 1987). Community-oriented and community-based learning is inherently problem-based and provides a natural environment to introduce PBL or to continue it in another setting. However, PBL is not inherently community-based or community-oriented. Introducing PBL into existing traditional curricula is difficult and time consuming (Mennin and Kaufman, 1989; Mennin and Kalishman, 1998; Schwartz, Mennin and Webb, 2001). Comments about PBL and Community-Based and Community-Oriented Learning Problems come in all forms, sizes, and shapes. Learning about health, wellness, and disease in communities requires students to listen and observe critically, and self-directed learning is a necessity. The process by which community health professionals engage problems is nearly identical to the process used by students in PBL. The problem comes first. Relevant facts are gathered, the problem and all participants are identified, hypotheses are formed, and existing knowledge is called into action, in this case collaboratively with community members. The testing of hypotheses is often evaluated based on the results and outcomes of actions taken by health professionals. It is possible, but more difficult, to learn about community health problems via paper cases in a PBL tutorial in a classroom based at a university. Compared to details of molecular biology or neuroscience, community problems can appear “soft” and lack well defined details. Further, sub specialist staff physicians and basic scientists are less likely to identify and write PBL cases centered around priority health problems of communities than are primary care and community-oriented practitioners. However, once out in the community, it is clear to students how little biology they actually know and how much they need to learn. Once again the need to know can be harnessed as a force for learning. In this case, it is fully integrated with behavior and population issues of the community. For students, the breadth of learning issues is much wider in communities. It is here that psychosocial issues and economic, environmental, and behavioral determinants of health and disease can be experienced and learned first-hand. In community learning programmes, students typically keep records of the patients they see, the problems they encounter, and the learning issues they study. In many programmes, students are also required to provide a community service or conduct a study or project in the community. The inextricable role of economics, education, transportation, employment, and public health are highlighted and, in the opinion of the present authors, learned best through direct and sustained experience in the community. The life style and community role of the health professional is experienced, and the importance of role models and mentors cannot be overestimated. What are some of the challenges? The growing popularity and increased application of PBL presents significant challenges for all of us. To date, research related to PBL has been more descriptive than predictive (Norman, 2000). Many questions remain unresolved, including: To what extent does PBL contribute to lifelong learning? What are the critical aspects of cases or problems that must be constant, and which aspects can be variable? What is the best way to select, prepare and sustain faculty/staff for their roles as tutors? How is assessment of learning best done where PBL and more traditional learning methods are combined? To what extent do we know if there is an optimal combination of pedagogical methods? What strategies work best for the transformation and integration of PBL into existing curricula? What are the benchmarks and criteria for a successful program? What data do we need to be collecting to be able to determine if PBL graduates practice differently compared to graduates from many other traditional program? To what extent will we be able to link educational pedagogy to health indices outcomes for a defined population? We look forward to the continued scholarly development, application, and transformation of knowledge in relation to problems of consequence as a means of improving medical education and the health of the public. Corresponding Authors on behalf of the writing group: Stewart Mennin and Gerard Majoor Email: smennin@salud.unm.edu / g.majoor@oifdg.unimaas.nl Recommended readings in PBL On the origins Barrows HS (2000). Problem-Based Learning Applied to Medical Education. Revised edition. Southern Illinois University School of Medicine, Springfield, Illinois. Neufeld VR, Barrows HS, (1974). “The McMaster philosophy”: An approach to medical education. Journal of Medical Education 49(11):1040-1050. On the theoretical and cognitive psychology background Bordage G (1994) Elaborated Knowledge: A key to successful diagnostic thinking. Academic Medicine 69(11):883-885. Bransford JD, Brown AL, Cocking RR (Eds.) (2000) How People Learn: Brain Mind, Experience, and School. National Academy of Sciences. National Academy Press, Washington, D.C. Norman GR, Schmidt HG. (2000) Effectiveness of problem-based learning curricula: theory practice and paper darts. Medical Education 69(9):557-565. Norman GR & Schmidt HG. (1992) The psychological basis of problem-based learning: A review of the evidence. Academic Medicine 67; 557-565. Regehr G, Norman GR (1996) Issues in cognitive psychology: Implications for professional education. Academic Medicine 71:998-1001. Schmidt HG (1993) Foundations of problem-based learning: some explanatory notes. Medical Education 27: 422-432. Schmidt HG (1983) Problem-based learning: rationale and description. Medical Education 17:11-16. Wilkerson L., Gijselaers WH (1996) Bringing problem-based learning to higher education: Theory and practice. New Directions for Teaching and Learning #68. Jossey-Bass Publishers, San Francisco, California. On curriculum design and development Almeida M, Feuerwerker L, Llanos MC (2001) Education of Health Professionals in Latin America. Network: TUFH Publications, Maastricht, The Netherlands. Barrow, HS (1999) Criteria for analyzing a problem-based learning curriculum. In: JA Rankin (ed). Handbook on Problem-Based Learning. Forbes Custom Publishing, NY, pp. 27-29. Bordage G (1987) The curriculum: Overloaded and too general? Medical Education 21:183-188 David T, Patel L, Burdett K, Rangachari P. (1999) Problem-Based Learning in Medicine: A practical guide for students and teachers. Royal Society of Medicine Press Ltd. London, England. De Grave WS, Moust JHC, Schmidt HG. (1984) Tutorials in Problem-Based Learning: A new direction in teaching the health professions. Volumes 1 and 2. Van Gorcum, Asses/Maastricht. Henry R, Byrne K, Engel C. (eds.) (1997) Imperatives in Medical Education: The Newcastle Approach. The University of Newcastle, Newcastle, Australia. Des Marchais JE (2001) Learning to Become a Physician at Sherbrooke: A full switch to a student-centered MD Program. Network: TUFH Publications, Maastricht, The Netherlands. Kantrowitz M, Kaufman A, Mennin S, Fulop T, Guilbert J-J. (1987) Innovative Tracks at Established Institutions for the Education of Health Personnel: An experimental approach to change relevant to health needs. WHO Offset Publication No. 101. Switzerland. Mennin S, Kaufman A. (1989) The Change Process and Medical Education. Medical Teacher 11:9 16. Mullins G, Wetherell J, Townsend G, Winning T, Greenwood F. (2001) Problem-Based Learning in Dentistry: The Adelaide Experience. The Dental School, The University of Adelaide. Adelaide, Australia Neufeld V, Pickering R, Simpson J (1997) Priority Health Problems in the Education of Health Professionals. Community-Partnerships Series: Volume 2. Network: TUFH Publications, Maastricht, The Netherlands. Richards R, Sayad J. (2001) Addressing the Needs of People: Best Practices in Community-Oriented Health Professions Education. Network: TUFH Publications, Maastricht, The Netherlands. Robinson V. (1993) Problem-Based Methodology: Research for the improvement of practice. Pergamon Press Ltd., Oxford. Schmidt H, Magzoub M, Feletti G, Nooman Z, Vluggen P (2000) Handbook of Community-Based Education: Theory and Practices. Network: TUFH Publications, Maastricht, The Netherlands. Snellen-Balendong HAM (1993) Rationale underlying the design of a problem-based curriculum. In: Bouhuijs PAJ et al. (Eds) Problem-Based Learning as an Educational Strategy. pp. 69-78. Maastricht: Network: TUFH Publications Snellen-Balendong HAM, Dolmans DHJM (1999) Block Construction. Maastricht: University Press. Schwartz P, Mennin S, Webb G (2001) Problem-Based Learning: Case Studies, Experience, Practice. Kogan Page, London. Woods DR. Problem-Based Learning: How to gain the most from PBL. (1994) W.L. Griffin Printing Ltd., Hamilton, Ontario, Canada. On the design of problems Dolmans DHJM et al. (1997) Seven principles of effective case design for a problem-based curriculum. Medical Teacher 19: 185-189 Hafler JP (1997) Case writing: Case writers’ perspectives. In: Boud D, Feletti, G. The Challenge of Problem-Based Learning, 2nd ed. Kogan Page, London, pp. 151-159. Hitchcock MA, Anderson AS. (1997) Dealing with dysfunctional tutorial groups. Teaching and Learning in Medicine 9:19-24. Jayawickramarajah PT (1996). Problems for Problem-Based Learning: A Comparative Study of Documents. Medical Education 30: 272-282. MacDonald, PJ. (1997) Selection of health problems for a problem-based curriculum. In: Boud D, Feletti, G. The Challenge of Problem-Based Learning, 2nd ed. Kogan Page, London, pp. 93-102. On the functioning of tutorial groups Evensen DH & Hmelo CE (Eds): Problem-Based Learning: A Research Perspective on Learning Interactions. pp. 19-52. Mahwah, NJ: Lawrence Erlbaum Lucero SM, et al.,(1985) Tutorial groups in problem-based learning. In: Kaufman A. (Ed) Implementing Problem-Based Medical Education. Lessons from Succesful Innovations. New York: Springer Publ. Pp.45-70 Peterson, M. (1998) Skills to enhance problem-based learning. Schmidt HG & Moust JHC (2000) Factors affecting small-group tutorial learning: A review of research. In: Evensen DH & Hmelo CE (Eds): Problem-Based Learning: A Research Perspective on Learning Interactions. pp. 19-52. Mahwah, NJ: Lawrence Erlbaum. Wilkerson L (1999) Tutors and small-groups in problem-based learning: Lessons from the literature. In: JA Rankin (ed). Handbook on Problem-Based Learning. Forbes Custom Publishing, NY, pp.69-79. Wilkerson L, Hundert EM (1997) Becoming a problem-based tutor: Increasing self-awareness through faculty development. In: Boud D, Feletti, G. The Challenge of Problem-Based Learning, 2nd ed. Kogan Page, London, pp. 160-172. On student assessment Blake JM et al. (1996) Introducing progress testing in McMaster University's problem-based medical curriculum: psychometric properties and effect on learning. Academic Medicine 71: 1002-1007. Nendaz MR, Tekain A (1999) Assessment in problem-based learning medical schools: A literature review. Teaching and Learning in Medicine 11(4):232-243. Swanson DV, Case SM, van der Vleuten CPM (1997) Strategies for student assessment. . In: Boud D, Feletti, G. The Challenge of Problem-Based Learning, 2nd ed. Kogan Page, London, pp.269-282. Van der Vleuten CPM et al. (1996) Fifteen years of experience with progress testing in a problem-based learning curriculum. Medical teacher 18: 103-109. On programme evaluation Friedman CP, deBliek R, Greer DS, Mennin SP, Norman GR, Sheps CG, Swanson DB, Woodward CA. (1990) Charting the winds of change: Evaluating innovative medical curricula. Academic Medicine 65:8-14. Mennin SP, Friedman M, Woodward CA (1992) Evaluating innovative medical education programmes: Common questions and problems. Annals of Community-Oriented Education 5:123-133. Schmidt HG et al. (1995) Theory-guided design of a rating scale for course evaluation in problem-based curricula. Teaching and Learning in Medicine 7: 82-91. Woodward CA (1992) Some reflections on evaluation of outcomes of innovative medical education programmes during the practice period. Annals of Community-Oriented Education 5:181-191. Woodward CA (1996) Problem-based learning in medical education: Developing a research agenda. Advances in Health Sciences Education 1:83-94. Woodward CA (1997) What can we learn from programme evaluation studies in medical education? In: Boud D, Feletti, G. The Challenge of Problem-Based Learning, 2nd ed. Kogan Page, London, pp.294-307. On programme management Bouhuijs PAJ (1990) The maintenance of educational innovations in medical schools. In: Nooman, ZM et al. (Eds) Innovation in Medical education: An Evaluation of its Present Status. New York: Springer Pub. pp. 175-188. Kaufman, A. 1985) Implementing Problem-Based Medical Education: Lessons from Successful Innovations. Springer Publishing Co., New York. Majoor GD & Kolle LFThM (1997) Faculty planning and control in problem-based learning. Education for Health 10: 189-197. Mennin SP, Kalishman S. (eds.) (1998 Issues and Strategies for Reform in Medical Education: Lessons from Eight Medical Schools. Academic Medicine (Supplement),73 (9). Mpofu DJS. (1999) Introducing problem-based learning into a traditional medical school Student and staff perceptions of the United Arab Emirates University’s innovation. Maastricht University Press, The Netherlands. Rossum HJM van (Ed) (2000) Curriculum 2000: The Patient-Oriented Curriculum at the University of Groningen. University of Groningen, Faculty of Medical Sciences, The Netherlands. Schwartz P, Mennin S, Webb G (2001). Problem-Based Learning: Case Studies, Experience and Practice. Kogan Press, London. On costs Mennin SP & Martinez-Burrola N (1986) The cost of problem-based vs. traditional medical education. Medical Education 20: 187-194. Donner RS & Bickley H (1990) Problem-based learning: An assessment of its feasibility and costs. Human Pathology 21: 881-885. Nieuwenhuijzen-Kruseman AC, Kolle LFJTh, Scherpbier AJJA (1997) Problem-based learning at Maastricht - An assessment of cost and outcome. Education for Health 10: 179-187. On critical appraisal and literature reviews Hemker CH (1998) Critical perceptions on problem-based learning. Advances in Health Sciences Education 3: 71-76. Albanese M (2000) Problem-based learning: Why curricula are likely to show little effect on knowledge and clinical skills. Medical Education 34: 729-738. Regan-Smith, MG, Obenshain, SS, Woodward, C, Richards, B, Zeitz, HJ, Small Jr. PA (1994) Rote learning in medical school. JAMA November 2, 1994, 272(17):1380-1381. Rideout E (2001) Transforming Nursing Education Through Problem-Based Learning. Jones and Bartlett Publishers, London. Schmidt HG, Dauphinee WD, Patel VL (1987) Comparing the effects of problem-based learning: A review of the evidence. Journal of Medical Education 62:305-315. Shin JH, Haynes RB, Johnston M. (1993) The effect of a problem-based self-directed undergraduate education on life-long-learning. Canadian Medical Association Journal 148:969-976. Vernon DTA, Blake RL (1993) Does problem-based learning work? A meta-analysis of evaluative research. Academic Medicine 68:550-563. Walton HJ, Matthews MB. (1989) Essentials of problem-based learning. Medical Education 23:542-558. Useful web sites http://www.samford.edu/pbl/pbl_main.html http://www.samford.edu/pbl/aboutus.html Project on the effectiveness of PBL http://www.hebes.mdx.ac.uk/teaching/Research/PEPBL/index.htm Centre for Instructional Support, University of Colorado http://www.uchsc.edu/CIS/PBL.html Queens University PBL site, Kingston Ontario http://meds-ss10.meds.queensu.ca/medicine/pbl/pblhome.htm Maastricht University PBL site http://www.unimaas.nl/pbl/ Australian PBL teaching & research network, University of Newcastle http://www.newcastle.edu.au/services/iesd/learndevelop/problarc/ University of Southern Illinois http://www.pbli.org/core.htm Coventry University http://heracles.coventry.ac.uk/pbl/ Regional sources of expertise (use 7 WHO regions) North America University of New Mexico School of Medicine (USA) University of Southern Illinois, School of Medicine (USA) McMaster University School of Medicine (Canada) Dalhousie University School of Medicine (Canada) University of Missouri (USA) University of California at Los Angeles (USA) Central and South America Fundação Municipal de Ensino Superior de Marilia (Brazil) Universidade Estadual de Londrina (UEL) (Brazil) Universidad Autónoma Metropolitana Xochimilco (Mexico) Africa Suez Canal University (Egypt) Australia University of Newcastle Flinders University School of Medicine, Adelaide, South Australia University of Adelaide, Adelaide, South Australia Mediterranean Arabian Gulf University (Bahrain) Dokuz Eylul University (Turkey) Pacific Ilo Ilo (Philippines) Gadjah Mada University (Indonesia) Europe University of Maastricht (the Netherlands) University of Linköping (Sweden) February 2002