Measuring the Impact of Educational Interventions: A Quantitative Approach
This chapter will discuss impact evaluation, an important method of measuring the effectiveness of an educational intervention. This form of evaluation represents a subset of program evaluation and focuses on outcomes and consequential events related to an educational intervention. In doing so, it incorporates several different quantitative methods and is typically reserved for stable, long-standing educational programs/curricula. Many of these methods are also used as part of program evaluation as a whole and in surgical research. Readers are directed to Chaps. 23 (“Demystifying Program Evaluation for Surgical Education”, Battista et al.) and 30 (“Researching in Surgical Education: An Orientation”, Ajjawi and McIllhenny) for more information on these subjects. In addition to providing a working definition of impact evaluation, this chapter will help define key concepts related to its successful use as well as aid in delineating the most useful quantitative methods to employ.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Subscribe and save
Springer+ Basic
€32.70 /Month
- Get 10 units per month
- Download Article/Chapter or eBook
- 1 Unit = 1 Article or 1 Chapter
- Cancel anytime
Buy Now
Price includes VAT (France)
eBook EUR 139.09 Price includes VAT (France)
Hardcover Book EUR 179.34 Price includes VAT (France)
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Promoting Excellence in Surgical Educational Research: A Review of Research Methods
Chapter © 2019
Assessing the Impact of Educational Programmes: An Evaluation of Research Validity
Chapter © 2014
Evaluation Paradigms
Chapter © 2023
References
- Patton, M. Q. (1997). Utilization-focused evaluation (4th ed.). Thousand Oaks: Sage Publications. Google Scholar
- Owen, J. M. (2006). Program evaluation: Forms and approaches (3rd ed.). Crows Nest: Allen and Unwin. Google Scholar
- Fudickar, A., et al. (2012). The effect of the WHO Surgical Safety Checklist on complication rate and communication. Deutsches Ärzteblatt International, 109(42), 695–701. Google Scholar
- Evers, U., et al. (2013). ‘Get your life back’: Process and impact evaluation of an asthma social marketing campaign targeting older adults. BMC Public Health, 13, 759–768. ArticleGoogle Scholar
- Tavakol, M., & Sanders, J. (2014). Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part I. Medical Teacher, 36(9), 746–756. ArticleGoogle Scholar
- Papaconstantinou, H. T., et al. (2013). Implementation of a surgical safety checklist: Impact on surgical team perspectives. The Oschner Journal, 13, 299–309. Google Scholar
- Seymour, N. E., et al. (2002). Virtual reality training improves operating room performance. Results of a randomized, double-blinded study. Annals of Surgery, 236(4), 458–464. ArticleGoogle Scholar
- Anastakis, D. J., et al. (1999). Assessment of technical skills transfer from the bench training model to the human model. American Journal of Surgery, 177(2), 167–170. ArticleGoogle Scholar
- Moulton, C. E., et al. (2006). Teaching surgical skills: What kind of practice makes perfect? A randomized, controlled trial. Annals of Surgery, 244(3), 400–409. Google Scholar
- Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings (1st ed.). Chicago: Rand McNally. Google Scholar
- Martling, A. L., et al. (2000). Effect of a surgical training programme on outcome of rectal cancer in the County of Stockholm. Lancet, 356(9224), 93–96. ArticleGoogle Scholar
- Rosser, J. C., et al. (2007). The impact of video games on training surgeons in the 21st century. Archives of Surgery, 142, 181–186. ArticleGoogle Scholar
- Tavakol, M., & Sanders, J. (2014). Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part II. Medical Teacher, 36(10), 838–848. ArticleGoogle Scholar
- Artino, A. R., et al. (2014). Developing questionnaires for educational research: AMEE Guide No.87. Medical Teacher, 36(6), 463–474. ArticleGoogle Scholar
- DeVellis, R. F. (2014). Scale development: Theory and applications (2nd ed.). Newbury Park: Sage Publications. Google Scholar
- Dillman, D., et al. (2009). Internet, mail and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken: Wiley. Google Scholar
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Google Scholar
- American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Google Scholar
- Yule, S., et al. (2008). Surgeons’ non-technical skills in the operating room: Reliability testing of the NOTSS behaviour rating system. World Journal of Surgery, 32, 548–556. ArticleGoogle Scholar
- Regher, G., et al. (1998). Comparing psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Academic Medicine, 73(9), 993–997. ArticleGoogle Scholar
- Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119, 166.e7–166.e16. ArticleGoogle Scholar
- Hojat, M., et al. (2002). Physician empathy: Definition, components, measurement and relationship to gender and speciality. American Journal of Psychiatry, 159(90), 1563–1569. ArticleGoogle Scholar
- Cook, D. A., & Hatala, R. (2016). Validation of educational assessments: A primer for simulation and beyond. Advances in Simulation, 1, 31. ArticleGoogle Scholar
- Martin, J. A., et al. (1997). Objective structured assessment of technical skill (OSATS) for surgical residents. British Journal of Surgery, 84, 273–278. ArticleGoogle Scholar
- Reznick, R., et al. (1997). Testing technical skill via an innovative ‘bench station’ examination. American Journal of Surgery, 173(3), 226–230. ArticleGoogle Scholar
Author information
Authors and Affiliations
- Medical Student Programs, Eastern Health Clinical School, Box Hill, Australia Jenepher A. Martin
- Faculty of Medicine Nursing and Health Sciences, Monash University, Clayton, Australia Jenepher A. Martin
- School of Medicine, Deakin University, Melbourne, Australia Jenepher A. Martin
- Jenepher A. Martin