Evaluation of HOTSPOTS system
Evaluation of HOTSPOTS using an Implementation Fidelity Method: Learning lessons for deployment in other settings
Category: Mental Wellbeing Innovation
Target audience: Medical students and clinical leaders on workplace training sites
Research innovator: Fiona Moir
The project
HOTSPOTS is a system developed to tackle workplace bullying, discrimination and harassment (BDH) experienced or witnessed by University of Auckland medical students, whilst on their workplace training sites.
It is an anonymous group reporting system, which involves all clinical students routinely completing a 6-monthly questionnaire where they allocate scores to each work placement. Students each submit 4 scores, for bullying, harassment, discrimination and lastly, inclusion and respect of students during the placement. All scores from all students are aggregated and compared, and any placements which stand out as outliers (with either excellent or concerning scores) are noted.
Medical school leaders make tailored reports for clinical leaders - one report for each Chief Medical Officer, and one for each University Head of Department. Meetings are arranged by medical school staff to discuss the reports, and leaders are asked about possible causes for the data, plus an action plan. The medical school writes a de-identified summary of all actions taken which is available to students and staff. HOTSPOTS won a national health and safety award in 2022.
Project Background
Implementation fidelity refers to the degree to which an intervention or programme is delivered as intended.
Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved. [1]
With proven success in the initial application of HOTSPOTS, Fiona and her team wanted to explore using HOTSPOTS in other settings. This research involved describing the experiences of the students and leaders who had used HOTSPOTS, along with documenting the patterns of HOTSPOTS response rates and data, and outlining actions taken by leaders to address problems experienced by students on clinical placements.
These research findings showed that the clinical leaders had a good understanding of how the HOTSPOTS system worked, and found the system fit for purpose. Students also appreciated the system, but there appeared to be confusion and uncertainty amongst parts of the student body as to aspects of the system's processes.
From this, Fiona and her team developed a HOTSPOTS implementation manual which will provide guidance for transferring the HOTSPOTS system to other institutions and workplace settings in the future.
Ember Innovations research funding impact
Ember’s research grant funding for this project is what enabled the research to go ahead. The findings from the research provided information that was used to develop the next project, which trialed moving HOTSPOTS to a new setting. The team also received funding for this second project.
The implementation manual, developed as a direct result of the Ember funding, enabled Fiona and her team to successfully transfer HOTSPOTS into the University of Auckland School of Nursing, benefitting both students and staff.
[Ember research grants] are vitally important, as they encourage researchers to 'think outside the box' and to develop new ways of thinking and addressing issues. As mental health problems affect so many of us, there is a huge potential for this type of research to make a meaningful impact in the lives of many people.
—Fiona Moir
Related publications and resources
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation science : IS, 2, 40. https://doi.org/10.1186/1748-5908-2-40