A comparative evaluation of 23 projects on mental health and wellbeing for veterans and first responders

Abstract

Background: Veterans and First Responders (VFR) are at risk of developing a range of mental health disorders because of cumulative exposure to critical incidents at work. Two Philanthropic organisations funded 15 organisations, which collectively implemented 23 highly heterogeneous and international early intervention mental ill-health and suicide prevention Projects. The aim was identify and collaborate with Projects with a multi-project evaluation. The evaluation examined multiple domains including intervention effectiveness but critically the implementation processes impacts for potential replication or scale up. This paper reports on the methods and evaluation results of implementation processes, impact analysis and sustainability.

Method: The evaluation involved ecosystems and complex systems approaches using novel methods and tools. There was multiple preparatory evaluation steps including developing indices for complexity and context. The Global Impact Analytics Framework (GIAF) toolkit was used to evaluate the implementation processes. Methodological tools included qualitative analysis, descriptive statistics, GIAF ladders/scales and checklists (qualitative and quantitative data).

Results: We provide the results on characteristics (organisational, Project and participants), GIAF process components (planning, pre-engagement, pre-readiness/readiness, dissemination/diffusion, usability/sustainability, adoption and uptake). All Project interventions were assessed as usable, adoptable and have capacity to be sustained, with financial resources. Uptake of the intervention was mostly high.

Conclusion: Complex multi-project evaluation of highly heterogenous Projects implemented in the real world across different countries is possible and provides valuable information and learnings. The evaluation results establish benchmarks including Project pre-engagement with potential end-users, continuous, frequent collaboration between Project and evaluation teams, adequate contract duration for sufficient recruitment and intervention.