Public health practitioners can use this flexible framework to guide the evaluation of the process and outcomes of a Community Health Worker (CHW) program.

Facilitating evaluation of community health worker programs

Evaluation is key to improving public health programs and maximizing program effectiveness. Public health professionals, however, do not always have the time, skills, or resources to design and conduct program evaluations, despite knowing that evaluation is important. Short-term funding cycles, understaffing, making sure programs get off the ground, and meeting clients’ needs often take precedence. This is especially the case for community health worker (CHW) programs. In our JPHMP article, “The More We Know, the More We’re Able to Help”: Participatory Development of an Evaluation Framework for Community Health Worker Programs, my colleagues – Mya Sherman and Maureen Lichtveld – and I describe the common components of CHW programs in community-based and clinical settings. We provide public health practitioners a flexible framework to guide the evaluation of the process and outcomes of a CHW program. Utilizing the framework will reduce the time needed to plan and ramp up an evaluation, and hopefully encourage evaluation in the first place. It also allows for tailoring data collection to program context.

Why evaluate CHW programs?

Among their many roles, CHWs play a critical role in reducing health disparities. They facilitate access to and enhance utilization of health care in underserved and under resourced communities. Evaluating CHW programs to understand how they can be improved or made more effective is thus important for achieving health equity. Evaluation also promotes accountability and continuous learning.

Our project

The development of the evaluation framework is rooted in a project we undertook in the aftermath of the 2010 Deepwater Horizon oil spill in the Gulf of Mexico. We implemented a community health worker (CHW) program in the coastal regions of four oil spill-impacted states (Louisiana, Mississippi, Alabama and Florida). Eighteen federally qualified health centers and community-based organizations were funded for four years to establish CHW programs. Our work was part of the Gulf Region Health Outreach Program (GRHOP), which sought to strengthen the region’s fragile public health infrastructure.

Developing the framework

With backgrounds in both public health practice and academic research and evaluation, we were eager to evaluate the GRHOP CHW program and to contribute to GRHOP’s enterprise evaluation. We learned that there was no one existing framework that could guide our evaluation of the funded CHW programs. We realized, though, that we had a unique opportunity to develop and pilot such a framework in collaboration with the CHWs and supervisors funded by the program.

Developing the framework relied on a qualitative research approach. Mya Sherman drew out the rich experiences of the CHWs and supervisors through semi-structured interviews and analysis of the transcripts. This work led to determining the framework’s evaluation categories: client relations, intra-organizational relations, inter-organizational relations, capacity development, program effectiveness, cost efficiency, and sustainability. I thematically analyzed surveys piloting the framework with the funded organization to identify best practices in CHW program implementation for each evaluation category. CHW programs can use our pilot survey and the indicators we wrote for their own evaluations. Both documents are available as supplementary digital content to our article.

Developing the framework took us about two years (not including the time needed to write and publish it). No wonder that program evaluation is daunting for frontline public health professionals! Our project, however, demonstrates a successful collaboration between researchers and practitioners for public health program evaluation. I would recommend that when warranted, public health practitioners based outside academia should reach out to researchers to form evaluation partnerships, or vice versa. We can advance public health program evaluation together.

Recommended Reading:

• Community Health Worker Initiatives: An Approach to Design and Measurement
• Innovative Methods for Designing Actionable Program Evaluation
• CDC Program Evaluation and Performance Office
• Community Toolbox, Chapter 36 Introduction to Evaluation
• Core competencies and a workforce framework for community health workers: A model for advancing the profession

You May Also Like:

• The Impact of Four State-Specific MBHCPs and Lessons Learned from a Multistate Collaborative Effort by Jennifer Langhinrichsen-Rohling
• Authors Discuss the Impact of the Primary Care Capacity Project by Heather Farb and Jamie Giepert
• Infographic: Collectively Enhancing Mental and Behavioral Health Capacity Throughout the Gulf Coast by Jennifer Langhinrichsen-Rohling
• The Gulf Region Health Outreach Program (GRHOP) and a Public Health Mantra by Maureen Lichtveld and Bernard Goldstein
• Gulf Region Health Outreach Program (GRHOP) Stands Out as an Outlier in Public Health Funding Support — Make Your Program One, Too! by Les Beitsch
• 15 Things You Need to Know About the Gulf Region Health Outreach Program (GRHOP) by Ayanna V. Buckner