Filling an Evaluation Gap: A Framework for Evaluating Community Health Worker Programs

Public health practitioners can use this flexible framework to guide the evaluation of the process and outcomes of a Community Health Worker (CHW) program.

Facilitating evaluation of community health worker programs

Evaluation is key to improving public health programs and maximizing program effectiveness. Public health professionals, however, do not always have the time, skills, or resources to design and conduct program evaluations, despite knowing that evaluation is important. Short-term funding cycles, understaffing, making sure programs get off the ground, and meeting clients’ needs often take precedence. This is especially the case for community health worker (CHW) programs. In our JPHMP article, “The More We Know, the More We’re Able to Help”: Participatory Development of an Evaluation Framework for Community Health Worker Programs, my colleagues – Mya Sherman and Maureen Lichtveld – and I describe the common components of CHW programs in community-based and clinical settings. We provide public health practitioners a flexible framework to guide the evaluation of the process and outcomes of a CHW program. Utilizing the framework will reduce the time needed to plan and ramp up an evaluation, and hopefully encourage evaluation in the first place. It also allows for tailoring data collection to program context.

Why evaluate CHW programs?

Among their many roles, CHWs play a critical role in reducing health disparities. They facilitate access to and enhance utilization of health care in underserved and under resourced communities. Evaluating CHW programs to understand how they can be improved or made more effective is thus important for achieving health equity. Evaluation also promotes accountability and continuous learning.

Our project

The development of the evaluation framework is rooted in a project we undertook in the aftermath of the 2010 Deepwater Horizon oil spill in the Gulf of Mexico. We implemented a community health worker (CHW) program in the coastal regions of four oil spill-impacted states (Louisiana, Mississippi, Alabama and Florida). Eighteen federally qualified health centers and community-based organizations were funded for four years to establish CHW programs. Our work was part of the Gulf Region Health Outreach Program (GRHOP), which sought to strengthen the region’s fragile public health infrastructure.

Developing the framework

With backgrounds in both public health practice and academic research and evaluation, we were eager to evaluate the GRHOP CHW program and to contribute to GRHOP’s enterprise evaluation. We learned that there was no one existing framework that could guide our evaluation of the funded CHW programs. We realized, though, that we had a unique opportunity to develop and pilot such a framework in collaboration with the CHWs and supervisors funded by the program.

Developing the framework relied on a qualitative research approach. Mya Sherman drew out the rich experiences of the CHWs and supervisors through semi-structured interviews and analysis of the transcripts. This work led to determining the framework’s evaluation categories: client relations, intra-organizational relations, inter-organizational relations, capacity development, program effectiveness, cost efficiency, and sustainability. I thematically analyzed surveys piloting the framework with the funded organization to identify best practices in CHW program implementation for each evaluation category. CHW programs can use our pilot survey and the indicators we wrote for their own evaluations. Both documents are available as supplementary digital content to our article.

Developing the framework took us about two years (not including the time needed to write and publish it). No wonder that program evaluation is daunting for frontline public health professionals! Our project, however, demonstrates a successful collaboration between researchers and practitioners for public health program evaluation. I would recommend that when warranted, public health practitioners based outside academia should reach out to researchers to form evaluation partnerships, or vice versa. We can advance public health program evaluation together.

Recommended Reading:

You May Also Like:

Author Profile

Hannah Covert
Hannah Covert, PhD, is a Research Assistant Professor in the Department of Environmental and Occupational Health at the University of Pittsburgh School of Public Health. She provides scientific and administrative leadership for research and capacity building portfolio focused on building resilience in communities facing environmental health threats and long-standing health disparities. Previously, she held a similar position at Tulane University. Her expertise is in environmental health literacy, climate and health, and community health worker programs. Hannah received her MA and PhD from the University of Florida and her BA from Middlebury College.