Evaluation

Evaluation is the process of examining if a program is meeting its goals, how it can improve, and what impact it has. It provides the evidence programs need to make informed decisions and demonstrates efficacy to program funders. Good evaluation supports accountability, improvement, and learning and is beneficial to program leadership and program participants.

As a postdoc at RIT with the PEER Institute, I learned about a valuable distinction between research and evaluation (see Eleanor Sayre's Research: A Practical Handbook). While research aims to generate new general knowledge for a scientific community, evaluation generates new local knowledge for a program’s stakeholders. As a STEM education researcher, promoting change can be a slow and diffusive process that can have large-scale impacts. As a program evaluator, I promote small-scale changes that may have a cumulative effect. I like to work at both of these scales!

I evaluate participants' and leaderships' experiences with an educational program, faculty/postdoc professional development, and educational interventions. This can be formative and/or summative. I'm open to other opportunities as well!

Interested in an external evaluator for your project? Contact me!

Topics

My focus has been in educational programs, but those vary in discipline and scale. Most recently, I've conducted evaluations of a scholarship program for pre-service teachers and a dual-credit high school program. I've been written into several grants to serve as an evaluator, including an REU and a faculty professional development program.

Activities

I regularly meet with project leadership to provide findings and receive program updates. Depending on the scale of the project and the evaluation goals, I usually provide formative feedback in 1-2 small reports throughout the project and one summative report at the end of the project.

Methods

I use semi-structured interviews, focus groups, surveys, and in-situ observations, and document review as the main tools for evaluation. In some evaluations, I use all of these methods; in others, I use a few depending on the evaluation questions and goals.

Why use an external evaluator?

Evaluation comes in a number of variations --- informal to formal, internal to external, and formative to summative (see Eleanor Sayre's Research: A Practical Handbook). All projects need evaluation, even research projects! It's a process that determines if the program is meeting its goals and what could be improved. This often happens informally and internally (e.g., "Am I answering my research question?", "Do participants like the program?", "How can we increase participation?").

External evaluation provides a new, independent perspective that ensures a project's implementation and impact is aligned with its goals. Hiring an external evaluator can provide insights that internal evaluations may overlook, bring expertise in conducting rigorous investigations of program efficacy, and build trust with stakeholders by providing a third-party to share their unfiltered program experiences with.

Additionally, many funders, including NSF and other federal agencies, require or strongly encourage external evaluation as a part of grant-funded projects. Having an external evaluator on your team demonstrates your commitment to accountability and improvement.