Blog

For museums to have meaningful impacts on the cultural problem of climate change, they will need to shift from a focus on outputs to outcomes. (Douglas Worts commenting on Joy Davis’ recent post Agentic Professionals: How individual museum workers can champion climate justice)

So what might a shift to outcomes-based program evaluation look like?

In 2013 I* had the privilege of working with the Ontario Museum Association in association with the Ontario Ministry of Tourism, Culture and Sport to develop an Outcomes-Based Program Planning & Evaluation Webinar and Workbook. These materials are online and available free of charge. Much of the following post is drawn from these resources.

What is Outcomes-Based Planning & Evaluation (OBPE)?

Outcomes-Based Program Evaluation is a systematic way to plan user-centered programs and to measure whether they have achieved their goals. OBPE goes beyond documenting what you did and measures what difference you made in the life of your audience. How has your audience changed?

Why OBPE for the Coalition?

The mission of the Coalition of Museums for Climate Justice is to mobilize and support Canadian museum workers and their organizations in building public awareness, mitigation and resilience in response to climate change. Ultimately, we hope the work of the Coalition will have an impact not only on museum workers and their organizations but also on their publics, the citizens of Canada and the world.

Planning for and evaluating the outcomes of museum programs and exhibits helps us to focus on and document the changes we want to see – public awareness, mitigation and resilience in response to climate change.

Being able to express and demonstrate what difference museums’ climate justice programs and exhibits might and do make is invaluable. Why?

1. Funds for museums are decreasing.

2. Community needs are increasing.
   

Previous evaluation measures focused on money, numbers of people served and on client satisfaction . These are outputs. These measures don’t really assess the impact of programs and exhibits on museum & cultural heritage audiences. We need to know if our efforts are making a difference in the lives of our citizens – and, if they are, to be able to share those results with others in a meaningful way. We need to know the outcomes and impacts.

For an example of one museum doing just this, see National Museums Liverpool – Making a Difference

So what are the components of OBPE?

Inputs – The materials and resources that the program uses in its activities. These are often easy to identify and are common to many organizations and programs. For example: equipment, staff, facilities, standards (Standards for Community Museums in Ontario), information, etc.

Activities – These are the tasks, or processes, that the program undertakes with/to the audience in order to meet the audience’s needs; for example, recruiting participants, promoting the program, networking, training. Note that when identifying the activities in a program, the focus is still pretty much on the organization or program itself, not on changes in the audience.

Outputs – These are the most immediate direct products or results of your program. Program outputs are typically measured in numbers. For example, the number of  … people who participated, classes who completed a program, materials developed, workshops given, supplies consumed, web site hits, etc. Outputs create the potential for outcomes to occur

Outcomes – These are changes that occur to people, organizations and communities as a result of your program. These changes, or outcomes, are usually expressed in terms of:

• Knowledge and skills
• Behaviours
• Values, conditions and status

Impact – This describes your vision of a preferred future and underlines why the project is important. It refers to the longer-term change that you hope your project will help create or to which your project may contribute.

Splash & Ripple

Imagine your program is like a rock thrown into a pond. Your program makes a splash. The different outcomes measured are the ripples.

Inputs are like the person & the rock.

Activities are like dropping the rock.

Outputs are like creating the splash.

The ripples spreading from the splash are like your Outcomes,

that later become your Impacts.

The edge of the pond represents the Situation.

(Plan:Net Limited & Stratecona Research Group 2008)

In my next post, I’ll focus on measuring outcomes.

In the meantime, check out these Free Resources …

Betts, Tracy. “The Nonprofit Social Media Decision Guide.” idealware. idealware. October 2011.

Diamond, Judy, Jessica J. Luke, and David H. Uttal. Practical Evaluation Guide: Tool for Museums and Other Informal Educational Settings. Second. Lanham: AltaMira Press, 2009.

Indiana University IUPUI & Institute for Museum & Library Services. What is Outcomes Based Planning & Evaluation? IMLS. 2006-2010.

McNamara, Carter. Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources.

Museums Association of Saskatchewan. Evaluation Workbook: A Basic Understanding of the Program Outcome Evaluation Model. Regina: Museums Association of Saskatchewan, 1999.

Patton, Michael Quinn. Utilization-Focused Evaluation: The New Century Text. 3rd. Beverly Hills: Sage Publications, 1997.

Plan:Net Limited & Stratecona Research Group. “Splash & Ripple: Using Outcomes to Design & Manage Community Activities.” Plan:Net. PLAN:NET LIMITED. 2008.

Plan:Net Limited. “Splash & Ripple.” Health Canada. n.d.

Thomson, Gareth, and Jenn Hoffman. “Measuring the Success of Environmental Education Programs.” Canadian Parks & Wilderness Society.

*I am M. Christine Castle, PhD, a Canadian museum educator and consultant and a member of the Coalition of Museums for Climate Justice Advisory Group. More info on me here.