Measuring Program Success
A couple of months ago I had a conversation with a colleague who questioned how can we measure the success of public outreach programs in archaeology. Specifically, she asked “How do we know that one shot archaeology week/month events are meaningful to the participants the day after? How can we tell if the events are successful?”
I have thought about her questions more of late. How do we measure success? At the C.H. Nash Museum, this question was particularly relevant as we completed our recent Museum Assessment Program sponsored by the American Association of Museums.
Here are some thoughts on measuring program success:
- Our staff concluded that attendance numbers and revenue dollars should not be the primary measure of success. It’s nice to see income come closer to offsetting our expenses but as a small nonprofit, we know that even doubling our attendance is not going to allow us to break even.
- Rather, we are thinking of ways to measure how well our programs and outreach align with our Mission Statement. The alignment is not readily measured in dollars and cents and attendance numbers. We also expect that to the extent we demonstrate an aggressive alignment with our mission, donor giving will increase.
- Educational and outreach opportunities are central to our Mission Statement. Schemes of pre and post testing visitors knowledge might not be a good gauge of success – particularly for our adult visitors. A recent article in the Boston Globe reported that the simple presentation of facts is not an effective means to change an individual’s perspective on an issue. But the article revealed an interesting point – the testing involved presumed experts telling people their assumptions were in error. The article did not discuss instances when individuals participated in a process of discovery about alternative explanations for a set of phenomena. The process described in the article is like standing up in front of a town meeting and saying that the beloved founder was a real scoundrel. An alternative and more engaged approach might be to provide the town folks copies of the beloved founder’s diaries, testimony from his/her spouse, etc. etc and let the citizens decide if he/she was a scoundrel – an engaged participatory approach.
- Specific to the Archaeology Week/Month event, here is an example – in years past I gave Archaeology Month presentations in Belzoni, Mississippi, location of the prehistoric Poverty Point Culture’s Jaketown site. The bulk of the site remains in private hands and is routinely collected by avocational archaeologists. In my presentations, I talked about the distributional patterns of artifact types at other Poverty Point culture sites. Attendees heads nodded in agreement about such patterns at Jaketown as well. After the talk, several of the collectors would tell me about the patterns they noted. We visited a bit, then I packed up my slides and drove the several hours back home to northeast Louisiana. I went back the next year for Archaeology Month with the same general story, resulting in more nodding heads, and more talks and visiting after the presentation – but despite my suggestions, none of the collectors thought to actually record the location of their newly collected materials. In hindsight, it’s probably the height of professional arrogance to think that a once a year preaching the Archaeology Gospel should result in everyone being saved.
- This comes full circle back to the conversation with my colleague on how to measure success. First, such changes likely are measurable only over extended periods of time. Perhaps the point is that we really should not expect more from Archaeology Month/Week events than success as measured by attendance data. Perhaps the real success is the extent that we use the Archaeology Day event as an opportunity to begin building a relationship with folks on an engaged and long-term basis where the less tangible measures can be made.
- We seem to have a choice – if we are expecting more substantive successes, then we likely will need to begin investing in more long-term strategies and commitments. For example, at the C.H. Nash Museum, we have gone from 5-10 people showing up for our monthly Volunteer Day events to over 40 this past month. We can talk about our short supply of staff and competing priorities, but building engaged relationships with these volunteers flows directly through our Mission Statement. To do otherwise is counter to our mission. The more sustained engagement today with the 40 is where we will develop the stakeholders who down the road will become advocates and participate in the Museum’s Mission.
- So, in addition to attendance and revenue numbers, are good measures of success the number of volunteer hours over time, feedbacks in social media, number of letters written to elected officials on cultural resource preservation, and so forth?
How do you measure your success in public outreach?