How to measure the effectiveness of GiveBig and other “day of giving” campaigns?

Today, May 15, is GiveBig, Seattle’s third annual “day of giving” event. Created by the Seattle Foundation in 2011, the idea is to focus attention on charitable giving, raise the public profile of the Seattle Foundation and of course raise some dough. There are similar events in many other cities now, and even a national “GivingTuesday” event right after Thanksgiving.

But how do we know whether GiveBig and similar day of giving type events are really working?

Seattle Foundation proudly trumpets the total number of dollars raised, which showed impressive growth from 2011 to 2012, and I’m sure will notch another nice gain this year. And of course, there are a nice slew of media hits to show for it, and I’m sure the Seattle Foundation website is seeing a delicious spike in traffic. All of these are good things. Nonprofits are usually below the radar unless there’s a scandal, and so it’s nice to break through with something positive.

And yet… shouldn’t the first and foremost metric of success for GiveBig be whether it can prove that participating in GiveBig causes nonprofits to raise more money than they would have otherwise?  Sure, it’s great that GiveBig raised $X million in a single day. But how much of that money was just “time-shifted” from other days in the year, and how much is genuinely “new?” This is a tricky question, but I think it really matters. GiveBig is a big investment for the Seattle Foundation, and participating groups spend a lot of time and energy on it.  And so I think it’s important that big investments in fundraising be able to show rigorously measured bottom-line results.

But how? I’ve been doing some thinking about this. It’s a tough problem, and I don’t presume to have solved it. But I wanted to put my ideas out there to start the conversation, and draw in smarter people than me.

Here’s the best approach I can come up with: Create two lists of Seattle nonprofits: ones that actively participated in GiveBig and ones that didn’t. Then, for each set of groups, look at six or seven years of total individual giving numbers from a large group of Seattle nonprofits, let’s say from 2006 – 2012. First look at the data from 2006-2010 (before GiveBig started). Find pairs of groups who had similar individual giving numbers over those years, and showed similar rates of growth. Then, track those pairs of groups for 2011-2012 and see if their individual giving totals diverge. Do groups that were performing similarly before GiveBig started suddenly show differences in fundraising performance?

To be sure, this is not a perfect approach. There could be many reasons why groups’ fundraising performance would diverge. ED or fundraising staff transitions, death of a major donor, etc. But if we have enough pairs of groups, these “random” errors should cancel each other out. Perhaps we won’t be able to match enough pairs of similar groups–I’m guessing that a lot of groups now participate, and relatively few don’t.

Smart friends: how would you approach evaluating whether GiveBig has an impact on total fundraising performance? Can you think of an experimental design that is simpler or better? I’d love to hear your thoughts.

Update: Upon some further reflection, I also think that tracking the number of new donors coming in through GiveBig would be an meaningful indicator of success–and even more valuable if we could get some numbers for the multi-year value of these new donors compared with donors acquired through other sources.

9 thoughts on “How to measure the effectiveness of GiveBig and other “day of giving” campaigns?”

  1. The other thing to keep in mind is that even if they did not result in any net gain in fundraising, there is value to nonprofits in shifting revenue to late spring from end of year. Helps both with cash flow and having more budgetary certainty.

  2. Jon – do you have the stats on how much was raised in 2011 and 2012? Also, what was the “stretch pot” in each of those years?

  3. I’m not sure the measure should be by group. I think there should be some way to poll individuals and see how effective the campaign was. Each group’s outreach can be muddled by the work done by the Seattle Foundation plus what was done on social media. I think the question is to a select group of individuals is “What did you hear/see”, “What made you give or not give?”, “Did you give today as a special gift?” and so on. I think that should be the approach, but I’m not sure how. Also some analysis if this move them up the engagement ladder.

  4. I’d enjoy checking the lists of overall donations. They would split into three lists–the folks who give separately to an NP and don’t use GiveBig, those who use GiveBig as a slide donation, and those who are new to giving to that NP because of GiveBig. Over three to five years watching that trend would be informative. A real wonk would want to compare this data pile to a similar United Way list, because I know several UWay people who smile grittily when GiveBig is mentioned, say something vaguely supportive, and gripe later that donations are just sliding from UWay over to GiveBig. It would be interesting to test that theory.

  5. Wouldn’t most of that data be in 990 forms already? Of course, the vagaries of the way those are completed may make it hard.

    How about this? If the Seattle Foundation (or the United Way) really want to know – how about offering a cash gift – a pretty good sized one – to all Puget Sound nonprofits? The price of entry would be entering data (which would be confidential, natch) into a tool or spreadsheet. And then we’d have the data to determine the overall effectiveness (if “more money” was the goal).

  6. Super interesting idea, Sophia!

    Patrick: yes, I think that in theory, some of the data would be in 990s, but as you say, it is often reported inconsistently, and of course there is a huge time delay. I think you are right that there would have to be some clear incentive for groups to cough up data–it could actually be a condition of GiveBig participation (for participants) and perhaps there could be some sort of cash reward for non-participating groups.

    Carol: I got a similar comment offline as well. While I think that would be interesting, too, it would be really hard to get individual donor data like that. Also, it would be really hard to establish a solid control group–people self-select into GiveBig participation. I’m most interested in the overall question: “Do day of giving events change the overall annual giving results from what they would have been otherwise?” That said, it would be really interesting to try to design an experiment in which a large group promoted GiveBig to a random half (or three quarters) of its members, but deliberately withheld from marketing to the remainder. Sure, there would be some “leakage” (e.g. marketing from other groups, or ambient social media) but there might still be some detectible effect.

  7. Oh, my comment was in the spirit of “wouldn’t it be nice”, and not “I really expect people would be willing”. Given the paucity of bucks NPs are dealing with right now, no one would intentionally cut their chances of getting even a penny less than they could by playing experimental chess with funding. The only way to make such a plan actually happen would indeed be to require the overall % lists as a condition of receiving the money. The lists would have to be general percentages and not specific donor names, or a number of NPs would probably be violating their own privacy policies. But, good to know we are all wonks!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Subscribe without commenting