GiveWell’s impact is a function of the quality of our research and the amount of money we direct to our recommended charities (our “money moved”). Historically, we’ve focused mostly on research because we felt that the quality of our recommendations was a greater constraint to our impact than our money moved.
This has changed. Outreach is now a major organizational priority. The goal of this work is to increase the amount of money we direct to our top-recommended charities.
In April 2014 I wrote about our work on outreach to explain why we hadn’t prioritized it: in brief, our growth had largely been driven by inbound interest in GiveWell, and proactive outreach efforts (beyond building relationships with existing donors) hadn’t yielded results that were worth the cost.
What changed?
- We believe that the amount of money we move is now a greater constraint to our impact than additional improvements in the quality of our research. Over the last two years, we’ve added five new top charities (three of which implement programs that weren’t previously represented on our top charities list), and we expect that our top charities, collectively, will have more than $200 million in unfilled funding gaps once they’ve received the funding that we expect to direct to them. (This calculation excludes GiveDirectly, which we believe could absorb and distribute hundreds of millions of dollars.) At the same time, the quality of our research and our capacity for research is higher than it’s ever been, so the returns to adding staff there (in terms of the pace at which we identify significantly better giving opportunities) are now lower.
- Increased capacity for outreach. In our 2014 post, we wrote that one of our key constraints was that senior staff (which at the time meant primarily GiveWell Co-Founder Holden Karnofsky and me) were necessary for most outreach-related work. This has changed. We now have capacity to take on outreach work as other staff have been hired and trained on this type of work.
- Better information on the impact of GiveWell’s outreach. We now have better information about the returns to outreach because:
- We’ve collected better data (via an improved donations processing system and outreach efforts) about where donors find out about us. Because of our ability to track donors, we know that a single appearance on NPR or major podcasts tends to drive $50,000+ in annual donations.
- More time passing has demonstrated that the lifetime value of the donations of a first time donor is higher than we expected. In several cases, we’ve seen major donors (i.e., those giving $10,000-$100,000) increase their annual giving by a factor of 10 or more.
We’re in the early stages of figuring out how we can proactively invest time and money in outreach to significantly increase our money moved. For now, we’ve taken some opportunities that we think will have positive returns; these are the three that we’ve invested the most time and money in to date:
- Podcast advertising. We’ve been advertising on podcasts that we believe our target audience listens to, based on interviews with current donors and GiveWell staff. In February and March, we ran a small experiment with a few ads on FiveThirtyEight’s Politics podcast and Vox’s The Weeds.1We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post.
In total, we spent approximately $20,000 on ads for this initial experiment. We ask donors who give via our website to tell us where they learned about GiveWell when they donate. GiveWell received approximately $8,000 in donations between February 1 and November 20 from donors who reported that they had learned about us via these podcasts.
The donations we received were from first-time donors; to assess the impact of our advertising, we need to estimate the lifetime value of acquiring a new donor. In work we’ve done to assess our retention rate, we’ve seen that (a) approximately 20-25% of the donors who make a first-time donation of less than $1,000 give again in the subsequent year but (b) because many first-time donors increase the size of their donation over time, collectively, the donors who recur give more than 100% of the value of what they give in their first year.
At higher donation levels ($1,000-$100,000), we measure 40-45% retention among donors, which leads to retention of approximately two-thirds of dollars given.2I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down.
We therefore estimated the net present value of expected future donations (over the next five years) from these podcasts ads as somewhere between approximately $20,000 (assuming two-thirds dollar retention for the first two years and 100% dollar retention subsequently) and $45,000 (assuming 100% dollar retention).3We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future.
A few additional facts are worth keeping in mind about the above figures:
- We ran this experiment in February and March; most donors give at the end of the calendar year. We consistently see donors who find out about GiveWell during the course of the year, but donate in December. Other things equal, we expect that our advertising would have had greater measured returns in December than earlier in the year.
- We are only able to track donors who (a) fill out our donation form telling us where they learned about us and (b) give directly through our website rather than to our top charities. Less than 50% of donors who give via credit card (and a smaller percentage of donors who give via check) tell us where they learned about GiveWell. Also, roughly speaking, approximately 50% of the donors and dollars we influence come through GiveWell rather than going to our top charities.4I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report.
- It’s certainly possible that donors who learn about us via podcast would be more likely to give through our website than an average donor, more likely to report on how they found us (since their source is clear), or less likely to be retained. My best guess is that donors who learn about us via podcast ads behave similarly to our other donors, but I won’t be surprised if they don’t.
With all that in mind, I believe that the impact of our podcast advertising is higher than what we directly measured.
The results we saw from February to November this year were promising enough that we decided to increase the size of our experiment by spending approximately $100,000 on podcast ads. We’re currently running ads on FiveThirtyEight’s Politics podcast and Ezra Klein’s podcast and The Weeds at Vox.
- Earned media outreach. Mentions of GiveWell in the media have historically been a strong driver of growth. We aimed to increase mentions of GiveWell in high-quality, high-profile media where we’ve had the most past success as measured by dollars donated (i.e., media like The New York Times, NPR, The Wall Street Journal, and Financial Times). We retained a PR firm that came strongly recommended; we also increased 1-to-1 outreach by GiveWell staff to members of the media who have covered GiveWell in the past. It’s very hard to attribute the impact of the additional effort we’ve invested—overall, our effort has been fairly limited, and it’s hard to easily draw the causal lines between our work and the stories that appear—but my guess is that our increased efforts have led to more coverage of GiveWell and our top charities this giving season than in the recent past.
- Website improvements. Companies that sell products online invest significant effort into optimizing their websites and checkout pages to maximize their revenues. We retained a marketing consultant, Will Wong of Mission Street, and we’ve been A/B testing different donation pages and plan to test other pages on our website such as our homepage or top charities page to see whether we can increase our conversion rate (i.e., the percentage of visitors to our website who give to one of our top charities). For context, our current conversion rate is 1%. Our understanding is that a standard conversion rate for e-commerce companies is 2%, and that international nonprofits have a similar conversion rate.5See Pg 51 of the study downloadable here. An increase in our conversion rate to the industry average would lead to a significant increase in the amount of money we direct to our top charities.
Notes
| 1. | ↑ | We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post. |
| 2. | ↑ | I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down. |
| 3. | ↑ | We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future. |
| 4. | ↑ | I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report. |
| 5. | ↑ | See Pg 51 of the study downloadable here. |
The post Update on our work on outreach appeared first on The GiveWell Blog.
from The GiveWell Blog http://ift.tt/2kMridx
0 comments:
Post a Comment