After 10 years of wear, I finally had to admit that my favorite Merrell clogs needed replacing. So I went online to Zappos.com to find a new pair. I am a big fan of Zappos, and not just because they have flattered me with a VIP membership. I can sort by size, color, brand, even heel height, to narrow down the 1,601 clog choices to a few.
But my favorite feature is sorting by customer rating to find the shoes other shoppers liked the best.
After about 15 minutes of shopping, I decided to go with a pair of Clarks because, in addition to the 5-star rating they received, one of the customers is a nurse and said they are comfortable all day long.
I have all that information on an $80 purchase.
But what if, instead of shoes, I wanted to invest $80, or $80,000, in a charity working in international aid? What information is available to me? Not nearly enough, in my opinion.
A lot of donors rely on the information provided on sites like Charity Navigator. Charity Navigator’s main goal is to inform potential donors about the financial health and “efficiency” of a charity. The efficiency rating comes from financial data in the annual tax forms most US nonprofits have to file every year with the federal government.
While the Charity Navigator ratings may tell you what percent of your donation goes to fundraising or how much the charity’s CEO gets paid, it will not tell you anything about what the charity actually accomplishes in the field. That’s why the Central Asia Institute, co- founded by Three Cups of Tea author, Greg Mortenson, had the highest (4-star) rating from Charity Navigator until media attention exposed questionable spending and alleged exaggerated accomplishments.
Hundreds of stories were written about this particular scandal, but there are many more less extreme examples of organizations out there who may look financially healthy and efficient, but actually have poor to mediocre programs in the field.
I was recently asked by Philanthropedia (owned by Guidestar) to be part of an expert opinion poll on which water and sanitation organizations are the best. I declined to participate in this crowd-sourcing exercise, because I don’t think it’s fair or accurate to evaluate an organization based on what I’ve heard or read about them. I base my opinions on actual field analysis. While I appreciate that they are trying to provide a more complete picture of a nonprofit’s activities than Charity Navigator, the result is what I would characterize as a popularity contest, not an independent assessment of a charity’s work.
Even Zappos reviews come from people who have actually worn the shoes.
Most donors ignore these rating systems and base their donations on information provided by the charity themselves. That’s the case for the majority of the supporters of the organization I work for, Water 1st. And while I believe Water 1st’s self-monitoring program works wells and fosters an environment of continuous program improvement, I realize that we are essentially filling out our own report card.
So, why shouldn’t you trust what charities report themselves? Well, I certainly can’t speak for all international aid, but I do have over 20 years experience evaluating global water and sanitation projects and here is what all donors to this sector should know:
It’s really hard to do a water and toilet project. It is estimated that 35-50% of projects stop functioning only two to five years after completion. There are many reasons that water projects fail, and none of them are that organizations who implement projects are not well-intentioned. Usually it’s because communities have not or cannot assume responsibility for maintenance and repairs.
Project follow-up is almost non-existent. I believe the underlying reason for the failure of water projects is that organizations that implement them are not investing in their own learning. While there is little information out there on how many organizations actually visit their projects after the ribbon-cutting ceremonies, expert opinion is that less than 5% of projects are visited post-construction and less than 1% receive any long-term monitoring. Follow-up allows organizations to improve the planning and execution of future projects, take any corrective action necessary to improve the functioning and impact of existing projects, and build a knowledge base of experience. While GPS technology and Google maps are very cool fundraising tools to connect donors with specific projects that they cannot visit themselves, placing a GPS point on a map at project completion does not count as monitoring and evaluation.
These failures mean that billions of dollars are wasted each year, forcing women and girls to return to the drudgery of carrying water from distant, contaminated sources. If my new shoes aren’t comfortable, I will simply send them back. But, poor people don’t have the luxury of the Zappos return policy – they rarely get a second chance at a major development project.
Nothing I’ve written here is news to those of us in the international water and sanitation sector. We have been quoting these same statistics and challenges for the past 30 years. Intellectually we all understand the need for better follow-up, but other activities – like fundraising – take priority.
I believe that until demonstrating projects are long-term successes impacts an organization’s bottom line, implementers of water and sanitation projects will continue business as usual.
In December, Water 1st piloted this idea of an independent audit of water and sanitation project outcomes in Honduras. We hope it will capture the attention of donors who are interested in having more information on which to make their philanthropic decisions.
About $10 billion is spent annually on water and sanitation projects. If half of these projects are failing, the potential of independent audits in this sector alone is enormous, in terms of steering donations to the best programs, and also incentivizing organizations to improve their performance.
The rating system Water 1st envisions for these independent audits is a little more complicated than the one provided by Zappos. But we would like the results to be as easy to find and interpret by donors. We can also imagine independent outcome-based rating systems being useful in other sectors, helping compassionate donors to make informed decisions about their support of programs to improve education or healthcare (and preventing organizations from making false or exaggerated claims).
Ultimately, we want all the money spent on international aid to be spent wisely. After all, shouldn’t development projects for the world’s poorest last at least as long as my Merrells?
Marla Smith-Nilson is the founder and Executive Director of Water 1st, a Seattle-based non-profit devoted to helping the world’s poor meet their needs for safe water. She has overseen the implementation of over 600 community water projects in Latin America, Asia and Africa benefiting 250,000 people. Water 1st has not been rated by Charity Navigator, which only rates charities with more than $1,000,000 in total revenue.