helping out, also consider the costs of your helping out and what alternatives there are.
Folding@home boasts of being the largest/most powerful distributed computing project in the world with >5 petaflops of capacity, focused on the NP-hard problem of protein folding. It is powered by volunteers running its client on their computers, or more specifically, their GPUs and PS3s. (Their architectures are more specialized than normal CPUs and, if a problem happens to match their architecture, can run an order of magnitude or two faster & more power-efficiently than a CPU would.)
The researchers solve their problems, the volunteers know their idle computing capacity is going to good use - everyone wins. Right?
But where is this free lunch coming from? What does it cost the volunteers to run the clients? Their time in setting this all up, of course, but more importantly: electricity. Lots and lots of electricity.
Wikipedians have already done the legwork on how much electricity consumption Folding@home causes. Each petaflop costs ~3 megawatts, so at >5 petaflops (6 as of November 2011), it uses >15 megawatts every second. To put it another way, every hour Folding@home uses up 15 megawatt-hours1.
Electricity doesn’t come from nowhere. If we are to do even the most simplistic cost-benefit analysis, we can’t simply assume the cost is 15 megawatts conjured out of nowhere or that the electricity would have been consumed anyway. What a silly defense that would be - what are the power companies doing, generating a fixed amount of electricity and if the Folding@homers shut down, the plants dump their electricity into the air? Of course the marginal demand from Folding@home causes the generation of megawatt-hours that would not otherwise have been generated. (Oh, but Folding@home users are so altruistic that they engage in conservation to offset their increased electricity consumption! Yeah, right.) And it doesn’t matter what other wastes of electricity may be going on - those wastes stand condemned as well (
two wrongs don’t make a right). No more distractions or excuses: what are the costs of that cost?
The most obvious cost is air pollution. It is major enough that we don’t even need to consider any other costs, because air pollution kills. The editors of Next Big Future have listed a number of interesting statistics from the WHO and other studies of electricity generation and the deadliness of air pollution (see also Markandya & Wilkinson 2007).
For example, half the world’s electricity production is done by that filthiest and most dangerous of sources, coal. (In comparison, nuclear power is a stroll in the park, which means that French users can run Folding@home with a 75% cleaner conscience than the rest of us, while Germany’s reaction to Fukushima demonstrates how fatal knee-jerk reactions can be, in the saddest and most utterly predictable way possible2.) We will - generously - pretend that everyone contributing to Folding@home is located in the US, with its relatively clean coal plants. Nevertheless, each terawatt-hour of coal kills 15 people. How often does Folding@home burn through a terawatt? Well, 1000 megawatt-hours is one gigawatt-hour, and 1000 gigawatt-hours is one terawatt-hour, and , so how fast does Folding@home burn through 1 million megawatt-hours? Each year, Folding@home uses megawatt-hours, or 131.5 gigawatts, or 13.15% of a terawatt-hour. Each terawatt-hour means, remember, 15 dead people; in our grim calculus, what is 13.15% of 15 dead people? 1.8 dead people.
So if the power is entirely derived from coal, Folding@home kills 2 people a year.
What if the power is from an oil plant? That’s worse! The listed deaths per terawatt-hour for oil is 36 dead people, for an annual death toll of 4.4 people. Hey, it could be much worse: if Folding@home had been invented and popularized in China, with a terawatt-hour death toll of 278, that’d be 34 deaths every year.
With coal and oil out of the way, we can look at the minority fuels which make up a small slice of the US power supply. Biofuel is pretty bad with a death toll of 12/TWh; hydro isn’t too bad at 1.4/TWh; solar, wind, and nuclear power have <1/TWh death tolls. But of course, we don’t live in an environmental fantasy where all our power is generated by those cleaner sources, and even if we did, that wouldn’t help the people in the past who have already been killed by Folding@home’s pollution.
The actual power mix of the USA in 2009 was 45% coal, 24% natural gas, 20% nuclear, and 7% hydro, so balancing our numbers that gives us 1.01 annual deaths for a USA power mix. Phew! Only one dead person. Doesn’t that make you feel better?
We already saw how much electricity Folding@home consumes: 15 megawatt-hours. But how much does each megawatt-hour cost? The EIA says the average US rate for 1 kilowatt-hour in November 2010 was $0.0962. A megawatt-hour is 1000 kilowatt-hours, so 1 megawatt-hour is , or $96.2, so = $1443/hr. And its annual bill is , or ~12,650,000 dollars per year3.
$12.65 million is a lot of money. What could we do with that? Meta-charity Givewell estimates that <$1000 could save one life; another source says
Cost-effectiveness estimates per death-averted are $64-294 for a range of countries4. (One modest proposal is to use this $1000 figure as the base unit of a new coinage: the DC or
dead child; it has the merit over the dollar of possibly ingraining an understanding of opportunity costs.) And these interventions are the kind of things that can absorb a lot of money. (There are a lot of people out there who could use some help.)
If <$1000 will buy 1 life, then $12.65m would buy ~12,650 lives. Quite a few, that. One wonders whether Folding@home was the best form of charity the 300k or so volunteers could have chosen to engage in. (Folding@home is lucky to be an academic project; otherwise we would see deeply ironic outcomes like Seti@home shutting down & begging for funding - while its volunteers burn 0.5 petaflops annually.)
Maybe they would have done better to donate a few dollars to a regular charity, and not run up their electricity bill. One might wonder, though, about the case where one isn’t paying for one’s electricity. So, either you are paying for all of your electrical bill or you’re not:
- If you are paying for all of it, then yes, you can donate your electrical costs! Just don’t run Folding@Home and send Oxfam a Paypal donation at the end of the year.
If you are not paying for all of it, if someone is sharing the bill or footing the bill entirely, then donating directly may harm your pocketbook, yes. But in such a situation, does it really still make sense to force the other to pay for all the electricity you are using?
The overall economics are bad per the original note, it’s an inefficient way to turn someone else’s money into charity. What right do you have to burn the electricity like there’s no tomorrow, for that matter? If you weren’t going to use a year’s worth of electricity, then whomever is paying for your electricity is poorer by that $10 as surely as if you had pick-pocketed him of $10; few would agree that you are Robin Hood who may steal from the rich and give to the charitable. He accepted that risk when he gave you access to his electricity and deserves however you can contrive to screw him over? What an attitude!
Other points can be dealt with similarly:
- perhaps one worries that overhead on a small donation of $10 will eliminate the value. But the overhead on financial transactions is usually only a few percent, and the difference between Folding@home and the best charities is a difference of far more than a few percent. A dollar is a dollar, no matter where it comes from. If you and 99 other people each donate $100 to 100 charities, then it’s the same as if each person donated $10,000 to just 1 charity. The only difference is whatever overhead there might be; and even if we say that our Folding@Home contributors lose 50% to overhead and the charities wind up getting only $6 million in their bank accounts to use, that’s still thousands more lives saved than by running Folding@Home and wasting the same amount of money!5
- perhaps one worries that it’s easier to run a Folding@home client than to donate regularly; leaving aside the basic fact that a one-time donation through Paypal is a lot easier than installing Folding@home, checking that it works, and perpetually sysadminning one’s computer for it, there are many ways to make donating easier. One could set up a recurring donation. One could annually flip a coin to decide whether to donate twice the usual amount. (Or one could roll a 1d10 dice every year, agreeing to donate 10x one’s annual donation.) One could ask for others to donate in lieu of a birthday or Christmas present; one could take advantage of employer matching plans. And so on. (If ease of contributing is one’s true reason, then good news - it is very easily dealt with!)
But hey, perhaps it’s done good research that will save even more lives. Biology, hell yeah!
Wikipedia has a partial list of 75 papers published drawing in some way on Folding@home. That is an average of 7.5 papers per year. The skeptic will notice that not a few (especially early papers, naturally) seem more concerned with Folding@home per se than with actual new results generated by it, and that project lead Vijay Pande seems to be author or co-author on almost all of the papers, which doesn’t indicate a large research community around the large investment of Folding@home. None of them seem important, and the number of publications seems to have peaked back in 2005-2006. The few actual compounds seem stalled in their test tubes. And a reminder, the wasted money amounts to many thousands of lives; for these sort of stakes, one would hope that one had good evidence, not mere possibility. But let’s lower standards and ask for ordinary evidence. What reason do we have to think Folding@Home has potential to save millions of lives? It has been operating for nearly 11 years. 11! And nothing that has yet saved a single life. (Readers are invited to provide a counter-example.) At what point do we stop talking about its potential to save millions of lives and about the possibility of teapots in orbit around Mercury?
Basic research is great and all, but at some point enough is enough. Wouldn’t it be better to redirect efforts to, say, Foldit - which besides not being polluting, is a fun puzzle game that can solve long-standing problems?6
As I am not a biologist nor omniscient, I can’t say for sure that the Folding@home work wasn’t useful, and I certainly couldn’t say it looks pretty worthless. But I feel much more comfortable asserting that the $12.65m could have been better spent on saving those 12,000 people (and the 12,000 people the year before, and the 12,000 the year before that, and…).
No one has yet shown me anything valuable done by Folding@home; but, they argue, it may yet produce something next year or perhaps the year after that. This is a statistical argument. There is an interesting perspective on this suggestion, known as the hope function. The idea is that you are looking at each time period (each year) for some sort of singular event (a discovery justifying Folding@home’s air-pollution and fatalities), but you also have a certain probability for the event never happening, for example because it is impossible. As each time period passes without the event, unsurprisingly, one’s probability that the event will happen the next period increases - as does the belief the event is impossible! (This isn’t contradictory.)
Interestingly, depending on one’s belief as to the probability distribution, the increases can be very small. One example I’ve run concerns the creation of Artificial Intelligence; AI enthusiasts have suggested AI could be invented anywhere in a very broad time span (Alan Turing, for example, putting it at post-1990). Many of those dates (like 1990) have already passed, and AI has clearly not happened. This has lead to a skepticism and belief that AI enthusiasts are irrational dreamers who refuse to update on the evidence; but what does the hope function say? I and others ran some estimates and found that for reasonable values, the failure of AI to happen only minimally supports the belief that AI is impossible!
For example, suppose one thought AI if possible would show up in the 100 years between 2000 and 2100 (a superset of many expert forecasts), that its possibility was 90% (0.9), but it was an essentially random breakthrough which could as easily happen in 2000 as 2099 (a uniform distribution). In x years, what is our new belief about its possibility? The equation goes:
We substitute in our 100 years and 90% belief:
We plug the year of interest into the hope function and find that in 2050, or in 50 years (), our original 90% belief AI is possible has fallen only 9%, to 81%! (Intuitively7, one might think that a half-century of failure would count for more, but it doesn’t.) Even in 2090, after 90 years of failure (), our view that AI is impossible has fallen only to 47%.
If we adopt a more realistic distribution like a bell curve centered in 2050, we get similar results - very small decreases each year until 2050, followed by a sudden plummet (as most of the chances get used up each year at or near the peak of the bell curve), and then a gradual curve downwards to 0% as 2100 draws near.
Setting up a hope function equation for Folding@home is conceptually clear: what is the full lifetime of Folding@home, how many years has it run without finding such a thing, what is the probability it will ever find anything useful, and what is the distribution of said probability?
- Internet distributed-computing projects are young enough that there’s no clear lifecycle. Seti@home looks like it will never shut down, other projects shut down after a year or two or finished their goals, like a number of the cryptographic brute-forcing projects set up to resolve the DES Challenges and prove DES insecure. (distributed.net has completed 9 such challenges but continues 2 long-term tasks). Let’s guess and say Folding@home has another decade to go, for a total n = 20
- Folding@home began in 2000 or so, so we can put x at 11
- This is the contentious one. Let’s see what 90% does - that is surely a favorable probability
- We could choose either the uniform or bell curve distribution; both are attractive as models (Folding@home as serendipity machine! Folding@home as project gearing up in the early years for its maturity and eventual decline!) But as it happens, we already put n = 20 and x = 11, so we’d be in the middle of the bell curve and that’s almost the same as the uniform distribution. So we’ll use the simpler uniform distribution.
We substitute in: or a fall of 10% in our belief that Folding@home will ever produce anything, which is a little sobering. Switching to the bell curve distribution will only make matters worse; recall that after the midpoint, the bell curve plummeted. With a bell curve centered on 2010 or 2011 and an end date of 2020, the hope function drops the 90% belief down to the <30% range by 2012 or 2013. (Further examples would not be enlightening; the reader can calculate out the variations himself.)
I leave some questions for the Folding@home enthusiast to ponder: Are these unreasonable assumptions? (Aren’t they favorable?) Do you have any real reason to believe that Folding@home’s discoveries ought to be heavily back-loaded, that one would expect it to take, say, 15 years (consuming ~2 terawatt-hours)? Would you have argued for this back-loading before reading about the hope function? If you do not dispute the assumptions, have you actually dropped your faith in Folding@home by ~10%? Or at all? What sort of evidence would convince you Folding@home is harmful?
So what’s the right way then? Look at a very similar grid computing project, Rosetta@home. Rosetta@home has only th the computing power of Folding@home and presumably consumes proportionately less electricity; hence it directly kills people a year and indirectly kills . This is three orders of magnitude fewer deaths. And with its applied focus, the benefits are a little bit better than Folding@home.
And Rosetta@home has made an additional contribution in demonstrating how new hybrid approaches to the protein-folding problem might work; I previously mentioned Foldit cracking a long-standing protein-folding problem, but Foldit did this by refining a Rosetta@home-generated approximation - and not a Folding@home approximation. Foldit failed on the other targets where it had no Rosetta@home starting point.8 (As far as I can tell, Folding@home has never participated in that particular competition, so one cannot compare its results with either Rosetta@home or Rosetta@home+Foldit.)
When you run a few numbers, this seems like a pretty uncontroversial conclusion. Lots of things are worse charities than the best charities (almost by definition); why be so wedded to the idea that Folding@home is not one of those charities? (Why do geeks in particular seem offended by criticism of Folding@home?) I think it has to do with our real reasons for a lot of things - social status9. Philanthropy is often for such worthless activities (does the MoMA really need donations from its board of directors so it can buy the latest artwork to have been priced into the stratosphere?) and people so uninterested in whether the charity actually helps10 that the truth of the matter - a straightforward cash-for-status bargain - is obvious11, but it’s not so obvious that charities themselves seek status-raising activities and so are biased towards funding bizarre & novel new activities12 - and what is more bizarre & novel than building a worldwide supercomputer to calculate the folding of proteins?
It is sad and pitiable that we spend so many billions on things like dog food and cosmetics rather than saving lives; but isn’t it even sadder that we can avoid that error, and try to do good, and still fail? The only thing sadder, I think, would be if we could know of our failure and go on supporting Folding@home. If charity truly was not about helping.
In 2011, Germany decided to begin shutting down its nuclear power supplies due to the Fukushima nuclear disaster; ignoring the points of disanalogies (Germany is not subject to tsunamis, is not running outdated designs from the 1960s, has more effective corporate governance etc), this was a predictable recipe for not replacing it all with renewable power (as environmentalists might hope) but increasing use of filthy fuel sources. This has indeed happened; from
Europe consuming more coal, Washington Post (7 February 2013):
In Germany, which by some measures is pursuing the most wide-ranging green goals of any major industrialized country, a 2011 decision to shutter nuclear power plants means that domestically produced lignite, also known as brown coal, is filling the gap . Power plants that burn the sticky, sulfurous, high-emissions fuel are running at full throttle, with many tallying 2012 as their highest-demand year since the early 1990s. Several new coal power plants have been unveiled in recent months - even though solar panel installations more than doubled last year…Lignite supplied 25.6% of Germany’s electricity in 2012, up from 22.7% in 2010. Hard black coal supplied an additional 19.1% last year, and it was also on the rise.
Above I quoted estimates that coal kills 15 people per terawatt-hour. Wikipedia says Germany used 590 terawatt-hours in 2010; lignite burning increased by %. So how many direct lignite-coal-based deaths have been caused by just 2 years of just Germans panicking over Fukushima? .
Wikipedia lists no direct deaths from Fukushima and 2-12 eventual plant worker deaths; Hoeve & Jacobson 2012 (discussion) estimated that total global radiation-related deaths from Fukushima will most likely be 180 and will not exceed 2500. Hence, Germany may already have killed more people due to its fear of Fukushima-like events than Fukushima killed.↩
$12.65m may be low for the cost of power. A March 2011 presentation by Kathy Yelick,
Exascale Computing: Technical Challenges, estimates each megawatt used at $1m, which obviously adds on another $2.5m to our estimate, which we estimated conservatively in any case.↩
A non-nutrient-based approach would be midwife training;
Even a small pilot project costing only $20,244 saved the lives of 97 infants, the authors estimated, meaning that it cost just $208 per life saved.↩
The marginal effectiveness of the best charities is huge; the best charities do orders of magnitude more good than mediocre or bad charities. A thought-example from
A hypothetical charity running programs like VillageReach’s which embezzled 95% of its budget and had correspondingly greatly reduced cost-effectiveness would still be doing far more good per dollar than the Make-A-Wish Foundation or the least effective developing world charities do. This example makes it clear how profoundly useless the overhead ratio is for assessing the relative quality of a charity.
This holds true for less mortal charities; Nicholas Kristof cites an example with 2 orders of magnitude difference:
In much of the developing world, most kids have intestinal worms, leaving them sick, anemic and more likely to miss school. Deworming is very cheap (a pill costing a few pennies), and, in the experiment he did with Edward Miguel, it resulted in 25% less absenteeism. Even years later, the kids who had been randomly chosen to be dewormed were earning more money than other kids. Kremer estimates that the cost of keeping a kid in school for an additional year by building schools or by subsidizing school uniforms is more than $100, while by deworming kids, the cost drops to $3.50. (In a pinch, kids can usually go to
schoolin a church or mosque without a uniform.)
Original 2011 paper:
Crystal structure of a monomeric retroviral protease solved by protein folding game players(PDF). General background on Foldit:
Predicting protein structures with a multiplayer online game2010.↩
Crystal structure of a monomeric retroviral protease solved by protein folding game players2011:
De novo structure prediction remains an exceptionally challenging problem, and very few predictions with atomic accuracy have been made in the history of [the competition] CASP. For CASP9 target T0581, starting from an extended chain, the Rosetta Server, which carried out a large-scale search for the lowest-energy structure using computing power from Rosetta@home volunteers (
http://boinc.bakerlab.org/rosetta/), produced a remarkably accurate model (Fig. 1a; compare red and blue). However, the server ranked this model fourth out of the five submissions. The Foldit Void Crushers team correctly selected this near-native model and further improved it by accurately moving the terminal helix, producing the best model for this target of any group and one of the best overall predictions at CASP9 (ref. 4) (Fig. 1a; compare yellow and blue). Thus, in a situation where one model out of several is in a near-native conformation, Foldit players can recognize it and improve it to become the best model. Unfortunately for the other Free Modeling targets, there were no similarly outstanding Rosetta Server starting models, so Foldit players simply tunneled to the nearest incorrect local minima.
Status as I use it here is a bit complex, more than a little idiosyncratic, and as much a paradigm as any simple property. To get an idea of what I mean, see Wikipedia, LessWrong/OvercomingBias, and Tyler Cowen’s description of Robin Hanson in Discover your Inner Economist.↩
It’s interesting how little effort goes into evaluating or ranking charities; the number of organizations can be counted on one hand, and one of the most prominent attempts, GiveWell, is amazingly recent (2006). In this vein, I was struck by a comment in the New York Time’s
For Federal Programs, a Taste of Market Discipline:
A recent review found that 10 major social programs had been rigorously evaluated over the past two decades, using the scientific gold standard of random assignment. Only one of the 10 — Early Head Start, for infants, toddlers pregnant women — was a clear success. Yet all 10 still exist, and largely in their original form.
The New Yorker, with its focus on New York City’s upper-crust, recently made this clear to me yet again with its coverage of the Koch brothers, who would more usually be greeted by said upper-crust with derision than acclaim; from 2010’s
Covert Operations: The billionaire brothers who are waging a war against Obama:
"On May 17th, a black-tie audience at the Metropolitan Opera House applauded as a tall, jovial-looking billionaire took the stage. It was the seventieth annual spring gala of American Ballet Theatre, and David H. Koch was being celebrated for his generosity as a member of the board of trustees; he had recently donated $2.5 million toward the company’s upcoming season, and had given many millions before that. Koch received an award while flanked by two of the gala’s co-chairs, Blaine Trump, in a peach-colored gown, and Caroline Kennedy Schlossberg, in emerald green. Kennedy’s mother, Jacqueline Kennedy Onassis, had been a patron of the ballet and, coincidentally, the previous owner of a Fifth Avenue apartment that Koch had bought, in 1995, and then sold, eleven years later, for thirty-two million dollars, having found it too small.
The gala marked the social ascent of Koch, who, at the age of seventy, has become one of the city’s most prominent philanthropists. In 2008, he donated a hundred million dollars to modernize Lincoln Center’s New York State Theatre building, which now bears his name. He has given twenty million to the American Museum of Natural History, whose dinosaur wing is named for him. This spring, after noticing the decrepit state of the fountains outside the Metropolitan Museum of Art, Koch pledged at least ten million dollars for their renovation. He is a trustee of the museum, perhaps the most coveted social prize in the city, and serves on the board of Memorial Sloan-Kettering Cancer Center, where, after he donated more than forty million dollars, an endowed chair and a research center were named for him.
One dignitary was conspicuously absent from the gala: the event’s third honorary co-chair, Michelle Obama. Her office said that a scheduling conflict had prevented her from attending. Yet had the First Lady shared the stage with Koch it might have created an awkward tableau…"
A quick calculation: , so a vastly incomplete tally of the Koch donations is $200 million or roughly 200,000 dead Africans. Does anyone want to argue that the Kochs’ philanthropy is even remotely close to being efficient, and that these donations were anything but purchasing status? In some cases, one wonders why they even pretend;
More Cash to Go to a Hall Than to Haiti, The New York Times:
Even if the event’s nearly $200,000 worth of tickets sell out, less than $8,000 from the sales will go to the cause. The concert, though, is expected to raise some money, thanks mainly to a $50,000 subsidy by the Montblanc company and $10,000 by CAMI Music, the concert’s presenter and Mr. Lang’s management agency…No hard and fast guidelines exist on how much money raised in a benefit should go for expenses, and it is not unusual for galas to raise little money or even lose it…In an accounting provided by CAMI Music, the costs will total $181,590. If the hall sells out, box office proceeds will total $189,793, excluding complimentary tickets.
From the GiveWell blog,
AfterExtraordinary and Unorthodox" comes the Valley of Death" (replacing the relevant adjectives with
statusis left as an exercise for the reader):
…it’s hard for me to see a big difference between it and the $100 million Gates Grand Challenges Explorations,a unique initiative that supports innovative research of unorthodox ideas" in global health (though the InnoCentive proposal above does not explicitly specify a sector, all three of its examples are in global health as well).
Speaking more informally, I’ve heard similar concepts emphasized by most major funders I’ve spoken with. Anyone who has dealt with major foundations should recognize the desire to find a completely new, revolutionary, neglected opportunity that just needs some seed funding to explode.
I do believe that the best opportunities are the under-funded ones. Yet I’m not sure that tiny, neglected innovations are the best places to look for these opportunities - precisely because that’s where all the major funders seem to be looking. I submit that the better place to look for neglected opportunities is the
valley of deathbetween proof of concept and large-scale rollout.
…There’s no glory in funding the VillageReach rollout. VillageReach already has shown that what it’s doing has worked; nobody can claim to be brilliant for spotting it. And VillageReach doesn’t need help designing its program (this has been cited to me explicitly as a drawback from the perspective of some major funders)."
Attitudes toward evidence seem less key than we would have guessed. When we started GiveWell, we and most of our supporters imagined that new customers could be found in certain industries where people are accustomed to using measurement to evaluate and learn from their decisions. We hoped these people would resonate with our desire to bring feedback loops into areas where feedback loops don’t naturally exist. But we’ve found that a lot of them don’t, largely because impact isn’t the main thing they’re aiming for when they give. People give for many reasons - to maintain friendships, to overcome guilt and cognitive dissonance, to achieve recognition - and a given donor is unlikely to be interested in GiveWell unless achieving impact is at the top of his/her list.
On the other hand,
GiveWell customers never seem interested in public recognition. In our first year, we considered posting acknowledgments to major supporters on our website, but there was no interest. Since then, we have had many customers who require anonymity (even when we ask them to take public credit for our sake) and no customers who’ve requested that we publicly thank them or otherwise help them get recognition.
It is a little difficult for Hansonian theories of charity to explain wholly anonymous charity. Possibly charity in such cases is due to signaling within a small group, rather than public signaling to thousands or millions of people (the general populace). And who knows; maybe there is genuine altruism in this world of dust and delusions.↩