Cost

All else equal, the less time it takes a scientist to make a breakthrough, the cheaper the breakthrough is - less opportunity cost to him, the sooner others can build on it, less maintenance & depreciation, less effort required to cover prerequisites and get up to speed, etc. Murray unfortunately does not give us a deep history of age of greatest accomplishment, but we can still look at the 20th century’s Nobel Prizes (eg. Chemistry ages); Jones & Weinberg 2011 matched winners with their age when they performed the award-winning work, summary:

A study of Nobel Laureates from 1901 to 2008 in these three fields examined the age at which scientists did their prize-winning work. Results showed that before 1905, about two-thirds of winners in all three fields did their prize-winning work before age 40, and about 20% did it before age 30. But by 2000, great achievements before age 30 nearly never occurred in any of the three fields. In physics, great achievements by age 40 only occurred in 19% of cases by the year 2000, and in chemistry, it nearly never occurred….Earlier work on creativity in the sciences has emphasized differences in the ages when creativity peaks across various scientific disciplines, assuming that those differences were stable over time, Weinberg said. But this new work suggests that the differences in the age of creativity peaks between fields like chemistry and physics are actually quite small compared to the differences in creativity peaks between time periods within each discipline…For the study, the researchers analyzed the complete set of 525 Nobel Prizes given between 1901 and 2008 in the three fields - 182 in physics, 153 in chemistry and 190 in medicine. Through extensive historical and biographical analysis, they determined the ages at which each Nobel Prize winner produced their prize-winning work. In general, there was an aging pattern over the 20th century as to when scientists made their breakthrough discoveries, although there were differences between the three fields. The most interesting case is physics, Weinberg said. In physics, there was an especially notable increase in the early 20th century in the frequency of young scientists producing prize-winning work. The proportion of physicists who did their prize-winning work by age 30 peaked in 1923 at 31%. Those that did their best work by age 40 peaked in 1934 at 78%. The proportion of physicists under age 30 or 40 producing Nobel Prize-winning work then declined throughout the rest of the century…Another reason that younger scientists may have made more significant contributions early in the 20th century is that they finished their training earlier in life. The majority of Nobel Laureates received their doctoral degrees by age 25 in the early 20th century, the researchers found. However, all three fields showed substantial declines in this tendency, with nearly no physics or chemistry laureates receiving their degrees that early in life by the end of the century. In another analysis, the researchers examined the age of studies referenced in important scientific papers in the three fields through the 20th century. They found that in the early part of the 1900s – the time when quantum mechanics made its mark – there was a strong tendency for physics to reference mostly recent work. The question is, how much old knowledge of the field do you need to know to make important scientific contributions in your field? Weinberg said. The fact that physicists in the early 20th century were citing mostly recent work suggests that older scientists didn’t have any advantage – their more complete knowledge of older work wasn’t necessary to make important contributions to the field. That could be one reason why younger scientists made such a mark. But now, physicists are more likely to cite older studies in their papers, he said. That means older scientists may have an advantage because of their depth of knowledge.

Rather, a researcher’s output tends to rise steeply in the 20’s and 30’s peak in the late 30’s or early 40’s and then trail off slowly through later years (Lehman, 1953; Simonton, 1991).

• Lehman HC (1953) Age and Achievement (Princeton University Press, Princeton, NJ)
• Simonton DK (1991) Career landmarks in science: Individual differences and interdisciplinary contrasts. Dev Psychol 27:119-130

Jones 2006 covered a similar upward shift in age for noted inventors

[Why peak in the early 40s? The age-related decline in intelligence is steady from 20s. There must be some balance between acquiring and exploiting information with one’s disappearing intelligence which produces a peak in the 40s before the decline kills productivity; the graph in aging section of the DNB curiously shows late 40s is where you hit 0 std-devs, intelligence/memory-wise, vis-a-vis the general population]

This reasoning is explored in Jones (2005), which studies ordinary inventors, looking at all U.S. patents in the 1975-2000 period. is rising at a rate of 6 years/century.

Jones 2005:

The estimates suggest that, on average, the great minds of the 20th Century typically became research active at age 23 at the start of the 20th Century, but only at age 31 at the end - an upward trend of 8 years. Meanwhile, there has been no compensating shift in the productivity of innovators beyond middle age.

The technological almanacs compile key advances in technology, by year, in several different categories such as electronics, energy, food & agriculture, materials, and tools & devices. The year (and therefore age) of great achievement is the year in which the key research was performed. For the technological almanacs, this is simply the year in which the achievement is listed.

The largest mass of great innovations in knowledge came in the 30’s (42%), but a substantial amount also came in the 40’s (30%), and some 14% came beyond the age of 50. Second, there are no observations of great achievers before the age of 19. Dirac and Einstein prove quite unusual, as only 7% of the sample produced a great achievement at or before the age of 26. Third, the age distribution for the Nobel Prize winners and the great inventors, which come from independent sources, are extremely similar over the entire distributions. Only 7% of individuals in the data appear in both the Nobel Prize and great inventors data sets.

While laboratory experiments do suggest that creative thinking becomes more difficult with age (e.g. Reese et al, 2001), the decline in innovative output at later ages may largely be due to declining effort, which a range of sociological, psychological, institutional, and economic theories have been variously proposed to explain (see Simonton 1996 for a review).

• Reese, H,W. Lee, L.J., Cohen, S. H., and Puckett, J.M. Effects of Intellectual Variables, Age, and Gender on Divergent Thinking in Adulthood, International Journal of Behavioral Development, November 2001, 25 (6), 491-500
• Simonton Creativity, in The Encyclopedia of Gerontology, San Diego, CA: Academic Press, 1996

In fact, aggregate data patterns, much debated in the growth literature, have noted long-standing declines in the per-capita output of R&D workers, both in terms of patent counts and productivity growth (Machlup 1962; Evenson, 1991; Jones 1995a; Kortum, 1997). Simple calculations from aggregate data suggest that the typical R&D worker contributes approximately 30% as much to aggregate productivity gains today as she did at the opening of the 20th Century.²⁹

29: Combining Machlup’s data on growth in knowledge producing occupations for 1900-1959 (Machlup 1962, Table X-4) with similar NSF data for 1959-1999 (National Science Foundation, 2005), we see that the total number of knowledge-producing workers in the United States has increased by a factor of approximately 19. Meanwhile, the U.S. per-capita income growth rate, which proxies for productivity growth over the long-run, suggests a 6-fold increase in productivity levels (based on a steady growth rate of 1.8%; see Jones 1995b). The average rate at which individual R&D workers contribute to productivity growth is A=LR , or gA=LR , where A is aggregate productivity, g is the productivity growth rate, and LR is the aggregate number of R&D workers. The average contribution of the individual R&D worker in the year 2000 is then a fraction A2000 =A1900 =(L2000 =L1900 ) = 6=19 (32%) of what it was in 1900.

Figure 5 compares the estimated life-cycle curves for the year 1900 and the year 2000, using specification (3). We see that the peak ability to produce great achievements in knowledge came around age 30 in 1900 but shifted to nearly age 40 by the end of the century. An interesting aspect of this graph is the suggestion that, other things equal, lifetime innovation potential has declined.

The first analysis looks directly at evidence from Ph.D. age and shows that Ph.D. age increases substantially over the 20th Century. The second analysis harnesses world wars, as exogenous interruptions to the young career, to test the basic idea that training is an important preliminary input to innovation. I show that, while the world wars do not explain the 20th century’s age trend, they do indicate the unavoidable nature of training: lost years of training appear to be made up after the war. The final analysis explores cross-field, cross-time variation. I show that variations in training duration predict variations in age at great invention

Indeed, several studies have documented upward trends in educational attainment among the general population of scientists. For example, the age at which individuals complete their doctorates rose generally across all major fields in a study of the 1967-1986 period, with the increase explained by longer periods in the doctoral program (National Research Council, 1990). The duration of doctorates as well as the frequency and duration of post-doctorates has been rising across the life-sciences since the 1960s (Tilghman et al, 1998). A study of electrical engineering over the course of the 20th century details a long-standing upward trend in educational attainment, from an initial propensity for bachelor degrees as the educational capstone to a world where Ph.D.’s are common (Terman, 1998).

• Terman, F.E. A Brief History of Electrical Engineering Education, Proceedings of the IEEE, August 1998, 86 (8), 1792-1800

Most strikingly, both achievement and Ph.D. age in Physics experienced a unique decline in the early 20th century. This unusual feature, beyond reinforcing the relationship between training and achievement age, may also serve to inform more basic theories for the underlying dynamics and differences across fields.

[that there was a fall in age for this radical and revolutionary period in physics validates the general approach]

First, mean life expectancy at age 10 was already greater than 60 in 1900, while it is clear from Sections 2 and 3 that innovation potential is modest beyond 60, so that adding years of life beyond this age would have at most mild effects on the optimization.²⁵ Related, even modest discounting would substantially limit the effect of gains felt 35+ years beyond the end of training on the marginal training decision. Next, common life expectancy changes cannot explain the unique cross-field and cross-time variation explored in Section 4, such as the unique behavior of physics. Moreover, Figure 7 suggests, if anything, accelerating age trends after the second world war, which is hard to explain with increased longevity, where post-war gains have slowed.

One proxy measure is research collaboration in patenting - measured as team size - which is increasing at over 10% per decade.28 A more direct measure of specialization considers the probability that an individual switches technological areas between consecutive patents. Jones (2005) shows that the probability of switching technological areas is substantially declining with time. These analyses indicate that training time, E, is rising, while measures of breadth, b, are simultaneously declining. It is then difficult to escape the conclusion that the distance to the knowledge frontier is rising.28 Large and general upward trends in research collaboration are also found in journal publications (e.g. Adams et al, 2004).

• Adams, James D., Black, Grant C., Clemmons, J.R., and Stephan, Paula E. Scientific Teams and Institutional Collaborations: Evidence from U.S. Universities, 1981-1999, NBER Working Paper #10640, July 2004

Analogously, problems that require more experiential training have older peak ages. For instance, Jones (2006) finds that the peak age for natural scientists has drifted higher over the twentieth century. Relative to 100 years ago, more experience now needs to be accumulated to reach the cutting edge of scientific fields.

lifespan increases cannot make up for this: http://lesswrong.com/lw/7jh/living_forever_is_hard_part_2_adult_longevity/

Jones, Benjamin F. The Burden of Knowledge and the Death of the Renaissance Man: Is Innovation Getting Harder? NBER Working Paper #11360, 2005

Upward trends in academic collaboration and lengthening doctorates, which have been noted in other research, can also be explained by the model, as can much-debated trends relating productivity growth and patent output to aggregate inventive effort. The knowledge burden mechanism suggests that the nature of innovation is changing, with negative implications for long-run economic growth.

Given this increasing educational attainment, innovators will only become more specialized if the burden of knowledge mechanism is sufficiently strong. More subtly, income arbitrage produces the surprising result that educational attainment will not vary across technological fields, regardless of variation in the burden of knowledge or innovative opportunities.

Any relation to Baumol’s cost disease?

As Science Evolves, How Can Science Policy?, Jones 2010; summary of all the Jones papers

First, R&D employment in leading economies has been rising dramatically, yet TFP growth has been flat (Jones, 1995b). Second, the average number of patents produced per R&D worker or R&D dollar has been falling over time across countries (Evenson 1984) and U.S. manufacturing industries (Kortum 1993). These aggregate data trends can be seen in the model as an effect of increasingly narrow expertise, where innovators are becoming less productive as individuals and are required to work in ever larger teams.

• Jones Time Series Tests of Endogenous Growth Models, Quarterly Journal of Economics, 1995b, 110, 495-525

Essentially, the greater the growth in the burden of knowledge, the greater must be the growth in the value of knowledge to compensate. Articulated views of why innovation may be getting harder in the growth literature (Kortum 1997, Segerstrom 1998) have focused on a fishing out idea; that is, on the parameter χ. The innovation literature also tends to focus on fishing out themes (e.g. Evenson 1991, Cockburn & Henderson, 1996). This paper offers the burden of knowledge as an alternative mechanism, one that makes innovation harder, acts similarly on the growth rate, and can explain aggregate data trends (see Section 4). Most importantly, the model makes specific predictions about the behavior of individual innovators, allowing one to get underneath the aggregate facts and test for a possible rising burden of knowledge using micro-data

• Kortum, Samuel S. Equilibrium R&D and the Decline in the Patent-R&D Ratio: U.S. Evidence, American Economic Review Papers and Proceedings, May 1993, 83, 450-457
• Evenson 1991 Patent Data by Industry: Evidence for Invention Potential Exhaustion? Technology and Productivity: The Challenge for Economic Policy, 1991, Paris: OECD, 233-248

This result is consistent with Henderson & Cockburn’s (1996) finding that researchers in the pharmaceutical industry are having a greater difficulty in producing innovations over time.

• Henderson, Rebecca and Cockburn, Iain. Scale, Scope, and Spillovers: The Determinants of Research Productivity in Drug Discovery, Rand Journal of Economics, Spring 1996, 27, 32-59.

The age at which individuals complete their doctorates rose generally across all major fields from 1967-1986, with the increase explained by longer periods in the doctoral program (National Research Council, 1990). The duration of doctorates as well as the frequency of post-doctorates has been rising across the life-sciences since the 1960s (Tilghman et al, 1998). An upward age trend has also been noted among the great inventors of the 20th Century at the age of their noted achievement (Jones, 2005), as shown in Table 1. Meanwhile, like the general trends in innovator teamwork documented here, upward trends in academic coauthorship have been documented in many academic literatures, including physics and biology (Zuckerman & Merton, 1973), chemistry (Cronin et al, 2004), mathematics (Grossman, 2002), psychology (Cronin et al, 2003), and economics (McDowell & Melvin, 1983; Hudson, 1996; Laband & Tollison, 2000). These coauthorship studies show consistent and, collectively, general upward trends, with some of the data sets going back as far as 1900.

• National Research Council, On Time to the Doctorate: A Study of the Lengthening Time to Completion for Doctorates in Science and Engineering, Washington, DC: National Academy Press, 1990
• Tilghman, Shirley (chair) et al. Trends in the Early Careers of Life Sciences, Washington, DC: National Academy Press, 1998
• Zuckerman, Harriet and Merton, Robert. Age, Aging, and Age Structure in Science, in Merton, Robert, The Sociology of Science, Chicago, IL: University of Chicago Press, 1973, 497-559
• Cronin et al, 2004 Visible, Less Visible, and Invisible Work: Patterns of Collaboration in 20th Century Chemistry, Journal of the American Society for Information Science and Technology, 2004, 55(2), 160-168
• Grossman, Jerry. The Evolution of the Mathematical Research Collaboration Graph, Congressus Numerantium, 2002, 158, 202-212
• Cronin, Blaise, Shaw, Debora, and La Barre, Kathryn. A Cast of Thousands: Coauthorship and Subauthorship Collaboration in the 20th Century as Manifested in the Scholarly Journal Literature of Psychology and Philosophy, Journal of the American Society for Information Science and Technology, 2003, 54(9), 855-871
• McDowell, John, and Melvin, Michael. The Determinants of Coauthorship: An Analysis of the Economics Literature, Review of Economics and Statistics, February 1983, 65, 155-160
• Hudson, John. Trends in Multi-Authored Papers in Economics, Journal of Economic Perspectives, Summer 1996, 10, 153-158
• Laband, David and Tollison, Robert. Intellectual Collaboration, Journal of Political Economy, June 2000, 108, 632-662

Of further interest is the drop in total patent production per total researchers, which has been documented across a range of countries and industries and may go back as far as 1900 and even before (Machlup 1962). Certainly, not all researchers are engaging in patentable activities, and it is possible that much of this trend is explained by relatively rapid growth of research in basic science.²² However, the results here indicate that among those specific individuals who produce patentable innovations, the ratio of patents to individuals is in fact declining. In particular, the recent drop in patents per U.S. R&D worker, a drop of about 50% since 1975 (see Segerstrom 1998), is roughly consistent in magnitude with the rise in team size over that period.

• Machlup, Fritz. The Production and Distribution of Knowledge in the United States, Princeton, NJ: Princeton University Press, 1962, 170-176
• Segerstrom, Paul. Endogenous Growth Without Scale Effects, American Economic Review, December 1998, 88, 1290-1310

TODO Performance Curve Database - many sigmoids or linear graphs?

Benefit

One warning sign is citation by contemporaries. If the percentage of papers which never get cited increases, this suggests that either the research itself is no good (even a null result is worth citing as information about what we know is not the case), or fellow researchers have a reason not to cite them (reasons which range from the malign like researchers are so overloaded that they cannot keep up with the literature, which implies diminishing marginal returns, or professional jealousy, which implies the process of science is being corrupted, to the merely possibly harmful, like length limits). Uncitedness is most famous in the humanities, which are not necessarily of major concern, but I have collected estimates that there are multiple hard fields like chemistry where uncitedness may range up to 70+%, and there’s a troubling indication that uncitedness may be increasing in even the top scientific journals.

demand for math inelastic and collaboration not helpful?

The Collapse of the Soviet Union and the Productivity of American Mathematicians, by George J. Borjas and Kirk B. Doran, NBER Working Paper No. 17800, February 2012

We use unique international data on the publications, citations, and affiliations of mathematicians to examine the impact of a large post-1992 influx of Soviet mathematicians on the productivity of their American counterparts. We find a negative productivity effect on those mathematicians whose research overlapped with that the Soviets. We also document an increased mobility rate (to lower-quality institutions and out of active publishing) and a reduced likelihood of producing home run papers. Although the total product of the pre-existing American mathematicians shrank, the Soviet contribution to American mathematics filled in the gap. However, there is no evidence that the Soviets greatly increased the size of the mathematics pie.

Ben Jones, a professor at the Kellogg School of Management, at Northwestern University, has quantified this trend. By analyzing 19.9 million peer-reviewed academic papers and 2.1 million patents from the past fifty years, he has shown that levels of teamwork have increased in more than ninety-five per cent of scientific subfields; the size of the average team has increased by about twenty per cent each decade. The most frequently cited studies in a field used to be the product of a lone genius, like Einstein or Darwin. Today, regardless of whether researchers are studying particle physics or human genetics, science papers by multiple authors receive more than twice as many citations as those by individuals. This trend was even more apparent when it came to so-called home-run papers-publications with at least a hundred citations. These were more than six times as likely to come from a team of scientists.

…A few years ago, Isaac Kohane, a researcher at Harvard Medical School, published a study that looked at scientific research conducted by groups in an attempt to determine the effect that physical proximity had on the quality of the research. He analyzed more than thirty-five thousand peer-reviewed papers, mapping the precise location of co-authors. Then he assessed the quality of the research by counting the number of subsequent citations. The task, Kohane says, took a small army of undergraduates eighteen months to complete. Once the data was amassed, the correlation became clear: when coauthors were closer together, their papers tended to be of significantly higher quality. The best research was consistently produced when scientists were working within ten metres of each other; the least cited papers tended to emerge from collaborators who were a kilometre or more apart. If you want people to work together effectively, these findings reinforce the need to create architectures that support frequent, physical, spontaneous interactions, Kohane says. Even in the era of big science, when researchers spend so much time on the Internet, it’s still so important to create intimate spaces.

http://www.newyorker.com/reporting/2012/01/30/120130fa_fact_lehrer?currentPage=all

Pharmaceuticals

Drugs are perhaps the most spectacular example of a sigmoid suddenly hitting and destroying a lot of expectations. If you read the transhumanist literature from the ’80s or ’90s, even the more sober and well-informed projections, it’s striking how much faith is put into ever-new miracle drugs coming out. And why not? The War On Cancer still didn’t seem like it was going too badly - perhaps it would take more than $1 billion, but real progress was being made - and a crop of nootropics came out in the ’70s and ’80s like modafinil, followed up with famous blockbusters like Viagra, and then the Human Genome Project would triumphantly finish in a decade or so, whereupon things would really get cooking.

What went unnoticed in all this was the diminishing marginal returns. First, it turned out that pharmaceutical companies were doing a pretty good job at searching all the small (lighter weight) chemicals:

What Do Medicinal Chemists Actually Make? A 50-Year Retrospective > The idea is to survey the field from a longer perspective than some of the other papers in this vein, and from a wider perspective than the papers that have looked at marketed drugs or structures reported as being in the clinic. I’m reproducing the plot for the molecular weights of the compounds, since it’s an important measure and representative of one of the trends that shows up. The prominent line is the plot of mean values, and a blue square shows that the mean for that period was statistically different than the 5-year period before it (it’s red if it wasn’t). The lower dashed line is the median. The dotted line, however, is the mean for actual launched drugs in each period with a grey band for the 95% confidence interval around it. > > > > As a whole, the mean molecular weight of a J. Med. Chem. has gone up by 25% over the 50-year period, with the steeped increase coming in 1990-1994. Why, that was the golden age of combichem, some of you might be saying, and so it was. Since that period, though, molecular weights have just increased a small amount, and may now be leveling off. Several other measures show similar trends. Fifty Years of Med-Chem Molecules: What Are They Telling Us?

The more atoms in a particular drug, the more possible permutations and arrangements; a logarithmic slowdown in average weight would be expected of a search through an exponentially increasing space of possibilities. Smaller drugs are much more desirable than larger ones: they often are easier & cheaper to synthesize, more often survive passage through the gut or can pass the blood-brain barrier, etc. So there are many more large drugs than small drugs, but the small ones are much more desirable; hence, one would expect a balance between them, with no clear shift - if we were far from exploiting all the low-hanging fruit. Instead, we see a very steady trend upwards, as if there were ever fewer worthwhile small drugs to be found.

(One could make an analogy to oil field exploration: the big oil fields are easiest to find and also the best, while small ones are both hard to find and the worst; if the big ones exist, the oil companies will exploit them as much as possible and neglect the small ones; hence, a chart showing ever decreasing average size of producing oil fields smells like a strong warning sign that there are few big oil fields left.)

Simultaneously with this indication that good drugs are getting harder to find, we find that returns are diminishing to each dollar spent on drug R&D (it takes more dollars to produce one drug):

Although modern pharmaceuticals are supposed to represent the practical payoff of basic research, the R&D to discover a promising new compound now costs about 100 times more (in inflation-adjusted dollars) than it did in 1950. (It also takes nearly three times as long.) This trend shows no sign of letting up: Industry forecasts suggest that once failures are taken into account, the average cost per approved molecule will top $3.8 billion by 2015. What’s worse, even these successful compounds don’t seem to be worth the investment. According to one internal estimate, approximately 85 percent of new prescription drugs approved by European regulators provide little to no new benefit. We are witnessing Moore’s law in reverse.⁴

The average drug developed by a major pharmaceutical company costs at least $4 billion, and it can be as much as $11 billion…The drug industry has been tossing around the $1 billion number for years. It is based largely on a study (supported by drug companies) by Joseph DiMasi of Tufts University…But as Bernard Munos of the InnoThink Center for Research In Biomedical Innovation has noted, just adjusting that estimate for current failure rates results in an estimate of $4 billion in research dollars spent for every drug that is approved…Forbes (that would be Scott DeCarlo and me) took Munos’ count of drug approvals for the major pharmas and combined it with their research and development spending as reported in annual earnings filings going back fifteen years…The range of money spent is stunning. AstraZeneca has spent $12 billion in research money for every new drug approved, as much as the top-selling medicine ever generated in annual sales; Amgen spent just $3.7 billion.⁵

Kindler’s attempts to figure out what to do about research were even more anguished. He was right that the old Pfizer model wasn’t working. Bigger wasn’t better when it came to producing new drugs. Studies by Bernard Munos, a retired strategist at Eli Lilly (LLY), show that both massive increases in research spending and corporate mergers have failed to increase R&D productivity. Between 2000 and 2008, according to Munos, Pfizer spent $60 billion on research and generated nine drugs that won FDA approval – an average cost of $6.7 billion per product. At that rate, Munos concluded, the company’s internal pipeline simply couldn’t sustain its profits.⁶

But the very opposite of Moore’s Law is happening at the downstream end of the R&D pipeline. The number of new molecules approved per billion dollars of inflation-adjusted R&D has declined inexorably at 9% a year and is now 1/100th of what it was in 1950. The nine biggest drug companies spend more than $60 billion a year on R&D but are finding new therapies at such a slow rate that, as a group, they’ve little chance of recouping that money. Meanwhile, blockbuster drugs are losing patent protection at an accelerating rate. The next few years will take the industry over a patent cliff of $170 billion in global annual revenue. On top of this, natural selection is producing resistant disease strains that undermine the efficacy not only of existing antibiotics and antivirals but (even faster) of anti-cancer drugs. Many people believe that something is terribly wrong with the way the industry works. The problem, some think, is that science-to mix clichés-is scraping the bottom of the biological barrel after plucking the low-hanging fruit.⁷

Eroom’s law: Diagnosing the decline in pharmaceutical R&D efficiency; Scannell et al 2012 (Lowe):

The past 60 years have seen huge advances in many of the scientific, technological and managerial factors that should tend to raise the efficiency of commercial drug research and development (R&D). Yet the number of new drugs approved per billion US dollars spent on R&D has halved roughly every 9 years since 1950, falling around 80-fold in inflation-adjusted terms. There have been many proposed solutions to the problem of declining R&D efficiency. However, their apparent lack of impact so far and the contrast between improving inputs and declining output in terms of the number of new drugs make it sensible to ask whether the underlying problems have been correctly diagnosed. Here, we discuss four factors that we consider to be primary causes, which we call the better than the Beatles problem; the cautious regulator problem; the throw money at it tendency; and the basic research-brute force bias. Our aim is to provoke a more systematic analysis of the causes of the decline in R&D efficiency.

similar graph: http://dl.dropbox.com/u/85192141/2008-wobbe.pdf Figure 3. New drugs discovered per billion dollars R&D spending and annual R&D spending

Drug development: Raise standards for preclinical cancer research, C. Glenn Begley & Lee M. Ellis 2012 - academic studies irreproducible

Education

Wikileaks diplomatic cable http://www.zerohedge.com/news/wikileaks-cable-reveals-chinese-warning-domestic-asset-bubbles-overcapacity-early-2010-bashing-

China will need to restructure its economy so that it has a significantly higher share of knowledge-based services, especially research and development. However China’s terrible educational system, which promotes copying and pasting over creative and independent thought, is the largest impediment the country faces on this front, our IFC contact said….

. (SBU) However, Lai [Consul General, the head of IFC’s Chengdu office, Lai Jinchang] identified China’s terrible educational system as presenting a serious impediment toward achieving a shift to a more knowledge-based economy. The current system promotes copying and pasting over creative and independent thought. Lai said that the system rewards students for thinking within a framework in order to get the grade. He described the normal process undertaken by students when writing as essentially collecting sentences from various sources without any original thinking. He compared the writing ability of a typical Chinese Phd as paling in comparison to his unskilled staff during his decade of work with the IFC in Africa.

R&D

Publication bias stronger in China? Local Literature Bias in Genetic Epidemiology: An Empirical Evaluation of the Chinese Literature, 2005:

We targeted 13 gene-disease associations, each already assessed by meta-analyses, including at least 15 non-Chinese studies. We searched the Chinese Journal Full-Text Database for additional Chinese studies on the same topics. We identified 161 Chinese studies on 12 of these gene-disease associations; only 20 were PubMed-indexed (seven English full-text). Many studies (14-35 per topic) were available for six topics, covering diseases common in China. With one exception, the first Chinese study appeared with a time lag (2-21 y) after the first non-Chinese study on the topic. Chinese studies showed significantly more prominent genetic effects than non-Chinese studies, and 48% were statistically significant per se, despite their smaller sample size (median sample size 146 versus 268, p< 0.001). The largest genetic effects were often seen in PubMed-indexed Chinese studies (65% statistically significant per se). Non-Chinese studies of Asian-descent populations (27% significant per se) also tended to show somewhat more prominent genetic effects than studies of non-Asian descent (17% significant per se).

Chinese Innovation Is a Paper Tiger: A closer look at China’s patent filings and R&D spending reveals a country that has a long way to go, WSJ, by Anil K. Gupta and Haiyan Wang:

China’s R&D expenditure increased to 1.5% of GDP in 2010 from 1.1% in 2002, and should reach 2.5% by 2020. Its share of the world’s total R&D expenditure, 12.3% in 2010, was second only to the U.S., whose share remained steady at 34%-35%. According to the World Intellectual Property Organization, Chinese inventors filed 203,481 patent applications in 2008. That would make China the third most innovative country after Japan (502,054 filings) and the U.S. (400,769)…According to the Organization for Economic Cooperation and Development, in 2008, the most recent year for which data are available, there were only 473 triadic patent filings from China versus 14,399 from the U.S., 14,525 from Europe, and 13,446 from Japan. Starkly put, in 2010 China accounted for 20% of the world’s population, 9% of the world’s GDP, 12% of the world’s R&D expenditure, but only 1% of the patent filings with or patents granted by any of the leading patent offices outside China. Further, half of the China-origin patents were granted to subsidiaries of foreign multinationals….A 2009 survey by the China Association for Science and Technology reported that half of the 30,078 respondents knew at least one colleague who had committed academic fraud. Such a culture inhibits serious inquiry and wastes resources.

Insiders agree; a dean at Tsinghua University (first or second best university in China):

In reality, however, rampant problems in research funding-some attributable to the system and others cultural-are slowing down China’s potential pace of innovation.

Although scientific merit may still be the key to the success of smaller research grants, such as those from China’s National Natural Science Foundation, it is much less relevant for the megaproject grants from various government funding agencies…the guidelines are often so narrowly described that they leave little doubt that the needs are anything but national; instead, the intended recipients are obvious. Committees appointed by bureaucrats in the funding agencies determine these annual guidelines. For obvious reasons, the chairs of the committees often listen to and usually cooperate with the bureaucrats.

Expert opinions simply reflect a mutual understanding between a very small group of bureaucrats and their favorite scientists. This top-down approach stifles innovation and makes clear to everyone that the connections with bureaucrats and a few powerful scientists are paramount, dictating the entire process of guideline preparation. To obtain major grants in China, it is an open secret that doing good research is not as important as schmoozing with powerful bureaucrats and their favorite experts.

This problematic funding system is frequently ridiculed by the majority of Chinese researchers. And yet it is also, paradoxically, accepted by most of them. Some believe that there is no choice but to accept these conventions. This culture even permeates the minds of those who are new returnees from abroad; they quickly adapt to the local environment and perpetuate the unhealthy culture. A significant proportion of researchers in China spend too much time on building connections and not enough time attending seminars, discussing science, doing research, or training students (instead, using them as laborers in their laboratories). Most are too busy to be found in their own institutions. Some become part of the problem: They use connections to judge grant applicants and undervalue scientific merit. editorial, China’s Research Culture, Science

An investigation by the Chinese Association of Scientists has revealed that only about 40 percent of the funds allocated for scientific research is used on the projects they are meant for. The rest is usually spent on things that have nothing to do with research. Some research project leaders use the money to buy furniture, home appliances and, hold your breath, even apartments. In the most appalling scandal, an accountant in the National Science Foundation of China misappropriated more than 200 million yuan ($3.12 million) in eight years until he was arrested in 2004. Besides, the degree of earnestness most scientists show in their research projects nowadays is questionable. Engaging in scientific research projects funded by the State has turned out to be an opportunity for some scientists to make money. There are examples of some scientists getting research funds because of their connections with officials rather than their innovation capacity. Qian Xuesen, known as the father of China’s atomic bomb and satellites, used to say during the last few years before his death in 2009 that the biggest problem is that Chinese universities cannot cultivate top-class scientists. Honest Research Needed, China Daily (government paper)

Zinch China, a consulting company that advises American colleges and universities about China, published a report last year that found cheating on college applications to be "pervasive in China, driven by hyper-competitive parents and aggressive agents.’’

From the survey’s introduction: "Our research indicates that 90 percent of recommendation letters are fake, 70 percent of essays are not written by the applicant, and 50 percent of high school transcripts are falsified.’’

http://rendezvous.blogs.nytimes.com/2012/02/05/sneaking-into-class-from-china/

But there’s growing evidence that the innovation shortfall of the past decade is not only real but may also have contributed to today’s financial crisis. Think back to 1998, the early days of the dot-com bubble. At the time, the news was filled with reports of startling breakthroughs in science and medicine, from new cancer treatments and gene therapies that promised to cure intractable diseases to high-speed satellite Internet, cars powered by fuel cells, micromachines on chips, and even cloning. These technologies seemed to be commercializing at Internet speed, creating companies and drawing in enormous investments from profit-seeking venture capitalists-and ordinarily cautious corporate giants. Federal Reserve Chairman Alan Greenspan summed it up in a 2000 speech: We appear to be in the midst of a period of rapid innovation that is bringing with it substantial and lasting benefits to our economy. With the hindsight of a decade, one thing is abundantly clear: The commercial impact of most of those breakthroughs fell far short of expectations-not just in the U.S. but around the world. No gene therapy has yet been approved for sale in the U.S. Rural dwellers can get satellite Internet, but it’s far slower, with longer lag times, than the ambitious satellite services that were being developed a decade ago. The economics of alternative energy haven’t changed much. And while the biotech industry has continued to grow and produce important drugs-such as Avastin and Gleevec, which are used to fight cancer-the gains in health as a whole have been disappointing, given the enormous sums invested in research. As Gary P. Pisano, a Harvard Business School expert on the biotech business, observes: It was a much harder road commercially than anyone believed.…With far fewer breakthrough products than expected, Americans had little new to sell to the rest of the world. Exports stagnated, stuck at around 11% of gross domestic product until 2006, while imports soared. That forced the U.S. to borrow trillions of dollars from overseas. The same surges of imports and borrowing also distorted economic statistics so that growth from 1998 to 2007, rather than averaging 2.7% per year, may have been closer to 2.3% per year.

…Even the sequencing of the human genome-an acclaimed scientific achievement-has not reduced the cost of developing profitable drugs. One indicator of the problem’s scope: 2008 was the first year that the U.S. biotech industry collectively made a profit, according to a recent report by Ernst & Young-and that performance is not expected to be repeated in 2009.

…If an innovation boom were truly happening, it would likely push up stock prices for companies in such leading-edge sectors as pharmaceuticals and information technology. Instead, the stock index that tracks the pharmaceutical, biotech, and life sciences companies in the Standard & Poor’s (MHP) 500-stock index dropped 32% from the end of 1998 to the end of 2007, after adjusting for inflation. The information technology index fell 29%. To pick out two major companies: The stock price of Merck declined 35% between the end of 1998 and the end of 2007, after adjusting for inflation, while the stock price of Cisco Systems (CSCO) was down 9%. Consider another indicator of commercially important innovation: the trade balance in advanced technology products. The Census Bureau tracks imports and exports of goods in 10 high-tech areas, including life sciences, biotech, advanced materials, and aerospace. In 1998 the U.S. had a $30 billion trade surplus in these advanced technology products; by 2007 that had flipped to a $53 billion deficit. Surprisingly, the U.S. was running a trade deficit in life sciences, an area where it is supposed to be a leader.

…The final clue: the agonizingly slow improvement in death rates by age, despite all the money thrown into health-care research. Yes, advances in health care can affect the quality of life, but one would expect any big innovation in medical care to result in a faster decline in the death rate as well. The official death-rate stats offer a mixed but mostly disappointing picture of how medical innovation has progressed since 1998. On the plus side, Americans 65 and over saw a faster decline in their death rate compared with previous decades. The bad news: Most age groups under 65 saw a slower fall in the death rate. For example, for children ages 1 to 4, the death rate fell at a 2.3% annual pace between 1998 and 2006, compared with a 4% decline in the previous decade. And surprisingly, the death rate for people in the 45-to-54 age group was slightly higher in 2006 than in 1998.

http://www.businessweek.com/magazine/content/09_24/b4135000953288.htm Mandel; the point about stock market is interesting, because a defender of no-declines could say that any failed predictions of innovations - like the ones surrounding the Human Genome Project - have been cherry-picked by declinists, but the stock market should be immune to such cherry-picking. Yet, it wasn’t.

As you develop more drugs, your standard for safety should go up because it becomes ever less likely that a new drug is superior to any of the old ones but the chance it fooled your tests remains pretty much the same.

Imagine you have a little random number generator 1-100, and you want to maximize the number you draw, but you also have, say, a 10% chance of misreading the number each time. Initially you’d keep discarding your number - pfft, a 50? pfft, a 65? I can do better than that! - but once you’ve successfully drawn a 95, then you want to start examining the numbers carefully. I just drew a 96, but the odds of getting a number >95 is just 4%! It’s more likely that I just misread this 96… oh wait, it was actually 69. My bad.

(I’m not sure how accurate this little model is, but it captures how I feel about it.)

genetics underachieving: http://blog.sethroberts.net/2012/03/18/genomics-confidential-the-faux-wonderland-of-iceland/

Question: Dick, would you care to comment on the relative effectiveness between giving talks, writing papers, and writing books?

Hamming: In the short-haul, papers are very important if you want to stimulate someone tomorrow. If you want to get recognition long-haul, it seems to me writing books is more contribution because most of us need orientation. In this day of practically infinite knowledge, we need orientation to find our way. Let me tell you what infinite knowledge is. Since from the time of Newton to now, we have come close to doubling knowledge every 17 years, more or less. And we cope with that, essentially, by specialization. In the next 340 years at that rate, there will be 20 doublings, i.e. a million, and there will be a million fields of specialty for every one field now. It isn’t going to happen. The present growth of knowledge will choke itself off until we get different tools. I believe that books which try to digest, coordinate, get rid of the duplication, get rid of the less fruitful methods and present the underlying ideas clearly of what we know now, will be the things the future generations will value. Public talks are necessary; private talks are necessary; written papers are necessary. But I am inclined to believe that, in the long-haul, books which leave out what’s not essential are more important than books which tell you everything because you don’t want to know everything. I don’t want to know that much about penguins is the usual reply. You just want to know the essence.

–Richard Hamming, You and Your Research