1987-miller-researchinsocialproblemsandpublicpolicy-v4.pdf: “Research in Social Problems and Public Policy, A Research Annual: Volume 4”, JoAnn L. Miller, Michael Lewis
1980-smith-thebenefitsofpsychotherapy.pdf: “The Benefits of Psychotherapy”, Mary Lee Smith, Gene V. Glass, Thomas I. Miller
1969-jensen.pdf: “How Much Can We Boost IQ and Scholastic Achievement?”, (1969-05-01; ):
Arthur Jensen argues that the failure of recent compensatory education efforts to produce lasting effects on children’s IQ and achievement suggests that the premises on which these efforts have been based should be reexamined. He begins by questioning a central notion upon which these and other educational programs have recently been based: that IQ differences are almost entirely a result of environmental differences and the cultural bias of IQ tests. After tracing the history of IQ tests, Jensen carefully defines the concept of IQ, pointing out that it appears as a common factor in all tests that have been devised thus far to tap higher mental processes. Having defined the concept of intelligence and related it to other forms of mental ability, Jensen employs an analysis of variance model to explain how IQ can be separated into genetic and environmental components. He then discusses the concept of “heritability”, a statistical tool for assessing the degree to which individual differences in a trait like intelligence can be accounted for by genetic factors. He analyzes several lines of evidence which suggest that the heritability of intelligence is quite high (ie., genetic factors are much more important than environmental factors in producing IQ differences). After arguing that environmental factors are not nearly as important in determining IQ as are genetic factors, Jensen proceeds to analyze the environmental influences which may be most critical in determining IQ. He concludes that prenatal influences may well contribute the largest environmental influence on IQ. He then discusses evidence which suggests that social class and racial variations in intelligence cannot be accounted for by differences in environment but must be attributed partially to genetic differences. After he has discussed the influence on the distribution of IQ in a society on its functioning, Jensen examines in detail the results of educational programs for young children, and finds that the changes in IQ produced by these programs are generally small. A basic conclusion of Jensen’s discussion of the influence of environment on IQ is that environment acts as a “threshold variable.” Extreme environmental deprivation can keep the child from performing up to his genetic potential, but an enriched educational program cannot push the child above that potential. Finally, Jensen examines other mental abilities that might be capitalized on in an educational program, discussing recent findings on diverse patterns of mental abilities between ethnic groups and his own studies of associative learning abilities that are independent of social class. He concludes that educational attempts to boost IQ have been misdirected and that the educational process should focus on teaching much more specific skills. He argues that this will be accomplished most effectively if educational methods are developed which are based on other mental abilities besides IQ.
2018-pianta.pdf: “Does Attendance in Private Schools Predict Student Outcomes at Age 15? Evidence From a Longitudinal Study”, (2018-01-01; ):
By tracking longitudinally a sample of American children (n = 1,097), this study examined the extent to which enrollment in private schools between kindergarten and ninth grade was related to students’ academic, social, psychological, and attainment outcomes at age 15. Results from this investigation revealed that in unadjusted models, children with a history of enrollment in private schools performed better on nearly all outcomes assessed in adolescence. However, by simply controlling for the sociodemographic characteristics that selected children and families into these schools, all of the advantages of private school education were eliminated. There was also no evidence to suggest that low-income children or children enrolled in urban schools benefited more from private school enrollment.
1963-clark.pdf: “Educational stimulation of racially disadvantaged children”, Kenneth B. Clark
1940-reymert.pdf: “The effect of a change to a relatively superior environment upon the IQs of one hundred children”, Martin L. Reymert, Ralph T. Hinton Jr.
1968-glass-biologyandbehaviorgenetics.pdf: “Biology and Behavior: Genetics”, David C. Glass
2006-sanbonmatsu.pdf: “Neighborhoods and Academic Achievement: Results from the Moving to Opportunity Experiment”, (2006-09-01; ):
Families originally living in public housing were assigned housing vouchers by lottery, encouraging moves to neighborhoods with lower poverty rates.
Although we had hypothesized that reading and math test scores would be higher among children in families offered vouchers (with larger effects among younger children), the results show no statistically-significant effects on test scores for any age group among more than 5,000 children aged 6 to 20 in 2002 who were assessed 4 to 7 years after randomization.
Program impacts on school environments were considerably smaller than impacts on neighborhoods, suggesting that achievement-related benefits from improved neighborhood environments alone are small.
Background: The creation of economically mixed communities has been proposed as one way to improve the life outcomes of children growing up in poverty. However, whether low-income children benefit from living alongside more affluent neighbors is unknown.
Method: Prospectively gathered data on over 1,600 children from the Environmental Risk (E-Risk) Longitudinal Twin Study living in urban environments is used to test whether living alongside more affluent neighbors (measured via high-resolution geo-spatial indices) predicts low-income children’s antisocial behavior (reported by mothers and teachers at the ages of 5, 7, 10, and 12).
Results: Results indicated that low-income boys (but not girls) surrounded by more affluent neighbors had higher levels of antisocial behavior than their peers embedded in concentrated poverty. The negative effect of growing up alongside more affluent neighbors on low-income boys’ antisocial behavior held across childhood and after controlling for key neighborhood and family-level factors.
Conclusions: Findings suggest that efforts to create more economically mixed communities for children, if not properly supported, may have iatrogenic effects on boys’ antisocial behavior.
2019-poulos.pdf: “Land lotteries, long-term wealth, and political selection”, Jason Poulos
1997-mayer-whatmoneycantbuy.pdf: “What Money Can't Buy: Family Income and Children's Life Chances”, Susan E. Mayer
2019-ager.pdf: “The Intergenerational Effects of a Large Wealth Shock: White Southerners After the Civil War”, Philipp Ager, Leah Platt Boustan, Katherine Eriksson
“Long-Run Effects of Lottery Wealth on Psychological Well-Being”, (2020-02-12):
We surveyed a large sample of Swedish lottery players about their psychological well-being 5–22 years after a major lottery event and analysed the data following pre-registered procedures. Relative to matched controls, large-prize winners experience sustained increases in overall life satisfaction that persist for over a decade and show no evidence of dissipating over time. The estimated treatment effects on happiness and mental health are statistically-significantly smaller. Follow-up analyses of domain-specific aspects of life satisfaction implicate financial life satisfaction as an important mediator for the long-run increase in overall life satisfaction.
2020-ostling.pdf: “Association Between Lottery Prize Size and Self-reported Health Habits in Swedish Lottery Players”, (2020-03-19; ):
- Question: Is unearned wealth from lottery winnings associated with more healthy habits and better overall health?
- Findings: This quasi-experimental cohort study of 3344 individuals in 3 Swedish lotteries found no statistically-significant differences in long-term (5–22 years) health behaviors or overall health among individuals who participated in the same lottery but who randomly won prizes of different magnitudes.
- Meaning: The findings suggest that large, random transfers of unearned wealth are unlikely to be associated with large, long-term changes in health habits or overall health.
substantially more prevalent among individuals with low income than among individuals with high income, but the underlying mechanisms are not well understood.
Objective: To evaluate whether changes to unearned wealth from lotteries are associated with long-term health behaviors and overall health.
Design, Setting, and Participants: In this quasi-experimental cohort study, 4820 participants (aged 18–70 years at the time of winning) in 3 Swedish lotteries were surveyed from September 1, 2016, to November 11, 2016, between 5 and 22 years after a lottery event. Outcomes of participants in the same lottery who were randomly assigned prizes of different magnitudes by the lotteries but were ex ante identical in terms of their probability of winning different prizes were compared. Data were analyzed from December 22, 2016, to November 21, 2019.
Exposures: Lottery prizes ranged from $0 for nonwinning players to $1.6 million.
Main Outcomes and Measures: 4 lifestyle factors (smoking, alcohol consumption, physical activity, and a healthy diet index) and 2 measures of overall health (subjective health and an index of total health derived from responses to questions about 35 health conditions).
Results: The survey was returned by 3344 of 4820 individuals (69%; 1722 [51.5%] male), which corresponded to 3362 observations. The mean (SD) age was 48 (11.8) years in the year of the lottery win and 60 (11.0) years at the time of the survey. There were no statistically-significant associations between prize amount won and any of the 6 long-term health outcomes. Estimated associations expressed in SD units per $100,000 won were as follows: smoking (−0.006, 95% CI, −0.038 to 0.026); alcohol consumption (0.003, 95% , −0.027 to 0.033); physical activity (0.001, 95% , −0.029 to 0.032); dietary quality (−0.007, 95% , −0.040 to 0.026); subjective health (0.013, 95% , −0.017 to 0.043); and index of total health (−0.003, 95% , −0.033 to 0.027).
Conclusions and Relevance: In this study of Swedish lottery players, unearned wealth from random lottery prize winnings was not associated with subsequent healthy lifestyle factors or overall health. The findings suggest that large, random transfers of unearned wealth are unlikely to be associated with large, long-term changes in health habits or overall health.
1987-fraker.pdf: “The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs”, (1987; ):
This study investigates empirically the strengths and limitations of using experimental versus nonexperimental designs for evaluating employment and training programs. The assessment involves comparing results from an experimental-design study-the National Supported Work Demonstration-with the estimated impacts of Supported Work based on analyses using comparison groups constructed from the Current Population Surveys.
The results indicate that nonexperimental designs cannot be relied on to estimate the effectiveness of employment programs. Impact estimates tend to be sensitive both to the comparison group construction methodology and to the analytic model used. There is currently no way a priori to ensure that the results of comparison group studies will be valid indicators of the program impacts.
[Keywords: public assistance programs, analytical models, analytical estimating, employment, control groups, estimation methods, random sampling, human resources, public works legislation,]
Since the establishment of the Institute for Education Sciences (IES) within the U.S. Department of Education in 2002, IES has commissioned a sizable number of well-conducted randomized controlled trials (RCTs) evaluating the effectiveness of diverse educational programs, practices, and strategies (“interventions”). These interventions have included, for example, various educational curricula, teacher professional development programs, school choice programs, educational software, and data-driven school reform initiatives. Largely as a result of these IES studies, there now exists—for the first time in U.S. education—a sizable body of credible knowledge about what works and what doesn’t work to improve key educational outcomes of American students. A clear pattern of findings in these IES studies is that the large majority of interventions evaluated produced weak or no positive effects compared to usual school practices. This pattern is consistent with findings in other fields where are frequently carried out, such as medicine and business,1 and underscores the need to test many different interventions so as to build the number shown to work.
1993-lipsey.pdf: “The Efficacy of Psychological, Educational, and Behavioral Treatment: Confirmation from Meta-Analysis”, (1993-12-01; ):
Conventional reviews of research on the efficacy of psychological, educational, and behavioral treatments often find considerable variation in outcome among studies and, as a consequence, fail to reach firm conclusions about the overall effectiveness of the interventions in question. In contrast, meta-analysis reviews show a strong, dramatic pattern of positive overall effects that cannot readily be explained as artifacts of meta-analytic technique or generalized placebo effects. Moreover, the effects are not so small that they can be dismissed as lacking practical or clinical-significance. Although meta-analysis has limitations, there are good reasons to believe that its results are more credible than those of conventional reviews and to conclude that well-developed psychological, educational, and behavioral treatment is generally efficacious.
2019-lortieforgues.pdf: “Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned?”, (2019-03-11; ):
There are a growing number of large-scale educational randomized controlled trials (effect size was 0.06 standard deviations. These sat within relatively large confidence intervals (mean width = 0.30 SDs), which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational often find small and uninformative effects.). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale commissioned by the UK-based Education Endowment Foundation and the U.S.-based National Center for Educational Evaluation and Regional Assistance, which evaluated interventions aimed at improving academic achievement in K–12 (141 ; 1,222,024 students). The mean
[Keywords: educational policy, evaluation, meta-analysis, program evaluation.]
We found that the adverse effect of neighbourhood deprivation on adolescent violent criminality and substance misuse in Sweden was not consistent with a causal inference. Instead, our findings highlight the need to control for familial confounding in multilevel studies of criminality and substance misuse.
2014-sariaslan-2.pdf: “Does Population Density and Neighborhood Deprivation Predict Schizophrenia? A Nationwide Swedish Family-Based Study of 2.4 Million Individuals”, (2014-07-22; ):
People living in densely populated and socially disorganized areas have higher rates of psychiatric morbidity, but the potential causal status of such factors is uncertain. We used nationwide Swedish longitudinal registry data to identify all children born 1967–1989 (n = 2361585), including separate datasets for all cousins (n = 1 715 059) and siblings (n = 1667 894). The nature of the associations between population density and neighborhood deprivation and individual risk for a schizophrenia diagnosis was investigated while adjusting for unobserved familial risk factors (through cousin and sibling comparisons) and then compared with similar associations for depression. We generated familial pedigree structures using the Multi-Generation Registry and identified study participants with schizophrenia and depression using the National Patient Registry. Fixed-effects logistic regression models were used to study within-family estimates. Population density, measured as ln(population size/km2), at age 15 predicted subsequent schizophrenia in the population (OR = 1.10; 95% : 1.09; 1.11). Unobserved familial risk factors shared by cousins within extended families attenuated the association (1.06; 1.03; 1.10), and the link disappeared entirely within nuclear families (1.02; 0.97; 1.08). Similar results were found for neighborhood deprivation as predictor and for depression as outcome. Sensitivity tests demonstrated that timing and accumulation effects of the exposures (mean scores across birth, ages 1–5, 6–10, and 11–15 years) did not alter the findings. Excess risks of psychiatric morbidity, particularly schizophrenia, in densely populated and socioeconomically deprived Swedish neighborhoods appear, therefore, to result primarily from unobserved familial selection factors. Previous studies may have overemphasized the etiological importance of these environmental factors.
“No causal associations between childhood family income and subsequent psychiatric disorders, substance misuse and violent crime arrests: a nationwide Finnish study of >650 000 individuals and their siblings”, (2021-05-29):
- The causal nature between childhood family income and subsequent risks for psychiatric disorders, substance misuse and violent crime remains unclear.
- In this Finnish cohort study of 650 680 individuals, we initially found that increased family income was associated with lower risks of psychiatric disorders, substance misuse and arrest for a violent crime.
- However, once we compared siblings who grew up in the same household but were exposed to varying income levels at specific ages, the associations were no longer present.
- Associations between family income and subsequent psychiatric disorders, substance misuse and violent crime arrest were therefore explained by shared familial risks and were not consistent with a causal interpretation.
Background: Childhood family income has been shown to be associated with later psychiatric disorders, substance misuse and violent crime, but the consistency, strength and causal nature of these associations remain unclear.
Methods: We conducted a nationwide cohort and co-sibling study of 650 680 individuals (426 886 siblings) born in Finland between 1986 and 1996 to re-examine these associations by accounting for unmeasured confounders shared between siblings. The participants were followed up from their 15th birthday until they either migrated, died, met criteria for the outcome of interest or reached the end of the study period (31 December 2017 or 31 December 2018 for substance misuse). The associations were adjusted for sex, birth year and birth order, and expressed as adjusted hazard ratios (aHRs). The outcomes included a diagnosis of a severe mental illness (-spectrum disorders or bipolar disorder), depression and anxiety. Substance misuse (eg. medication prescription, hospitalization or death due to a substance use disorder or arrest for drug-related crime) and violent crime arrests were also examined. Stratified Cox regression models accounted for unmeasured confounders shared between differentially exposed siblings.
Results: For each $15,000 increase in family income at age 15 years, the risks of the outcomes were reduced by between 9% in severe mental illness (aHR = 0.91; 95% confidence interval: 0.90–0.92) and 23% in violent crime arrests (aHR = 0.77; 0.76–0.78). These associations were fully attenuated in the sibling-comparison models (aHR range: 0.99–1.00). Sensitivity analyses confirmed the latter findings.
Conclusions: Associations between childhood family income and subsequent risks for psychiatric disorders, substance misuse and violent crime arrest were not consistent with a causal interpretation.
[Keywords: socio-economic status, family income, schizophrenia, bipolar disorder, depression, anxiety, substance-use disorders, violence, quasi-experimental research designs, public health] [See also “Childhood family income, adolescent violent criminality and substance misuse: quasi-experimental total population study”, Sariaslan et al 2014; “Parental income and mental disorders in children and adolescents: prospective register-based study”, Kinge et al 2021.]
2015-martorell.pdf: “Investing in Schools: Capital Spending, Facility Conditions, and Student Achievement”, kstange
2004-jacob.pdf: “Public Housing, Housing Vouchers, and Student Achievement: Evidence from Public Housing Demolitions in Chicago”, (2004-03-01; ):
This paper utilizes a plausibly exogenous source of variation in housing assistance generated by public housing demolitions in Chicago to examine the impact of high-rise public housing on student outcomes. I find that children in households affected by the demolitions do no better or worse than their peers on a wide variety of achievement measures. Because the majority of households that leave high-rise public housing in response to the demolitions move to neighborhoods and schools that closely resemble those they left, the zero effect of the demolitions may be interpreted as the independent impact of public housing.
2016-depew.pdf: “Born on the wrong day? School entry age and juvenile crime”, Briggs Depew, Ozkan Eren
Self-esteem has become a household word. Teachers, parents, therapists, and others have focused efforts on boosting self-esteem, on the assumption that high self-esteem will cause many positive outcomes and benefits—an assumption that is critically evaluated in this review.
Appraisal of the effects of self-esteem is complicated by several factors. Because many people with high self-esteem exaggerate their successes and good traits, we emphasize objective measures of outcomes. High self-esteem is also a heterogeneous category, encompassing people who frankly accept their good qualities along with narcissistic, defensive, and conceited individuals.
The modest correlations between self-esteem and school performance do not indicate that high self-esteem leads to good performance. Instead, high self-esteem is partly the result of good school performance. Efforts to boost the self-esteem of pupils have not been shown to improve academic performance and may sometimes be counterproductive. Job performance in adults is sometimes related to self-esteem, although the correlations vary widely, and the direction of causality has not been established. Occupational success may boost self-esteem rather than the reverse. Alternatively, self-esteem may be helpful only in some job contexts. Laboratory studies have generally failed to find that self-esteem causes good task performance, with the important exception that high self-esteem facilitates persistence after failure.
People high in self-esteem claim to be more likable and attractive, to have better relationships, and to make better impressions on others than people with low self-esteem, but objective measures disconfirm most of these beliefs. Narcissists are charming at first but tend to alienate others eventually. Self-esteem has not been shown to predict the quality or duration of relationships.
High self-esteem makes people more willing to speak up in groups and to criticize the group’s approach. Leadership does not stem directly from self-esteem, but self-esteem may have indirect effects. Relative to people with low self-esteem, those with high self-esteem show stronger in-group favoritism, which may increase prejudice and discrimination.
Neither high nor low self-esteem is a direct cause of violence. Narcissism leads to increased aggression in retaliation for wounded pride. Low self-esteem may contribute to externalizing behavior and delinquency, although some studies have found that there are no effects or that the effect of self-esteem vanishes when other variables are controlled. The highest and lowest rates of cheating and bullying are found in different subcategories of high self-esteem.
Self-esteem has a strong relation to happiness. Although the research has not clearly established causation, we are persuaded that high self-esteem does lead to greater happiness. Low self-esteem is more likely than high to lead to depression under some circumstances. Some studies support the buffer hypothesis, which is that high self-esteem mitigates the effects of stress, but other studies come to the opposite conclusion, indicating that the negative effects of low self-esteem are mainly felt in good times. Still others find that high self-esteem leads to happier outcomes regardless of stress or other circumstances.
High self-esteem does not prevent children from smoking, drinking, taking drugs, or engaging in early sex. If anything, high self-esteem fosters experimentation, which may increase early sexual activity or drinking, but in general effects of self-esteem are negligible. One important exception is that high self-esteem reduces the chances of bulimia in females.
Overall, the benefits of high self-esteem fall into two categories: enhanced initiative and pleasant feelings. We have not found evidence that boosting self-esteem (by therapeutic interventions or school programs) causes benefits. Our findings do not support continued widespread efforts to boost self-esteem in the hope that it will by itself foster improved outcomes. In view of the heterogeneity of high self-esteem, indiscriminate praise might just as easily promote narcissism, with its less desirable consequences. Instead, we recommend using praise to boost self-esteem as a reward for socially desirable behavior and self-improvement.
1974-martinson.pdf: “What works? - questions and answers about prison reform”, Robert Martinson
We use experimental data to estimate impacts on school readiness of different kinds of preschool curricula—a largely neglected preschool input and measure of preschool quality. We find that the widely-used “whole-child” curricula found in most Head Start and pre-K classrooms produced higher classroom process quality than did locally-developed curricula, but failed to improve children’s school readiness. A curriculum focused on building mathematics skills increased both classroom math activities and children’s math achievement relative to the whole-child curricula. Similarly, curricula focused on literacy skills increased literacy achievement relative to whole-child curricula, despite failing to boost measured classroom process quality.
2018-chaiyachati.pdf: “Association of Rideshare-Based Transportation Services and Missed Primary Care AppointmentsA Clinical Trial”, American Medical Association
“Delayed Impact of Fair Machine Learning”, (2018-03-12):
Fairness in machine learning has predominantly been studied in static classification settings without concern for how decisions change the underlying population over time. Conventional wisdom suggests that fairness criteria promote the long-term well-being of those groups they aim to protect.
We study how static fairness criteria interact with temporal indicators of well-being, such as long-term improvement, stagnation, and decline in a variable of interest. We demonstrate that even in a one-step feedback model, common fairness criteria in general do not promote improvement over time, and may in fact cause harm in cases where an unconstrained objective would not.
We completely characterize the delayed impact of three standard criteria, contrasting the regimes in which these exhibit qualitatively different behavior. In addition, we find that a natural form of measurement error broadens the regime in which fairness criteria perform favorably.
Our results highlight the importance of measurement and temporal modeling in the evaluation of fairness criteria, suggesting a range of new challenges and trade-offs.
2014-flyvbjerg.pdf: “What You Should Know About Megaprojects and Why: An Overview”, (2014-04-07; ):
This paper takes stock of megaproject management, an emerging and hugely costly field of study. First, it answers the question of how large megaprojects are by measuring them in the units mega, giga, and tera, concluding we are presently entering a new “tera era” of trillion-dollar projects. Second, total global megaproject spending is assessed, at USD 6–9 trillion annually, or 8 percent of total global GDP, which denotes the biggest investment boom in human history. Third, four “sublimes” —political, technological, economic, and aesthetic—are identified to explain the increased size and frequency of megaprojects. Fourth, the “iron law of megaprojects” is laid out and documented: Over budget, over time, over and over again. Moreover, the “break-fix model” of megaproject management is introduced as an explanation of the iron law. Fifth, Albert O. Hirschman’s theory of the Hiding Hand is revisited and critiqued as unfounded and corrupting for megaproject thinking in both the academy and policy. Sixth, it is shown how megaprojects are systematically subject to “survival of the unfittest”, explaining why the worst projects get built instead of the best. Finally, it is argued that the conventional way of managing megaprojects has reached a “tension point”, where tradition is challenged and reform is emerging.
2019-ichino.pdf: “Cognitive and Non-Cognitive Costs of Daycare [Age] 0–2 for Children in Advantaged Families”, Andrea Ichino, Margherita Fort, Giulio Zanella
1972-page.pdf: “How We *All* Failed In Performance Contracting”, Ellis B. Page
1975-gramlich-educationalperformancecontracting.pdf: “Educational Performance Contracting: An Evaluation of an Experiment”, Edward M. Gramlich, Patricia P. Koshel
2017-mercier.pdf: “How Gullible are We? A Review of the Evidence from Psychology and Social Science”, (2017-05-18; ):
A long tradition of scholarship, from ancient Greece to Marxism or some contemporary social psychology, portrays humans as strongly gullible—wont to accept harmful messages by being unduly deferent. However, if humans are reasonably well adapted, they should not be strongly gullible: they should be vigilant toward communicated information. Evidence from experimental psychology reveals that humans are equipped with well-functioning mechanisms of epistemic vigilance. They check the plausibility of messages against their background beliefs, calibrate their trust as a function of the source’s competence and benevolence, and critically evaluate arguments offered to them. Even if humans are equipped with well-functioning mechanisms of epistemic vigilance, an adaptive lag might render them gullible in the face of new challenges, from clever marketing to omnipresent propaganda. I review evidence from different cultural domains often taken as proof of strong gullibility: religion, demagoguery, propaganda, political campaigns, advertising, erroneous medical beliefs, and rumors. Converging evidence reveals that communication is much less influential than often believed—that religious proselytizing, propaganda, advertising, and so forth are generally not very effective at changing people’s minds. Beliefs that lead to costly behavior are even less likely to be accepted. Finally, it is also argued that most cases of acceptance of misguided communicated information do not stem from undue deference, but from a fit between the communicated information and the audience’s preexisting beliefs.
[Keywords: epistemic vigilance, gullibility, trust]
A global priority for the behavioural sciences is to develop cost-effective, scalable interventions that could improve the academic outcomes of adolescents at a population level, but no such interventions have so far been evaluated in a population-generalizable sample. Here we show that a short (less than one hour), online growth mindset intervention—which teaches that intellectual abilities can be developed—improved grades among lower-achieving students and increased overall enrolment to advanced mathematics courses in a nationally representative sample of students in secondary education in the United States. Notably, the study identified school contexts that sustained the effects of the growth mindset intervention: the intervention changed grades when peer norms aligned with the messages of the intervention. Confidence in the conclusions of this study comes from independent data collection and processing, pre-registration of analyses, and corroboration of results by a blinded Bayesian analysis.
2019-kristal.pdf: “What we can learn from five naturalistic field experiments that failed to shift commuter behaviour”, (2019-12-23; ):
Across five field experiments with employees of a large organization (n = 68,915), we examined whether standard behavioural interventions (‘nudges’) successfully reduced single-occupancy vehicle commutes. In Studies 1 and 2, we sent letters and emails with nudges designed to increase carpooling. These interventions failed to increase carpool sign-up or usage. In Studies 3a and 4, we examined the efficacy of other well-established behavioural interventions: non-cash incentives and personalized travel plans. Again, we found no positive effect of these interventions. Across studies,ranged from Cohen’s d = −0.01 to d = 0.05. Equivalence testing, using study-specific smallest of interest, revealed that the treatment effects observed in 4 out of 5 of our experiments were statistically equivalent to zero (p < 0.04). The failure of these well-powered experiments designed to nudge commuting behaviour highlights both the difficulty of changing commuter behaviour and the importance of publishing null results to build cumulative knowledge about how to encourage sustainable travel.
2020-warne.pdf: “Beliefs About Human Intelligence in a Sample of Teachers and Nonteachers”, (2020-03-24; ):
Research in educational psychology consistently finds a relationship between intelligence and academic performance. However, in recent decades, educational fields, including gifted education, have resisted intelligence research, and there are some experts who argue that intelligence tests should not be used in identifying giftedness. Hoping to better understand this resistance to intelligence research, we created a survey of beliefs about intelligence and administered it online to a sample of the general public and a sample of teachers. We found that there are conflicts between currently accepted intelligence theory and beliefs from the American public and teachers, which has important consequences on gifted education, educational policy, and the effectiveness of interventions.
2020-okuyama.pdf: “Fast food outlets, physical activity facilities, and obesity among adults: a nationwide longitudinal study from Sweden”, (2020-05-19; ):
Background: While neighborhood deprivation is a well-known predictor of obesity, the mechanisms behind this association are unclear and these are important to clarify before designing interventions focusing on modifiable neighborhood environmental factors in order to reduce obesity risk.
Objectives: This study examined the longitudinal association between availability of fast-food outlets and physical activity facilities and the risk of obesity among adults.
Methods: This study used multiple national register data from Sweden. During the 11-year follow-up period between 2005 and 2015, data from 1,167,449 men and 542,606 women, aged 20–55 years, were accessible for inclusion in this analysis. Incidence of obesity was identified based on a diagnosis of obesity during the follow-up period derived from clinical register data. Neighborhood availability of fast-food outlets and physical activity facilities were assessed in 2005 and Cox regression was used in the statistical analysis. Individual socio-demographic factors and neighborhood deprivation were used as covariates.
Results: There were no meaningful associations between neighborhood fast-food outlets or physical activity facilities and obesity in men or women. Neighborhood deprivation was, however, consistently and strongly associated with incidence of obesity in both men and women.
Conclusions: Availability of fast-food outlets and lack of physical activity facilities appear unlikely to cause obesity in Swedish adults. Other potentially modifiable environmental factors within specific social and cultural settings that may influence obesity risk should be examined in future studies.
2020-oster.pdf: “Health Recommendations and Selection in Health Behaviors”, (2020-06-01; ):
Consider a case in which a new research finding links a health behavior with good health outcomes. A possible consequence is take-up of this behavior among individuals who engage in other positive health behaviors. If this occurs, later analyses of observational data may be biased by the change in selection. This paper evaluates these dynamic biases in empirical settings. Using data from vitamin supplementation and diet, I show that selection responds endogenously to health recommendations. These results highlight how spurious findings on health behaviors can be self-reinforcing.
[Examples: vitamin E, vitamin D, sugar consumption, fat consumption, and the Mediterranean diet.]
2020-yanguas.pdf: “Technology and educational choices: Evidence from a one-laptop-per-child program (OLPC)”, (2020-06-01; ):
This paper provides the first causal estimates of the effect of children’s access to computers and the internet on educational outcomes in early adulthood, such as schooling and choice of major. I exploit cross-cohort variation in access to technology among primary and middle school students in Uruguay, the first country to implement a nationwide one-laptop-per-child program. Despite a notable increase in computer access, educational attainment has not increased; the schooling gap between private and public school students has persisted, despite closing the technology gap. Among college students, those who had been exposed to the program as children were less likely to enroll in science and technology.
[Keywords: education policy, education and inequality, government expenditures and education]
2019-yanguas.pdf: “Essays in Applied Microeconomics (OLPC, natural-disasters/growth, Silent Spring)”, (2019-01-01; ):
This dissertation contains three essays in Applied Microeconomics. Chapter 1 provides the first causal estimates of the effect of children’s access to computers and the internet on adult educational outcomes such as schooling and choice of major. I exploit cross-cohort variation in access to technology among primary and middle school students in Uruguay, the first country to implement a nationwide one-laptop-per-child program. Despite a notable increase in computer access, educational attainment has not increased. However, college students who had been exposed to the program as children, were more likely to select majors with good employment prospects. Chapter 2 provides the first empirical evidence of the historical effects of natural disasters on economic activity in the United States. Although the literature has focused on salient natural disasters, more than one hounded strike the country every year, causing extensive property destruction and loss of life. My coauthors and I construct an 80 year panel data set that includes the universe of natural disasters in the United States from 1930 to 2010 and study how these shocks affected migration rates, home prices and poverty rates at the county level. Severe disasters increased out-migration rates by 1.5 percentage points and lowered housing prices/rents by 2.5–5.0%, but milder disasters had little effect on economic outcomes. Chapter 3 exploits the 1962 publication of Silent Spring, the first successful environmental science book, to investigate whether public information can influence popular demand for environmental regulation. Protecting the environment is often plagued by collective-action problems, so it is important to understand what motivates politicians to act. Combining historical U.S. congressional roll-call votes and census data, I find that the propensity of politicians to vote in favor of pro-environmental regulation increased by 5 to 33 percentage points after the publication of the book. The response to the informational shock varies with the constituency’s level of education, income, and exposure to pollution.
“Technology and Educational Choices: Evidence from a One-Laptop-per-Child Program”
This paper provides the first causal estimates of the effect of children’s access to computers and the internet on adult educational outcomes such as schooling and choice of major. I exploit cross-cohort variation in access to technology among primary and middle school students in Uruguay, the first country to implement a nationwide one-laptop-per-child program. Despite a notable increase in computer access, educational attainment has not increased. However, college students who had been exposed to the program as children, were more likely to select majors with good employment prospects.
“The Effect of Natural Disasters on Economic Activity in US Counties: A Century of Data”
More than 100 natural disasters strike the United States every year, causing extensive property destruction and loss of life. We construct an 80 year panel data set that includes the universe of natural disasters in the United States from 1930 to 2010 and study how these shocks affected migration rates, home prices and poverty rates at the county level. Severe disasters increased out-migration rates by 1.5 percentage points and lowered housing prices/rents by 2.5–5.0%, but milder disasters had little effect on economic outcomes.
“From Awareness to Action: Informational Shocks and Demand for Environmental Regulation”
Protecting the environment is often plagued by collective-action problems, so it is important to understand what motivates politicians to act. This paper exploits the 1962 publication of Silent Spring, the first influential environmental science book, to investigate whether public information can influence popular demand for environmental regulation. Combining historical U.S. congressional roll-call votes and census data, I find that the propensity of politicians to vote in favor of pro-environmental regulation increased by 5 to 33 percentage points after the publication of the book. The response to the informational shock varies with the constituency’s level of education, income, and exposure to pollution.
2019-rea.pdf: “New Evidence On The Heckman Curve”, (2019-12-17; ):
The Heckman Curve characterizes the rate of return to public investments in human capital as rapidly diminishing with age. For the disadvantaged, it describes investments early in the life course as having substantially higher rates of return compared to later in life. This paper assesses the Heckman Curve using estimates of program benefit cost ratios from the Washington State Institute for Public Policy. We find no support for the claim that social policy programs targeted early in the life course have the largest benefit cost ratios, or that on average the benefits of adult programs are less than the cost of the intervention.
- Many assume that high family income protects against the risk of youth obesity.
- I exploit Pennsylvania’s Marcellus Shale economic boom to test this causal theory.
- Youth obesity rates are unchanged with exogenous income gains, even in poor areas.
- There is no causal effect of income on youth obesity in this setting.
Low family income is frequently assumed to be a primary social determinant of youth obesity in the U.S. But while the observed correlation between family income and youth obesity is consistently negative, the true causal relationship is unclear.
I take advantage of a natural experiment—the boom economy created by development of the Marcellus Shale geological formation for natural gas extraction—to study whether income gains affect youth obesity rates among Pennsylvania students. To test this relationship, I compile data from geological, administrative, Census and other governmental sources and estimate cross-sectional OLS regression models, longitudinal fixed effects models, and two-stage instrumental variable models within a difference-in-differences framework. Falsification tests indicate that children’s location relative to the Marcellus Shale’s geological boundaries is a valid instrument for income gains. Yet plausibly exogenous income gains do not alter youth obesity rates, regardless of the community’s initial level of poverty or affluence and regardless of the child’s grade level.
Thus, the observed disparities in youth obesity by area income in Pennsylvania do not result from simple differences in disposable income and the relative cost of “healthy” versus “unhealthy” goods and services.
[Keywords: Youth obesity, income, health disparities, Natural experiment]
2020-dellavigna.pdf: “RCTs to Scale: Comprehensive Evidence from Two Nudge Units”, (2020-07-01; ):
Nudge interventions have quickly expanded from academic studies to larger implementation in so-called Nudge Units in governments. This provides an opportunity to compare interventions in research studies, versus at scale. We assemble a unique data set of 126 covering over 23 million individuals, including all trials run by 2 of the largest Nudge Units in the United States. We compare these trials to a sample of nudge trials published in academic journals from 2 recent meta-analyses.
In papers published in academic journals, the average impact of a nudge is very large—an 8.7 percentage point take-up effect, a 33.5% increase over the average control. In the Nudge Unit trials, the average impact is still sizable and highly statistically-significant, but smaller at 1.4 percentage points, an 8.1% increase [8.7 / 1.4 = 6.2×].
We consider 5 potential channels for this gap: statistical power, selective publication, academic involvement, differences in trial features and in nudge features. Publication bias in the academic journals, exacerbated by low statistical power, can account for the full difference in. Academic involvement does not account for the difference. Different features of the nudges, such as in-person versus letter-based communication, likely reflecting institutional constraints, can partially explain the different .
We conjecture that larger sample sizes and institutional constraints, which play an important role in our setting, are relevant in other at-scale implementations. Finally, we compare these results to the predictions of academics and practitioners. Most forecasters overestimate the impact for the Nudge Unit interventions, though nudge practitioners are almost perfectly calibrated.
…In this paper, we present the results of a unique collaboration with 2 of the major “Nudge Units”: BIT North America operating at the level of US cities and SBST/OES for the US Federal government. These 2 units kept a comprehensive record of all trials that they ran from inception in 2015 to July 2019, for a total of 165 trials testing 349 nudge treatments and a sample size of over 37 million participants. In a remarkable case of administrative transparency, each trial had atrial report, including in many cases a pre-analysis plan. The 2 units worked with us to retrieve the results of all the trials. Importantly, over 90% of these trials have not been documented in working paper or academic publication format. [emphasis added]
…Since we are interested in comparing the Nudge Unit trials to nudge papers in the literature, we aim to find broadly comparable studies in academic journals, without hand-picking individual papers. We lean on 2 recent meta-analyses summarizing over 100 RCTs across many different applications (Benartzi et al 2017, and Hummel & Maedche 2019). We apply similar restrictions as we did in the Nudge Unit sample, excluding lab or hypothetical experiments and non- , treatments with financial incentives, requiring treatments with binary dependent variables, and excluding default effects. This leaves a final sample of 26 , including 74 nudge treatments with 505,337 participants. Before we turn to the results, we stress that the features of behavioral interventions in academic journals do not perfectly match with the nudge treatments implemented by the Nudge Units, a difference to which we indeed return below. At the same time, overall interventions conducted by Nudge Units are fairly representative of the type of nudge treatments that are run by researchers.
What do we find? In the sample of 26 papers in the Academic Journals sample, we compute the average (unweighted) impact of a nudge across the 74 nudge interventions. We find that on average a nudge intervention increases the take up by 8.7 (s.e. = 2.5) percentage points, out of an average control take up of 26.0 percentage points.
Turning to the 126 trials by Nudge Units, we estimate an unweighted impact of 1.4 percentage points (s.e. = 0.3), out of an average control take up of 17.4 percentage points. While this impact is highly statistically-significantly different from 0 and sizable, it is about 1⁄6th the size of the estimated nudge impact in academic papers. What explains this large difference in the impact of nudges?
We discuss 3 features of the 2 samples which could account for this difference. First, we document a large difference in the sample size and thus statistical power of the interventions. The median nudge intervention in the Academic Journals sample has treatment arm sample size of 484 participants and a minimum detectable (MDE, the that can be detected with 80% power) of 6.3 percentage points. In contrast, the nudge interventions in the Nudge Units have a median treatment arm sample size of 10,006 participants and MDE of 0.8 percentage points. Thus, the statistical power for the trials in the Academic Journals sample is nearly an order of magnitude smaller. This illustrates a key feature of the “at scale” implementation: the implementation in an administrative setting allows for a larger sample size. Importantly, the smaller sample size for the Academic Journals papers could lead not just to noisier estimates, but also to upward-biased point estimates in the presence of publication bias.
A second difference, directly zooming into publication bias, is the evidence of selective publication of studies with statistically-significant results (t > 1.96), versus studies that are not(t < 1.96). In the sample of Academic Journals nudges, there are over 4× as many studies with a t-statistic for the most statistically-significant nudge between 1.96 and 2.96, versus the number of studies with the most nudge with at between 0.96 and 1.96. Interestingly, the publication bias appears to operate at the level of the most statistically-significant treatment arm within a paper. By comparison, we find no evidence of a discontinuity in the distribution of t-statistics for the Nudge Unit sample, consistent with the fact that the Nudge Unit registry contains the comprehensive sample of all studies run. We stress here that with “publication bias” we include not just whether a journal would publish a paper, but also whether a researcher would write up a study (the “file drawer” problem). In the Nudge Units sample, all these selective steps are removed, as we access all studies that were run.
2013-carrell.pdf: “From Natural Variation to Optimal Policy? The Importance of Endogenous Peer Group Formation”, (2013-05-16; ):
We take cohorts of entering freshmen at the United States Air Force Academy and assign half to peer groups designed to maximize the academic performance of the lowest ability students. Our assignment algorithm uses nonlinear peer effects estimates from the historical pre-treatment data, in which students were randomly assigned to peer groups.
We find a negative andtreatment effect for the students we intended to help. We provide evidence that within our “optimally” designed peer groups, students avoided the peers with whom we intended them to interact and instead formed more homogeneous subgroups.
These results illustrate how policies that manipulate peer groups for a desired social outcome can be confounded by changes in the endogenous patterns of social interactions within the group.
[Keywords: peer effects, social network formation, homophily]
…We first identify nonlinear peer effects at the United States Air Force Academy (USAFA) using pre-treatment data in which students were randomly assigned to peer groups (squadrons) of about 30 students. These estimates showed that low ability students benefited statistically-significantly from being with peers who have high SAT Verbal scores. We use these estimates to create optimally designed peer groups intended to improve academic achievement of the bottom one-third of incoming students by academic ability while not harming achievement of students at other points in the distribution. [This objective function was determined by USAFA senior leadership, who had a strong desire to reduce the academic probation rate, then at roughly 20%.] Using an experimental design, we sorted the incoming college freshman cohorts at USAFA into peer groups during the fall semesters of 2007 and 2008. Half of the students were placed in the control group and randomly assigned to squadrons, as was done with preceding entering classes. The other half of students (the treatment group) were sorted into squadrons in a manner intended to maximize the academic achievement of the students in the lowest third of the predicted grade point average (GPA) distribution. To do so, low ability students were placed into squadrons with a high fraction of peers with high SAT Verbal scores. We refer to these as bimodal squadrons. In the process, the sorting algorithm also created a set of treatment squadrons consisting largely of middle ability students. We call these homogeneous squadrons.
The reduced form coefficients (using the pre-treatment data) predicted a Pareto-improving allocation in which grades of students in the bottom third of the academic distribution would rise, on average, 0.053 grade points while students with higher predicted achievement would be unaffected. Despite this prediction, actual outcomes from the experiment yielded quite different results. For the lowest ability students, we observe a negative andtreatment effect of −0.061 (p = 0.055). For the middle ability students, expected to be unaffected, we observe a positive and statistically-significant treatment effect of 0.082 (p = 0.041). High ability students are unaffected by the treatment.
High and low ability students in the treatment squadrons appear to have segregated themselves into separate social networks, resulting in decreased beneficial social interactions among group members. Survey responses following the experiment show that, compared to the control group, low ability students in the treatment group were much more likely to sort into study (friendship) groups with other low ability students. For the middle ability students, evidence suggests that the positive treatment effect occurred because these students did not interact with low ability students after being placed into the homogeneous squadrons.
…Well known difficulties exist in the application of policy to affect a desired outcome. General equilibrium responses as in Lucas 1976 or Acemoglu 2010 can undo effects predicted by more simple partial equilibrium models. Large policy interventions can also lead to political responses by actors and interest groups (Acemoglu 2010). However, we see in our results a different mechanism at work; policy interventions can affect patterns of endogenous social interaction. As such, we believe that endogenous responses to large policy interventions are a major obstacle to foreseeing the effects of manipulating peer groups for a desired social outcome.
…Students in the control group were randomly assigned to one of the 20 control squadrons according to an algorithm that has been used by USAFA since the summer of 2000. The algorithm provides an even distribution of students by demographic characteristics. Specifically, the USAFA admissions office implements a stratified random assignment process where females are first randomly assigned to squadrons. Next, male ethnic and racial minorities are randomly assigned, followed by male non-minority recruited athletes. Students who attended a military preparatory school are then randomly assigned. Finally, all remaining students are randomly assigned to squadrons. Students with the same last name, including siblings, are not placed in the same squadron. This stratified process is accomplished to ensure demographic diversity across peer groups.