“The Bitter Fight Over the Benefits of Bilingualism” (So Bialystok is refusing a collaboration because a pre-registered protocol is unscientific and she refuses to work with someone so ‘biased’ that they would damage the non-pre-registered results, and besides, publication bias doesn’t exist—“there is absolutely no evidence”. I see…)
Interview with short seller Jim Chanos (Short sellers are always so interesting. Few people are so motivated to see through the miasma of lies and bias and self-serving optimism in the business and finance world.)
On Speed: The Many Lives of Amphetamine, Rasmussen 2008 (particularly interesting for the inside information about how early American pharmacorps & drug development worked, but much weaker past the ’70s; at least, if you can get past Rasmussen’s huge bias against amphetamines and blind support for the disastrous War on Drugs)
Megazone 23 (a strange relict. It’s as if Macross cuckolded Streets of Fire with Akira while Miyazaki perved at the window and the Gainax boys took notes. Episode 1 in particular crosses so many genres and does so much in so little time that even non-fans of 1980s anime might find it worthwhile.)
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/changes/additions.
Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular /r/Gwern subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).
A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.
People’s differences in cognitive functions are partly heritable and are associated with important life outcomes. Previous genome-wide association (GWA) studies of cognitive functions have found evidence for polygenic effects yet, to date, there are few replicated genetic associations. Here we use data from the UK Biobank sample to investigate the genetic contributions to variation in tests of three cognitive functions and in educational attainment. GWA analyses were performed for verbal-numerical reasoning (n = 36 035), memory (n = 112 067), reaction time (n = 111 483) and for the attainment of a college or a university degree (n = 111 114). We report genome-wide significant single-nucleotide polymorphism (SNP)-based associations in 20 genomic regions, and significant gene-based findings in 46 regions. These include findings in the ATXN2, CYP2DG, APBA1 and CADM2 genes. We report replication of these hits in published GWA studies of cognitive function, educational attainment and childhood intelligence. There is also replication, in UK Biobank, of SNP hits reported previously in GWA studies of educational attainment and cognitive function. GCTA-GREML analyses, using common SNPs (minor allele frequency>0.01), indicated significant SNP-based heritabilities of 31% (s.e.m.=1.8%) for verbal-numerical reasoning, 5% (s.e.m. = 0.6%) for memory, 11% (s.e.m. = 0.6%) for reaction time and 21% (s.e.m. = 0.6%) for educational attainment. Polygenic score analyses indicate that up to 5% of the variance in cognitive test scores can be predicted in an independent cohort. The genomic regions identified include several novel loci, some of which have been associated with intracranial volume, neurodegeneration, Alzheimer’s disease and schizophrenia.
Statistical folklore asserts that “everything is correlated”: in any real-world dataset, most or all measured variables will have non-zero correlations, even between variables which appear to be completely independent of each other, and that these correlations are not merely sampling error flukes but will appear in large-scale datasets to arbitrarily designated levels of statistical-significance or posterior probability.
This raises serious questions for null-hypothesis statistical-significance testing, as it implies the null hypothesis of 0 will always be rejected with sufficient data, meaning that a failure to reject only implies insufficient data, and provides no actual test or confirmation of a theory. Even a directional prediction is minimally confirmatory since there is a 50% chance of picking the right direction at random.
It also has implications for conceptualizations of theories & causal models, interpretations of structural models, and other statistical principles such as the “sparsity principle”.
Disorders of the brain can exhibit considerable epidemiological comorbidity and often share symptoms, provoking debate about their etiologic overlap. We quantified the genetic sharing of 25 brain disorders from genome-wide association studies of 265,218 patients and 784,643 control participants and assessed their relationship to 17 phenotypes from 1,191,588 individuals. Psychiatric disorders share common variant risk, whereas neurological disorders appear more distinct from one another and from the psychiatric disorders. We also identified significant sharing between disorders and a number of brain phenotypes, including cognitive measures. Further, we conducted simulations to explore how statistical power, diagnostic misclassification, and phenotypic heterogeneity affect genetic correlations. These results highlight the importance of common genetic variation as a risk factor for brain disorders and the value of heritability-based methods in understanding their etiology.
“Physical and neurobehavioral determinants of reproductive onset and success”, Felix R. Day, Hannes Helgason, Daniel I. Chasman, Lynda M. Rose, Po-Ru Loh, Robert A. Scott, Agnar Helgason, Augustine Kong, Gisli Masson, Olafur Th Magnusson, Daniel Gudbjartsson, Unnur Thorsteinsdottir, Julie E. Buring, Paul M. Ridker, Patrick Sulem, Kari Stefansson, Ken K. Ong & John R. B. Perry (2016-04-18):
The ages of puberty, first sexual intercourse and first birth signify the onset of reproductive ability, behavior and success, respectively. In a genome-wide association study of 125,667 UK Biobank participants, we identify 38 loci associated (p < 5 × 10−8) with age at first sexual intercourse. These findings were taken forward in 241,910 men and women from Iceland and 20,187 women from the Women’s Genome Health Study. Several of the identified loci also exhibit associations (p < 5 × 10−8) with other reproductive and behavioral traits, including age at first birth (variants in or near ESR1 and RBM6–SEMA3F), number of children (CADM2 and ESR1), irritable temperament (MSRA) and risk-taking propensity (CADM2). Mendelian randomization analyses infer causal influences of earlier puberty timing on earlier first sexual intercourse, earlier first birth and lower educational attainment. In turn, likely causal consequences of earlier first sexual intercourse include reproductive, educational, psychiatric and cardiometabolic outcomes.
Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time. We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function. This simple approach complements the recent success of residual networks. It reduces training time substantially and improves the test error significantly on almost all data sets that we used for evaluation. With stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4.91
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a shallow RNN is exactly equivalent to a very deep ResNet with weight sharing among the layers. A direct implementation of such a RNN, although having orders of magnitude fewer parameters, leads to a performance similar to the corresponding ResNet. We propose 1) a generalization of both RNN and ResNet architectures and 2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex. We demonstrate the effectiveness of the architectures by testing them on the CIFAR-10 dataset.
We show that adversarial examples, i.e., the visually imperceptible perturbations that result in Convolutional Neural Networks (CNNs) fail, can be alleviated with a mechanism based on foveations—applying the CNN in different image regions. To see this, first, we report results in ImageNet that lead to a revision of the hypothesis that adversarial perturbations are a consequence of CNNs acting as a linear classifier: CNNs act locally linearly to changes in the image regions with objects recognized by the CNN, and in other regions the CNN may act non-linearly. Then, we corroborate that when the neural responses are linear, applying the foveation mechanism to the adversarial example tends to significantly reduce the effect of the perturbation. This is because, hypothetically, the CNNs for ImageNet are robust to changes of scale and translation of the object produced by the foveation, but this property does not generalize to transformations of the perturbation. As a result, the accuracy after a foveation is almost the same as the accuracy of the CNN without the adversarial perturbation, even if the adversarial perturbation is calculated taking into account a foveation.
“Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner, Dharmendra S. Modha (2016-03-28):
Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classification accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1200 and 2600 frames per second and using between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. For the first time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
Measuring the causal effects of digital advertising remains challenging despite the availability of granular data. Unobservable factors make exposure endogenous, and advertising’s effect on outcomes tends to be small. In principle, these concerns could be addressed using randomized controlled trials (RCTs). In practice, few online ad campaigns rely on RCTs and instead use observational methods to estimate ad effects. We assess empirically whether the variation in data typically available in the advertising industry enables observational methods to recover the causal effects of online advertising. Using data from 15 U.S. advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions, we contrast the experimental results to those obtained from multiple observational models. The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables. In our setting, advances in causal inference methods do not allow us to isolate the exogenous variation needed to estimate the treatment effects. We also characterize the incremental explanatory power our data would require to enable observational methods to successfully measure advertising effects. Our findings suggest that commonly used observational approaches based on the data usually available in the industry often fail to accurately measure the true effect of advertising.
During the 1850s and 1860s, engineers carried out a piecemeal raising of the level of central Chicago. Streets, sidewalks, and buildings were physically raised on jackscrews. The work was funded by private property owners and public funds.
A common objection against starting a large-scale biomedical war on aging is the fear of catastrophic population consequences (overpopulation). This fear is only exacerbated by the fact that no detailed demographic projections for radical life extension scenario have been conducted so far. This study explores different demographic scenarios and population projections, in order to clarify what could be the demographic consequences of a successful biomedical war on aging. A general conclusion of this study is that population changes are surprisingly slow in their response to a dramatic life extension. For example, we applied the cohort-component method of population projections to 2005 Swedish population for several scenarios of life extension and a fertility schedule observed in 2005. Even for very long 100-year projection horizon, with the most radical life extension scenario (assuming no aging at all after age 60), the total population increases by 22% only (from 9.1 to 11.0 million). Moreover, if some members of society reject to use new anti-aging technologies for some religious or any other reasons (inconvenience, non-compliance, fear of side effects, costs, etc.), then the total population size may even decrease over time. Thus, even in the case of the most radical life extension scenario, population growth could be relatively slow and may not necessarily lead to overpopulation. Therefore, the real concerns should be placed not on the threat of catastrophic population consequences (overpopulation), but rather on such potential obstacles to a success of biomedical war on aging, as scientific, organizational, and financial limitations.
Pirates of Silicon Valley is a 1999 American biographical drama television film directed by Martyn Burke and starring Noah Wyle as Steve Jobs and Anthony Michael Hall as Bill Gates. Spanning the years 1971–1997 and based on Paul Freiberger and Michael Swaine's 1984 book Fire in the Valley: The Making of the Personal Computer, it explores the impact that the rivalry between Jobs and Gates (Microsoft) had on the development of the personal computer. The film premiered on TNT on June 20, 1999.
Megazone 23 is a four-part Japanese cyberpunk original video animation created by Noboru Ishiguro, written by Hiroyuki Hoshiyama and Emu Arii, and directed by Ishiguro, Ichiro Itano, Kenichi Yatagai, and Shinji Aramaki. The series debuted in 1985. It was originally titled Omega Zone 23 but the title was changed just before release.
Neuro: Supernatural Detective, known in Japan as Majin Tantei Nōgami Neuro, is a Japanese manga series written and illustrated by Yūsei Matsui. The series follows Neuro Nōgami, a demon who depends on mysteries for sustenance. Having consumed all the mysteries in the demon world, Neuro travels to the human world in search of more. There, Neuro recruits high school student Yako Katsuragi as a facade for a detective agency. The supernatural-themed manga was created because Matsui considered himself unable to draw humans.
Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.
Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)