N/A
2016-03-15–2021-01-04
finished
certainty: log
importance: 0
This is the April 2016 edition of the Gwern.net newsletter; previous, March 2016. This is a summary of the revision-history RSS feed, overlapping with Changelog & /
Writings
- Nothing completed
Media
Links
Genetics:
Everything Is Heritable:
“Genome-wide association study of cognitive functions and educational attainment in UK Biobank (n = 112151)”, Davies et al 2016
“Variants near CHRNA3/
5 and APOE have age- and sex-related effects on human lifespan” , Joshi et al 2016“National Happiness and Genetic Distance: A Cautious Exploration”, Proto & Oswald 2015 (dubious)
Friends are as genetically similar as fourth cousins (media)
everything is correlated: Continuation of the recent phenome theme of inter-correlations of negative traits and inter-correlation of positive traits:
- “Analysis of shared heritability in common disorders of the brain”, Anttila et al 2016
- “Genetic variants associated with subjective well-being, depressive symptoms, and neuroticism identified through genome-wide analyses”, Okbay et al 2016
- “Physical and neurobehavioral determinants of reproductive onset and success”, Day et al 2016
- “Association between stressful life events and psychotic experiences in adolescence: evidence for gene–environment correlations”, Shakoor et al 2016
Engineering:
- “Introducing precise genetic modifications into human 3PN embryos by CRISPR/
Cas-mediated genome editing” , Kang et al 2016 (see also Liang et al 2015; both describe the CRISPR state of the art as of early 2014, 2+ years ago) - “Monkey kingdom: China is positioning itself as a world leader in primate research”
- “Introducing precise genetic modifications into human 3PN embryos by CRISPR/
Politics/
- Book Review: Albion’s Seed
- “Peer review: Troubled from the start”
- “My Year in San Francisco’s $2 Million Secret Society Startup”
AI:
- “Deep Networks with Stochastic Depth”, Huang et al 2016 (dropout of random layers, which does not just work, but works better and enables >1200-layer residual networks. This might be related to Liao & Poggio 2016 claim that deep residual nets are effectively unrolled RNNs which apply slight variants of an algorithm to many timesteps.)
- “Generating Large Images from Latent Vectors”
- “Foveation-based Mechanisms Alleviate Adversarial Examples”, Luo et al 2016
- “Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Esser et al 2016
Statistics/
- “A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook”, Gordon et al 2019 (How often does correlation=causation? <50% of the time in Internet advertising, with gross over/
underestimates even with detailed covariate & advanced modeling.) - “The Bitter Fight Over the Benefits of Bilingualism” (So Bialystok is refusing a collaboration because a pre-registered protocol is unscientific and she refuses to work with someone so ‘biased’ that they would damage the non-pre-registered results, and besides, publication bias doesn’t exist—“there is absolutely no evidence”. I see…)
- “Why the National Institutes of Health Should Replace Peer Review With a Lottery: A new study shows that peer-review scores for grant proposals are random anyway”
- “Bandit based Monte-Carlo planning”, Kocsis & Szepesvári 2006
- “How Kalman Filters Work, Part 1”/
part 2/ part 3 - “The importance of ‘gold standard’ studies for consumers of research”
Psychology/
- “The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments”, Fryer 2016 (education; lots of small effects)
- “A Meta-Analysis of Blood Glucose Effects on Human Decision Making”, Orquin & Kurzban 2016
- “Is Ego-Depletion a Replicable Effect? A Forensic Meta-Analysis of 165 Ego Depletion Articles”
- “Indoor air quality and academic performance”, Stafford 2015 (???; apropos of harms of CO2 discussion)
- “Seven Pervasive Statistical Flaws in Cognitive Training Interventions”, Moreau et al 2016
- “The Voyeur’s Motel”
Technology:
Economics:
- “Demographic Consequences of Defeating Aging”, Gavrilov & Gavrilova 2010
- Interview with short seller Jim Chanos (Short sellers are always so interesting. Few people are so motivated to see through the miasma of lies and bias and self-serving optimism in the business and finance world.)
Books
Nonfiction:
- A Life of Sir Francis Galton (review)
- On Speed: The Many Lives of Amphetamine, Rasmussen 2008 (particularly interesting for the inside information about how early American pharmacorps & drug development worked, but much weaker past the ’70s; at least, if you can get past Rasmussen’s huge bias against amphetamines and blind support for the disastrous War on Drugs)
Film/TV
Live-action:
Anime:
- Megazone 23 (a strange relict. It’s as if Macross cuckolded Streets of Fire with Akira while Miyazaki perved at the window and the Gainax boys took notes. Episode 1 in particular crosses so many genres and does so much in so little time that even non-fans of 1980s anime might find it worthwhile.)
- Majin Tantei Nougami Neuro (review)
Music
Touhou:
- “音の瓶詰—河童の里” (ジャム; 守矢幻燈録 ~ Separation of Spirit {R12}) [classical]
- “Reincarnation” (Mitty; Imitation Circus {C89}) [classical]
- “Anois –” (Casket/
Escarmew; 幻想パブ:午後11時23分58秒 {科学世紀のカフェテラス6}) [Celtic] - “デザイアドライブ” (Marasy; 幻想遊戯<神> ~ Museum of marasy {R12}) [classical]
- “古きユアンシェン” (Marasy; 幻想遊戯<神> ~ Museum of marasy {R12}) [classical]
- “聖徳伝説 ~ True Administrator” (Marasy; 幻想遊戯<神> ~ Museum of marasy {R12}) [classical]
- “欲深き霊魂” (Marasy; 幻想遊戯<神> ~ Museum of marasy {R12}) [classical]
Link Bibliography
Bibliography of page links in reading order (with annotations when available):
“March 2016 News”, (2016-03-01):
N/A
“Changelog”, (2013-09-15):
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/
changes/ additions. Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular /
r/ subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).Gwern “/r/gwern subreddit”, (2018-10-01):
A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.
“Genome-wide association study of cognitive functions and educational attainment in UK Biobank (n = 112 151)”, (2016-04-05):
People’s differences in cognitive functions are partly heritable and are associated with important life outcomes. Previous genome-wide association (GWA) studies of cognitive functions have found evidence for polygenic effects yet, to date, there are few replicated genetic associations. Here we use data from the UK Biobank sample to investigate the genetic contributions to variation in tests of three cognitive functions and in educational attainment. GWA analyses were performed for verbal-numerical reasoning (n = 36 035), memory (n = 112 067), reaction time (n = 111 483) and for the attainment of a college or a university degree (n = 111 114). We report genome-wide significant single-nucleotide polymorphism (SNP)-based associations in 20 genomic regions, and significant gene-based findings in 46 regions. These include findings in the ATXN2, CYP2DG, APBA1 and CADM2 genes. We report replication of these hits in published GWA studies of cognitive function, educational attainment and childhood intelligence. There is also replication, in UK Biobank, of SNP hits reported previously in GWA studies of educational attainment and cognitive function. GCTA-GREML analyses, using common SNPs (minor allele frequency>0.01), indicated significant SNP-based heritabilities of 31% (s.e.m.=1.8%) for verbal-numerical reasoning, 5% (s.e.m. = 0.6%) for memory, 11% (s.e.m. = 0.6%) for reaction time and 21% (s.e.m. = 0.6%) for educational attainment. Polygenic score analyses indicate that up to 5% of the variance in cognitive test scores can be predicted in an independent cohort. The genomic regions identified include several novel loci, some of which have been associated with intracranial volume, neurodegeneration, Alzheimer’s disease and schizophrenia.
“Everything Is Correlated”, (2014-09-12):
Statistical folklore asserts that “everything is correlated”: in any real-world dataset, most or all measured variables will have non-zero correlations, even between variables which appear to be completely independent of each other, and that these correlations are not merely sampling error flukes but will appear in large-scale datasets to arbitrarily designated levels of statistical-significance or posterior probability.
This raises serious questions for null-hypothesis statistical-significance testing, as it implies the null hypothesis of 0 will always be rejected with sufficient data, meaning that a failure to reject only implies insufficient data, and provides no actual test or confirmation of a theory. Even a directional prediction is minimally confirmatory since there is a 50% chance of picking the right direction at random.
It also has implications for conceptualizations of theories & causal models, interpretations of structural models, and other statistical principles such as the “sparsity principle”.
“Analysis of shared heritability in common disorders of the brain”, (2018-06-22):
Disorders of the brain can exhibit considerable epidemiological comorbidity and often share symptoms, provoking debate about their etiologic overlap. We quantified the genetic sharing of 25 brain disorders from genome-wide association studies of 265,218 patients and 784,643 control participants and assessed their relationship to 17 phenotypes from 1,191,588 individuals. Psychiatric disorders share common variant risk, whereas neurological disorders appear more distinct from one another and from the psychiatric disorders. We also identified significant sharing between disorders and a number of brain phenotypes, including cognitive measures. Further, we conducted simulations to explore how statistical power, diagnostic misclassification, and phenotypic heterogeneity affect genetic correlations. These results highlight the importance of common genetic variation as a risk factor for brain disorders and the value of heritability-based methods in understanding their etiology.
“Physical and neurobehavioral determinants of reproductive onset and success”, (2016-04-18):
The ages of puberty, first sexual intercourse and first birth signify the onset of reproductive ability, behavior and success, respectively. In a genome-wide association study of 125,667 UK Biobank participants, we identify 38 loci associated (p < 5 × 10−8) with age at first sexual intercourse. These findings were taken forward in 241,910 men and women from Iceland and 20,187 women from the Women’s Genome Health Study. Several of the identified loci also exhibit associations (p < 5 × 10−8) with other reproductive and behavioral traits, including age at first birth (variants in or near ESR1 and RBM6–SEMA3F), number of children (CADM2 and ESR1), irritable temperament (MSRA) and risk-taking propensity (CADM2). Mendelian randomization analyses infer causal influences of earlier puberty timing on earlier first sexual intercourse, earlier first birth and lower educational attainment. In turn, likely causal consequences of earlier first sexual intercourse include reproductive, educational, psychiatric and cardiometabolic outcomes.
http:/
/ bjp.rcpsych.org/ content/ early/ 2016/ 03/ 17/ bjp.bp.114.159079.abstract https:/
/ slatestarcodex.com/ 2016/ 04/ 27/ book-review-albions-seed/ http:/
/ www.nature.com/ news/ peer-review-troubled-from-the-start-1.19763 https:/
/ motherboard.vice.com/ read/ my-year-in-san-franciscos-2-million-secret-society-startup “Deep Networks with Stochastic Depth”, (2016-03-30):
Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time. We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function. This simple approach complements the recent success of residual networks. It reduces training time substantially and improves the test error significantly on almost all data sets that we used for evaluation. With stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4.91
“Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex”, (2016-04-13):
We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a shallow RNN is exactly equivalent to a very deep ResNet with weight sharing among the layers. A direct implementation of such a RNN, although having orders of magnitude fewer parameters, leads to a performance similar to the corresponding ResNet. We propose 1) a generalization of both RNN and ResNet architectures and 2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex. We demonstrate the effectiveness of the architectures by testing them on the CIFAR-10 dataset.
http:/
/ blog.otoro.net/ 2016/ 04/ 01/ generating-large-images-from-latent-vectors/ “Foveation-based Mechanisms Alleviate Adversarial Examples”, (2015-11-19):
We show that adversarial examples, i.e., the visually imperceptible perturbations that result in Convolutional Neural Networks (CNNs) fail, can be alleviated with a mechanism based on foveations—applying the CNN in different image regions. To see this, first, we report results in ImageNet that lead to a revision of the hypothesis that adversarial perturbations are a consequence of CNNs acting as a linear classifier: CNNs act locally linearly to changes in the image regions with objects recognized by the CNN, and in other regions the CNN may act non-linearly. Then, we corroborate that when the neural responses are linear, applying the foveation mechanism to the adversarial example tends to significantly reduce the effect of the perturbation. This is because, hypothetically, the CNNs for ImageNet are robust to changes of scale and translation of the object produced by the foveation, but this property does not generalize to transformations of the perturbation. As a result, the accuracy after a foveation is almost the same as the accuracy of the CNN without the adversarial perturbation, even if the adversarial perturbation is calculated taking into account a foveation.
“Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, (2016-03-28):
Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classification accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1200 and 2600 frames per second and using between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. For the first time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
“A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook”, (2019-05-04):
Measuring the causal effects of digital advertising remains challenging despite the availability of granular data. Unobservable factors make exposure endogenous, and advertising’s effect on outcomes tends to be small. In principle, these concerns could be addressed using randomized controlled trials (RCTs). In practice, few online ad campaigns rely on RCTs and instead use observational methods to estimate ad effects. We assess empirically whether the variation in data typically available in the advertising industry enables observational methods to recover the causal effects of online advertising. Using data from 15 U.S. advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions, we contrast the experimental results to those obtained from multiple observational models. The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables. In our setting, advances in causal inference methods do not allow us to isolate the exogenous variation needed to estimate the treatment effects. We also characterize the incremental explanatory power our data would require to enable observational methods to successfully measure advertising effects. Our findings suggest that commonly used observational approaches based on the data usually available in the industry often fail to accurately measure the true effect of advertising.
http:/
/ www.theatlantic.com/ science/ archive/ 2016/ 02/ the-battle-over-bilingualism/ 462114/ https:/
/ web.engr.oregonstate.edu/ ~afern/ classes/ cs533/ notes/ uct.pdf http:/
/ www.anuncommonlab.com/ articles/ how-kalman-filters-work/ http:/
/ www.anuncommonlab.com/ articles/ how-kalman-filters-work/ part2.html http:/
/ www.anuncommonlab.com/ articles/ how-kalman-filters-work/ part3.html http:/
/ scholar.harvard.edu/ files/ fryer/ files/ handbook_fryer_03.25.2016.pdf http:/
/ lesswrong.com/ r/ discussion/ lw/ nk0/ what_is_up_with_carbon_dioxide_and_cognition_an/ http:/
/ journal.frontiersin.org/ article/ 10.3389/ fnhum.2016.00153/ full http:/
/ www.newyorker.com/ magazine/ 2016/ 04/ 11/ gay-talese-the-voyeurs-motel https:/
/ eaves.ca/ 2013/ 11/ 07/ what-werewolf-teaches-us-about-trust-security/ “Raising of Chicago”, (2020-12-22):
During the 1850s and 1860s, engineers carried out a piecemeal raising of the level of central Chicago. Streets, sidewalks, and buildings were physically raised on jackscrews. The work was funded by private property owners and public funds.
“Demographic Consequences of Defeating Aging”, (2010-04):
A common objection against starting a large-scale biomedical war on aging is the fear of catastrophic population consequences (overpopulation). This fear is only exacerbated by the fact that no detailed demographic projections for radical life extension scenario have been conducted so far. This study explores different demographic scenarios and population projections, in order to clarify what could be the demographic consequences of a successful biomedical war on aging. A general conclusion of this study is that population changes are surprisingly slow in their response to a dramatic life extension. For example, we applied the cohort-component method of population projections to 2005 Swedish population for several scenarios of life extension and a fertility schedule observed in 2005. Even for very long 100-year projection horizon, with the most radical life extension scenario (assuming no aging at all after age 60), the total population increases by 22% only (from 9.1 to 11.0 million). Moreover, if some members of society reject to use new anti-aging technologies for some religious or any other reasons (inconvenience, non-compliance, fear of side effects, costs, etc.), then the total population size may even decrease over time. Thus, even in the case of the most radical life extension scenario, population growth could be relatively slow and may not necessarily lead to overpopulation. Therefore, the real concerns should be placed not on the threat of catastrophic population consequences (overpopulation), but rather on such potential obstacles to a success of biomedical war on aging, as scientific, organizational, and financial limitations.
http:/
/ www.amazon.com/ Life-Sir-Francis-Galton-Exploration/ dp/ 0195143655 http:/
/ www.amazon.com/ Speed-Many-Lives-Amphetamine/ dp/ 0814776396 “Pirates of Silicon Valley”, (2020-12-28):
Pirates of Silicon Valley is a 1999 American biographical drama television film directed by Martyn Burke and starring Noah Wyle as Steve Jobs and Anthony Michael Hall as Bill Gates. Spanning the years 1971–1997 and based on Paul Freiberger and Michael Swaine's 1984 book Fire in the Valley: The Making of the Personal Computer, it explores the impact that the rivalry between Jobs and Gates (Microsoft) had on the development of the personal computer. The film premiered on TNT on June 20, 1999.
“Megazone 23”, (2020-12-28):
Megazone 23 is a four-part Japanese cyberpunk original video animation created by Noboru Ishiguro, written by Hiroyuki Hoshiyama and Emu Arii, and directed by Ishiguro, Ichiro Itano, Kenichi Yatagai, and Shinji Aramaki. The series debuted in 1985. It was originally titled Omega Zone 23 but the title was changed just before release.
“Neuro: Supernatural Detective”, (2020-12-28):
Neuro: Supernatural Detective, known in Japan as Majin Tantei Nōgami Neuro, is a Japanese manga series written and illustrated by Yūsei Matsui. The series follows Neuro Nōgami, a demon who depends on mysteries for sustenance. Having consumed all the mysteries in the demon world, Neuro travels to the human world in search of more. There, Neuro recruits high school student Yako Katsuragi as a facade for a detective agency. The supernatural-themed manga was created because Matsui considered himself unable to draw humans.
“Gwern.net newsletter (Substack subscription page)”, (2013-12-01):
Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.
“Gwern.net newsletter archives”, (2013-12-01):
Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)