Link Bibliography

1. https://gwern.substack.com

2. “March 2016 News”, Gw­ern Bran­wen (2016-03-01):

   N/A

3. “Changelog”, Gw­ern Bran­wen (2013-09-15):

   This page is a changelog for Gw­ern.net: a monthly re­verse chrono­log­i­cal list of re­cent ma­jor writings/changes/additions.

   Fol­low­ing my writ­ing can be a lit­tle diffi­cult be­cause it is often so in­cre­men­tal. So every mon­th, in ad­di­tion to my reg­u­lar /r/Gwern sub­red­dit sub­mis­sions, I write up rea­son­ably-in­ter­est­ing changes and send it out to the mail­ing list in ad­di­tion to a com­pi­la­tion of links & re­views (archives).

4. “/r/gwern subreddit”, Gw­ern Bran­wen (2018-10-01):

   A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.

5. https://www.patreon.com/gwern

6. “Genome-wide association study of cognitive functions and educational attainment in UK Biobank (n = 112 151)”, G. Davies, R. E. Mar­i­oni, D. C. Liewald, W. D. Hill, S. P. Ha­ge­naars, S. E. Har­ris, S. J. Ritchie, M. Lu­ciano, C. Fawn­s-Ritchie, D. Lyall, B. Cul­len, S. R. Cox, C. Hay­ward, D. J. Por­te­ous, J. Evans, A. M. McIn­tosh, J. Gal­lacher, N. Crad­dock, J. P. Pell, D. J. Smith, C. R. Gale, I. J. Deary (2016-04-05):

   People’s differences in cognitive functions are partly heritable and are associated with important life outcomes. Previous genome-wide association (GWA) studies of cognitive functions have found evidence for polygenic effects yet, to date, there are few replicated genetic associations. Here we use data from the UK Biobank sample to investigate the genetic contributions to variation in tests of three cognitive functions and in educational attainment. GWA analyses were performed for verbal-numerical reasoning (n = 36 035), memory (n = 112 067), reaction time (n = 111 483) and for the attainment of a college or a university degree (n = 111 114). We report genome-wide significant single-nucleotide polymorphism (SNP)-based associations in 20 genomic regions, and significant gene-based findings in 46 regions. These include findings in the ATXN2, CYP2DG, APBA1 and CADM2 genes. We report replication of these hits in published GWA studies of cognitive function, educational attainment and childhood intelligence. There is also replication, in UK Biobank, of SNP hits reported previously in GWA studies of educational attainment and cognitive function. GCTA-GREML analyses, using common SNPs (minor allele frequency>0.01), indicated significant SNP-based heritabilities of 31% (s.e.m.=1.8%) for verbal-numerical reasoning, 5% (s.e.m. = 0.6%) for memory, 11% (s.e.m. = 0.6%) for reaction time and 21% (s.e.m. = 0.6%) for educational attainment. Polygenic score analyses indicate that up to 5% of the variance in cognitive test scores can be predicted in an independent cohort. The genomic regions identified include several novel loci, some of which have been associated with intracranial volume, neurodegeneration, Alzheimer’s disease and schizophrenia.

7. https://www.nature.com/articles/ncomms11174

8. /docs/genetics/selection/2015-proto.pdf

9. http://www.pnas.org/content/111/Supplement_3/10796.long

10. https://www.washingtonpost.com/national/health-science/research-human-friendships-based-on-genetic-similarities-beyond-the-superficial/2014/07/14/8aea04fe-0ab5-11e4-8c9a-923ecc0c7d23_story.html

11. “Everything Is Correlated”, Gw­ern Bran­wen (2014-09-12):

    

    Sta­tis­ti­cal folk­lore as­serts that “every­thing is cor­re­lated”: in any re­al-world dataset, most or all mea­sured vari­ables will have non-zero cor­re­la­tions, even be­tween vari­ables which ap­pear to be com­pletely in­de­pen­dent of each oth­er, and that these cor­re­la­tions are not merely sam­pling er­ror flukes but will ap­pear in large-s­cale datasets to ar­bi­trar­ily des­ig­nated lev­els of sta­tis­ti­cal-sig­nifi­cance or pos­te­rior prob­a­bil­i­ty.

    This raises se­ri­ous ques­tions for nul­l-hy­poth­e­sis sta­tis­ti­cal-sig­nifi­cance test­ing, as it im­plies the null hy­poth­e­sis of 0 will al­ways be re­jected with suffi­cient data, mean­ing that a fail­ure to re­ject only im­plies in­suffi­cient data, and pro­vides no ac­tual test or con­fir­ma­tion of a the­o­ry. Even a di­rec­tional pre­dic­tion is min­i­mally con­fir­ma­tory since there is a 50% chance of pick­ing the right di­rec­tion at ran­dom.

    It also has im­pli­ca­tions for con­cep­tu­al­iza­tions of the­o­ries & causal mod­els, in­ter­pre­ta­tions of struc­tural mod­els, and other sta­tis­ti­cal prin­ci­ples such as the “spar­sity prin­ci­ple”.

12. “Analysis of shared heritability in common disorders of the brain”, The Brain­storm Con­sor­tium (2018-06-22):

    Disorders of the brain can exhibit considerable epidemiological comorbidity and often share symptoms, provoking debate about their etiologic overlap. We quantified the genetic sharing of 25 brain disorders from genome-wide association studies of 265,218 patients and 784,643 control participants and assessed their relationship to 17 phenotypes from 1,191,588 individuals. Psychiatric disorders share common variant risk, whereas neurological disorders appear more distinct from one another and from the psychiatric disorders. We also identified significant sharing between disorders and a number of brain phenotypes, including cognitive measures. Further, we conducted simulations to explore how statistical power, diagnostic misclassification, and phenotypic heterogeneity affect genetic correlations. These results highlight the importance of common genetic variation as a risk factor for brain disorders and the value of heritability-based methods in understanding their etiology.

13. /docs/genetics/correlation/2016-okbay.pdf

14. “Physical and neurobehavioral determinants of reproductive onset and success”, Fe­lix R. Day, Hannes Hel­ga­son, Daniel I. Chas­man, Lynda M. Rose, Po-Ru Loh, Robert A. Scott, Ag­nar Hel­ga­son, Au­gus­tine Kong, Gisli Mas­son, Ola­fur Th Mag­nus­son, Daniel Gud­b­jartsson, Un­nur Thorsteins­dot­tir, Julie E. Bur­ing, Paul M. Rid­ker, Patrick Sulem, Kari Ste­fansson, Ken K. Ong & John R. B. Perry (2016-04-18):

    The ages of puberty, first sexual intercourse and first birth signify the onset of reproductive ability, behavior and success, respectively. In a genome-wide association study of 125,667 UK Biobank participants, we identify 38 loci associated (p < 5 × 10⁻⁸) with age at first sexual intercourse. These findings were taken forward in 241,910 men and women from Iceland and 20,187 women from the Women’s Genome Health Study. Several of the identified loci also exhibit associations (p < 5 × 10⁻⁸) with other reproductive and behavioral traits, including age at first birth (variants in or near ESR1 and RBM6–SEMA3F), number of children (CADM2 and ESR1), irritable temperament (MSRA) and risk-taking propensity (CADM2). Mendelian randomization analyses infer causal influences of earlier puberty timing on earlier first sexual intercourse, earlier first birth and lower educational attainment. In turn, likely causal consequences of earlier first sexual intercourse include reproductive, educational, psychiatric and cardiometabolic outcomes.

15. http://bjp.rcpsych.org/content/early/2016/03/17/bjp.bp.114.159079.abstract

16. /docs/genetics/editing/2016-kang.pdf

17. http://www.nature.com/news/monkey-kingdom-1.19762

18. https://slatestarcodex.com/2016/04/27/book-review-albions-seed/

19. http://www.nature.com/news/peer-review-troubled-from-the-start-1.19763

20. https://motherboard.vice.com/read/my-year-in-san-franciscos-2-million-secret-society-startup

21. “Deep Networks with Stochastic Depth”, Gao Huang, Yu Sun, Zhuang Liu, Daniel Se­dra, Kil­ian Wein­berger (2016-03-30):

    Very deep convolutional networks with hundreds of layers have led to significant reductions in error on competitive benchmarks. Although the unmatched expressiveness of the many layers can be highly desirable at test time, training very deep networks comes with its own set of challenges. The gradients can vanish, the forward flow often diminishes, and the training time can be painfully slow. To address these problems, we propose stochastic depth, a training procedure that enables the seemingly contradictory setup to train short networks and use deep networks at test time. We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function. This simple approach complements the recent success of residual networks. It reduces training time substantially and improves the test error significantly on almost all data sets that we used for evaluation. With stochastic depth we can increase the depth of residual networks even beyond 1200 layers and still yield meaningful improvements in test error (4.91

22. https://old.reddit.com/r/MachineLearning/comments/4fgk8k/160403640v1_bridging_the_gaps_between_residual/d29ezsc

23. “Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex”, Qianli Liao, Tomaso Pog­gio (2016-04-13):

    We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex. We begin with the observation that a shallow RNN is exactly equivalent to a very deep ResNet with weight sharing among the layers. A direct implementation of such a RNN, although having orders of magnitude fewer parameters, leads to a performance similar to the corresponding ResNet. We propose 1) a generalization of both RNN and ResNet architectures and 2) the conjecture that a class of moderately deep RNNs is a biologically-plausible model of the ventral stream in visual cortex. We demonstrate the effectiveness of the architectures by testing them on the CIFAR-10 dataset.

24. http://blog.otoro.net/2016/04/01/generating-large-images-from-latent-vectors/

25. “Foveation-based Mechanisms Alleviate Adversarial Examples”, Yan Luo, Xavier Boix, Gemma Roig, Tomaso Pog­gio, Qi Zhao (2015-11-19):

    We show that adversarial examples, i.e., the visually imperceptible perturbations that result in Convolutional Neural Networks (CNNs) fail, can be alleviated with a mechanism based on foveations—applying the CNN in different image regions. To see this, first, we report results in ImageNet that lead to a revision of the hypothesis that adversarial perturbations are a consequence of CNNs acting as a linear classifier: CNNs act locally linearly to changes in the image regions with objects recognized by the CNN, and in other regions the CNN may act non-linearly. Then, we corroborate that when the neural responses are linear, applying the foveation mechanism to the adversarial example tends to significantly reduce the effect of the perturbation. This is because, hypothetically, the CNNs for ImageNet are robust to changes of scale and translation of the object produced by the foveation, but this property does not generalize to transformations of the perturbation. As a result, the accuracy after a foveation is almost the same as the accuracy of the CNN without the adversarial perturbation, even if the adversarial perturbation is calculated taking into account a foveation.

26. “Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing”, Steven K. Es­ser, Paul A. Merol­la, John V. Arthur, An­drew S. Cas­sidy, Rathi­naku­mar Ap­puswamy, Alexan­der An­dreopoulos, David J. Berg, Jeffrey L. McK­instry, Tim­o­thy Melano, Davis R. Barch, Carmelo di Nol­fo, Pal­lab Dat­ta, Arnon Amir, Brian Taba, My­ron D. Flick­n­er, Dhar­men­dra S. Modha (2016-03-28):

    Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that i) approach state-of-the-art classification accuracy across 8 standard datasets, encompassing vision and speech, ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1200 and 2600 frames per second and using between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. For the first time, the algorithmic power of deep learning can be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.

27. “A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook”, Brett R. Gor­don, Flo­rian Zettelmey­er, Neha Bhar­gava, Dan Chap­sky (2019-05-04):

    Measuring the causal effects of digital advertising remains challenging despite the availability of granular data. Unobservable factors make exposure endogenous, and advertising’s effect on outcomes tends to be small. In principle, these concerns could be addressed using randomized controlled trials (RCTs). In practice, few online ad campaigns rely on RCTs and instead use observational methods to estimate ad effects. We assess empirically whether the variation in data typically available in the advertising industry enables observational methods to recover the causal effects of online advertising. Using data from 15 U.S. advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions, we contrast the experimental results to those obtained from multiple observational models. The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables. In our setting, advances in causal inference methods do not allow us to isolate the exogenous variation needed to estimate the treatment effects. We also characterize the incremental explanatory power our data would require to enable observational methods to successfully measure advertising effects. Our findings suggest that commonly used observational approaches based on the data usually available in the industry often fail to accurately measure the true effect of advertising.

28. http://www.theatlantic.com/science/archive/2016/02/the-battle-over-bilingualism/462114/

29. https://psmag.com/why-the-national-institutes-of-health-should-replace-peer-review-with-a-lottery-41d09bcce5d8

30. https://web.engr.oregonstate.edu/~afern/classes/cs533/notes/uct.pdf

31. http://www.anuncommonlab.com/articles/how-kalman-filters-work/

32. http://www.anuncommonlab.com/articles/how-kalman-filters-work/part2.html

33. http://www.anuncommonlab.com/articles/how-kalman-filters-work/part3.html

34. http://blog.givewell.org/2016/01/19/the-importance-of-gold-standard-studies-for-consumers-of-research/

35. http://scholar.harvard.edu/files/fryer/files/handbook_fryer_03.25.2016.pdf

36. /docs/psychology/2016-orquin.pdf

37. https://replicationindex.wordpress.com/2016/04/18/is-ego-depletion-a-replicable-effect-a-forensic-meta-analysis-of-165-ego-depletion-articles/

38. /docs/co2/2015-stafford.pdf

39. http://lesswrong.com/r/discussion/lw/nk0/what_is_up_with_carbon_dioxide_and_cognition_an/

40. http://journal.frontiersin.org/article/10.3389/fnhum.2016.00153/full

41. http://www.newyorker.com/magazine/2016/04/11/gay-talese-the-voyeurs-motel

42. https://eaves.ca/2013/11/07/what-werewolf-teaches-us-about-trust-security/

43. “Raising of Chicago”, Eng­lish Wikipedia (2020-12-22):

    During the 1850s and 1860s, engineers carried out a piecemeal raising of the level of central Chicago. Streets, sidewalks, and buildings were physically raised on jackscrews. The work was funded by private property owners and public funds.

44. “Demographic Consequences of Defeating Aging”, Leonid A. Gavrilov, Na­talia S. Gavrilova (2010-04):

    A common objection against starting a large-scale biomedical war on aging is the fear of catastrophic population consequences (overpopulation). This fear is only exacerbated by the fact that no detailed demographic projections for radical life extension scenario have been conducted so far. This study explores different demographic scenarios and population projections, in order to clarify what could be the demographic consequences of a successful biomedical war on aging. A general conclusion of this study is that population changes are surprisingly slow in their response to a dramatic life extension. For example, we applied the cohort-component method of population projections to 2005 Swedish population for several scenarios of life extension and a fertility schedule observed in 2005. Even for very long 100-year projection horizon, with the most radical life extension scenario (assuming no aging at all after age 60), the total population increases by 22% only (from 9.1 to 11.0 million). Moreover, if some members of society reject to use new anti-aging technologies for some religious or any other reasons (inconvenience, non-compliance, fear of side effects, costs, etc.), then the total population size may even decrease over time. Thus, even in the case of the most radical life extension scenario, population growth could be relatively slow and may not necessarily lead to overpopulation. Therefore, the real concerns should be placed not on the threat of catastrophic population consequences (overpopulation), but rather on such potential obstacles to a success of biomedical war on aging, as scientific, organizational, and financial limitations.

45. https://web.archive.org/web/20161021040312/https://ftalphaville-cdn.ft.com/wp-content/uploads/2016/04/Alphachatterbox-Chanos-transcript.pdf

46. http://www.amazon.com/Life-Sir-Francis-Galton-Exploration/dp/0195143655

47. /Book-reviews#a-life-of-sir-francis-galton-gillham-2001

48. http://www.amazon.com/Speed-Many-Lives-Amphetamine/dp/0814776396

49. “Pirates of Silicon Valley”, Eng­lish Wikipedia (2020-12-28):

    

    Pirates of Silicon Valley is a 1999 American biographical drama television film directed by Martyn Burke and starring Noah Wyle as Steve Jobs and Anthony Michael Hall as Bill Gates. Spanning the years 1971–1997 and based on Paul Freiberger and Michael Swaine's 1984 book Fire in the Valley: The Making of the Personal Computer, it explores the impact that the rivalry between Jobs and Gates (Microsoft) had on the development of the personal computer. The film premiered on TNT on June 20, 1999.

50. /Movie-reviews#pirates-of-silicon-valley

51. “Megazone 23”, Eng­lish Wikipedia (2020-12-28):

    

    Megazone 23 is a four-part Japanese cyberpunk original video animation created by Noboru Ishiguro, written by Hiroyuki Hoshiyama and Emu Arii, and directed by Ishiguro, Ichiro Itano, Kenichi Yatagai, and Shinji Aramaki. The series debuted in 1985. It was originally titled Omega Zone 23 but the title was changed just before release.

52. “Neuro: Supernatural Detective”, Eng­lish Wikipedia (2020-12-28):

    

    Neuro: Supernatural Detective, known in Japan as Majin Tantei Nōgami Neuro, is a Japanese manga series written and illustrated by Yūsei Matsui. The series follows Neuro Nōgami, a demon who depends on mysteries for sustenance. Having consumed all the mysteries in the demon world, Neuro travels to the human world in search of more. There, Neuro recruits high school student Yako Katsuragi as a facade for a detective agency. The supernatural-themed manga was created because Matsui considered himself unable to draw humans.

53. /Anime-reviews#majin-tantei-nougami-neuro

54. https://www.dropbox.com/s/6rtjgo76begffsp/%E3%82%B8%E3%83%A3%E3%83%A0-separationofspirit-%E9%9F%B3%E3%81%AE%E7%93%B6%E8%A9%B0%E6%B2%B3%E7%AB%A5%E3%81%AE%E9%87%8C.ogg

55. https://www.youtube.com/watch?v=baIb72Klcio

56. https://www.dropbox.com/s/6o10gcnvsqjus9g/casket-%E5%B9%BB%E6%83%B3%E3%83%91%E3%83%96%E5%8D%88%E5%BE%8C11%E6%99%8223%E5%88%8658%E7%A7%92-anois.ogg

57. https://www.dropbox.com/s/k9puqk4ae3giv5k/marasy-%E5%B9%BB%E6%83%B3%E9%81%8A%E6%88%AF%EF%BC%9C%E7%A5%9E%EF%BC%9E-%E3%83%87%E3%82%B6%E3%82%A4%E3%82%A2%E3%83%89%E3%83%A9%E3%82%A4%E3%83%96.ogg

58. https://www.dropbox.com/s/pme31ntlg30n96k/marasy-%E5%B9%BB%E6%83%B3%E9%81%8A%E6%88%AF%EF%BC%9C%E7%A5%9E%EF%BC%9E-%E5%8F%A4%E3%81%8D%E3%83%A6%E3%82%A2%E3%83%B3%E3%82%B7%E3%82%A7%E3%83%B3.ogg

59. https://www.dropbox.com/s/4rpp3pzsfo0cj7v/marasy-%E5%B9%BB%E6%83%B3%E9%81%8A%E6%88%AF%EF%BC%9C%E7%A5%9E%EF%BC%9E-%E8%81%96%E5%BE%B3%E4%BC%9D%E8%AA%AC%EF%BD%9Etrueadministrator.ogg

60. https://www.dropbox.com/s/gajpzajn4tdz9sk/marasy-%E5%B9%BB%E6%83%B3%E9%81%8A%E6%88%AF%EF%BC%9C%E7%A5%9E%EF%BC%9E-%E6%AC%B2%E6%B7%B1%E3%81%8D%E9%9C%8A%E9%AD%82.ogg

61. “Gwern.net newsletter (Substack subscription page)”, Gw­ern Bran­wen (2013-12-01):

    Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.

62. “Gwern.net newsletter archives”, Gw­ern Bran­wen (2013-12-01):

    Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)

63. https://old.reddit.com/r/gwern/.rss