newsletter/2019/04 (Link Bibliography)

“newsletter/​2019/​04” links:

  1. 04

  2. https://gwern.substack.com

  3. 03

  4. newsletter

  5. Changelog

  6. https://www.patreon.com/gwern

  7. Everything

  8. ⁠, Saskia Selzam, Stuart J. Ritchie, Jean-Baptiste Pingault, Chandra A. Reynolds, Paul F. O’Reilly, Robert Plomin (2019-04-10):

    Polygenic scores are a popular tool for prediction of complex traits. However, prediction estimates in samples of unrelated participants can include effects of population stratification, assortative mating and environmentally mediated parental genetic effects, a form of genotype-environment correlation (rGE). Comparing genome-wide (GPS) predictions in unrelated individuals with predictions between siblings in a within-family design is a powerful approach to identify these different sources of prediction.

    Here, we compared within-family to between-family GPS predictions of eight life outcomes (anthropometric, cognitive, personality and health) for eight corresponding GPSs. The outcomes were assessed in up to 2,366 dizygotic (DZ) twin pairs from the Twins Early Development Study from age 12 to age 21. To account for family clustering, we used mixed-effects modelling, simultaneously estimating within-family and between-family effects for target-trait and cross-trait GPS prediction of the outcomes.

    There were three main findings: (1) DZ twin GPS differences predicted DZ differences in height, ⁠, intelligence, educational achievement and symptoms; (2) target and cross-trait analyses indicated that GPS prediction estimates for cognitive traits (intelligence and educational achievement) were on average 60% greater between families than within families, but this was not the case for non-cognitive traits; and (3) this within-family and between-family difference for cognitive traits disappeared after controlling for family (SES), suggesting that SES is a source of between-family prediction through rGE mechanisms.

    These results provide novel insights into the patterns by which rGE contributes to GPS prediction, while ruling out confounding due to population stratification and mating.

  9. https://www.cell.com/cell/fulltext/S0092-8674(19)30290-9

  10. ⁠, R. Cheesman, J. Coleman, C. Rayner, K. L. Purves, G. Morneau-Vaillancourt, K. Glanville, S. W. Choi, G. Breen, T. C. Eley (2019-03-20):

    Genome-wide studies often exclude family members, even though they are a valuable source of information. We identified parent-offspring pairs, siblings and couples in the and implemented a family-based DNA-derived heritability method to capture additional genetic effects and multiple sources of environmental influence on neuroticism and years of education. Compared to estimates from unrelated individuals, heritability increased from 10% to 27% and from 19% to 57% for neuroticism and education respectively by including family-based genetic effects. We detected no family environmental influences on neuroticism, but years of education was substantially influenced by couple similarity (38%). Overall, our genetic and environmental estimates closely replicate previous findings from an independent sample, but more research is required to dissect contributions to the additional heritability, particularly rare and structural genetic effects and residual environmental ⁠. The latter is especially relevant for years of education, a highly socially-contingent variable, for which our heritability estimate is at the upper end of twin estimates in the literature. Family-based genetic effects narrow the gap between twin and DNA-based heritability methods, and could be harnessed to improve polygenic prediction.

  11. ⁠, Philip R. Jansen, Mats Nagel, Kyoko Watanabe, Yongbin Wei, Jeanne E. Savage, Christiaan A. de Leeuw, Martijn P. van den Heuvel, Sophie van der Sluis, Danielle Posthuma (2019-04-19):

    The phenotypic correlation between human intelligence and brain volume (BV) is considerable (r≈0.40), and has been shown to be due to shared genetic factors1. To further examine specific genetic factors driving this correlation, we present genomic analyses of the genetic overlap between intelligence and BV using genome-wide association study () results. First, we conducted the largest BV GWAS meta-analysis to date (n = 54,407 individuals), followed by functional annotation and gene-mapping. We identified 35 genomic loci (27 novel), implicating 362 genes (346 novel) and 23 biological pathways for BV. Second, we used an existing GWAS for intelligence (n = 269,867 individuals2), and estimated the (rg) between BV and intelligence to be 0.23. We show that the rg is driven by physical overlap of GWAS hits in 5 genomic loci. We identified 67 shared genes between BV and intelligence, which are mainly involved in important signaling pathways regulating cell growth. Out of these 67 we prioritized 32 that are most likely to have functional impact. These results provide new information on the genetics of BV and provide biological insight into BV’s shared genetic etiology with intelligence.

  12. 2019-ager.pdf: “The Intergenerational Effects of a Large Wealth Shock: White Southerners After the Civil War”⁠, Philipp Ager, Leah Platt Boustan, Katherine Eriksson

  13. https://openai.com/blog/how-to-train-your-openai-five/

  14. https://arena.openai.com/#/results

  15. https://old.reddit.com/r/reinforcementlearning/comments/bctqmv/n_openai_five_dota2_finals_match_livestream_has/

  16. 2019-anumanchipalli.pdf: ⁠, Gopala K. Anumanchipalli, Josh Chartier, Edward F. Chang (2019-04-24; ai):

    Technology that translates neural activity into speech would be transformative for people who are unable to communicate as a result of neurological impairments. Decoding speech from neural activity is challenging because speaking requires very precise and rapid multi-dimensional control of vocal tract articulators.

    Here we designed a neural decoder that explicitly leverages kinematic and sound representations encoded in human cortical activity to synthesize audible speech. Recurrent neural networks first decoded directly recorded cortical activity into representations of articulatory movement, and then transformed these representations into speech acoustics. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. Intermediate articulatory dynamics enhanced performance even with limited data. Decoded articulatory representations were highly conserved across speakers, enabling a component of the decoder to be transferable across participants. Furthermore, the decoder could synthesize speech when a participant silently mimed sentences.

    These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication.

  17. ⁠, Christine Payne () (2019-04-25):

    We’ve created MuseNet, a deep neural network that can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles. MuseNet was not explicitly programmed with our understanding of music, but instead discovered patterns of harmony, rhythm, and style by learning to predict the next token in hundreds of thousands of MIDI files. MuseNet uses the same general-purpose unsupervised technology as ⁠, a large-scale transformer model trained to predict the next token in a sequence, whether audio or text.

    [See also: ⁠, Child et al 2019

    Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length. In this paper we introduce sparse factorizations of the attention matrix which reduce this to 𝒪(n ⋅ √n). We also introduce (a) a variation on architecture and initialization to train deeper networks, (b) the recomputation of attention matrices to save memory, and (c) fast attention kernels for training. We call networks with these changes Sparse ⁠, and show they can model sequences tens of thousands of timesteps long using hundreds of layers. We use the same architecture to model images, audio, and text from raw bytes, setting a new state of the art for density modeling of Enwik8, CIFAR-10, and ImageNet-64. We generate unconditional samples that demonstrate global coherence and great diversity, and show it is possible in principle to use self-attention to model sequences of length one million or more. ]

  18. ⁠, Rewon Child, Scott Gray, Alec Radford, Ilya Sutskever (2019-04-23):

    Transformers are powerful sequence models, but require time and memory that grows quadratically with the sequence length.

    In this paper we introduce sparse factorizations of the attention matrix which reduce this to 𝑂(nn). We also introduce (1) a variation on architecture and initialization to train deeper networks, (2) the recomputation of attention matrices to save memory, and (3) fast attention kernels for training. We call networks with these changes Sparse Transformers, and show they can model sequences tens of thousands of timesteps long using hundreds of layers.

    We use the same architecture to model images, audio, and text from raw bytes, setting a new state of the art for density modeling of Enwik8, CIFAR-10, and -64.

    We generate unconditional samples that demonstrate global coherence and great diversity, and show it is possible in principle to use self-attention to model sequences of length one million or more.

  19. https://www.twitch.tv/videos/416276005#openai

  20. https://news.ycombinator.com/item?id=19749767

  21. ⁠, Andrew Cutler, Brian Kulis (2018-05-22):

    This paper explores the use of language models to predict 20 human traits from users’ Facebook status updates. The data was collected by the myPersonality project, and includes user statuses along with their personality, gender, political identification, religion, race, satisfaction with life, IQ, self-disclosure, fair-mindedness, and belief in astrology. A single interpretable model meets state of the art results for well-studied tasks such as predicting gender and personality; and sets the standard on other traits such as IQ, sensational interests, political identity, and satisfaction with life. Additionally, highly weighted words are published for each trait. These lists are valuable for creating hypotheses about human behavior, as well as for understanding what information a model is extracting. Using performance and extracted features we analyze models built on social media. The real world problems we explore include gendered classification bias and Cambridge Analytica’s use of psychographic models.

  22. #gwern-everything

  23. 2019-soto.pdf: ⁠, Christopher J. Soto (2019-01-01; psychology):

    The personality traits have been linked to dozens of life outcomes. However, metascientific research has raised questions about the replicability of behavioral science. The Life Outcomes of Personality Replication (LOOPR) Project was therefore conducted to estimate the replicability of the personality-outcome literature. Specifically, I conducted ⁠, high-powered (median n = 1,504) replications of 78 previously published trait–outcome associations. Overall, 87% of the replication attempts were in the expected direction. The replication effects were typically 77% as strong as the corresponding original effects, which represents a significant decline in ⁠. The replicability of individual effects was predicted by the effect size and design of the original study, as well as the sample size and of the replication. These results indicate that the personality-outcome literature provides a reasonably accurate map of trait–outcome associations but also that it stands to benefit from efforts to improve replicability.

  24. https://old.reddit.com/r/slatestarcodex/comments/bers5o/using_gpt2_to_sort_by_controversial/

  25. https://old.reddit.com/r/slatestarcodex/comments/bo26lv/simulated_culture_war_roundup_thread_using_gpt2/

  26. 2019-vrselja.pdf: ⁠, Zvonimir Vrselja, Stefano G. Daniele, John Silbereis, Francesca Talpo, Yury M. Morozov, André M. M. Sousa, Brian S. Tanaka, Mario Skarica, Mihovil Pletikos, Navjot Kaur, Zhen W. Zhuang, Zhao Liu, Rafeed Alkawadri, Albert J. Sinusas, Stephen R. Latham, Stephen G. Waxman, Nenad Sestan (2019-05-17; longevity):

    The brains of humans and other mammals are highly vulnerable to interruptions in blood flow and decreases in oxygen levels. Here we describe the restoration and maintenance of microcirculation and molecular and cellular functions of the intact pig brain under ex vivo normothermic conditions up to four hours post-mortem. We have developed an extracorporeal pulsatile-perfusion system and a haemoglobin-based, acellular, non-coagulative, echogenic, and cytoprotective perfusate that promotes recovery from anoxia, reduces reperfusion injury, prevents oedema, and metabolically supports the energy requirements of the brain. With this system, we observed preservation of cytoarchitecture; attenuation of cell death; and restoration of vascular dilatory and glial inflammatory responses, spontaneous synaptic activity, and active cerebral metabolism in the absence of global electrocorticographic activity. These findings demonstrate that under appropriate conditions the isolated, intact large mammalian brain possesses an underappreciated capacity for restoration of microcirculation and molecular and cellular activity after a prolonged post-mortem interval.

  27. https://www.nature.com/articles/d41586-019-01216-4

  28. https://www.nature.com/articles/d41586-019-01168-9

  29. https://www.nytimes.com/2019/04/17/science/brain-dead-pigs.html

  30. 2019-obrien.pdf: {#linkBibliography-o’brien-2019 .docMetadata doi=“10.1037/​​pspa0000147”}, E. O'Brien (2019; culture):

    What would it be like to revisit a museum, restaurant, or city you just visited? To rewatch a movie you just watched? To replay a game you just played? People often have opportunities to repeat hedonic activities. Seven studies (total N = 3,356) suggest that such opportunities may be undervalued: Many repeat experiences are not as dull as they appear. Studies 1–3 documented the basic effect. All participants first completed a real-world activity once in full (Study 1, museum exhibit; Study 2, movie; Study 3, video game). Then, some predicted their reactions to repeating it whereas others actually repeated it. Predictors underestimated Experiencers’ enjoyment, even when experienced enjoyment indeed declined. Studies 4 and 5 compared mechanisms: neglecting the pleasurable byproduct of continued exposure to the same content (eg., fluency) versus neglecting the new content that manifests by virtue of continued exposure (eg., discovery), both of which might dilute uniform dullness. We found stronger support for the latter: The misprediction was moderated by stimulus complexity (Studies 4 and 5) and mediated by the amount of novelty discovered within the stimulus (Study 5), holding exposure constant. Doing something once may engender an inflated sense that one has now seen “it”, leaving people naïve to the missed nuances remaining to enjoy. Studies 6 and 7 highlighted consequences: Participants incurred costs to avoid repeats so to maximize enjoyment, in specific contexts for which repetition would have been as enjoyable (Study 6) or more enjoyable (Study 7) as the provided novel alternative. These findings warrant a new look at traditional assumptions about hedonic adaptation and novelty preferences. Repetition too could add an unforeseen spice to life.

  31. https://www.pnas.org/content/early/2019/03/27/1817407116

  32. https://www.newyorker.com/magazine/2019/04/08/the-day-the-dinosaurs-died

  33. https://www.sciencemag.org/news/2019/04/astonishment-skepticism-greet-fossils-claimed-record-dinosaur-killing-asteroid-impact

  34. https://www.smithsonianmag.com/science-nature/fossil-site-captures-dinosaur-killing-impact-its-only-beginning-story-180971868/

  35. https://www.nytimes.com/2019/03/29/science/dinosaurs-extinction-asteroid.html

  36. https://www.nationalgeographic.com/science/2019/03/fossils-found-from-day-dinosaurs-died-chicxulub-tanis-cretaceous-extinction/

  37. https://www.economist.com/science-and-technology/2019/04/06/stony-evidence-of-the-hellfire-that-drove-dinosaurs-to-extinction

  38. ⁠, Ut Na Sio, Thomas C. Ormerod (2009):

    A meta-analytic review of empirical studies that have investigated incubation effects on problem solving is reported. Although some researchers have reported increased solution rates after an incubation period (ie., a period of time in which a problem is set aside to further attempts to solve), others have failed to find effects. The analysis examined the contributions of moderators such as problem type, presence of solution-relevant or misleading cues, and lengths of preparation and incubation periods to incubation effect sizes. The authors identified a positive incubation effect, with divergent thinking tasks benefiting more than linguistic and visual insight tasks from incubation. Longer preparation periods gave a greater incubation effect, whereas filling an incubation period with high cognitive demand tasks gave a smaller incubation effect. Surprisingly, low cognitive demand tasks yielded a stronger incubation effect than did rest during an incubation period when solving linguistic insight problems. The existence of multiple types of incubation effect provides evidence for differential invocation of knowledge-based vs. strategic solution processes across different classes of problem, and it suggests that the conditions under which incubation can be used as a practical technique for enhancing problem solving must be designed with care.

  39. https://www.theatlantic.com/magazine/archive/1999/02/a-new-germ-theory/377430/

  40. https://pdfs.semanticscholar.org/2fa2/17097f8efff6ca2181521d44212cc361be74.pdf

  41. https://old.reddit.com/r/CucumbersScaringCats/

  42. https://www.youtube.com/watch?v=8S0FDjFBj8o

  43. https://archivebox.io/

  44. https://kk.org/thetechnium/was-moores-law/

  45. https://www.wired.com/story/hansa-dutch-police-sting-operation/

  46. https://motherboard.vice.com/amp/en_us/article/9kx59a/silk-road-2-founder-dread-pirate-roberts-2-caught-jailed-for-5-years

  47. https://www.polygon.com/2019/3/27/18281082/elder-scrolls-morrowind-oral-history-bethesda

  48. http://sigbovik.org/2019/

  49. http://sigbovik.org/2019/proceedings.pdf#page=38

  50. http://sigbovik.org/2019/proceedings.pdf#page=48

  51. http://sigbovik.org/2019/proceedings.pdf#page=90

  52. http://sigbovik.org/2019/proceedings.pdf#page=148

  53. http://sigbovik.org/2019/proceedings.pdf#page=158

  54. http://sigbovik.org/2019/proceedings.pdf#page=178

  55. http://sigbovik.org/2019/proceedings.pdf#page=219

  56. https://www.newyorker.com/magazine/2012/07/23/the-strongest-man-in-the-world

  57. ⁠, Alex Churchill, Stella Biderman, Austin Herrick (2019-03-24):

    is a popular and famously complicated trading card game about magical combat. In this paper we show that optimal play in real-world Magic is at least as hard as the Halting Problem, solving a problem that has been open for a decade.

    To do this, we present a methodology for embedding an arbitrary Turing machine into a game of Magic such that the first player is guaranteed to win the game if and only if the Turing machine halts. Our result applies to how real Magic is played, can be achieved using standard-size tournament-legal decks, and does not rely on stochasticity or hidden information. Our result is also highly unusual in that all moves of both players are forced in the construction. This shows that even recognising who will win a game in which neither player has a non-trivial decision to make for the rest of the game is undecidable.

    We conclude with a discussion of the implications for a unified computational theory of games and remarks about the playability of such a board in a tournament setting.

  58. Turing-complete

  59. 2008-changizi.pdf: ⁠, Mark Changizi (2008-01-01; cs):

    Might it be possible to harness the visual system to carry out artificial computations, somewhat akin to how DNA has been harnessed to carry out computation? I provide the beginnings of a research programme attempting to do this. In particular, new techniques are described for building ‘visual circuits’ (or ‘visual software’) using wire, NOT, OR, and AND gates in a visual modality such that our visual system acts as ‘visual hardware’ computing the circuit, and generating a resultant perception which is the output.

  60. https://old.reddit.com/r/cogsci/comments/b8l7ml/example_visual_circuit_using_visual_stimuli_and/

  61. https://www.straighttalkonevidence.org/2019/04/03/beware-the-pitfalls-of-short-term-program-effects-they-often-fade/

  62. https://www.wired.com/2004/10/tail/

  63. http://dresdencodak.com/

  64. http://dresdencodak.com/2005/06/08/the-tomorrow-man/

  65. http://dresdencodak.com/2005/06/14/lil-werner/

  66. http://dresdencodak.com/2005/06/22/rupert-and-hubert/

  67. http://dresdencodak.com/2005/11/29/secular-heaven/

  68. http://dresdencodak.com/2006/01/30/epilogue/

  69. http://dresdencodak.com/2006/02/13/trouble-in-memphis/

  70. http://dresdencodak.com/2006/07/24/girl-vs-bear/

  71. http://dresdencodak.com/2006/08/30/traversing-the-luminiferous-aether/

  72. http://dresdencodak.com/2006/10/07/summer-dream-job/

  73. http://dresdencodak.com/2006/12/03/dungeons-and-discourse/

  74. http://dresdencodak.com/2009/01/27/advanced-dungeons-and-discourse/

  75. http://dresdencodak.com/2008/05/02/copan/

  76. http://dresdencodak.com/2008/10/02/epilogue-2/

  77. http://dresdencodak.com/2009/02/16/exorcising-laplaces-demon/

  78. http://dresdencodak.com/2009/05/11/42-essential-3rd-act-twists/

  79. http://dresdencodak.com/2009/05/15/a-thinking-apes-critique-of-trans-simianism-repost/

  80. http://dresdencodak.com/2009/09/22/caveman-science-fiction/

  81. http://dresdencodak.com/2009/07/12/fabulous-prizes/

  82. http://dresdencodak.com/2009/08/06/youre-a-good-man-charlie-darwin-2/

  83. http://dresdencodak.com/2009/09/07/the-sleepwalkers/

  84. http://dresdencodak.com/2009/12/16/lantern-season/

  85. http://dresdencodak.com/2010/02/09/the-process/

  86. 2010-vasseur-dresdencodakguestcomic.html

  87. http://dresdencodak.com/2010/06/03/dark-science-01/

  88. http://dresdencodak.com/2010/06/10/dark-science-02/

  89. http://dresdencodak.com/2011/04/19/dark-science-09/

  90. http://dresdencodak.com/2015/07/08/dark-science-49-exode/

  91. ⁠, Tanner Greer (2019-04-19):

    The second point probably deserves more space than I was able to give in the LA Review of Books. Consider, for a moment, the typical schedule of a Beijing teenager:

    She will (depending on the length of her morning commute) wake up somewhere between 5:30 and 7:00 AM. She must be in her seat by 7:45, 15 minutes before classes start. With bathroom breaks and gym class excepted, she will not leave that room until the 12:00 lunch hour and will return to the same spot after lunch is ended for another four hours of instruction. Depending on whether she has after-school tests that day, she will be released from her classroom sometime between 4:10 and 4:40. She then has one hour to get a start on her homework, eat, and travel to the evening cram school her parents have enrolled her in. Math, English, Classical Chinese—there are cram schools for every topic on the gaokao. On most days of the week she will be there studying from 6:00 to 9:00 PM (if the family has the money, she will spend another six hours at these after-school schools on Saturday and Sunday mornings). Our teenager will probably arrive home somewhere around 10:00 PM, giving her just enough time to spend two or three hours on that day’s homework before she goes to bed. Rinse and repeat, day in and day out, for six years. The strain does not abate until she has defeated—or has been defeated by—the gaokao.

    This is well known, but I think the wrong aspects of this experience are emphasized. Most outsiders look at this and think: see how much pressure these Chinese kids are under. I look and think: how little privacy and independence these Chinese kids are given!

    To put this another way: Teenage demands for personal space are hardly unique to China. What makes China distinctive is the difficulty its teenagers have securing this goal. Chinese family life is hemmed in narrow bounds. The urban apartments that even well-off Chinese call their homes are tiny and crowded. Few have more than two bedrooms. Teenagers are often forced to share their bedroom with a grandparent. So small was the apartment of one 16-year-old I interviewed that she slept, without apparent complaint, in the same bed as her parents for her entire first year of high school. Where can a teenager like her go, what door could she slam, when she was angry with her family? Within the walls of her home there was no escape from the parental gaze.

    A Chinese teen has few better options outside her home. No middle-class Chinese teenager has a job. None have cars. The few that have boyfriends or girlfriends go about it as discreetly as possible. Apart from the odd music lesson here or there, what Americans call “extra-curricular activities” are unknown. One a recent graduate of a prestigious international high school in Beijing once explained to me the confusion she felt when she was told she would need to excel at an after-school activity to be competitive in American university admissions:

    “In tenth grade our home room teacher told us that American universities cared a lot about the things we do outside of school, so from now on we would need to find time to ‘cultivate a hobby.’ I remember right after he left the girl sitting at my right turned to me and whispered, ‘I don’t know how to cultivate a hobby. Do you?’”

  92. https://enki2.tumblr.com/stories

  93. https://rpo.library.utoronto.ca/poems/long-legged-fly

  94. https://www.amazon.com/Cultural-Revolution-Peoples-History-1962_1976/dp/1632864231

  95. Cultural-Revolution

  96. 1989-huey-kyogokutamekane.pdf: “Kyōgoku Tamekane: Poetry and Politics in Late Kamakura Japan”⁠, Robert N. Huey

  97. Movies#freaks

  98. https://old.reddit.com/r/TOUHOUMUSIC/search?q=author%3Agwern&sort=new&restrict_sr=on&t=all

  99. https://www.youtube.com/watch?v=bh0_5NFvldY

  100. https://glassyocean.coln.biz/glov0014/

  101. https://old.reddit.com/r/Vocaloid/search?q=author%3Agwern&sort=new&restrict_sr=on&t=all

  102. https://www.youtube.com/watch?v=ssEJoUTtz6o

  103. https://mrkittyngp.bandcamp.com/album/a-i

  104. https://poniesatdawn.bandcamp.com/track/starlight-sanctuary

  105. https://poniesatdawn.bandcamp.com/album/skyward

  106. https://www.youtube.com/watch?v=TMhsfA6FUPI

  107. https://www.youtube.com/watch?v=D5S3rxp5MSw

  108. https://poniesatdawn.bandcamp.com/track/rituals-and-dances-of-the-pegasi-feat-koron-korak

  109. https://www.youtube.com/watch?v=8-H2oQ_ihIQ&t=0s

  110. https://www.youtube.com/watch?v=8-H2oQ_ihIQ&t=238s

  111. https://www.youtube.com/watch?v=PBlg-mbCVB0

  112. https://www.youtube.com/watch?v=geogRhkrsco

  113. https://www.youtube.com/watch?v=5MlCM0mR8DQ