Link Bibliography

1. https://gwern.substack.com

2. “September 2016 News”, Gw­ern Bran­wen (2016-08-18):

   N/A

3. “Changelog”, Gw­ern Bran­wen (2013-09-15):

   This page is a changelog for Gw­ern.net: a monthly re­verse chrono­log­i­cal list of re­cent ma­jor writings/changes/additions.

   Fol­low­ing my writ­ing can be a lit­tle diffi­cult be­cause it is often so in­cre­men­tal. So every mon­th, in ad­di­tion to my reg­u­lar /r/Gwern sub­red­dit sub­mis­sions, I write up rea­son­ably-in­ter­est­ing changes and send it out to the mail­ing list in ad­di­tion to a com­pi­la­tion of links & re­views (archives).

4. “/r/gwern subreddit”, Gw­ern Bran­wen (2018-10-01):

   A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.

5. https://www.patreon.com/gwern

6. “Genetic correlation”, Eng­lish Wikipedia (2020-12-22):

   In multivariate quantitative genetics, a genetic correlation is the proportion of variance that two traits share due to genetic causes, the correlation between the genetic influences on a trait and the genetic influences on a different trait estimating the degree of pleiotropy or causal overlap. A genetic correlation of 0 implies that the genetic effects on one trait are independent of the other, while a correlation of 1 implies that all of the genetic influences on the two traits are identical. The bivariate genetic correlation can be generalized to inferring genetic latent variable factors across > 2 traits using factor analysis. Genetic correlation models were introduced into behavioral genetics in the 1970s–1980s.

7. http://ije.oxfordjournals.org/content/early/2016/09/28/ije.dyw216.full

8. “Educational attainment and personality are genetically intertwined”, René Mõt­tus, Anu Re­alo, Uku Vainik, Jüri Al­lik, Tõnu Esko (2016-09-28):

   It is possible that heritable variance in personality characteristics does not reflect (only) genetic and biological processes specific to personality per se. We tested the possibility that Five-Factor Model personality domains and facets, as rated by people themselves and their knowledgeable informants, reflect polygenic influences that have been previously associated with educational attainment. In a sample of over 3,000 adult Estonians, polygenic scores for educational attainment, based on small contributions from more than 150,000 genetic variants, were correlated with various personality traits, mostly from the Neuroticism and Openness domains. The correlations of personality characteristics with educational attainment-related polygenic influences reflected almost entirely their correlations with phenotypic educational attainment. Structural equation modeling of the associations between polygenic risk, personality (a weighed aggregate of education-related facets) and educational attainment lent relatively strongest support to the possibility of educational attainment mediating (explaining) some of the heritable variance in personality traits.
   

9. /docs/genetics/heritable/2016-ganna.pdf

10. “Genome-wide analyses of empathy and systemizing: heritability and correlates with sex, education, and psychiatric risk”, Varun War­ri­er, Roberto Toro, Bhis­madev Chakrabar­ti, Na­dia Lit­ter­man, David A. Hinds, Thomas Bourg­eron, Si­mon Baron-Co­hen (2016-04-29):

    Empathy is the drive to identify the mental states of others and respond to these with an appropriate emotion. Systemizing is the drive to analyse or build lawful systems. Difficulties in empathy have been identified in different psychiatric conditions including autism and schizophrenia. In this study, we conducted genome-wide association studies of empathy and systemizing using the Empathy Quotient (EQ) (n = 46,861) and the Systemizing Quotient-Revised (SQ-R) (n = 51,564) in participants from 23andMe, Inc. We confirmed significant sex-differences in performance on both tasks, with a male advantage on the SQ-R and female advantage on the EQ. We found highly significant heritability explained by single nucleotide polymorphisms (SNPs) for both the traits (EQ: 0.11±0.014; p = 1.7 × 10^-14 and SQ-R: 0.12±0.012; p = 1.2 × 10^-20) and these were similar for males and females. However, genes with higher expression in the male brain appear to contribute to the male advantage for the SQ-R. Finally, we identified significant genetic correlations between high score for empathy and risk for schizophrenia (p = 2.5 × 10^-5), and correlations between high score for systemizing and higher educational attainment (p = 5 × 10^-4). These results shed light on the genetic contribution to individual differences in empathy and systemizing, two major cognitive functions of the human brain.

11. “Genome-wide meta-analysis of cognitive empathy: heritability, and correlates with sex, neuropsychiatric conditions and brain anatomy”, Varun War­ri­er, Ka­t­rina Gras­by, Flo­rina Uze­fovsky, Roberto Toro, Paula Smith, Bhis­madev Chakrabar­ti, Jy­oti Khadake, Na­dia Lit­ter­man, Jouke-Jan Hot­ten­ga, Gitta Lubke, Dor­ret I. Booms­ma, Nicholas G. Mar­t­in, Pe­ter K. Hatemi, Sarah E. Med­land, David A. Hinds, Thomas Bourg­eron, Si­mon Baron-Co­hen (2016-10-19):

    We conducted a genome-wide meta-analysis of cognitive empathy using the ‘Reading the Mind in the Eyes’ Test (Eyes Test) in 88,056 Caucasian research participants (44,574 females and 43,482 males) from 23andMe Inc., and an additional 1,497 Caucasian participants (891 females and 606 males) from the Brisbane Longitudinal Twin Study (BLTS). We confirmed a female advantage on the Eyes Test (Cohen’s d = 0.21, p &;t; 0.001), and identified a locus in 3p26.1 that is associated with scores on the Eyes Test in females (rs7641347, p_meta = 1.57 × 10⁻⁸). Common single nucleotide polymorphisms (SNPs) explained 20% of the twin heritability and 5.6% (±0.76; p = 1.72 × 10⁻¹³) of the total trait variance in both sexes. Finally, we identified significant genetic correlation between the Eyes Test and measures of empathy (the Empathy Quotient), openness (NEO-Five Factor Inventory), and different measures of educational attainment and cognitive aptitude, and show that the genetic determinants of striatal volumes (caudate nucleus, putamen, and nucleus accumbens) are positively correlated with the genetic determinants of performance on the Eyes Test.

12. https://www.cambridge.org/core/journals/twin-research-and-human-genetics/article/personality-polygenes-positive-affect-and-life-satisfaction/4DB2BE673BF122FB9A0AF2147EED80C0/core-reader

13. /docs/genetics/selection/2016-fan.pdf

14. https://economics.stanford.edu/sites/default/files/djprice_jmp.pdf

15. /docs/sociology/2016-vonrueden.pdf

16. /docs/psychology/2016-nieuwkamp.pdf

17. /docs/rl/2016-graves.pdf#deepmind

18. https://deepmind.com/blog/differentiable-neural-computers/

19. “Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes”, Jack W. Rae, Jonathan J. Hunt, Tim Harley, Ivo Dani­helka, An­drew Se­nior, Greg Wayne, Alex Graves, Tim­o­thy P. Lil­l­i­crap (2016-10-27):

    Neural networks augmented with external memory have the ability to learn algorithmic solutions to complex tasks. These models appear promising for applications such as language modeling and machine translation. However, they scale poorly in both space and time as the amount of memory grows — limiting their applicability to real-world domains. Here, we present an end-to-end differentiable memory access scheme, which we call Sparse Access Memory (SAM), that retains the representational power of the original approaches whilst training efficiently with very large memories. We show that SAM achieves asymptotic lower bounds in space and time complexity, and find that an implementation runs $1,000\times$ faster and with $3,000\times$ less physical memory than non-sparse models. SAM learns with comparable data efficiency to existing models on a range of synthetic tasks and one-shot Omniglot character recognition, and can scale to tasks requiring $100,000$s of time steps and memories. As well, we show how our approach can be adapted for models that maintain temporal associations between memories, as with the recently introduced Differentiable Neural Computer.

20. “Achieving Human Parity in Conversational Speech Recognition”, W. Xiong, J. Drop­po, X. Huang, F. Sei­de, M. Seltzer, A. Stol­cke, D. Yu, G. Zweig (2016-10-17):

    Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human error rate on the widely used NIST 2000 test set, and find that our latest automated system has reached human parity. The error rate of professional transcribers is 5.9 which newly acquainted pairs of people discuss an assigned topic, and 11.3 the CallHome portion where friends and family members have open-ended conversations. In both cases, our automated system establishes a new state of the art, and edges past the human benchmark, achieving error rates of 5.8 11.0 convolutional and LSTM acoustic model architectures, combined with a novel spatial smoothing method and lattice-free MMI acoustic training, multiple recurrent neural network language modeling approaches, and a systematic use of system combination.

21. “Video Pixel Networks”, Nal Kalch­bren­ner, Aaron van den Oord, Karen Si­monyan, Ivo Dani­helka, Oriol Vinyals, Alex Graves, Ko­ray Kavukcuoglu (2016-10-03):

    We propose a probabilistic video model, the Video Pixel Network (VPN), that estimates the discrete joint distribution of the raw pixel values in a video. The model and the neural architecture reflect the time, space and color structure of video tensors and encode it as a four-dimensional dependency chain. The VPN approaches the best possible performance on the Moving MNIST benchmark, a leap over the previous state of the art, and the generated videos show only minor deviations from the ground truth. The VPN also produces detailed samples on the action-conditional Robotic Pushing benchmark and generalizes to the motion of novel objects.

22. “Asynchronous Methods for Deep Reinforcement Learning”, Volodymyr Mnih, Adrià Puig­domènech Ba­dia, Mehdi Mirza, Alex Graves, Tim­o­thy P. Lil­l­i­crap, Tim Harley, David Sil­ver, Ko­ray Kavukcuoglu (2016-02-04):

    We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

23. “Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates”, Shix­i­ang Gu, Ethan Hol­ly, Tim­o­thy Lil­l­i­crap, Sergey Levine (2016-10-03):

    Reinforcement learning holds the promise of enabling autonomous robots to learn large repertoires of behavioral skills with minimal human intervention. However, robotic applications of reinforcement learning often compromise the autonomy of the learning process in favor of achieving training times that are practical for real physical systems. This typically involves introducing hand-engineered policy representations and human-supplied demonstrations. Deep reinforcement learning alleviates this limitation by training general-purpose neural network policies, but applications of direct deep reinforcement learning algorithms have so far been restricted to simulated settings and relatively simple tasks, due to their apparent high sample complexity. In this paper, we demonstrate that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. We demonstrate that the training times can be further reduced by parallelizing the algorithm across multiple robots which pool their policy updates asynchronously. Our experimental evaluation shows that our method can learn a variety of 3D manipulation skills in simulation and a complex door opening skill on real robots without any prior demonstrations or manually designed representations.

24. https://sites.google.com/site/deeproboticmanipulation/

25. https://research.googleblog.com/2016/10/how-robots-can-acquire-new-skills-from.html

26. http://mlg.eng.cam.ac.uk/yarin/blog_2248.html

27. “Sim-to-Real Robot Learning from Pixels with Progressive Nets”, An­drei A. Rusu, Mel Ve­cerik, Thomas Rothörl, Nico­las Heess, Raz­van Pas­canu, Raia Had­sell (2016-10-13):

    Applying end-to-end learning to solve complex, interactive, pixel-driven control tasks on a robot is an unsolved problem. Deep Reinforcement Learning algorithms are too slow to achieve performance on a real robot, but their potential has been demonstrated in simulated environments. We propose using progressive networks to bridge the reality gap and transfer learned policies from simulation to the real world. The progressive net approach is a general framework that enables reuse of everything from low-level visual features to high-level policies for transfer to new tasks, enabling a compositional, yet simple, approach to building complex skills. We present an early demonstration of this approach with a number of experiments in the domain of robot manipulation that focus on bridging the reality gap. Unlike other proposed approaches, our real-world experiments demonstrate successful task learning from raw visual input on a fully actuated robot manipulator. Moreover, rather than relying on model-based trajectory optimisation, the task learning is accomplished using only deep reinforcement learning and sparse rewards.

28. http://www.allinea.com/blog/201610/deep-learning-episode-4-supercomputer-vs-pong-ii

29. “Image Synthesis from Yahoo's open_nsfw”, Gabriel Goh (2016):

    Yahoo’s recently open sourced neural network, open_nsfw, is a fine tuned Residual Network which scores images on a scale of 0 to 1 on its suitability for use in the workplace…What makes an image NSFW, according to Yahoo? I explore this question with a clever new visualization technique by Nguyen et al…Like Google’s Deep Dream, this visualization trick works by maximally activating certain neurons of the classifier. Unlike deep dream, we optimize these activations by performing descent on a parameterization of the manifold of natural images.

    [Demonstration of an unusual use of backpropagation to ‘optimize’ a neural network: instead of taking a piece of data to input to a neural network and then updating the neural network to change its output slightly towards some desired output (such as a correct classification), one can instead update the input so as to make the neural net output slightly more towards the desired output. When using a image classification neural network, this reversed form of optimization will ‘hallucinate’ or ‘edit’ the ‘input’ to make it more like a particular class of images. In this case, a porn/NSFW-detecting NN is reversed so as to make images more (or less) “porn-like”. Goh runs this process on various images like landscapes, musical bands, or empty images; the maximally/minimally porn-like images are disturbing, hilarious, and undeniably pornographic in some sense.]

30. http://blog.mldb.ai/blog/posts/2016/10/deepteach/

31. http://burrsettles.com/pub/settles.activelearning.pdf

32. “Why Tool AIs Want to Be Agent AIs”, Gw­ern Bran­wen (2016-09-07):

    Au­tonomous AI sys­tems (A­gent AIs) trained us­ing re­in­force­ment learn­ing can do harm when they take wrong ac­tions, es­pe­cially su­per­in­tel­li­gent Agent AIs. One so­lu­tion would be to elim­i­nate their agency by not giv­ing AIs the abil­ity to take ac­tions, con­fin­ing them to purely in­for­ma­tional or in­fer­en­tial tasks such as clas­si­fi­ca­tion or pre­dic­tion (Tool AIs), and have all ac­tions be ap­proved & ex­e­cuted by hu­mans, giv­ing equiv­a­lently su­per­in­tel­li­gent re­sults with­out the risk.

    I ar­gue that this is not an effec­tive so­lu­tion for two ma­jor rea­sons. First, be­cause Agent AIs will by de­fi­n­i­tion be bet­ter at ac­tions than Tool AIs, giv­ing an eco­nomic ad­van­tage. Sec­ond­ly, be­cause Agent AIs will be bet­ter at in­fer­ence & learn­ing than Tool AIs, and this is in­her­ently due to their greater agen­cy: the same al­go­rithms which learn how to per­form ac­tions can be used to se­lect im­por­tant dat­a­points to learn in­fer­ence over, how long to learn, how to more effi­ciently ex­e­cute in­fer­ence, how to de­sign them­selves, how to op­ti­mize hy­per­pa­ra­me­ters, how to make use of ex­ter­nal re­sources such as long-term mem­o­ries or ex­ter­nal soft­ware or large data­bases or the In­ter­net, and how best to ac­quire new da­ta. All of these ac­tions will re­sult in Agent AIs more in­tel­li­gent than Tool AIs, in ad­di­tion to their greater eco­nomic com­pet­i­tive­ness. Thus, Tool AIs will be in­fe­rior to Agent AIs in both ac­tions and in­tel­li­gence, im­ply­ing use of Tool AIs is a even more highly un­sta­ble equi­lib­rium than pre­vi­ously ar­gued, as users of Agent AIs will be able to out­com­pete them on two di­men­sions (and not just one).

33. http://andrewgelman.com/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/

34. http://www.nytimes.com/interactive/2016/09/20/upshot/the-error-the-polling-world-rarely-talks-about.html

35. http://www.nature.com/news/close-but-no-nobel-the-scientists-who-never-won-1.20781

36. /docs/statistics/2016-cook.pdf

37. https://idl.cs.washington.edu/files/2015-BeyondWebersLaw-InfoVis.pdf

38. /docs/iq/2016-dutton.pdf

39. /docs/longevity/2016-moon.pdf

40. /docs/iq/smpy/2016-lubinski.pdf

41. http://edition.cnn.com/2016/10/06/health/rapamycin-dog-live-longer/index.html

42. http://andrewgelman.com/wp-content/uploads/2015/12/Current-Directions-in-Psychological-Science-2015-Jussim-490-7.pdf

43. http://www.nytimes.com/2016/10/25/health/cmv-cytomegalovirus-pregnancy.html

44. /docs/creatine/2016-meregefilho.pdf

45. “Functional MRI in awake dogs predicts suitability for assistance work”, Gre­gory S. Berns, An­drew M. Brooks, Mark Spi­vak, Kerinne Levy (2016-10-12):

    The overall goal of this work was to measure the efficacy of fMRI for predicting whether a dog would be a successful service dog. The training and imaging were performed in 50 dogs entering advanced training at 17-21 months of age. FMRI responses were measured while each dog observed hand signals indicating either reward or no reward and given by both a familiar handler and a stranger. 49 dogs successfully completed fMRI training and scanning. Of these, 33 eventually completed service training and were matched with a person, while 10 were released for behavioral reasons. Using anatomically defined regions-of-interest in the ventral caudate, amygdala, and visual cortex, we developed a classifier based on the dogs9 outcomes. We found that responses in the stranger condition were sufficient to develop an accurate brain-based classifier. On all data, the classifier had a positive predictive value of 96% with 10% false positives. The area under the receiver operating characteristic curve was 0.90 (0.79 with 4-fold cross-validation, p = 0.02), indicating a significant diagnostic capability. Within the stranger condition, the differential response to [reward – no reward] in ventral caudate was positively correlated with a successful outcome, while the differential response in the amygdala was negatively correlated to outcome. These results show that successful service dogs transfer knowledge to strangers as indexed by ventral caudate activity without excessive arousal as measured in the amygdala.

46. http://www.instructables.com/id/DIY-Kitty-Crack%3a--ultra-potent-catnip-extract/?ALLSTEPS

47. http://www.newyorker.com/magazine/2016/10/10/sam-altmans-manifest-destiny

48. https://nitter.net/TheJoinery_jp

49. http://www.spoon-tamago.com/2016/10/04/animated-gifs-illustrating-the-art-of-japanese-wood-joinery/

50. https://www.technologyreview.com/s/602662/ubers-ad-toting-drones-are-heckling-drivers-stuck-in-traffic/

51. https://www.sigmobile.org/mobicom/2015/papers/p90-aliA.pdf

52. http://researcher.watson.ibm.com/researcher/files/us-liup/apex.pdf

53. https://www.buzzfeed.com/chrishamby/super-court

54. http://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.30.4.171

55. http://www.nytimes.com/interactive/2016/10/09/magazine/big-food-photo-essay.html

56. /docs/economics/2016-kelly.pdf

57. http://econlog.econlib.org/archives/2016/10/what_do_crimina.html

58. http://www.vanityfair.com/news/1998/10/gulfstream-199810

59. https://80000hours.org/articles/money-and-happiness/

60. http://www.nytimes.com/2016/09/25/world/asia/bangalore-india-women-factories.html

61. “Montaillou (book)”, Eng­lish Wikipedia (2020-12-28):

    

    Montaillou is a book by the French historian Emmanuel Le Roy Ladurie first published in 1975. It was first translated into English in 1978 by Barbara Bray, and has been subtitled The Promised Land of Error and Cathars and Catholics in a French Village.

62. “Emmanuel Le Roy Ladurie”, Eng­lish Wikipedia (2020-12-28):

    

    Emmanuel Bernard Le Roy Ladurie is a French historian whose work is mainly focused upon Languedoc in the Ancien Régime, particularly the history of the peasantry. One of the leading historians of France, Le Roy Ladurie has been called the "standard-bearer" of the third generation of the Annales school and the "rock star of the medievalists", noted for his work in social history.

63. https://www.amazon.com/Riddle-Labyrinth-Quest-Crack-Ancient/dp/0062228862

64. /Book-reviews#the-riddle-of-the-labyrinth-fox-2013

65. “Confessions of an English Opium-Eater”, Eng­lish Wikipedia (2020-12-28):

    

    Confessions of an English Opium-Eater (1821) is an autobiographical account written by Thomas De Quincey, about his laudanum addiction and its effect on his life. The Confessions was "the first major work De Quincey published and the one which won him fame almost overnight..."

66. /Book-reviews#confessions-of-an-english-opium-eater-quincey-2003

67. “The Tales of Ise”, Eng­lish Wikipedia (2020-12-22):

    The Tales of Ise is a Japanese uta monogatari, or collection of waka poems and associated narratives, dating from the Heian period. The current version collects 125 sections, with each combining poems and prose, giving a total of 209 poems in most versions.

68. /Book-reviews#tales-of-ise-anonymous-1968

69. https://www.youtube.com/watch?v=Nlqg8l6VNqM

70. https://www.youtube.com/watch?v=KHaCE3SEIVo

71. https://www.youtube.com/watch?v=XZRbh3_hibo

72. https://www.dropbox.com/s/up6dmutwl4h94re/rando-radialdiversesystemfifteenthanniversary-heurebleue.ogg

73. https://www.dropbox.com/s/q8nfgj7yxkbaa3j/sta-radialdiversesystemfifteenthanniversary-hallofmirrors.ogg

74. https://www.dropbox.com/s/aicai14gdz14p6i/tigerlily-radialdiversesystemfifteenthanniversary-parabola.ogg

75. https://www.dropbox.com/s/fslrukxwwz3gapc/b-radialdiversesystemfifteenthanniversary-16777215.ogg?

76. https://www.youtube.com/watch?v=sNMy2UHpnEI

77. https://www.youtube.com/watch?v=DPFXk4zJZ1g

78. https://www.youtube.com/watch?v=ShjSplbL3FQ

79. https://www.dropbox.com/s/co1x45wjnmfod2h/nbuna-walkingonthemoon-epilogue.ogg

80. https://www.dropbox.com/s/v5uq5nwuppewy5o/morrigan-from080723discedition-%E3%81%9D%E3%82%8C%E3%81%AF%E6%9C%AC%E5%BD%93%E3%81%AB%E6%95%91%E6%B8%88%E3%81%AA%E3%81%AE%E3%81%8B.ogg

81. “Gwern.net newsletter (Substack subscription page)”, Gw­ern Bran­wen (2013-12-01):

    Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.

82. “Gwern.net newsletter archives”, Gw­ern Bran­wen (2013-12-01):

    Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)

83. https://old.reddit.com/r/gwern/.rss

84. “Reinforcement learning”, Eng­lish Wikipedia (2020-12-22):

    Reinforcement learning (RL) is an area of machine learning concerned with how software agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.