Link Bibliography

1. https://gwern.substack.com

2. “March 2017 News”, Gw­ern Bran­wen (2017-02-23):

   N/A

3. “Gwern.net newsletter archives”, Gw­ern Bran­wen (2013-12-01):

   Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)

4. “Changelog”, Gw­ern Bran­wen (2013-09-15):

   This page is a changelog for Gw­ern.net: a monthly re­verse chrono­log­i­cal list of re­cent ma­jor writings/changes/additions.

   Fol­low­ing my writ­ing can be a lit­tle diffi­cult be­cause it is often so in­cre­men­tal. So every mon­th, in ad­di­tion to my reg­u­lar /r/Gwern sub­red­dit sub­mis­sions, I write up rea­son­ably-in­ter­est­ing changes and send it out to the mail­ing list in ad­di­tion to a com­pi­la­tion of links & re­views (archives).

5. “/r/gwern subreddit”, Gw­ern Bran­wen (2018-10-01):

   A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.

6. https://www.patreon.com/gwern

7. “Tea/Water Experiment: Blinded Randomized Racing-Arms Experiment of Mineral Water Taste”, Gw­ern Bran­wen (2017-04-04):

   The kind of water used in tea is claimed to make a difference in the flavor: mineral water being better than tap water or distilled water. However, mineral water is vastly more expensive than tap water. To test the claim, I run a preliminary test of pure water to see if any water differences are detectable at all. Compared my tap water, 3 distilled water brands (Great Value, Nestle Pure Life, & Poland Spring), 1 osmosis-purified brand (Aquafina), and 3 non-carbonated mineral water brands (Evian, Voss, & Fiji) in a series of n = 67 blinded randomized comparisons of water flavor. The comparisons are modeled using a Bradley-Terry competitive model implemented in Stan; comparisons were chosen using an adaptive Bayesian best-arm sequential trial (racing) method designed to locate the best-tasting water in the minimum number of samples by preferentially comparing the best-known arm to potentially superior arms. Blinding & randomization are achieved by using a Lazy Susan to physically randomize two identical (but marked in a hidden spot) cups of water. The final posterior distribution indicates that some differences between waters are likely to exist but are small & imprecisely estimated and of little practical concern.

8. “Soft sweeps are the dominant mode of adaptation in the human genome”, Daniel R. Schrid­er, An­drew D. Kern (2017-04-27):

   The degree to which adaptation in recent human evolution shapes genetic variation remains controversial. This is in part due to the limited evidence in humans for classic “hard selective sweeps,” wherein a novel beneficial mutation rapidly sweeps through a population to fixation. However, positive selection may often proceed via “soft sweeps” acting on mutations already present within a population. Here we examine recent positive selection across six human populations using a powerful machine learning approach that is sensitive to both hard and soft sweeps. We found evidence that soft sweeps are widespread and account for the vast majority of recent human adaptation. Surprisingly, our results also suggest that linked positive selection affects patterns of variation across much of the genome, and may increase the frequencies of deleterious mutations. Our results also reveal insights into the role of sexual selection, cancer risk, and central nervous system development in recent human evolution.

9. “Comparative genomic evidence for self-domestication in Homo sapiens”, Con­stan­tina The­o­fanopoulou, Si­mone Gastal­don, Thomas O’Rourke, Brid­get D. Samuels, An­gela Mess­ner, Pe­dro Tiago Mar­tins, Francesco De­l­ogu, Saleh Alam­ri, Boeckx Cedric (2017-04-09):

   This study identifies and analyzes statistically significant overlaps between selective sweep screens in anatomically modern humans and several domesticated species. The results obtained suggest that (paleo-) genomic data can be exploited to complement the fossil record and support the idea of self-domestication in Homo sapiens, a process that likely intensified as our species populated its niche. Our analysis lends support to attempts to capture the “domestication syndrome” in terms of alterations to certain signaling pathways and cell lineages, such as the neural crest.

10. /docs/genetics/selection/2017-owers.pdf

11. http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1006616

12. /docs/genetics/selection/2017-librado.pdf

13. https://www.nytimes.com/2017/04/27/science/horses-genetics-domestication-scythians.html

14. /docs/genetics/correlation/2017-docherty.pdf

15. “Syphilis in renaissance Europe: rapid evolution of an introduced sexually transmitted disease?”, Knell, Robert J (2004):

    When syphilis first appeared in Europe in 1495, it was an acute and extremely unpleasant disease. After only a few years it was less severe than it once was, and it changed over the next 50 years into a milder, chronic disease. The severe early symptoms may have been the result of the disease being introduced into a new host population without any resistance mechanisms, but the change in virulence is most likely to have happened because of selection favouring milder strains of the pathogen. The symptoms of the virulent early disease were both debilitating and obvious to potential sexual partners of the infected, and strains that caused less obvious or painful symptoms would have enjoyed a higher transmission rate.

16. https://www.theatlantic.com/magazine/archive/2017/04/pleistocene-park/517779/

17. /docs/genetics/heritable/2017-mcconnell.pdf

18. http://sciencebulletin.org/archives/9946.html

19. http://www.nationalgeographic.com/magazine/2017/04/evolution-genetics-medicine-brain-technology-cyborg/

20. “Recurrent Environment Simulators”, Sil­via Chi­ap­pa, Sébastien Racaniere, Daan Wier­stra, Shakir Mo­hamed (2017-04-07):

    Models that can simulate how environments change in response to actions can be used by agents to plan and act efficiently. We improve on previous environment simulators from high-dimensional pixel observations by introducing recurrent neural networks that are able to make temporally and spatially coherent predictions for hundreds of time-steps into the future. We present an in-depth analysis of the factors affecting performance, providing the most extensive attempt to advance the understanding of the properties of these models. We address the issue of computationally inefficiency with a model that does not need to generate a high-dimensional image at each time-step. We show that our approach can be used to improve exploration and is adaptable to many diverse environments, namely 10 Atari games, a 3D car racing environment, and complex 3D mazes.

21. https://old.reddit.com/r/MachineLearning/comments/66x02v/d_rl_gans_as_mcts_environment_simulator_for_deep/

22. “Using Human Brain Activity to Guide Machine Learning”, Ruth Fong, Wal­ter Scheir­er, David Cox (2017-03-16):

    Machine learning is a field of computer science that builds algorithms that learn. In many cases, machine learning algorithms are used to recreate a human ability like adding a caption to a photo, driving a car, or playing a game. While the human brain has long served as a source of inspiration for machine learning, little effort has been made to directly use data collected from working brains as a guide for machine learning algorithms. Here we demonstrate a new paradigm of "neurally-weighted" machine learning, which takes fMRI measurements of human brain activity from subjects viewing images, and infuses these data into the training process of an object recognition learning algorithm to make it more consistent with the human brain. After training, these neurally-weighted classifiers are able to classify images without requiring any additional neural data. We show that our neural-weighting approach can lead to large performance gains when used with traditional machine vision features, as well as to significant improvements with already high-performing convolutional neural network features. The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.

23. /docs/ai/2017-jouppi.pdf

24. “Communication-Efficient Learning of Deep Networks from Decentralized Data”, H. Bren­dan McMa­han, Ei­der Moore, Daniel Ra­m­age, Seth Hamp­son, Blaise Agüera y Ar­cas (2016-02-17):

    Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device. For example, language models can improve speech recognition and text entry, and image models can automatically select good photos. However, this rich data is often privacy sensitive, large in quantity, or both, which may preclude logging to the data center and training there using conventional approaches. We advocate an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning.

    We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. These experiments demonstrate the approach is robust to the unbalanced and non-IID data distributions that are a defining characteristic of this setting. Communication costs are the principal constraint, and we show a reduction in required communication rounds by 10-100× as compared to synchronized stochastic gradient descent.

25. “Visual Attribute Transfer through Deep Image Analogy”, Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, Sing Bing Kang (2017-05-02):

    We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure. By visual attribute transfer, we mean transfer of visual information (such as color, tone, texture, and style) from one image to another. For example, one image could be that of a painting or a sketch while the other is a photo of a real scene, and both depict the same type of scene.

    Our technique finds semantically-meaningful dense correspondences between two input images. To accomplish this, it adapts the notion of "image analogy" with features extracted from a Deep Convolutional Neutral Network for matching; we call our technique Deep Image Analogy. A coarse-to-fine strategy is used to compute the nearest-neighbor field for generating the results. We validate the effectiveness of our proposed method in a variety of cases, including style/texture transfer, color/style swap, sketch/painting to photo, and time lapse.

26. https://liaojing.github.io/html/data/analogy_supplemental.pdf

27. “Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting”, Ning Xie, Hi­ro­taka Hachiya, Masashi Sugiyama (2012-06-18):

    Oriental ink painting, called Sumi-e, is one of the most appealing painting styles that has attracted artists around the world. Major challenges in computer-based Sumi-e simulation are to abstract complex scene information and draw smooth and natural brush strokes. To automatically find such strokes, we propose to model the brush as a reinforcement learning agent, and learn desired brush-trajectories by maximizing the sum of rewards in the policy search framework. We also provide elaborate design of actions, states, and rewards tailored for a Sumi-e agent. The effectiveness of our proposed approach is demonstrated through simulated Sumi-e experiments.

28. “Understanding Synthetic Gradients and Decoupled Neural Interfaces”, Wo­j­ciech Mar­ian Czar­necki, Grze­gorz Świrszcz, Max Jader­berg, Si­mon Osin­dero, Oriol Vinyals, Ko­ray Kavukcuoglu (2017-03-01):

    When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking—without waiting for a true error gradient to be backpropagated—resulting in Decoupled Neural Interfaces (DNIs). This unlocked ability of being able to update parts of a neural network asynchronously and with only local information was demonstrated to work empirically in Jaderberg et al (2016). However, there has been very little demonstration of what changes DNIs and SGs impose from a functional, representational, and learning dynamics point of view. In this paper, we study DNIs through the use of synthetic gradients on feed-forward networks to better understand their behaviour and elucidate their effect on optimisation. We show that the incorporation of SGs does not affect the representational strength of the learning system for a neural network, and prove the convergence of the learning system for linear and deep linear models. On practical problems we investigate the mechanism by which synthetic gradient estimators approximate the true loss, and, surprisingly, how that leads to drastically different layer-wise representations. Finally, we also expose the relationship of using synthetic gradients to other error approximation techniques and find a unifying language for discussion and comparison.

29. “Machine Learning Predicts Laboratory Earthquakes”, Bertrand Rou­et-Le­duc, Clau­dia Hul­bert, Nicholas Lub­bers, Kip­ton Bar­ros, Colin Humphreys, Paul A. John­son (2017-02-19):

    Forecasting fault failure is a fundamental but elusive goal in earthquake science. Here we show that by listening to the acoustic signal emitted by a laboratory fault, machine learning can predict the time remaining before it fails with great accuracy. These predictions are based solely on the instantaneous physical characteristics of the acoustical signal, and do not make use of its history. Surprisingly, machine learning identifies a signal emitted from the fault zone previously thought to be low-amplitude noise that enables failure forecasting throughout the laboratory quake cycle. We hypothesize that applying this approach to continuous seismic data may lead to significant advances in identifying currently unknown signals, in providing new insights into fault physics, and in placing bounds on fault failure times.

30. https://www.technologyreview.com/s/603785/machine-learning-algorithm-predicts-laboratory-earthquakes/

31. https://old.reddit.com/r/MachineLearning/comments/c4ylga/d_misuse_of_deep_learning_in_nature_journals/

32. https://jacquesmattheij.com/sorting-two-metric-tons-of-lego

33. https://news.ycombinator.com/item?id=14226889

34. https://jacquesmattheij.com/sorting-lego-the-software-side

35. /docs/statistics/causality/2017-adams.pdf

36. /docs/history/2016-caro.html

37. http://blogs.discovermagazine.com/gnxp/2012/08/historical-dynamics-and-contingent-conditions-of-religion/

38. http://80000hours.org/blog/66-social-interventions-gone-wrong

39. https://www.theguardian.com/uk-news/2017/mar/16/what-happens-when-queen-elizabeth-dies-london-bridge

40. “Sex differences in the adult human brain: Evidence from 5,216 UK Biobank participants”, Stu­art J. Ritchie, Si­mon R. Cox, Xueyi Shen, Michael V. Lom­bar­do, Lianne M. Reus, Clara Al­loza, Matthew A. Har­ris, He­len L. Alder­son, Stu­art Hunter, Emma Neil­son, David C. M. Liewald, Bon­nie Auye­ung, Heather C. Whal­ley, Stephen M. Lawrie, Catharine R. Gale, Mark E. Bastin, An­drew M. McIn­tosh, Ian J. Deary (2017-04-04):

    Sex differences in human brain structure and function are of substantial scientific interest because of sex-differential susceptibility to psychiatric disorders [1,2,3] and because of the potential to explain sex differences in psychological traits [4]. Males are known to have larger brain volumes, though the patterns of differences across brain subregions have typically only been examined in small, inconsistent studies [5]. In addition, despite common findings of greater male variability in traits like intelligence [6], personality [7], and physical performance [8], variance differences in the brain have received little attention. Here we report the largest single-sample study of structural and functional sex differences in the human brain to date (2,750 female and 2,466 male participants aged 44-77 years). Males had higher cortical and sub-cortical volumes, cortical surface areas, and white matter diffusion directionality; females had thicker cortices and higher white matter tract complexity. Considerable overlap between the distributions for males and females was common, and subregional differences were smaller after accounting for global differences. There was generally greater male variance across structural measures. The modestly higher male score on two cognitive tests was partly mediated via structural differences. Functional connectome organization showed stronger connectivity for males in unimodal sensorimotor cortices, and stronger connectivity for females in the default mode network. This large-scale characterisation of neurobiological sex differences provides a foundation for attempts to understand the causes of sex differences in brain structure and function, and their associated psychological and psychiatric consequences.

41. /docs/longevity/2017-trepanowski.pdf

42. https://link.springer.com/article/10.1007/s11357-017-9972-z

43. http://www.fadedpage.com/books/20160325/html.php#Page_107

44. http://lesswrong.com/r/discussion/lw/oql/in_support_of_yak_shaving/

45. /docs/psychology/2012-mannes.pdf

46. https://www.andrew.cmu.edu/user/twildenh/PowerPointTM/Paper.pdf

47. https://www.youtube.com/watch?v=uNjxe8ShM-8

48. https://linusakesson.net/scene/a-mind-is-born/

49. https://web.archive.org/web/20160903194154/https://medium.com/stubborn-attachments/stubborn-attachments-full-text-8fc946b694d

50. https://www.nytimes.com/2017/03/29/magazine/those-indecipherable-medical-bills-theyre-one-reason-health-care-costs-so-much.html

51. https://www.theatlantic.com/technology/archive/2017/04/the-tragedy-of-google-books/523320/

52. /docs/economics/1999-kohn.pdf

53. https://www.amazon.com/Moondust-Search-Men-Fell-Earth/dp/0007155425

54. /Book-reviews#moondust-smith-2006

55. http://drmcninja.com/newreaders.php

56. “Real Ultimate Power”, Eng­lish Wikipedia (2020-12-22):

    The Official Ninja Webpage: Real Ultimate Power is a satire website. It was launched in 2002 by the pseudonymous Robert Hamburger. Written using the persona of a 13-year-old boy, the site is a parody of adolescent fascination with Ninjas. Warren St. John, columnist for The New York Times described it as "a satirical ode to the masculine prowess of ninjas".

57. “Ninja Burger”, Eng­lish Wikipedia (2020-12-28):

    

    Ninja Burger (忍者バーガー) is a parody website started in late 1999, purporting that a sect of noble ninja have taken to secretly delivering fast food meals, anywhere, anytime, within 30 minutes or less. Failure to deliver within the ascribed time limit results in seppuku. Some of Ninja Burger's rivals include Pirate Pizza, Otaku Bell, and Samurai Burger. The site riffs on many of the same points as Real Ultimate Power, another ninja parody website.

58. “Megatokyo”, Eng­lish Wikipedia (2020-12-28):

    

    Megatokyo (メガトーキョー) is an English-language webcomic created by Fred Gallagher and Rodney Caston. Megatokyo debuted on August 14, 2000, and has been written and illustrated solely by Gallagher since July 17, 2002. Gallagher's style of writing and illustration is heavily influenced by Japanese manga. Megatokyo is freely available on its official website. The stated schedule for updates is Tuesday and Friday, but they typically are posted just once or twice a month on non-specific days. Recently, this schedule has slipped further, due to the health issues of Sarah Gallagher (Seraphim), Fred's wife. Megatokyo was also published in book-format by CMX, although the first three volumes were published by Dark Horse. For February 2005, sales of the comic's third printed volume were ranked third on BookScan's list of graphic novels sold in bookstores, then the best showing for an original English-language manga.

59. “Unbelievable Gwenpool”, Eng­lish Wikipedia (2020-12-28):

    

    The Unbelievable Gwenpool, more commonly called Unbelievable Gwenpool, is a comic book series published by Marvel Comics, featuring Gwenpool as its main protagonist. The series was a spin-off from the character's feature in a Howard the Duck comic, and was Gwenpool's first solo series. The series lasted 26 issues, #1-25 and a special #0 that collected her intro material. The series ran from June 2016 to April 2018.

60. “The Tale of the Princess Kaguya”, Eng­lish Wikipedia (2020-12-28):

    

    The Tale of the Princess Kaguya is a 2013 Japanese animated fantasy drama film co-written and directed by Isao Takahata, based on The Tale of the Bamboo Cutter, a 10th-century Japanese literary tale. It was produced by Studio Ghibli for Nippon Television Network, Dentsu, Hakuhodo DYMP, Walt Disney Japan, Mitsubishi, Toho and KDDI, and distributed by Toho.

61. /Anime-reviews#the-tale-of-the-princess-kaguya

62. https://www.youtube.com/watch?v=1dVv5RMwzuo

63. /Movie-reviews#dna-dreams

64. https://www.dropbox.com/s/hbw9e5zd9h12n9g/%E7%8E%96%E5%8E%9F%E3%80%80%E3%82%A4%E3%83%85%E3%83%8A-%E6%B0%B8%E5%8A%AB%E6%BA%80%E6%9C%88phantasystarsymphony-bulletraid.ogg?dl=0

65. https://www.youtube.com/watch?v=3Q6a7gNu4lk

66. https://www.youtube.com/watch?v=Mbuo324g-7w

67. https://www.youtube.com/watch?v=1istntu1IA4

68. https://www.youtube.com/watch?v=tAONfTdJ9i4

69. https://www.youtube.com/watch?v=JTScPAEyeuY

70. https://www.youtube.com/watch?v=1xJ0GAz7ylE

71. https://www.youtube.com/watch?v=3WuetN1UCZg

72. https://www.youtube.com/watch?v=xaIXzS2xTQo

73. https://www.dropbox.com/s/okqgmpdc7x6okyq/itm-koebuticmusicfes-higheroriginalmix.ogg?dl=0

74. https://www.dropbox.com/s/3qmnbyrvewum7qu/nash-koebuticmusicfes-drag.ogg?dl=0

75. https://www.dropbox.com/s/wdz69l6vvl7ylli/%E3%82%A2%E3%82%AF%E3%83%AA%E3%83%AD-koebuticmusicfes-pearflowers%EF%BD%9E%E6%A2%A8%E8%A1%A3%E3%81%97%E3%82%87%E3%82%93%E3%81%AE%E8%8A%B1%EF%BD%9E.ogg?dl=0

76. https://www.dropbox.com/s/4zwnjfd8pu1mb4a/%E3%81%A0-koebuticmusicfes-calmforest.ogg?dl=0

77. https://www.dropbox.com/s/8p7j0j86i7cj290/%E3%83%9E%E3%83%B3%E3%83%80%E3%83%AA%E3%83%B3-koebuticmusicfes-%E4%B8%83%E8%89%B2%E3%81%AE%E6%97%A5%E5%B8%B8.ogg?dl=0

78. “Gwern.net newsletter (Substack subscription page)”, Gw­ern Bran­wen (2013-12-01):

    Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.

79. https://old.reddit.com/r/gwern/.rss