newsletter/2017/12 (Link Bibliography)

“newsletter/​2017/​12” links:

  1. https://gwern.substack.com

  2. 11

  3. newsletter

  4. Changelog

  5. https://www.patreon.com/gwern

  6. ⁠, Jakob Grove, Stephan Ripke, Thomas D. Als, Manuel Mattheisen, Raymond Walters, Hyejung Won, Jonatan Pallesen, Esben Agerbo, Ole A. Andreassen, Richard Anney, Rich Belliveau, Francesco Bettella, Joseph D. Buxbaum, Jonas Bybjerg-Grauholm, Marie Bækved-Hansen, Felecia Cerrato, Kimberly Chambert, Jane H. Christensen, Claire Churchhouse, Karin Dellenvall, Ditte Demontis, Silvia De Rubeis, Bernie Devlin, Srdjan Djurovic, Ashle Dumont, Jacqueline Goldstein, Christine S. Hansen, Mads Engel Hauberg, Mads V. Hollegaard, Sigrun Hope, Daniel P. Howrigan, Hailiang Huang, Christina Hultman, Lambertus Klei, Julian Maller, Joanna Martin, Alicia R. Martin, Jennifer Moran, Mette Nyegaard, Terje Nærland, Duncan S. Palmer, Aarno Palotie, Carsten B. Pedersen, Marianne G. Pedersen, Timothy Poterba, Jesper B. Poulsen, Beate St Pourcain, Per Qvist, Karola Rehnström, Avi Reichenberg, Jennifer Reichert, Elise B. Robinson, Kathryn Roeder, Panos Roussos, Evald Saemundsen, Sven Sandin, F. Kyle Satterstrom, George D. Smith, Hreinn Stefansson, Kari Stefansson, Stacy Steinberg, Christine Stevens, Patrick F. Sullivan, Patrick Turley, G. Bragi Walters, Xinyi Xu, Autism Spectrum Disorders Working Group of The Psychiatric Genomics Consortium, BUPGEN, Major Depressive Disorder Working Group of the Psychiatric Genomics Consortium, 23andMe Research Team, Daniel Geschwind, Merete Nordentoft, David M. Hougaard, Thomas Werge, Ole Mors, Preben Bo Mortensen, Benjamin M. Neale, Mark J. Daly, Anders D. Børglum (2017-11-25):

    Autism spectrum disorder (ASD) is a highly heritable and heterogeneous group of neurodevelopmental phenotypes diagnosed in more than 1% of children. Common genetic variants contribute substantially to ASD susceptibility, but to date no individual variants have been robustly associated with ASD. With a marked sample size increase from a unique Danish population resource, we report a genome-wide association of 18,381 ASD cases and 27,969 controls that identifies five genome-wide statistically-significant loci. Leveraging results from three phenotypes with significantly overlapping genetic architectures (⁠, major depression, and educational attainment), seven additional loci shared with other traits are identified at equally strict significance levels. Dissecting the polygenic architecture we find both quantitative and qualitative polygenic heterogeneity across ASD subtypes, in contrast to what is typically seen in other complex disorders. These results highlight biological insights, particularly relating to neuronal function and corticogenesis and establish that GWAS performed at scale will be much more productive in the near term in ASD, just as it has been in a broad range of important psychiatric and diverse medical phenotypes.

  7. https://www.wired.com/story/ancestrys-genetic-testing-kits-are-heading-for-your-stocking-this-year/

  8. http://www.aging-us.com/article/101334/text

  9. 2017-dudbridge.pdf

  10. https://gnxp.nofe.me/2017/12/12/most-people-say-they-think-nurture-is-more-important-than-nature-especially-white-americans/

  11. 2017-weigel.pdf: “A 100-Year Review: Methods and impact of genetic selection in dairy cattle - From daughter-dam comparisons to deep learning algorithms”⁠, K. A. Weigel, P. M. VanRaden, H. D. Norman, H. Grosu

  12. https://www.bbc.com/news/health-42337396

  13. https://www.nejm.org/doi/full/10.1056/NEJMoa1708483

  14. https://www.wired.com/story/crispr-therapeutics-plans-its-first-clinical-trial-for-genetic-disease/

  15. ⁠, Tim Beissinger, Jochen Kruppa, David Cavero, Ngoc-Thuy Ha, Malena Erbe, Henner Simianer (2017-12-21):

    Important traits in agricultural, natural, and human populations are increasingly being shown to be under the control of many genes that individually contribute only a small proportion of genetic variation. However, the majority of modern tools in quantitative and ⁠, including genome wide association studies and selection mapping protocols, are designed to target the identification of individual genes with large effects. We have developed an approach to identify traits that have been under selection and are controlled by large numbers of loci. In contrast to existing methods, our technique utilizes additive effects estimates from all available markers, and relates these estimates to allele frequency change over time. Using this information, we generate a composite statistic, denoted Ĝ, which can be used to test for significant evidence of selection on a trait. Our test requires genotypic data from multiple time points but only a single time point with phenotypic information. Simulations demonstrate that Ĝ is powerful for identifying selection, particularly in situations where the trait being tested is controlled by many genes, which is precisely the scenario where classical approaches for selection mapping are least powerful. We apply this test to breeding populations of maize and chickens, where we demonstrate the successful identification of selection on traits that are documented to have been under selection.

  16. ⁠, David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis (2017-12-05):

    The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa from games of self-play. In this paper, we generalise this approach into a single algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

  17. https://old.reddit.com/r/reinforcementlearning/comments/7hvin5/mastering_chess_and_shogi_by_selfplay_with_a/

  18. 2017-12-24-meme-nnlayers-alphagozero.jpg

  19. ⁠, Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger (2017-09-19):

    In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.

  20. ⁠, Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, Olivier Bousquet (2017-11-28):

    Generative adversarial networks () are a powerful subclass of generative models. Despite a very rich research activity leading to numerous interesting GAN algorithms, it is still very hard to assess which algorithm(s) perform better than others. We conduct a neutral, multi-faceted large-scale empirical study on state-of-the art models and evaluation measures. We find that most models can reach similar scores with enough hyperparameter optimization and random restarts. This suggests that improvements can arise from a higher computational budget and tuning more than fundamental algorithmic changes. To overcome some limitations of the current metrics, we also propose several data sets on which precision and recall can be computed. Our experimental results suggest that future GAN research should be based on more systematic and objective evaluation procedures. Finally, we did not find evidence that any of the tested algorithms consistently outperforms the non-saturating GAN introduced in .

  21. ⁠, Gábor Melis, Chris Dyer, Phil Blunsom (2017-07-18):

    Ongoing innovations in architectures have provided a steady influx of apparently state-of-the-art results on language modelling benchmarks. However, these have been evaluated using differing code bases and limited computational resources, which represent uncontrolled sources of experimental variation. We reevaluate several popular architectures and regularisation methods with large-scale automatic black-box hyperparameter tuning and arrive at the somewhat surprising conclusion that standard LSTM architectures, when properly regularised, outperform more recent models. We establish a new state of the art on the Penn Treebank and Wikitext-2 corpora, as well as strong baselines on the Hutter Prize dataset.

  22. ⁠, Jeff Dean (2017):

    Slide deck for Google Brain presentation on Machine Learning and the future of ML development processes. Conclusions: ML hardware is at its infancy. Even faster systems and wider deployment will lead to many more breakthroughs across a wide range of domains. Learning in the core of all of our computer systems will make them better/​​​​more adaptive. There are many opportunities for this.

  23. ⁠, Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, Neoklis Polyzotis (2017-12-04):

    Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not. In this exploratory research paper, we start from this premise and posit that all existing index structures can be replaced with other types of models, including deep-learning models, which we term learned indexes. The key idea is that a model can learn the sort order or structure of lookup keys and use this signal to effectively predict the position or existence of records. We theoretically analyze under which conditions learned indexes outperform traditional index structures and describe the main challenges in designing learned index structures. Our initial results show, that by using neural nets we are able to outperform cache-optimized B-Trees by up to 70% in speed while saving an order-of-magnitude in memory over several real-world data sets. More importantly though, we believe that the idea of replacing core components of a data management system through learned models has far reaching implications for future systems designs and that this work just provides a glimpse of what might be possible.

  24. Tool-AI

  25. https://distill.pub/2017/aia/

  26. ⁠, Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro (2017-11-30):

    We present a new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs). Conditional GANs have enabled a variety of applications, but the results are often limited to low-resolution and still far from realistic. In this work, we generate 2048×1024 visually appealing results with a novel adversarial loss, as well as new multi-scale generator and discriminator architectures. Furthermore, we extend our framework to interactive visual manipulation with two additional features. First, we incorporate object instance segmentation information, which enables object manipulations such as removing/​​​​adding objects and changing the object category. Second, we propose a method to generate diverse results given the same input, allowing users to edit the object appearance interactively. Human opinion studies demonstrate that our method significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing.

  27. https://tcwang0509.github.io/pix2pixHD/

  28. https://www.theatlantic.com/magazine/archive/2018/01/whats-college-good-for/546590/

  29. https://slatestarcodex.com/2017/11/13/book-review-legal-systems-very-different-from-ours/

  30. https://www.newyorker.com/magazine/2017/11/13/where-the-small-town-american-dream-lives-on

  31. https://meaningness.com/metablog/post-apocalyptic-health-care

  32. https://www.smbc-comics.com/comic/healthcare

  33. https://www.nytimes.com/2017/12/09/health/drug-prices-generics-insurance.html

  34. https://www.wired.com/story/age-of-social-credit/

  35. https://medium.com/@tonyszhou/postmortem-1b338537fabc

  36. Copyright-deadweight

  37. https://www.samharris.org/blog/item/the-fireplace-delusion

  38. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.510.4296&rep=rep1&type=pdf

  39. https://vizhub.healthdata.org/gbd-compare/#settings=1c5235ed0e4875665b1e2005644f6a7694b65e96

  40. https://www.nytimes.com/2017/12/26/health/measles-deaths-vaccination.html

  41. http://eprints.lse.ac.uk/86619/1/dp1522.pdf

  42. 2017-flynn.pdf: “IQ decline and Piaget_ Does the rot start at the top?”⁠, James R. Flynn, Michael Shayer

  43. https://www.frontiersin.org/articles/10.3389/fnhum.2017.00588/full

  44. 2017-boland.pdf: “Meta-Analysis of the Antidepressant Effects of Acute Sleep Deprivation”⁠, Elaine M. Boland, Hengyi Rao, David F. Dinges, Rachel V. Smith, Namni Goel, John A. Detre, Mathias Basner, Yvette I. Sheline, Michael E. Thase, Philip R. Gehrman

  45. http://ftp.iza.org/dp11110.pdf

  46. https://www.nytimes.com/2017/12/14/health/zika-babies-brazil.html

  47. https://www.newyorker.com/magazine/2017/12/18/chinas-selfie-obsession

  48. https://www.si.com/vault/2009/03/23/105789480/how-and-why-athletes-go-broke

  49. https://www.bmj.com/content/359/bmj.j5451

  50. https://www.wired.com/story/mirai-botnet-minecraft-scam-brought-down-the-internet/

  51. https://www.filfre.net/2017/12/games-on-the-net-before-the-web-part-2-mud/

  52. ⁠, Dan Luu (2017-12):

    I’ve had this nagging feeling that the computers I use today feel slower than the computers I used as a kid. As a rule, I don’t trust this kind of feeling because human perception has been shown to be unreliable in empirical studies, so I carried around a high-speed camera and measured the response latency of devices I’ve run into in the past few months. These are tests of the latency between a keypress and the display of a character in a terminal (see appendix for more details)…If we look at overall results, the fastest machines are ancient. Newer machines are all over the place. Fancy gaming rigs with unusually high refresh-rate displays are almost competitive with machines from the late 70s and early 80s, but “normal” modern computers can’t compete with thirty to forty year old machines.

    …Almost every computer and mobile device that people buy today is slower than common models of computers from the 70s and 80s. Low-latency gaming desktops and the iPad Pro can get into the same range as quick machines from thirty to forty years ago, but most off-the-shelf devices aren’t even close.

    If we had to pick one root cause of latency bloat, we might say that it’s because of “complexity”. Of course, we all know that complexity is bad. If you’ve been to a non-academic non-enterprise tech conference in the past decade, there’s a good chance that there was at least one talk on how complexity is the root of all evil and we should aspire to reduce complexity.

    Unfortunately, it’s a lot harder to remove complexity than to give a talk saying that we should remove complexity. A lot of the complexity buys us something, either directly or indirectly. When we looked at the input of a fancy modern keyboard vs. the Apple 2 keyboard, we saw that using a relatively powerful and expensive general purpose processor to handle keyboard inputs can be slower than dedicated logic for the keyboard, which would both be simpler and cheaper. However, using the processor gives people the ability to easily customize the keyboard, and also pushes the problem of “programming” the keyboard from hardware into software, which reduces the cost of making the keyboard. The more expensive chip increases the manufacturing cost, but considering how much of the cost of these small-batch artisanal keyboards is the design cost, it seems like a net win to trade manufacturing cost for ease of programming.

  53. ⁠, Dan Luu (2017-10-16; technology⁠, cs⁠, design):

    [Dan Luu continues his investigation of why computers feel so laggy and have such high latency compared to old computers (⁠, ⁠, ⁠, cf text editor analysis).

    He measures 21 keyboard latencies using a logic analyzer, finding a range of 15–60ms (!), representing a waste of a large fraction of the available ~100–200ms latency budget before a user notices and is irritated (“the median keyboard today adds as much latency as the entire end-to-end pipeline as a fast machine from the 70s.”). The latency estimates are surprising, and do not correlate with advertised traits. They simply have to be measured empirically.]

    We can see that, even with the limited set of keyboards tested, there can be as much as a 45ms difference in latency between keyboards. Moreover, a modern computer with one of the slower keyboards attached can’t possibly be as responsive as a quick machine from the 70s or 80s because the keyboard alone is slower than the entire response pipeline of some older computers. That establishes the fact that modern keyboards contribute to the latency bloat we’ve seen over the past forty years…Most keyboards add enough latency to make the user experience noticeably worse, and keyboards that advertise speed aren’t necessarily faster. The two gaming keyboards we measured weren’t faster than non-gaming keyboards, and the fastest keyboard measured was a minimalist keyboard from Apple that’s marketed more on design than speed.

  54. https://medium.com/future-crunch/99-reasons-2017-was-a-good-year-d119d0c32d19

  55. https://ourworldindata.org/famines/

  56. https://www.adamsmith.org/research/back-in-the-ussr

  57. https://reason.com/archives/2017/11/28/in-search-of-the-elusive-bitco

  58. https://www.newyorker.com/magazine/2017/12/18/jim-simons-the-numbers-king

  59. https://www.alicemaz.com/writing/minecraft.html

  60. https://www.washingtonian.com/2017/12/06/spies-dossiers-insane-lengths-restaurants-go-track-influence-food-critics-tom-sietsema/

  61. https://www.amazon.com/Sunset-Spider-Poetry-Ancient-Korea/dp/0030120713/

  62. https://www.amazon.com/Selected-Poems-Knowledge-Directions-Paperbook/dp/0811201910

  63. Movies#icarus

  64. Movies#rollerball

  65. https://www.youtube.com/watch?v=pKqENDuZS6A

  66. https://www.youtube.com/watch?v=gX4Q26OzO_Q

  67. https://www.youtube.com/watch?v=JLQmiYVomcg

  68. https://www.youtube.com/watch?v=CeaHtXykY2Q

  69. https://mega.nz/#!RKYGyTQY!15PMlngSzKpM6vg0gC_NrR6I7uBydhk0vZIVmX7UEmc

  70. https://mega.nz/#!tKYRRAQJ!ZRNqWlsMuWYHx1IJbOTlwdEslmb1SrSMhdb2eXVv7ig

  71. https://mega.nz/#!JHZTCBqT!G9dLXKYG-jptH8lP_YHIsBA_V_8byExciGnm_lX5wNU

  72. https://www.youtube.com/watch?v=kkD4yuxehHU

  73. https://www.dropbox.com/s/2x98cx7ovifclad/akt-%E6%AE%8B%E9%9F%BF-zankyo-%E3%83%94%E3%82%A2%E3%83%8E%E3%82%BD%E3%83%8A%E3%82%BF%E3%80%8C%E6%AE%8B%E9%9F%BF%E3%80%8D-iii-scherzoprestovivace.ogg?dl=0

  74. https://www.dropbox.com/s/g2p8axw80c740mw/akt-%E6%AE%8B%E9%9F%BF-zankyo-%E3%83%94%E3%82%A2%E3%83%8E%E3%82%BD%E3%83%8A%E3%82%BF%E3%80%8C%E6%AE%8B%E9%9F%BF%E3%80%8D-iv-rondoallegretto.ogg

  75. https://www.dropbox.com/s/6ce2ahac07lo59q/akt-%E6%AE%8B%E9%9F%BF-zankyo-%E4%BA%8C%E5%8F%B0%E3%81%AE%E3%83%81%E3%82%A7%E3%83%AD%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE3%E3%81%A4%E3%81%AE%E5%B9%BB%E6%83%B3%E6%9B%B2-02%E3%80%8C%E6%A0%84%E8%8F%AF%E3%80%8D.ogg?dl=0

  76. https://www.dropbox.com/s/9zj5tdvnvthoy5e/akt-%E6%AE%8B%E9%9F%BF-zankyo-%E4%BA%8C%E5%8F%B0%E3%81%AE%E3%83%81%E3%82%A7%E3%83%AD%E3%81%AE%E3%81%9F%E3%82%81%E3%81%AE3%E3%81%A4%E3%81%AE%E5%B9%BB%E6%83%B3%E6%9B%B2-03%E3%80%8C%E9%9D%99%E5%AF%82%E3%80%8D.ogg?dl=0

  77. https://www.dropbox.com/s/xkaulrvd3pok2nx/akt-%E6%AE%8B%E9%9F%BF-zankyo-%E3%83%94%E3%82%A2%E3%83%8E%E4%BA%94%E9%87%8D%E5%A5%8F%E6%9B%B2%E3%80%8C%E8%A8%98%E6%86%B6%E3%80%8D-iiiadagioedolce.ogg?dl=0

  78. https://www.dropbox.com/s/sleyvqgdl221fln/%E9%98%BF%E8%89%AF%E8%89%AF%E6%9C%A8%E5%81%A5feat.luotianyi-lovetheory-%E6%9C%80%E5%90%8E%E7%9A%84%E6%AD%8Clalala.ogg?dl=0