---
title: January 2020 News
description: January 2020 Gwern.net newsletter with 5 writeups, and links on AI scaling, videos-for-cats, and art; 1 book and 1 opera review.
created: 2019-12-26
status: finished
confidence: log
css-extension: dropcaps-de-zs
backlink: False
...

January 2020's [Gwern.net](/newsletter/2020/01 "'January 2020 News', Gwern 2019") [newsletter](https://gwern.substack.com/ "'Gwern.net newsletter (Substack subscription page)', Gwern 2013") is now out; previous, [December 2019](/newsletter/2019/12 "'December 2019 News', Gwern 2019")/[2019 round-up](/newsletter/2019/13 "'2019 News', Gwern 2019") ([archives](/doc/newsletter/index)).
This is a collation of links and summary of major changes, overlapping with my [Changelog](/changelog); brought to you by my donors on [Patreon](https://www.patreon.com/gwern).

# Writings

- ["Danbooru2019: A Large-Scale Crowdsourced & Tagged Anime Illustration Dataset"](/danbooru2021#danbooru2019 "Danbooru2019 is a large-scale anime image database with 3.69m+ images annotated with 108m+ tags; it can be useful for machine learning purposes such as image recognition and generation.")
- [Preference Learning GPT-2 Music: Null Result](/gpt-2-preference-learning "Experiments with OpenAI's 'preference learning' approach, which trains a NN to predict global quality of datapoints, and then uses reinforcement learning to optimize that directly, rather than proxies. I am unable to improve quality, perhaps due to too-few ratings.")
- [This Waifu Does Not Exist](https://www.thiswaifudoesnotexist.net/ "'ThisWaifuDoesNotExist.net', Gwern 2019")[v3](/twdne#twdnev3 "'This Waifu Does Not Exist § TWDNEv3', Gwern 2019"): 100k [StyleGAN 2](/face#stylegan-2 "'Making Anime Faces With StyleGAN § StyleGAN 2', Gwern 2019") anime portrait samples
- [Subreddit Simulator: GPT-2-1.5b upgrade](https://www.reddit.com/r/SubSimulatorGPT2Meta/comments/entfgx/update_upgrading_to_15b_gpt2_and_adding_22_new/ "Subreddit Simulator imitates Redditor discusses using GPT-2; we train GPT-2-1.5b on 10GB from ~100 subreddits & provide an upgrade.")
- [14 Internet Search Case Studies](/search-case-studies "14 case studies of difficult-to-find resources or citations, and how I went about locating them, giving practical demonstrations of the various Internet search techniques described previously and showing how to think about searches.")
- **Gwern.net**: margin notes are now inlined on mobile

# Media

## Links

**Genetics**:

- [Everything Is Heritable]{.smallcaps}:

    - ["Genomic Relationships, Novel Loci, and Pleiotropic Mechanisms across 8 Psychiatric Disorders"](/doc/genetics/heritable/correlation/2019-pgc.pdf "'Genomic Relationships, Novel Loci, and Pleiotropic Mechanisms across Eight Psychiatric Disorders', Consortium 2019"), PGC 2019 ([everything is correlated](/everything))
    - ["Investigating the Genetic Architecture of Non-Cognitive Skills Using GWAS-by-Subtraction"](https://www.biorxiv.org/content/10.1101/2020.01.14.905794.full), Demange et al 2020
- [Recent Evolution]{.smallcaps}:

    - ["The Exposome in Human Evolution: From Dust to Diesel"](/doc/genetics/selection/natural/human/2019-trumble.pdf), Trumble & Finch 2019 ([media](https://www.nytimes.com/2020/01/13/science/air-pollution-fires-genes.html "Why Is Air Pollution So Harmful? DNA May Hold the Answer: It's not just a modern problem. Airborne toxins are so pernicious that they may have shaped human evolution"); previously, [Hubbard et al 2016](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5026259/ "Divergent Ah receptor ligand selectivity during hominin evolution"))
- [Engineering]{.smallcaps}:

    - ["Utility and First Clinical Application of Screening Embryos for Polygenic Disease Risk Reduction"](https://www.frontiersin.org/journals/endocrinology/articles/10.3389/fendo.2019.00845/full), Treff et al 2019 (Genomic Prediction)
    - ["CC, world's first cloned cat, turns 18 years old"](https://theeagle.com/news/local/cc-world-s-first-cloned-cat-turns-years-old/article_d2aeac6e-2471-11ea-a5f2-7b6c21b2b4b4.html), Katz

**AI**:

- ["2019 AI Alignment Literature Review and Charity Comparison"](https://www.lesswrong.com/posts/SmDziGM9hBjW9DKmf/2019-ai-alignment-literature-review-and-charity-comparison), Larks
- [Matters Of Scale]{.smallcaps}:

    - ["Scaling Laws for Neural Language Models"](https://arxiv.org/abs/2001.08361#openai), Kaplan et al 2020 (NN LMs appear to be nowhere near saturation nor training infeasibility as larger models show predictable gains and are both more compute-efficient & data-efficient (!)---onwards to GPT-3?)
    - ["Big Transfer (BiT): Large Scale Learning of General Visual Representations for Transfer"](https://arxiv.org/abs/1912.11370#google "'Big Transfer (BiT): General Visual Representation Learning', Kolesnikov et al 2019"), Kolesnikov et al 2019 ([blog](https://research.google/blog/open-sourcing-bit-exploring-large-scale-pre-training-for-computer-vision/ "'Open-Sourcing BiT: Exploring Large-Scale Pre-training for Computer Vision', Beyer et al 2020"); JFT-300M showing larger=better: large benchmark gains, transfer everywhere, much more robust representations, more plausible errors, etc.)
    - ["DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames"](https://arxiv.org/abs/1911.00357#facebook), Wijmans et al 2019 ([blog](https://ai.meta.com/blog/near-perfect-point-goal-navigation-from-25-billion-frames-of-experience/ "'Near-perfect point-goal navigation from 2.5 billion frames of experience', Wijmans & Kadian 2020"); "[Mishkin et al 2019](https://arxiv.org/abs/1901.10915#intel "Benchmarking Classic and Learned Navigation in Complex 3D Environments") benchmarked classical (mapping + planning) and learning-based methods...and showed that classical methods outperform learning-based. However, they trained for 'only' 5 million steps...[Savva et al 2019](https://arxiv.org/abs/1904.01201#facebook "Habitat: A Platform for Embodied AI Research") then scaled this training to 75 million steps and found that this trend *reverses*...Fig. 1 shows an agent does not saturate before 1 billion steps, suggesting that previous studies were incomplete by 1--2 *orders of magnitude*.")
    - ["Meena: Towards a Human-like Open-Domain Chatbot"](https://arxiv.org/abs/2001.09977#google "'Towards a Human-like Open-Domain Chatbot', Adiwardana et al 2020"), Adiwardana et al 2020 ([blog](https://research.google/blog/towards-a-conversational-agent-that-can-chat-aboutanything/ "Towards a Conversational Agent that Can Chat About...Anything"); 2.6b parameters trained on 341GB text, although Kaplan et al 2020 suggests they'd've done better to go much bigger & trained much less than 164 epoches; likelihood loss near identical with human-rated performance? Likelihood loss seems ultimately a flawed metric, but maybe we're still far from hitting its limits...)
- ["A Very Unlikely Chess Game"](https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-game/) (GPT-2-1.5b shenanigans; [rival implementation](https://github.com/ricsonc/transformers-play-chess/blob/master/README.md "Transformers Play Chess"))

**Statistics/Meta-Science**:

- ["Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study"](/doc/statistics/bias/2020-devito.pdf "‘Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study’, DeVito et al 2020"), DeVito et al 2020; ["FDA and NIH let clinical trial sponsors keep results secret and break the law"](https://www.science.org/content/article/fda-and-nih-let-clinical-trial-sponsors-keep-results-secret-and-break-law), _Science_
- ["A national experiment reveals where a growth mindset improves achievement"](https://www.nature.com/articles/s41586-019-1466-y), Yeager et al 2019 (the incredible shrinking 'growth mindset' effect & the [Stainless Steel Law](/doc/sociology/1987-rossi "'The Iron Law Of Evaluation And Other Metallic Rules', Rossi 2012"))
- ["Backlash Over Meat Dietary Recommendations Raises Questions About Corporate Ties to Nutrition Scientists"](https://jamanetwork.com/journals/jama/article-abstract/2759201), Rubin 2019 (criticizing the critics of [Carroll & Doherty 2019](https://www.acpjournals.org/doi/10.7326/M19-2620 "Meat Consumption and Health: Food for Thought") / [Zeraatkar et al 2019a](https://www.acpjournals.org/doi/10.7326/M19-0655 "Red and Processed Meat Consumption and Risk for All-Cause Mortality and Cardiometabolic Outcomes: A Systematic Review and Meta-analysis of Cohort Studies") / [Han et al 2019](https://www.acpjournals.org/doi/10.7326/M19-0699 "Reduction of Red and Processed Meat Intake and Cancer Mortality and Incidence: A Systematic Review and Meta-analysis of Cohort Studies") / [Vernooij et al 2019](https://www.acpjournals.org/doi/10.7326/M19-1583 "Patterns of Red and Processed Meat Consumption and Risk for Cardiometabolic and Cancer Outcomes: A Systematic Review and Meta-analysis of Cohort Studies") / [Zeraatkar et al 2019b](/doc/longevity/2019-zeraatkar.pdf "Effect of Lower Versus Higher Red Meat Intake on Cardiometabolic and Cancer Outcomes: A Systematic Review of Randomized Trials"){#zeraatkar-2} / [Valli et al 2019](https://www.acpjournals.org/doi/10.7326/M19-1326 "Health-Related Values and Preferences Regarding Meat Consumption: A Mixed-Methods Systematic Review") / [Johnston et al 2019](https://www.acpjournals.org/doi/10.7326/M19-1621 "Unprocessed Red Meat and Processed Meat Consumption: Dietary Guideline Recommendations From the Nutritional Recommendations (NutriRECS) Consortium")---"what's sauce for the goose is sauce for the gander")
- ["Follow-up: I found two identical packs of Skittles, among 468 packs with a total of 27,740 Skittles"](https://possiblywrong.wordpress.com/2019/04/06/follow-up-i-found-two-identical-packs-of-skittles-among-468-packs-with-a-total-of-27740-skittles/) (empirically verifying the [birthday paradox](https://en.wikipedia.org/wiki/Birthday_problem) in [Skittles](https://en.wikipedia.org/wiki/Skittles_\(confectionery\)) bags)

**Politics/religion**:

- ["The rape of men: the darkest secret of war"](https://www.theguardian.com/society/2011/jul/17/the-rape-of-men "Sexual violence is one of the most horrific weapons of war, an instrument of terror used against women. Yet huge numbers of men are also victims. In this harrowing report, Will Storr travels to Uganda to meet traumatised survivors, and reveals how male rape is endemic in many of the world's conflicts") (case study: [Libya](https://www.theguardian.com/world/2017/nov/03/revealed-male-used-systematically-in-libya-as-instrument-of-war "'Revealed: male rape used systematically in Libya as instrument of war: Videos and testimony expose brutal tactics used by several factions in fractured country', Allegra 2017"))
- ["Under the Weather: As Psychiatrists And Philosophers Begin To Define A Pervasive Mental Health Crisis Triggered By Climate Change, They Ask Who Is Really Sick: The Individual Or Society?"](https://www.thebeliever.net/climate-grief-anxiety/)
- ["Statistical Reliability Analysis For A Most Dangerous Occupation: Roman Emperor"](https://www.nature.com/articles/s41599-019-0366-y "'Statistical reliability analysis for a most dangerous occupation: Roman emperor', Saleh 2019"), Saleh 2019
- ["Parachuting For Charity: Is It Worth The Money? A 5-Year Audit Of Parachute Injuries In Tayside And The Cost To The NHS"](/doc/sociology/1999-lee.pdf "'Parachuting for charity: is it worth the money? A 5-year audit of parachute injuries in Tayside and the cost to the NHS', Lee et al 1999"), Lee et al 1999
- ["What's in a Font?: Ideological Perceptions of Typography"](https://www.tandfonline.com/doi/full/10.1080/10510974.2019.1692884), Haenschen & Tamul 2019 ('[Sunrise](https://www.dafont.com/sunrise-2.font) is the future liberals want')

**Psychology/biology**:

- ["A Meta-Analysis of Procedures to Change Implicit Measures"](/doc/psychology/cognitive-bias/2019-forscher.pdf), Forscher et al 2019
- ["Attention And Awareness In Stage Magic: Turning Tricks Into Research"](/doc/psychology/cognitive-bias/illusion-of-depth/2008-macknik.pdf "'Attention and awareness in stage magic: turning tricks into research', Macknik et al 2008"), Macknik et al 2008
- ["A World Without Pain: Does hurting make us human?"](https://www.newyorker.com/magazine/2020/01/13/a-world-without-pain) (cf. my [essay on pain](/backstop "Evolution as Backstop for Reinforcement Learning"))
- ["What Intellectual Progress Did I Make In The 2010s?"](https://slatestarcodex.com/2020/01/08/what-intellectual-progress-did-i-make-in-the-2010s/), Scott Alexander (a look back on how his ideas/beliefs evolved over the past decade of psychiatry blogging)
- ["Three cases of giant panda attack on humans at Beijing Zoo"](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4276236/ "'Three cases giant panda attack on human at Beijing Zoo', Zhang et al 2014"), Zhang et al 2014 (graphic images)

**Technology**:

- ["Cats, Once YouTube Stars, Are Now an 'Emerging Audience': They're addicted to channels like Little Kitty & Family, Handsome Nature, and Videos for Your Cat---provided their owners switch on the iPad first"](https://www.wired.com/story/cats-watch-youtube/) (After reading this, I gave videos-for-cats another try with my cat, since he gets stir-crazy in winter. I full-screened it, in landscape mode; he continued resolutely ignoring the screen as always, until I left my earphones out and he *heard* the birds chirping---he went nuts, convinced a bird had gotten into the apartment, until he *finally* noticed the screen, and I could see the instant the lightbulb went on and he was hooked. He'll watch videos like [Paul Dinning's "8 Hour Bird Bonanza"](https://www.youtube.com/watch?v=xbs7FT7dXYc) for as many hours as I'll leave it on, hunting behind the monitor for the birds, and makes a nuisance of himself sitting in front of it, waiting for the birds to come back---which he is doing as I try to write this! These are truly superstimuli: cats would never see these many birds this close up so unsuspecting in the wild.)

**Economics**:

- ["Clustering of health, crime and social-welfare inequality in 4 million citizens from two nations"](/doc/sociology/2020-richmondrakerd.pdf), Richmond-Rakerd et al 2020 (everything is correlated: ["Figure 4: Aggregation of poor health, crime and social-welfare dependency"](/doc/genetics/heritable/correlation/2020-richmondrakerd-figure4-correlations.png){.invert}; previously, [Belsky et al 2016](/doc/genetics/heritable/correlation/2016-belsky.pdf "The Genetics of Success: How Single-Nucleotide Polymorphisms Associated With Educational Attainment Relate to Life-Course Development") & [Caspi et al 2016](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5505663/ "Childhood forecasting of a small segment of the population with large economic burden"))
- ["The Exquisitely English (and Amazingly Lucrative) World of London Clerks"](/doc/economics/2017-akam-theexquisitelyenglishandamazinglylucrativeworldoflondonclerks.html "It's a Dickensian profession that can still pay upwards of $650,000 per year.")

**Philosophy**:

- [Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal](/subscript "'Subscripts For Citations', Gwern 2020")

**Fiction**:

- ["Behind the Sensationalism: Images of a Decaying Corpse in Japanese Buddhist Art"](/doc/japan/art/2005-kanda.pdf), Kanda 2005 (on _kusozu_, cf. [Maraṇasati](https://en.wikipedia.org/wiki/Mara%E1%B9%87asati); graphic images; famous examples: [_Body of a Courtesan in 9 stages_](https://strangeremains.com/2014/06/24/body-of-a-courtesan-in-nine-stages-a-19th-century-study-of-decomposition/ "'"Body of a courtesan in 9 stages": A 19<sup>th</sup> century study of decomposition', Remains 2014"), [Kobayashi Eitaku](https://en.wikipedia.org/wiki/Kobayashi_Eitaku) c. 1870s, and [_The Death Of A Noble Lady And The Decay Of Her Body_](https://strangeremains.com/2015/03/06/the-beauty-of-decomposition-in-japanese-watercolor/ "'The beauty of human decomposition in Japanese watercolor', Remains 2015"), c. 1700s)
- ["Having Had No Predecessor to Imitate, He Had No Successor Capable of Imitating Him"](https://www.fantasticanachronism.com/2020/01/17/having-had-no-predecessor-to-imitate/index.html), Alvaro de Menard (summary of the [Homeric Question](https://en.wikipedia.org/wiki/Homeric_Question), resolved by [Parry 1933](/doc/history/1933-parry.pdf "'Whole Formulaic Verses in Greek and Southslavic Heroic Song', Perry 1933"); also worth reading: Borges on the literary merits of different translations of [Homer](/doc/borges/1932-borges-thehomericversions.pdf "'The Homeric Versions', Borges 1932") & [_The Thousand and One Nights_](/doc/borges/1936-borges-thetranslatorsofthethousandandonenights.pdf "'The Translators of <em>The Thousand and One Nights</em>', Borges 1936"))
- ["Choose Your Own Adventure: One Book, Many Readings"](https://samizdat.co/cyoa/), [Christian Swinehart](https://samizdat.co/me/) 2009 (visualizing paths through the classic [CYOA](https://en.wikipedia.org/wiki/Choose_Your_Own_Adventure) [gamebooks](https://en.wikipedia.org/wiki/Gamebook); see also: ["These Maps Reveal the Hidden Structures of _Choose Your Own Adventure_ Books: If you decide to see more, click on this story"](https://www.atlasobscura.com/articles/cyoa-choose-your-own-adventure-maps "'These Maps Reveal the Hidden Structures of <em>Choose Your Own Adventure</em> Books: If you decide to see more, click on this story', Laskow 2017"), _Atlas Obscura_ 2017)
- ["_Master of Orion_"](https://www.filfre.net/2020/01/master-of-orion/ "'<em>Master of Orion</em>', Maher 2020"), Jimmy Maher (review of the [seminal 4X strategy game](https://en.wikipedia.org/wiki/Master_of_Orion))

**Misc**:

- ["Dorodango, the Japanese art of making mud balls: _Dorodango_ author Bruce Gardner shares the story of how he discovered the Japanese art of hikaru dorodango"](https://www.laurenceking.com/blog/2019/09/26/dorodango-blog/ "'Dorodango, the Japanese art of making mud balls: <em>Dorodango</em> author Bruce Gardner shares the story of how he discovered the Japanese art of hikaru dorodango', Gardner 2019") ([WP](https://en.wikipedia.org/wiki/Dorodango); [Gibson](/doc/japan/art/2002-gibson "'Shiny balls of Mud: William Gibson Looks at Japanese Pursuits of Perfection', Gibson 2012"))
- "[It Had to Be Her: Review of _Passionate Spirit: The Life of_](https://www.nybooks.com/articles/2020/01/16/alma-mahler-it-had-to-be-her/ "'It Had to Be Her: Review of <em>Passionate Spirit: The Life of Alma Mahler</em>, Haste 2019', Schine 2020") [_Alma Mahler_](https://en.wikipedia.org/wiki/Alma_Mahler)"

## Books

**Nonfiction**:

- _An Introduction to Japanese Court Poetry_, Miner 1968 ([review](/review/book#an-introduction-to-japanese-court-poetry-miner-1968){.include})

## Film/TV

**Live-action**:

- [_They Live_](https://en.wikipedia.org/wiki/They_Live) (1988; [review](/review/movie#they-live){.include})
- [_Wozzeck_](https://en.wikipedia.org/wiki/Wozzeck) ([review](/review/opera#wozzeck){.include})