--- title: November 2020 News description: 'November 2020 Gwern.net newsletter with links on DL and genomics scaling, dark mode rewrite, 1 essay, and 1 opera review (_The Ring_ cycle).' created: 2019-12-26 modified: 2021-01-09 status: finished previous: /newsletter/2020/10 next: /newsletter/2020/12 confidence: log cssExtension: dropcaps-de-zs backlink: False ... November 2020's [Gwern.net newsletter](/newsletter/2020/11 "'November 2020 News', Branwen 2019") is now out; previous, [October 2020](/newsletter/2020/10 "'October 2020 News', Branwen 2019") ([archives](/doc/newsletter/index)). This is a summary of the updates RSS feed, overlapping with my [Changelog](/changelog); brought to you by my donors on [Patreon](https://www.patreon.com/gwern). # Writings - [On Development Hell](/review/anime#on-development-hell) - **Gwern.net**: dark-mode rewrite complete (fixes page load flash & laggy scrolling) [arabesque](https://commons.wikimedia.org/wiki/File:Filet_arabesque.svg) navigation bar in footer with JS keyboard shortcuts; [IBM Plex](https://en.wikipedia.org/wiki/IBM_Plex) Mono font & custom [ALGOL](https://en.wikipedia.org/wiki/ALGOL_60)-like syntax highlighting for code blocks; new [sun](https://en.wikipedia.org/wiki/Vergina_Sun)/[moon](https://commons.wikimedia.org/wiki/File:Japanese_crest_Tsuki_ni_Hoshi.svg) icons for horizontal rulers; images in Wikipedia popups; new internal link/citation convention for multiple citations # Links ## AI [Matters Of Scale](https://www.reddit.com/r/mlscaling/ "'ML Scaling subreddit', Branwen 2020"): - ["Exploring the limits of Concurrency in ML Training on Google TPUs"](https://arxiv.org/abs/2011.03641#google), Kumar et al 2020 (BERT in 23s on a TPU-4096; "We view the current competition in language understanding as a modern-day Space Race, with competing organizations assembling both giant machines and giant models in the quest for an Artificial General Intelligence breakthrough.") - ["When Do You Need Billions of Words of Pretraining Data?"](https://arxiv.org/abs/2011.04946), Zhang et al 2020 (how do NNs learn from language as _n_ increases? The [blessings of scale](/scaling-hypothesis#blessings-of-scale) again: superficial linguistic competence is learned easily with mere millions of words, but, like in GPT-3, the interesting capabilities only start to show up at billions+) - ["Measuring Progress in Deep Reinforcement Learning Sample Efficiency"](https://arxiv.org/abs/2102.04881), Anonymous et al 2020 (ALE halving: 10--18mo; continuous state (Half-Cheetah): 5--24mo; continuous pixel (Walker): 4--9mo) - ["Contrastive Representation Learning: A Framework and Review"](https://arxiv.org/abs/2010.05113), Le-Khac et al 2020 - ["Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus"](https://arxiv.org/abs/2010.14571#google "'Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus', Caswell et al 2020"), Caswell et al 2020 (Internet data is noisy: every problem you can think of & more you haven't); ["Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study"](https://arxiv.org/abs/2008.13533#google), Bahri et al 2020 - ["Scaling Laws from the Data Manifold Dimension"](https://arxiv.org/abs/2004.10802), Sharma & Kaplan 2020 - ["Understanding RL Vision"](https://distill.pub/2020/understanding-rl-vision/#diversity-hypothesis "'Understanding RL Vision: With diverse environments, we can analyze, diagnose and edit deep reinforcement learning models using attribution', Hilton et al 2020"), Hilton et al 2020 ('the blessings of scale': agent vision generalizes better & is more interpretable with more kinds of levels, even with fixed sample size, past a certain point) - ["Towards Playing Full MOBA Games with Deep Reinforcement Learning"](https://arxiv.org/abs/2011.12692#tencent), Ye et al 2020 (pro-level on 5×5 MOBA [_Honor of Kings_](https://en.wikipedia.org/wiki/Honor_of_Kings) using 250k CPU-cores/2k GPUs); ["TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game"](https://arxiv.org/abs/2011.13729#tencent), Han et al 2020 (12.6k CPU-cores/0.3k GPUs) - ["VDVAE: Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images"](https://arxiv.org/abs/2011.10650#openai "'Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images', Child 2020"), Child 2020 (was all VAEs really needed was much larger models & some stabilizing tricks like ["gradient skipping"](https://arxiv.org/pdf/2011.10650.pdf#page=6&org=openai)?); ["NVAE: A Deep Hierarchical Variational Autoencoder"](https://arxiv.org/abs/2007.03898#nvidia), Vahdat & Kautz 2020 (using spectral regularization instead of gradient skipping) ## Genetics Everything Is Heritable: - ["Discovery of rare variants associated with blood pressure regulation through meta-analysis of 1.3 million individuals"](/doc/genetics/heritable/rare/2020-surendran.pdf), Surendran et al 2020; ["Largest GWAS (_N_ = 1,126,563) of Alzheimer’s Disease Implicates Microglia and Immune Cells"](https://www.medrxiv.org/content/10.1101/2020.11.20.20235275.full "'Largest GWAS (N=1,126,563) of Alzheimer’s Disease Implicates Microglia and Immune Cells', Wightman et al 2020"), Wightman et al 2020 - ["Genome-wide meta-analysis of brain volume identifies genomic loci and genes shared with intelligence"](https://www.nature.com/articles/s41467-020-19378-5), Jansen et al 2020 - ["Genetic predictors of educational attainment and intelligence test performance predict voter turnout"](/doc/genetics/heritable/correlation/mendelian-randomization/2020-aroe.pdf), Aarøe et al 2020 - ["Estimation of non-additive genetic variance in human complex traits from a large sample of unrelated individuals"](https://www.biorxiv.org/content/10.1101/2020.11.09.375501.full), Hivert et al 2020 (non-additive variance still trivial: 0% dominance; 6% epistasis) - ["An integrative analysis of genomic and exposomic data for complex traits and phenotypic prediction"](https://www.biorxiv.org/content/10.1101/2020.11.09.373704.full), Zhou & Lee 2020 Engineering: - ["An antiviral self-replicating molecular heterotroph"](https://www.biorxiv.org/content/10.1101/2020.08.12.248997.full), Shapiro et al 2020 - ["Engineering Brain Parasites (_Toxoplasma_) for Intracellular Delivery of Therapeutic Proteins"](https://www.biorxiv.org/content/10.1101/481192.full "'Engineering Brain Parasites for Intracellular Delivery of Therapeutic Proteins', Bracha et al 2018"), Bracha et al 2018 (see also [Del Giudice 2019](/doc/genetics/selection/natural/2019-delguidice.pdf "Invisible Designers: Brain Evolution Through the Lens of Parasite Manipulation") ([SSC](https://slatestarcodex.com/2019/08/19/maybe-your-zoloft-stopped-working-because-a-liver-fluke-tried-to-turn-your-nth-great-grandmother-into-a-zombie/ "Maybe Your Zoloft Stopped Working Because A Liver Fluke Tried To Turn Your Nth-Great-Grandmother Into A Zombie"))) ## Statistics/Meta-Science - ["Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability"](/doc/statistics/bias/2020-ebersole.pdf), Ebersole et al 2020 (despite the claims from researchers' whose work doesn't replicate, the "experimenter competence" moderator doesn't exist) - ["Bayesian workflow"](http://www.stat.columbia.edu/~gelman/research/unpublished/Bayesian_Workflow_article.pdf "'Bayesian Workflow', Gelman et al 2020"), Gelman et al 2020 - [Berkson’s paradox](https://en.wikipedia.org/wiki/Berkson%27s_paradox) (anywhere there is *selection* or *optimization* and you are not using total population samples---be on guard) ## Psychology/Biology - ["Evening home lighting adversely impacts the circadian system and sleep"](https://www.nature.com/articles/s41598-020-75622-4), Cain et al 2020 (between-housing light differences can drive large individual differences in melatonin) - ["Memorising Milton's _Paradise Lost_: A study of a septuagenarian exceptional memoriser"](/doc/psychology/spaced-repetition/2010-seamon.pdf "'Memorising Milton's Paradise Lost: A study of a septuagenarian exceptional memoriser', Seamon et al 2020"), Seamon 2010 ## Technology - ["Survey of Alternative Displays"](/doc/technology/2016-09-19-neal-surveyofalternativedisplays.html), Blair Neal (beyond LEDs) - ["Structural Typography: Type as both language and composition"](/doc/design/typography/2018-10-09-heck-structuraltypography.html), Bethany Heck (using fonts as integral part of posters' form & meaning) - ["The Relevance of Classic Fuzz Testing: Have We Solved This One?"](https://arxiv.org/abs/2008.06537), Miller et al 2020 ('no', they aren't even robust to the [feline form](/review/cat#fuzz-testing "'Cat Psychology & Domestication: Are We Good Owners? § Fuzz Testing', Branwen 2018") of [fuzzing](https://en.wikipedia.org/wiki/Fuzzing); followup to [Miller et al 1990](/doc/cs/security/1990-miller.pdf "An empirical study of the reliability of UNIX utilities")) ## Economics - ["The daily grind: Before millstones were invented, the preparation of flour for food was an arduous task largely carried out by women for hours every day. How did it affect their lives and why does it remain a tradition in some places even today?"](https://worksinprogress.co/issue/the-daily-grind/), Rachel Laudan ("all that is solid melts into thin air...") ## Fiction - ["We Are What We Watch: Movie Plots Predict the Personalities of Those who ‘Like’ Them"](https://osf.io/preprints/psyarxiv/wsdu8/), Nave et al 2020 ## Miscellaneous - [Béla Kiss](https://en.wikipedia.org/wiki/B%C3%A9la_Kiss) # Film/TV **Live-action**: - [_Das Rheingold_](https://en.wikipedia.org/wiki/Das_Rheingold), [_Siegfried_](https://en.wikipedia.org/wiki/Siegfried_\(opera\)), [_Götterdämmerung_](https://en.wikipedia.org/wiki/G%C3%B6tterd%C3%A4mmerung) ([review](/review/opera#rheingold-siegfried-gotterdammerung){.include-replace-container}) - [_Singin' In The Rain_](https://en.wikipedia.org/wiki/Singin%27_in_the_Rain) ([review](/review/movie#singin-in-the-rain){.include-replace-container}) - [_Rosemary's Baby_](https://en.wikipedia.org/wiki/Rosemary%27s_Baby_\(film\)) (1968) - [_Ex Machina_](https://en.wikipedia.org/wiki/Ex_Machina_\(film\)) (2014) - [_The Cabinet of Dr. Caligari_](https://en.wikipedia.org/wiki/The_Cabinet_of_Dr._Caligari) **Animated**: - [_Azumanga Daioh_](https://en.wikipedia.org/wiki/Azumanga_Daioh) - [_Planetes_](https://en.wikipedia.org/wiki/Planetes)