newsletter/2020/11 (Link Bibliography)

“newsletter/​2020/​11” links:

  1. 11

  2. 10

  3. newsletter

  4. Changelog


  6. Anime#on-development-hell



  9. ⁠, Gwern Branwen (2020-10-30):

    Subreddit for discussing AI, machine learning, or deep learning approaches involving big numbers: billions of parameters, millions of n, petaflops, etc. eg ⁠. Most research is conducted at much smaller scale; this subreddit is for research analogous to ‘high energy physics’, requiring specialized approaches, large investments, consortium, etc.

    Topics: How? Who? Why do they work? What are they good for? What resources are available? Who will pay & how? What is the future of such approaches? What global consequences will there be?

  10. ⁠, Sameer Kumar, James Bradbury, Cliff Young, Yu Emma Wang, Anselm Levskaya, Blake Hechtman, Dehao Chen, HyoukJoong Lee, Mehmet Deveci, Naveen Kumar, Pankaj Kanwar, Shibo Wang, Skye Wanderman-Milne, Steve Lacy, Tao Wang, Tayo Oguntebi, Yazhou Zu, Yuanzhong Xu, Andy Swing (2020-11-07):

    Recent results in language understanding using neural networks have required training hardware of unprecedented scale, with thousands of chips cooperating on a single training run. This paper presents techniques to scale ML models on the Google TPU Multipod, a mesh with 4096 chips. We discuss model parallelism to overcome scaling limitations from the fixed batch size in data parallelism, communication/​​​​collective optimizations, distributed evaluation of training metrics, and host input processing scaling optimizations. These techniques are demonstrated in both the TensorFlow and JAX programming frameworks. We also present performance results from the recent Google submission to the MLPerf-v0.7 benchmark contest, achieving record training times from 16 to 28 seconds in 4 MLPerf models on the Google TPU-v3 Multipod machine.

    Figure 11: End-to-end time speedups of MLPerf benchmarks over 16 accelerator chips of their own types.
  11. ⁠, Yian Zhang, Alex Warstadt, Haau-Sing Li, Samuel R. Bowman (2020-11-09):

    NLP is currently dominated by general-purpose pretrained language models like ⁠, which achieve strong performance on NLU tasks through pretraining on billions of words. But what exact knowledge or skills do Transformer LMs learn from large-scale pretraining that they cannot learn from less data?

    We adopt four probing methods—classifier probing, information-theoretic probing, unsupervised relative acceptability judgment, and fine-tuning on NLU tasks—and draw learning curves that track the growth of these different measures of linguistic ability with respect to pretraining data volume using the MiniBERTas, a group of RoBERTa models pretrained on 1M, 10M, 100M and 1B words.

    We find that LMs require only about 10M or 100M words to learn representations that reliably encode most syntactic and semantic features we test. A much larger quantity of data is needed in order to acquire enough common-sense knowledge and other skills required to master typical downstream NLU tasks. The results suggest that, while the ability to encode linguistic features is almost certainly necessary for language understanding, it is likely that other forms of knowledge are the major drivers of recent improvements in language understanding among large pretrained models.

  12. Scaling-hypothesis#blessings-of-scale

  13. ⁠, Anonymous (2020-09-28):

    We measure progress in deep sample efficiency using training curves from published papers. Sampled environment transitions are a critical input to deep reinforcement learning (DRL) algorithms. Current DRL benchmarks often allow for the cheap and easy generation of large amounts of samples such that perceived progress in DRL does not necessarily correspond to improved sample efficiency. As simulating real world processes is often prohibitively hard and collecting real world experience is costly, sample efficiency is an important indicator for economically relevant applications of DRL. We investigate progress in sample efficiency on Atari games and continuous control tasks by comparing the amount of samples that a variety of algorithms need to reach a given performance level according to training curves in the corresponding publications. We find exponential progress in sample efficiency with estimated doubling times of around 10 to 18 months on Atari [ALE], 5 to 24 months on state-based continuous control [HalfCheetah] and of around 4 to 9 months on pixel-based continuous control [Walker Walk] depending on the specific task and performance level.

    The amount of samples used to train DRL agents on the ALE and the speed at which samples are generated has increased rapidly. since was first trained on the majority of the now standard 57 Atari games in 2015 (Mnih et al2015), the amount of samples per game used by the most ambitious projects to train their agents on the ALE has increased by a factor of 450 from 200 million to 90 billion as shown in figure 1 (a). This corresponds to a doubling time in sample use of around 7 months. Converted into real game time, it represents a jump from 38.6 hours (per game) to 47.6 years which was enabled by the fast speed of the simulators and running large amounts of simulations in parallel to reduce the wall-clock time needed for processing that many frames. In fact, the trend in wall-clock training time is actually reversed as can be seen in table 1: while DQN was trained for a total of 9.5 days, MuZero took only 12 hours of training to process 20 billion frames (), which represents a speedup in utilized frames per second of 1900 in less than five years. This demonstrates that using larger and larger amounts of samples has become a lot more popular as well as feasible over time.

    Figure 1: (a): Amount of frames per game used for results on Atari over time plotted on a log scale…(b): Median human-normalized score on 57 Atari games over time plotted on a log scale.

    …While the exact slopes of the fitted trend lines are fairly uncertain due to the limited amount of data points, especially for the unrestricted benchmark, it seems like progress on the unrestricted benchmarks is around twice as fast. This can be interpreted as roughly half of progress coming from increased sample use, while the other half comes from a combination of algorithmic improvements and more compute usage [In the form of larger neural networks or reusing samples for multiple training passes.].

    Figure 2: (a): Amount of frames needed per game to reach the same median human-normalized score as DQN over 57 games in the Arcade Learning Environment (ALE) (Bellemare et al 2013). Grey dots indicate measurements and blue dots indicate the SOTA in sample efficiency at the time of a measurement. The linear fit on the log scale plot for the SOTA (blue dots) indicates exponential progress in sample efficiency. It corresponds to a doubling time of 11 months. (b): Pareto front concerning training frames per game and the median-normalized score on Atari on a doubly logarithmic scale. The dotted lines indicate an interpolation from the data points. Results for less than 10 million frames consider 26 rather than all 57 games.
  14. ⁠, Phuc H. Le-Khac, Graham Healy, Alan F. Smeaton (2020-10-10):

    Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.

  15. ⁠, Isaac Caswell, Theresa Breiner, Daan van Esch, Ankur Bapna (2020-10-27):

    Large text corpora are increasingly important for a wide variety of Natural Language Processing (NLP) tasks, and automatic language identification (LangID) is a core technology needed to collect such datasets in a multilingual context. LangID is largely treated as solved in the literature, with models reported that achieve over 90% average F1 on as many as 1,366 languages. We train LangID models on up to 1,629 languages with comparable quality on held-out test sets, but find that human-judged LangID accuracy for web-crawl text corpora created using these models is only around 5% for many lower-resource languages, suggesting a need for more robust evaluation. Further analysis revealed a variety of error modes, arising from domain mismatch, class imbalance, language similarity, and insufficiently expressive models. We propose two classes of techniques to mitigate these errors: wordlist-based tunable-precision filters (for which we release curated lists in about 500 languages) and transformer-based semi-supervised LangID models, which increase median dataset precision from 5.5% to 71.2%. These techniques enable us to create an initial data set covering 100K or more relatively clean sentences in each of 500+ languages, paving the way towards a 1,000-language web text corpus.

  16. ⁠, Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Cliff Brunk, Andrew Tomkins (2020-08-17):

    Large generative language models such as are well-known for their ability to generate text as well as their utility in supervised downstream tasks via fine-tuning. Our work is twofold: firstly we demonstrate via human evaluation that classifiers trained to discriminate between human and machine-generated text emerge as unsupervised predictors of “page quality”, able to detect low quality content without any training. This enables fast bootstrapping of quality indicators in a low-resource setting. Secondly, curious to understand the prevalence and nature of low quality pages in the wild, we conduct extensive qualitative and quantitative analysis over 500 million web articles, making this the largest-scale study ever conducted on the topic.

  17. ⁠, Utkarsh Sharma, Jared Kaplan (2020-04-22):

    When data is plentiful, the loss achieved by well-trained neural networks scales as a LN−α in the number of network parameters N. This empirical scaling law holds for a wide variety of data modalities, and may persist over many orders of magnitude. The scaling law can be explained if neural models are effectively just performing regression on a data manifold of intrinsic dimension d. This simple theory predicts that the scaling exponents α ≈ 4⁄d for and mean-squared error losses. We confirm the theory by independently measuring the intrinsic dimension and the scaling exponents in a teacher/​​​​student framework, where we can study a variety of d and α by dialing the properties of random teacher networks. We also test the theory with image classifiers on several datasets and with GPT-type language models.

  18. ⁠, Jacob Hilton, Nick Cammarata, Shan Carter, Gabriel Goh, Chris Olah (2020-11-17):

    In this article, we apply interpretability techniques to a reinforcement learning (RL) model trained to play the video game CoinRun. Using attribution combined with dimensionality reduction, we build an interface for exploring the objects detected by the model, and how they influence its value function and policy. We leverage this interface in several ways.

    • Dissecting failure: We perform a step-by-step analysis of the agent’s behavior in cases where it failed to achieve the maximum reward, allowing us to understand what went wrong, and why. For example, one case of failure was caused by an obstacle being temporarily obscured from view.
    • Hallucinations: We find situations when the model “hallucinated” a feature not present in the observation, thereby explaining inaccuracies in the model’s value function. These were brief enough that they did not affect the agent’s behavior.
    • Model editing: We hand-edit the weights of the model to blind the agent to certain hazards, without otherwise changing the agent’s behavior. We verify the effects of these edits by checking which hazards cause the new agents to fail. Such editing is only made possible by our previous analysis, and thus provides a quantitative validation of this analysis.

    Our results depend on levels in CoinRun being procedurally-generated, leading us to formulate a diversity hypothesis for interpretability. If it is correct, then we can expect RL models to become more interpretable as the environments they are trained on become more diverse. We provide evidence for our hypothesis by measuring the relationship between interpretability and generalization.

    …All of the above analysis uses the same hidden layer of our network, the third of five convolutional layers, since it was much harder to find interpretable features at other layers. Interestingly, the level of abstraction at which this layer operates—finding the locations of various in-game objects—is exactly the level at which CoinRun levels are randomized using procedural generation. Furthermore, we found that training on many randomized levels was essential for us to be able to find any interpretable features at all.

    This led us to suspect that the diversity introduced by CoinRun’s randomization is linked to the formation of interpretable features. We call this the diversity hypothesis:

    Interpretable features tend to arise (at a given level of abstraction) if and only if the training distribution is diverse enough (at that level of abstraction).

    Our explanation for this hypothesis is as follows. For the forward implication (“only if”), we only expect features to be interpretable if they are general enough, and when the training distribution is not diverse enough, models have no incentive to develop features that generalize instead of overfitting. For the reverse implication (“if”), we do not expect it to hold in a strict sense: diversity on its own is not enough to guarantee the development of interpretable features, since they must also be relevant to the task. Rather, our intention with the reverse implication is to hypothesize that it holds very often in practice, as a result of generalization being bottlenecked by diversity.

  19. ⁠, Deheng Ye, Guibin Chen, Wen Zhang, Sheng Chen, Bo Yuan, Bo Liu, Jia Chen, Zhao Liu, Fuhao Qiu, Hongsheng Yu, Yinyuting Yin, Bei Shi, Liang Wang, Tengfei Shi, Qiang Fu, Wei Yang, Lanxiao Huang, Wei Liu (2020-11-25):

    MOBA games, e.g., Honor of Kings, League of Legends, and Dota 2, pose grand challenges to AI systems such as multi-agent, enormous state-action space, complex action control, etc. Developing AI for playing MOBA games has raised much attention accordingly. However, existing work falls short in handling the raw game complexity caused by the explosion of agent combinations, i.e., lineups, when expanding the hero pool in case that OpenAI’s Dota AI limits the play to a pool of only 17 heroes. As a result, full MOBA games without restrictions are far from being mastered by any existing AI system. In this paper, we propose a MOBA AI learning paradigm that methodologically enables playing full MOBA games with deep reinforcement learning. Specifically, we develop a combination of novel and existing learning techniques, including curriculum self-play learning, ⁠, off-policy adaption, multi-head value estimation, and Monte-Carlo tree-search, in training and playing a large pool of heroes, meanwhile addressing the scalability issue skillfully. Tested on Honor of Kings, a popular MOBA game, we show how to build superhuman AI agents that can defeat top esports players. The superiority of our AI is demonstrated by the first large-scale performance test of MOBA AI agent in the literature.

  20. ⁠, Lei Han, Jiechao Xiong, Peng Sun, Xinghai Sun, Meng Fang, Qingwei Guo, Qiaobo Chen, Tengfei Shi, Hongsheng Yu, Zhengyou Zhang (2020-11-27):

    StarCraft, one of the most difficult esport games with long-standing history of professional tournaments, has attracted generations of players and fans, and also, intense attentions in artificial intelligence research. Recently, Google’s DeepMind announced ⁠, a grandmaster level AI in StarCraft II. In this paper, we introduce a new AI agent, named TStarBot-X, that is trained under limited computation resources and can play competitively with expert human players. TStarBot-X takes advantage of important techniques introduced in AlphaStar, and also benefits from substantial innovations including new league training methods, novel multi-agent roles, rule-guided policy search, lightweight neural network architecture, and importance sampling in imitation learning, etc. We show that with limited computation resources, a faithful reimplementation of AlphaStar can not succeed and the proposed techniques are necessary to ensure TStarBot-X’s competitive performance. We reveal all technical details that are complementary to those mentioned in AlphaStar, showing the most sensitive parts in league training, reinforcement learning and imitation learning that affect the performance of the agents. Most importantly, this is an open-sourced study that all codes and resources (including the trained model parameters) are publicly accessible via this URL⁠. We expect this study could be beneficial for both academic and industrial future research in solving complex problems like StarCraft, and also, might provide a sparring partner for all StarCraft II players and other AI agents.

    [See also ⁠, Sun et al 2020.]

  21. ⁠, Rewon Child (2020-11-20):

    We present a hierarchical VAE that, for the first time, generates samples quickly while outperforming the PixelCNN in log-likelihood on all natural image benchmarks. We begin by observing that, in theory, VAEs can actually represent autoregressive models, as well as faster, better models if they exist, when made sufficiently deep. Despite this, autoregressive models have historically outperformed VAEs in log-likelihood. We test if insufficient depth explains why by scaling a VAE to greater stochastic depth than previously explored and evaluating it CIFAR-10, ⁠, and FFHQ. In comparison to the PixelCNN, these very deep VAEs achieve higher likelihoods, use fewer parameters, generate samples thousands of times faster, and are more easily applied to high-resolution images. Qualitative studies suggest this is because the VAE learns efficient hierarchical visual representations. We release our source code and models at https:/​​​​/​​​​​​​​openai/​​​​vdvae.


  23. ⁠, Arash Vahdat, Jan Kautz (2020-07-08):

    Normalizing flows, autoregressive models, variational autoencoders (VAEs), and deep energy-based models are among competing likelihood-based frameworks for deep generative learning. Among them, VAEs have the advantage of fast and tractable sampling and easy-to-access encoding networks. However, they are currently outperformed by other models such as normalizing flows and autoregressive models. While the majority of the research in VAEs is focused on the statistical challenges, we explore the orthogonal direction of carefully designing neural architectures for hierarchical VAEs. We propose Nouveau VAE (NVAE), a deep hierarchical VAE built for image generation using depth-wise separable convolutions and batch normalization. NVAE is equipped with a residual parameterization of and its training is stabilized by spectral regularization. We show that NVAE achieves state-of-the-art results among non-autoregressive likelihood-based models on the MNIST, CIFAR-10, CelebA 64, and CelebA HQ datasets and it provides a strong baseline on ⁠. For example, on CIFAR-10, NVAE pushes the state-of-the-art from 2.98 to 2.91 bits per dimension, and it produces high-quality images on CelebA HQ. To the best of our knowledge, NVAE is the first successful VAE applied to natural images as large as 256×256 pixels. The source code is available at https:/​​​​/​​​​​​​​NVlabs/​​​​NVAE .

  24. ⁠, L. Yengo, A. R. Wood, S. Vedantam, E. Marouli, J. Sidorenko, S. Sakaue, S. Raghavan, G. Lettre, Y. Okada, J. N. Hirschhorn, P. M. Visscher, Genetic Investigation of Anthropometric Traits (GIANT) consortium (2020-10-26):

    Human height is a highly heritable and polygenic trait, with an estimated -based heritability (h2SNP) from common SNPs between 40% and 50% in individuals of European ancestry. Genome-wide (GWS) SNPs identified by previous account for 25% of phenotypic variation, that is less than 50% of h2SNP. Here, as part of a study across 5 major ancestries, we perform the largest GWAS of height to date, in 4.1 million individuals of European ancestry. We identify ~10,000 conditionally independent associations, which altogether explains 40% of height (ie. between 75% and 100% of h2SNP). We find that 1,547/​​​​1,703 (ie. ~90%) of European-ancestry (LD) blocks across the genome contain at least one association and that the genomic distribution of GWS SNPs is not random, such that a few genomic loci contain >20 independent associations within 100 kb of one another, consistent with the presence of multiple causal variants. We find a ~3.2-fold enrichment (p < 0.001) of genes with pathogenic mutations causing extreme height or abnormal skeletal growth syndromes near genomic loci with a high density of GWS SNPs. A (PGS) of height calculated from all GWS SNPs has a within-ancestry prediction accuracy R2 of 40% (S.E. 3.1%) and combining that PGS with the average height of parents reaches an unprecedented accuracy of 54% (S.E. 3.0%), consistent with theoretical ⁠. Altogether, our results show that GWAS of height in individuals of European ancestry approaches saturation for common variants and suggest, when the experimental sample size is sufficiently large, that genetic variation can be resolved to the contribution of individual variants.

  25. ⁠, Jocelyn Kaiser (Science) (2020-11-03):

    For height, DNA is largely destiny. Studies of identical and fraternal twins suggest up to 80% of variation in height is genetic. But the genes responsible have largely eluded researchers. Now, by amassing genome data for 4 million people—the largest such study ever—geneticists have accounted for a major share of this “missing heritability”, at least for people of European ancestry. In this group, they’ve identified nearly 10,000 DNA markers that appear to fully explain the influence of common genetic variants over height. “This is a genuine landmark”, says Daniel MacArthur of the Garvan Institute of Medical Research in Australia.

    …The mystery of missing heritability dates back to the late 2000s…A number of possible explanations emerged, including rare gene variants missed by the GWA studies, gene-gene interactions, and that the twin studies were wrong. But Peter Visscher, leader of Yengo’s team, argued it was partly a matter of finding many more common variants with very small effects. He that such variants should account for 40% to 50% of the genetic component of height. Picking out the faint signals would require studying the DNA of a huge number of people, however.

    By 2018, Visscher’s team and other members of a global consortium called GIANT had pooled DNA data for 700,000 people and found 3300 common markers that explained in height. Now, by looking across DNA from 201 GWA studies with 4.1 million participants, GIANT has brought the total to roughly 9900 common markers, accounting for 40% of the variation. Other markers located nearby and likely inherited together account for another 10% of height variability. That’s still short of the 80% predicted by twin studies. But last year, Visscher’s group drew on whole-genome sequencing data of a smaller number of people to demonstrate that rare variants—those carried by fewer than one in 100 people—should explain another 30% of height’s variation. (The result was released in a that the team is revising.)

    Some geneticists say they aren’t surprised that heritability gaps can be filled once enough people had their DNA scanned. “It was expected”, says Aravinda Chakravarti of New York University.

  26. 2020-surendran.pdf: ⁠, Praveen Surendran, Elena V. Feofanova, Najim Lahrouchi, Ioanna Ntalla, Savita Karthikeyan, James Cook, Lingyan Chen, Borbala Mifsud, Chen Yao, Aldi T. Kraja, James H. Cartwright, Jacklyn N. Hellwege, Ayush Giri, Vinicius Tragante, Gudmar Thorleifsson, Dajiang J. Liu, Bram P. Prins, Isobel D. Stewart, Claudia P. Cabrera, James M. Eales, Artur Akbarov, Paul L. Auer, Lawrence F. Bielak, Joshua C. Bis, Vickie S. Braithwaite, Jennifer A. Brody, E. Warwick Daw, Helen R. Warren, Fotios Drenos, Sune Fallgaard Nielsen, Jessica D. Faul, Eric B. Fauman, Cristiano Fava, Teresa Ferreira, Christopher N. Foley, Nora Franceschini, He Gao, Olga Giannakopoulou, Franco Giulianini, Daniel F. Gudbjartsson, Xiuqing Guo, Sarah E. Harris, Aki S. Havulinna, Anna Helgadottir, Jennifer E. Huffman, Shih-Jen Hwang, Stavroula Kanoni, Jukka Kontto, Martin G. Larson, Ruifang Li-Gao, Jaana Lindström, Luca A. Lotta, Yingchang Lu, Jian’an Luan, Anubha Mahajan, Giovanni Malerba, Nicholas G. D. Masca, Hao Mei, Cristina Menni, Dennis O. Mook-Kanamori, David Mosen-Ansorena, Martina Müller-Nurasyid, Guillaume Paré, Dirk S. Paul, Markus Perola, Alaitz Poveda, Rainer Rauramaa, Melissa Richard, Tom G. Richardson, Nuno Sepúlveda, Xueling Sim, Albert V. Smith, Jennifer A. Smith, James R. Staley, Alena Stanáková, Patrick Sulem, Sébastien Thériault, Unnur Thorsteinsdottir, Stella Trompet, Tibor V. Varga, Digna R. Velez Edwards, Giovanni Veronesi, Stefan Weiss, Sara M. Willems, Jie Yao, Robin Young, Bing Yu, Weihua Zhang, Jing-Hua Zhao, Wei Zhao, Wei Zhao, Evangelos Evangelou, Stefanie Aeschbacher, Eralda Asllanaj, Stefan Blankenberg, Lori L. Bonnycastle, Jette Bork-Jensen, Ivan Brandslund, Peter S. Braund, Stephen Burgess, Kelly Cho, Cramer Christensen, John Connell, Renée de Mutsert, Anna F. Dominiczak, Marcus Dörr, Gudny Eiriksdottir, Aliki-Eleni Farmaki, J. Michael Gaziano, Niels Grarup, Megan L. Grove, Göran Hallmans, Torben Hansen, Christian T. Have, Gerardo Heiss, Marit E. Jørgensen, Pekka Jousilahti, Eero Kajantie, Mihir Kamat, AnneMari Käräjämäki, Fredrik Karpe, Heikki A. Koistinen, Csaba P. Kovesdy, Kari Kuulasmaa, Tiina Laatikainen, Lars Lannfelt, I-Te Lee, Wen-Jane Lee, LifeLines Cohort Study, Allan Linneberg, Lisa W. Martin, Marie Moitry, Girish Nadkarni, Matt J. Neville, Colin N. A. Palmer, George J. Papanicolaou, Oluf Pedersen, James Peters, Neil Poulter, Asif Rasheed, Katrine L. Rasmussen, N. William Rayner, Reedik Mägi, Frida Renström, Rainer Rettig, Jacques Rossouw, Pamela J. Schreiner, Peter S. Sever, Emil L. Sigurdsson, Tea Skaaby, Yan V. Sun, Johan Sundstrom, Gudmundur Thorgeirsson, Tõnu Esko, Elisabetta Trabetti, Philip S. Tsao, Tiinamaija Tuomi, Stephen T. Turner, Ioanna Tzoulaki, Ilonca Vaartjes, Anne-Claire Vergnaud, Cristen J. Willer, Peter W. F. Wilson, Daniel R. Witte, Ekaterina Yonova-Doing, He Zhang, Naheed Aliya, Peter Almgren, Philippe Amouyel, Folkert W. Asselbergs, Michael R. Barnes, Alexandra I. Blakemore, Michael Boehnke, Michiel L. Bots, Erwin P. Bottinger, Julie E. Buring, John C. Chambers, Yii-Der Ida Chen, Rajiv Chowdhury, David Conen, Adolfo Correa, George Davey Smith, Rudolf A. de Boer, Ian J. Deary, George Dedoussis, Panos Deloukas, Emanuele Di Angelantonio, Paul Elliott, EPIC-CVD, EPIC-InterAct, Stephan B. Felix, Jean Ferrières, Ian Ford, Myriam Fornage, Paul W. Franks, Stephen Franks, Philippe Frossard, Giovanni Gambaro, Tom R. Gaunt, Leif Groop, Vilmundur Gudnason, Tamara B. Harris, Caroline Hayward, Branwen J. Hennig, Karl-Heinz Herzig, Erik Ingelsson, Jaakko Tuomilehto, Marjo-Riitta Järvelin, J. Wouter Jukema, Sharon L. R. Kardia, Frank Kee, Jaspal S. Kooner, Charles Kooperberg, Lenore J. Launer, Lars Lind, Ruth J. F. Loos, Abdulla al Shafi. Majumder, Markku Laakso, Mark I. McCarthy, Olle Melander, Karen L. Mohlke, Alison D. Murray, Børge Grønne Nordestgaard, Marju Orho-Melander, Chris J. Packard, Sandosh Padmanabhan, Walter Palmas, Ozren Polasek, David J. Porteous, Andrew M. Prentice, Michael A. Province, Caroline L. Relton, Kenneth Rice, Paul M. Ridker, Olov Rolandsson, Frits R. Rosendaal, Jerome I. Rotter, Igor Rudan, Veikko Salomaa, Nilesh J. Samani, Naveed Sattar, Wayne H.-H. Sheu, Blair H. Smith, Nicole Soranzo, Timothy D. Spector, John M. Starr, Sylvain Sebert, Kent D. Taylor, Timo A. Lakka, Nicholas J. Timpson, Martin D. Tobin, Understanding Society Scientific Group, Pim van der Harst, Peter van der Meer, Vasan S. Ramachandran, Niek Verweij, Jarmo Virtamo, Uwe Völker, David R. Weir, Eleftheria Zeggini, Fadi J. Charchar, Million Veteran Program⁠, Nicholas J. Wareham, Claudia Langenberg, Maciej Tomaszewski, Adam S. Butterworth, Mark J. Caulfield, John Danesh, Todd L. Edwards, Hilma Holm, Adriana M. Hung, Cecilia M. Lindgren, Chunyu Liu, Alisa K. Manning, Andrew P. Morris, Alanna C. Morrison, Christopher J. O’Donnell, Bruce M. Psaty, Danish Saleheen, Kari Stefansson, Eric Boerwinkle, Daniel I. Chasman, Daniel Levy, Christopher Newton-Cheh, Patricia B. Munroe, Joanna M. M. Howson (2020-11-23; genetics  /​ ​​ ​heritable):

    Genetic studies of blood pressure (BP) to date have mainly analyzed common variants (minor allele frequency > 0.05). In a meta-analysis of up to ~1.3 million participants, we discovered 106 new BP-associated genomic regions and 87 rare (minor allele frequency ≤ 0.01) variant BP associations (p <5 × 10−8), of which 32 were in new BP-associated loci and 55 were independent BP-associated single-nucleotide variants within known BP-associated regions. Average effects of rare variants (44% coding) were ~8 times larger than common variant effects and indicate potential candidate causal genes at new and known loci (for example, GATA5 and PLCB3). BP-associated variants (including rare and common) were enriched in regions of active chromatin in fetal tissues, potentially linking fetal development with BP regulation in later life. Multivariable Mendelian randomization suggested possible inverse effects of elevated systolic and diastolic BP on large artery stroke. Our study demonstrates the utility of rare-variant analyses for identifying candidate genes and the results highlight potential therapeutic targets.

  27. ⁠, Douglas P. Wightman, Iris E. Jansen, Jeanne E. Savage, Alexey A. Shadrin, Shahram Bahrami, Arvid Rongve, Sigrid Børte, Bendik S. Winsvold, Ole Kristian Drange, Amy E. Martinsen, Anne Heidi Skogholt, Cristen Willer, Geir Bråthen, Ingunn Bosnes, Jonas Bille Nielsen, Lars Fritsche, Laurent F. Thomas, Linda M. Pedersen, Maiken E. Gabrielsen, Marianne Bakke Johnsen, Tore Wergel, Meisingset, Wei Zhou, Petra Proitsi, Angela Hodges, Richard Dobson, Latha Velayudhan, 23andMe Research Team, Julia M. Sealock, Lea K. Davis, Nancy L. Pedersen, Chandra A. Reynolds, Ida K. Karlsson, Sigurdur Magnusson, Hreinn Stefansson, Steinunn Thordardottir, Palmi V. Jonsson, Jon Snaedal, Anna Zettergren, Ingmar Skoog, Silke Kern, Margda Waern, Henrik Zetterberg, Kaj Blennow, Eystein Stordal, Kristian Hveem, John-Anker Zwart, Lavinia Athanasiu, Ingvild Saltvedt, Sigrid B. Sando, Ingun Ulstein, Srdjan Djurovic, Tormod Fladby, Dag Aarsland, Geir Selbæk, Stephan Ripke, Kari Stefansson, Ole A. Andreassen, Danielle Posthuma (2020-11-23):

    Late-onset Alzheimer’s disease is a prevalent age-related polygenic disease that accounts for 50–70% of dementia cases1. Late-onset Alzheimer’s disease is caused by a combination of many genetic variants with small and environmental influences. Currently, only a fraction of the genetic variants underlying Alzheimer’s disease have been identified2,3. Here we show that increased sample sizes allowed for identification of seven novel genetic loci contributing to Alzheimer’s disease. We highlighted eight potentially causal genes where gene expression changes are likely to explain the association. Human microglia were found as the only cell type where the gene expression pattern was statistically-significantly associated with the Alzheimer’s disease association signal. Gene set analysis identified four independent pathways for associated variants to influence disease pathology. Our results support the importance of microglia, amyloid and tau aggregation, and immune response in Alzheimer’s disease. We anticipate that through collaboration the results from this study can be included in larger meta-analyses of Alzheimer’s disease to identify further genetic variants which contribute to Alzheimer’s pathology. Furthermore, the increased understanding of the mechanisms that mediate the effect of genetic variants on disease progression will help identify potential pathways and gene-sets as targets for drug development.

  28. ⁠, Philip R. Jansen, Mats Nagel, Kyoko Watanabe, Yongbin Wei, Jeanne E. Savage, Christiaan A. de Leeuw, Martijn P. van den Heuvel, Sophie van der Sluis, Danielle Posthuma (2020-11-05):

    The phenotypic correlation between human intelligence and brain volume (BV) is considerable (r ≈ 0.40), and has been shown to be due to shared genetic factors. To further examine specific genetic factors driving this correlation, we present genomic analyses of the genetic overlap between intelligence and BV using genome-wide association study (GWAS) results. First, we conduct a large BV GWAS (n = 47,316 individuals), followed by functional annotation and gene-mapping. We identify 18 genomic loci (14 not previously associated), implicating 343 genes (270 not previously associated) and 18 biological pathways for BV. Second, we use an existing GWAS for intelligence (n = 269,867 individuals), and estimate the (rg) between BV and intelligence to be 0.24. We show that the rg is partly attributable to physical overlap of GWAS hits in 5 genomic loci. We identify 92 shared genes between BV and intelligence, which are mainly involved in signaling pathways regulating cell growth. Out of these 92, we prioritize 32 that are most likely to have functional impact. These results provide information on the genetics of BV and provide biological insight into BV’s shared genetic etiology with intelligence.

  29. 2020-aroe.pdf: ⁠, Lene Aarøe, Vivek Appadurai, Kasper M. Hansen, Andrew J. Schork, Thomas Werge, Ole Mors, Anders D. Børglum, David M. Hougaard, Merete Nordentoft, Preben B. Mortensen, Wesley Kurt Thompson, Alfonso Buil, Esben Agerbo, Michael Bang Petersen (2020-11-09; genetics  /​ ​​ ​correlation):

    Although the genetic influence on voter turnout is substantial (typically 40–50%), the underlying mechanisms remain unclear. Across the social sciences, research suggests that ‘resources for politics’ (as indexed notably by educational attainment and intelligence test performance) constitute a central cluster of factors that predict electoral participation. Educational attainment and intelligence test performance are heritable. This suggests that the genotypes that enhance these phenotypes could positively predict turnout. To test this, we conduct a genome-wide complex trait analysis of individual-level turnout. We use two samples from the Danish iPSYCH case-cohort study, including a nationally representative sample as well as a sample of individuals who are particularly vulnerable to political alienation due to psychiatric conditions (n = 13,884 and n = 33,062, respectively). Using validated individual-level turnout data from the administrative records at the polling station, genetic correlations and ⁠, we show that there is a substantial genetic overlap between voter turnout and both educational attainment and intelligence test performance.

  30. ⁠, Valentin Hivert, Julia Sidorenko, Florian Rohart, Michael E. Goddard, Jian Yang, Naomi R. Wray, Loic Yengo, Peter M. Visscher (2020-11-09):

    Non-additive genetic variance for complex traits is traditionally estimated from data on relatives. It is notoriously difficult to estimate without bias in non-laboratory species, including humans, because of possible with environmental covariance among relatives. In principle, non-additive variance attributable to common DNA variants can be estimated from a random sample of unrelated individuals with genome-wide SNP data. Here, we jointly estimate the proportion of variance explained by additive (h[^2^~SNP~]{.supsub}), dominance (δ[^2^~SNP~]{.supsub}) and additive-by-additive (η[^2^~SNP~]{.supsub}) genetic variance in a single analysis model. We first show by simulations that our model leads to unbiased estimates and provide new theory to predict standard errors estimated using either least squares or maximum likelihood. We then apply the model to 70 complex traits using 254,679 unrelated individuals from the and 1.1M genotyped and imputed SNPs. We found strong evidence for additive variance (average across traits ħ[^2^~SNP~]{.supsub} = 0.207) . In contrast, the average estimate of δ[^2^~SNP~]{.supsub} across traits was 0.001, implying negligible dominance variance at causal variants tagged by common SNPs. The average epistatic variance η[^2^~SNP~]{.supsub} across the traits was 0.058, not statistically-significantly different from zero because of the large sampling variance. Our results provide new evidence that genetic variance for complex traits is predominantly additive, and that sample sizes of many millions of unrelated individuals are needed to estimate epistatic variance with sufficient precision.

  31. ⁠, Xuan Zhou, S. Hong Lee (2020-11-10):

    Complementary to the genome, the concept of exposome has been proposed to capture the totality of human environmental exposures. While there has been some recent progress on the construction of the exposome, few tools exist that can integrate the genome and exposome for complex trait analyses. Here we propose a linear mixed model approach to bridge this gap, which jointly models the random effects of the two omics layers on phenotypes of complex traits. We illustrate our approach using traits from the UK Biobank (e.g., & height for n ~ 40,000) with a small fraction of the exposome that comprises 28 lifestyle factors. The joint model of the genome and exposome explains substantially more phenotypic variance and significantly improves phenotypic prediction accuracy, compared to the model based on the genome alone. The additional phenotypic variance captured by the exposome includes its additive effects as well as non-additive effects such as genome-exposome (gxe) and exposome-exposome (exe) interactions. For example, 19% of variation in BMI is explained by additive effects of the genome, while additional 7.2% by additive effects of the exposome, 1.9% by exe interactions and 4.5% by gxe interactions. Correspondingly, the prediction accuracy for BMI, computed using Pearson’s correlation between the observed and predicted phenotypes, improves from 0.15 (based on the genome alone) to 0.35 (based on the genome & exposome). We also show, using established theories, integrating genomic and exposomic data is essential to attaining a clinically meaningful level of prediction accuracy for disease traits. In conclusion, the genomic and exposomic effects can contribute to phenotypic variation via their relationships, i.e. genome-exposome correlation, and gxe and exe interactions, and modelling these effects has a great potential to improve phenotypic prediction accuracy and thus holds a great promise for future clinical practice.

  32. ⁠, Anastasia Shapiro, Alexander Rosenberg, Adva Levy-Zamir, Liron Bassali, Shmulik Ittah, Almogit Abu-Horowitz, Ido Bachelet (2020-08-14):

    We report the synthesis of a molecular machine, fabricated from nucleic acids, which is capable of digesting viral RNA and utilizing it to assemble additional copies of itself inside living cells. The machine’s body plan combines several parts that build upon the target RNA, assembling an immobile, DNA:RNA 4-way junction, which contains a single gene encoding a hammerhead ribozyme (HHR). Full assembly of the machine’s body from its parts enables the subsequent elongation of the gene and transcription of HHR molecules, followed by HHR-mediated digestion of the target molecule. This digestion converts the target to a building block suitable for participation in the assembly of more copies of the machine, mimicking biological heterotrophy. In this work we describe the general design of a prototypical machine, characterize its activity cycle and kinetics, and show that it can be efficiently and safely delivered into live cells. As a proof of principle, we constructed a machine that targets the Autographa californica multicapsid nucleopolyhedrovirus (AcMNPV) GP64 gene, and show that it effectively suppresses viral propagation in a cell population, exhibiting predator/​​​​prey-like dynamics with the infecting virus. In addition, the machine significantly reduced viral infection, stress signaling, and innate immune activation inside virus-infected animals. This preliminary design could control the behavior of antisense therapies for a range of applications, particularly against dynamic targets such as viruses and cancer.

  33. ⁠, Shahar Bracha, Karoliina Hassi, Paul D. Ross, Stuart Cobb, Lilach Sheiner, Oded Rechavi (2018-12-03):

    Protein therapy has the potential to alleviate many neurological diseases; however, delivery mechanisms for the central nervous system (CNS) are limited, and intracellular delivery poses additional hurdles. To address these challenges, we harnessed the protist parasite Toxoplasma gondii, which can migrate into the CNS and secrete proteins into cells. Using a fusion protein approach, we engineered T. gondii to secrete therapeutic proteins for human neurological disorders. We tested two secretion systems, generated fusion proteins that localized to T. gondii’s secretory organelles and assessed their intracellular targeting in various mammalian cells including neurons. We show that T. gondii expressing GRA16 fused to the Rett syndrome protein MeCP2 deliver a fusion protein that mimics the endogenous MeCP2, binding heterochromatic DNA in neurons. This demonstrates the potential of T. gondii as a therapeutic protein vector, which could provide either transient or chronic, in situ synthesis and delivery of intracellular proteins to the CNS.

  34. 2019-delguidice.pdf: ⁠, Marco Del Giudice (2019-09; genetics  /​ ​​ ​selection):

    The ability of parasites to manipulate host behavior to their advantage has been studied extensively, but the impact of parasite manipulation on the evolution of neural and endocrine mechanisms has remained virtually unexplored. If selection for countermeasures has shaped the evolution of nervous systems, many aspects of neural functioning are likely to remain poorly understood until parasites—the brain’s invisible designers—are included in the picture.

    This article offers the first systematic discussion of brain evolution in light of parasite manipulation. After reviewing the strategies and mechanisms employed by parasites, the paper presents a taxonomy of host countermeasures with four main categories, namely: restrict access to the brain; increase the costs of manipulation; increase the complexity of signals; and increase robustness. For each category, possible examples of countermeasures are explored, and the likely evolutionary responses by parasites are considered.

    The article then discusses the metabolic, computational, and ecological constraints that limit the evolution of countermeasures. The final sections offer suggestions for future research and consider some implications for basic neuroscience and psychopharmacology.

    The paper aims to present a novel perspective on brain evolution, chart a provisional way forward, and stimulate research across the relevant disciplines.

    [Keywords: behavior, brain evolution, hormones, neurobiology, parasite-host interactions, parasite manipulation]


  36. 2020-ebersole.pdf: ⁠, Charles R. Ebersole, Maya B. Mathur, Erica Baranski, Diane-Jo Bart-Plange, Nicholas R. Buttrick, Christopher R. Chartier, Katherine S. Corker, Martin Corley, Joshua K. Hartshorne, Hans IJzerman, Ljiljana B. Lazarević, Hugh Rabagliati, Ivan Ropovik, Balazs Aczel, Lena F. Aeschbach, Luca Andrighetto, Jack D. Arnal, Holly Arrow, Peter Babincak, Bence E. Bakos, Gabriel Baník, Ernest Baskin, Radomir Belopavlović, Michael H. Bernstein, Michał Białek, Nicholas G. Bloxsom, Bojana Bodroža, Diane B. V. Bonfiglio, Leanne Boucher, Florian Brühlmann, Claudia C. Brumbaugh, Erica Casini, Yiling Chen, Carlo Chiorri, William J. Chopik, Oliver Christ, Antonia M. Ciunci, Heather M. Claypool, Sean Coary, Marija V. Čolić, W. Matthew Collins, Paul G. Curran, Chris R. Day, Benjamin Dering, Anna Dreber, John E. Edlund, Filipe Falcão, Anna Fedor, Lily Feinberg, Ian R. Ferguson, Máire Ford, Michael C. Frank, Emily Fryberger, Alexander Garinther, Katarzyna Gawryluk, Kayla Ashbaugh, Mauro Giacomantonio, Steffen R. Giessner, Jon E. Grahe, Rosanna E. Guadagno, Ewa Hałasa, Peter J. B. Hancock, Rias A. Hilliard, Joachim Hüffmeier, Sean Hughes, Katarzyna Idzikowska, Michael Inzlicht, Alan Jern, William Jiménez-Leal, Magnus Johannesson, Jennifer A. Joy-Gaba, Mathias Kauff, Danielle J. Kellier, Grecia Kessinger, Mallory C. Kidwell, Amanda M. Kimbrough, Josiah P. J. King, Vanessa S. Kolb, Sabina Kołodziej, Marton Kovacs, Karolina Krasuska, Sue Kraus, Lacy E. Krueger, Katarzyna Kuchno, Caio Ambrosio Lage, Eleanor V. Langford, Carmel A. Levitan, Tiago Jessé Souza de Lima, Hause Lin, Samuel Lins, Jia E. Loy, Dylan Manfredi, Łukasz Markiewicz, Madhavi Menon, Brett Mercier, Mitchell Metzger, Venus Meyet, Ailsa E. Millen, Jeremy K. Miller, Andres Montealegre, Don A. Moore, Rafał Muda, Gideon Nave, Austin Lee Nichols, Sarah A. Novak, Christian Nunnally, Ana Orlić, Anna Palinkas, Angelo Panno, Kimberly P. Parks, Ivana Pedović, Emilian Pękala, Matthew R. Penner, Sebastiaan Pessers, Boban Petrović, Thomas Pfeiffer, Damian Pieńkosz, Emanuele Preti, Danka Purić, Tiago Ramos, Jonathan Ravid, Timothy S. Razza, Katrin Rentzsch, Juliette Richetin, Sean C. Rife, Anna Dalla Rosa, Kaylis Hase Rudy, Janos Salamon, Blair Saunders, Przemysław Sawicki, Kathleen Schmidt, Kurt Schuepfer, Thomas Schultze, Stefan Schulz-Hardt, Astrid Schütz, Ani N. Shabazian, Rachel L. Shubella, Adam Siegel, Rúben Silva, Barbara Sioma, Lauren Skorb, Luana Elayne Cunha de Souza, Sara Steegen, L. A. R. Stein, R. Weylin Sternglanz, Darko Stojilović, Daniel Storage, Gavin Brent Sullivan, Barnabas Szaszi, Peter Szecsi, Orsolya Szöke, Attila Szuts, Manuela Thomae, Natasha D. Tidwell, Carly Tocco, Ann-Kathrin Torka, Francis Tuerlinckx, Wolf Vanpaemel, Leigh Ann Vaughn, Michelangelo Vianello, Domenico Viganola, Maria Vlachou, Ryan J. Walker, Sophia C. Weissgerber, Aaron L. Wichman, Bradford J. Wiggins, Daniel Wolf, Michael J. Wood, David Zealley, Iris Žeželj, Mark Zrubka, Brian A. Nosek (2020-11-13; statistics  /​ ​​ ​bias):

    Replication studies in psychological science sometimes fail to reproduce findings. If these studies use methods that are unfaithful to the original study or ineffective in eliciting the phenomenon of interest, then a failure to replicate may be a failure of the protocol rather than a challenge to the original finding. Formal pre-data-collection peer review by experts may address shortcomings and increase replicability rates. We selected 10 replication studies from the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015) for which the original authors had expressed concerns about the replication designs before data collection; only one of these studies had yielded a statistically-significant effect (p < 0.05). Commenters suggested that lack of adherence to expert review and low-powered tests were the reasons that most of these RP:P studies failed to replicate the original effects. We revised the replication protocols and received formal peer review prior to conducting new replication studies. We administered the RP:P and revised protocols in multiple laboratories (median number of laboratories per original study = 6.5, range = 3–9; median total sample = 1,279.5, range = 276–3,512) for high-powered tests of each original finding with both protocols. Overall, following the preregistered analysis plan, we found that the revised protocols produced effect sizes similar to those of the RP:P protocols (Δr = 0.002 or 0.014, depending on analytic approach). The median effect size for the revised protocols (r = 0.05) was similar to that of the RP:P protocols (r = 0.04) and the original RP:P replications (r = 0.11), and smaller than that of the original studies (r = 0.37). Analysis of the cumulative evidence across the original studies and the corresponding three replication attempts provided very precise estimates of the 10 tested effects and indicated that their effect sizes (median r = 0.07, range = 0.00–0.15) were 78% smaller, on average, than the original effect sizes (median r = 0.37, range = 0.19–0.50).

    [Keywords: replication, reproducibility, metascience, peer review, ⁠, open data, preregistered]

  37. ⁠, Andrew Gelman, Aki Vehtari, Daniel Simpson, Charles C. Margossian, Bob Carpenter, Yuling Yao, Lauren Kennedy, Jonah Gabry, Paul-Christian Bürkner, Martin Modrák (2020-11-02):

    The to data analysis provides a powerful way to handle uncertainty in all observations, model parameters, and model structure using probability theory. Probabilistic programming languages make it easier to specify and fit Bayesian models, but this still leaves us with many options regarding constructing, evaluating, and using these models, along with many remaining challenges in computation. Using Bayesian inference to solve real-world problems requires not only statistical skills, subject matter knowledge, and programming, but also awareness of the decisions made in the process of data analysis. All of these aspects can be understood as part of a tangled workflow of applied Bayesian statistics. Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model comparison. We review all these aspects of workflow in the context of several examples, keeping in mind that in practice we will be fitting many models for any given problem, even if only a subset of them will ultimately be relevant for our conclusions.

    • Introduction

      • From Bayesian inference to Bayesian workflow
      • Why do we need a Bayesian workflow?
      • “Workflow” and its relation to statistical theory and practice
      • Organizing the many aspects of Bayesian workflow
      • Aim and structure of this article
    • Before fitting a model

      • Choosing an initial model
      • Modular construction
      • Scaling and transforming the parameters
      • Prior predictive checking
      • Generative and partially generative models
    • Fitting a model

      • Initial values, adaptation, and warmup
      • How long to run an iterative algorithm
      • Approximate algorithms and approximate models
      • Fit fast, fail fast
    • Using constructed data to find and understand problems

      • Fake-data simulation
      • Simulation-based calibration
      • Experimentation using constructed data
    • Addressing computational problems

      • The folk theorem of statistical computing
      • Starting at simple and complex models and meeting in the middle
      • Getting a handle on models that take a long time to fit
      • Monitoring intermediate quantities
      • Stacking to reweight poorly mixing chains
      • Posterior distributions with multimodality and other difficult geometry
      • Reparameterization
      • Marginalization
      • Adding prior information
      • Adding data
    • Evaluating and using a fitted model

      • Posterior predictive checking
      • Cross validation and influence of individual data points and subsets of the data
      • Influence of prior information
      • Summarizing inference and propagating uncertainty
    • Modifying a model

      • Constructing a model for the data
      • Incorporating additional data
      • Working with prior distributions
      • A topology of models
    • Understanding and comparing multiple models

      • Visualizing models in relation to each other
      • Cross validation and model averaging
      • Comparing a large number of models
    • Modeling as software development

      • Version control smooths collaborations with others and with your past self
      • Testing as you go
      • Making it essentially reproducible
      • Making it readable and maintainable
    • Example of workflow involving model building and expansion: Golf putting

      • First model: logistic regression
      • Modeling from first principles
      • Testing the fitted model on new data
      • A new model accounting for how hard the ball is hit
      • Expanding the model by including a fudge factor
      • General lessons from the golf example
    • Example of workflow for a model with unexpected multimodality: Planetary motion

      • Mechanistic model of motion
      • Fitting a simplified model
      • Bad Markov chain, slow Markov chain?
      • Building up the model
      • General lessons from the planetary motion example
    • Discussion

      • Different perspectives on statistical modeling and prediction
      • Justification of iterative model building
      • Model selection and overfitting
      • Bigger datasets demand bigger models
      • Prediction, generalization, and poststratification
      • Going forward
  38. ⁠, Sean W. Cain, Elise M. McGlashan, Parisa Vidafar, Jona Mustafovska, Simon P. N. Curran, Xirun Wang, Anas Mohamed, Vineetha Kalavally, Andrew J. K. Phillips (2020-11-05):

    The regular rise and fall of the sun resulted in the development of 24-h rhythms in virtually all organisms. In an evolutionary heartbeat, humans have taken control of their light environment with electric light. Humans are highly sensitive to light, yet most people now use light until bedtime. We evaluated the impact of modern home lighting environments in relation to sleep and individual-level light sensitivity using a new wearable spectrophotometer. We found that nearly half of homes had bright enough light to suppress melatonin by 50%, but with a wide range of individual responses (0–87% suppression for the average home). Greater evening light relative to an individual’s average was associated with increased wakefulness after bedtime. Homes with energy-efficient lights had nearly double the melanopic illuminance of homes with incandescent lighting. These findings demonstrate that home lighting substantially affects sleep and the circadian system, but the impact of lighting for a specific individual in their home is highly unpredictable.

  39. 2010-seamon.pdf: ⁠, John G. Seamon, Paawan J. Punjabi, Emily A. Busch (2020-04-23; spaced-repetition):

    At age 58, JB [John Basinger] began memorizing Milton’s epic poem ⁠. 9 years and thousands of study hours later, he completed this process in 2001 and recalled from memory all 12 books of this 10,565-line poem over a 3-day period. Now 74, JB continues to recite this work. We tested his memory accuracy by cueing his recall with two lines from the beginning or middle of each book and asking JB to recall the next 10 lines. JB is an exceptional memoriser of Milton, both in our laboratory tests in which he did not know the specific tests or procedures in advance, and in our analysis of a videotaped, prepared performance. Consistent with deliberate practice theory, JB achieved this remarkable ability by deeply analysing the poem’s structure and meaning over lengthy repetitions. Our findings suggest that exceptional memorizers such as JB are made, not born, and that cognitive expertise can be demonstrated even in later adulthood.

    [Keywords: Exceptional memory, Prose memory, Age and memory]

  40. 2016-09-19-neal-surveyofalternativedisplays.html: ⁠, Blair Neal (2016-09-19; technology):

    The purpose of this article is to collect and consolidate a list of these alternative methods of working with displays, light and optics. This will by no means be an exhaustive list of the possibilities available—depending on how you categorize, there could be dozens or hundreds of ways. There are historical mainstays, oddball one-offs, expensive failures and techniques that are only beginning to come into their own.

    [Survey of visual display technologies going beyond the familiar CRT or LED display. See also ⁠. Contents:

    • Notes on Standard Displays
    • Brief Note on Holograms
    • Pepper’s Ghost
    • Projection on Static Transparent Materials/​​​​​​Scrims
    • Projection on Water or Fog
    • Volumetric Projection
    • Diffusion and Distortion Techniques
    • Transparent LCD/​​​​​​OLED
    • LCDs with modified polarization layers
    • Volumetric Displays (Mechanical/​​​​​​Persistence of Vision)
    • Volumetric Displays (Layered screens)
    • Electronic Paper
    • Flexible Displays
    • Laser Projectors
    • Head Mounted Displays (VR/​​​​​​AR/​​​​​​Mixed Reality)
    • Plasma Combustion
    • Physical/​​​​​​Mechanical Displays
    • Appendix and Other References]
  41. 2018-10-09-heck-structuraltypography.html: ⁠, Bethany Heck (2018-10-09; design):

    Words matter (or so I’m told). Some of my favorite typographic pieces are the ones that use typography not only to deliver a message but to serve as the compositional foundation that a design centers around. Letterforms are just as valuable as graphic elements as they are representations of language, and asking type to serve multiples roles in a composition is a reliable way to elevate the quality of your work…I’ve pulled out a few of my favorite designs that use type in this way and grouped them into shared themes so we can analyze the range of techniques different designers have used to let typography guide their work. Let’s dive in!…

    • Type Informing Grid: Using one typographic element to influence other pieces of the design
    • Type as Representation: Rendering type as a manifestation of an object or ideal
    • Reinforcing Imagery: Type can extend the impact of imagery in a design
    • Large Type Does Not Mean Structural Type: Big type can be lazy type (Lastly, I wanted to show a few examples that aren’t good examples of type as structure…)

    …There’s something freeing about starting a design with a commitment to only using type and words to communicate effectively. I hope this essay demystifies some of the thought processes that can go into improving how you handle type in a variety of situations and leaves you with a different perspective on the pieces discussed, as well as a new toolkit of process-starters for your design work going forward.

  42. ⁠, Barton P. Miller, Mengxiao Zhang, Elisa R. Heymann (2020-08-14):

    As fuzz testing has passed its 30th anniversary, and in the face of the incredible progress in fuzz testing techniques and tools, the question arises if the classic, basic fuzz technique is still useful and applicable? In that tradition, we have updated the basic fuzz tools and testing scripts and applied them to a large collection of utilities on Linux, ⁠, and MacOS. As before, our failure criteria was whether the program crashed or hung. We found that 9 crash or hang out of 74 utilities on Linux, 15 out of 78 utilities on FreeBSD, and 12 out of 76 utilities on MacOS. A total of 24 different utilities failed across the three platforms. We note that these failure rates are somewhat higher than our in previous 1995, 2000, and 2006 studies of the reliability of command line utilities. In the basic fuzz tradition, we debugged each failed utility and categorized the causes the failures. Classic categories of failures, such as pointer and array errors and not checking return codes, were still broadly present in the current results. In addition, we found a couple of new categories of failures appearing. We present examples of these failures to illustrate the programming practices that allowed them to happen. As a side note, we tested the limited number of utilities available in a modern programming language (Rust) and found them to be of no better reliability than the standard ones.

  43. Cat-Sense#fuzz-testing

  44. 1990-miller.pdf: ⁠, Barton P. Miller, Louis Fredriksen, Bryan So (1990; cs):

    Operating system facilities, such as the kernel and utility programs, are typically assumed to be reliable. In our recent experiments, we have been able to crash 25–33% of the utility programs on any version of UNIX that was tested. This report describes these tests and the program bugs that caused the crashes…The following section describes the tools we built to test the utilities. These tools include the fuzz (random character) generator, ptyjig (to test interactive utilities), and scripts to automate the testing process. Next, we will describe the tests we performed, giving the types of input we presented to the utilities. Results from the tests will follow along with an analysis of the results, including identification and classification of the program bugs that caused the crashes. The final section presents concluding remarks, including suggestions for avoiding the types of problems detected by our study and some commentary on the bugs we found. We include an Appendix with the user manual pages for fuzz and ptyjig.

  45. ⁠, Rachel Laudan (2020-10-19):

    …By starting with flour, pandemic cooks dodged all the preliminary stages of turning grains into flour. Even the few hardy souls equipped with metal hand grinders or tabletop electric mills started with cleaned, threshed, and winnowed grain. Forgotten were the thousands of years when grain was laboriously pounded and ground into something edible, usually by women. Although in most societies those labor costs have been effectively eliminated by successive spurts of technological innovation, in far too many others women are still condemned to the daily grind.

    …The many virtues of the grains came with the accompanying costs of processing. That processing food post-harvest or slaughter was laborious was nothing new: the hunter gatherer way of life had never been one of leisure. What was new was the kind of cost of removing the layers of scratchy husks and tough hulls that make grains impossible to chew and to digest. This requires one, or more often a series of different kinds of violent mechanical processing depending on the particular features of each grain variety: repeated threshing with a heavy object to get rid of the outer layers; pounding by standing to lift a long pestle above the head and allowing it to fall into a mortar; and or kneeling to grind dry or wet on a stone. For hard grains such as wheat and barley, grindstones were essential. The people of Lake Kinneret placed their seeds on a flat stone, then thrust a second stone across them to reduce them to flour. While this lateral grindstone (or saddle quern or metate) has been abandoned in Europe and the Middle East, variants of it are still used elsewhere particularly where grains are soaked or boiled before grinding…My first reaction when I tried imitating Margarita was this is easy. While my movements were nothing like as practiced the stones worked efficiently and soon I accumulated a tiny heap of wet paste. Quickly, however, I began to feel queasy and light headed. Five minutes left me exhausted and breathless. Margarita allowed herself a little smirk when she saw that I could not possibly produce the 1 to 2 lbs an hour that she could turn out.

    Quite how long it takes a woman to grind for a family, apart from the time husking and shucking the maize, collecting the cooking water, and shaping and cooking the tortillas, depends on her skill and strength, the age and number of family members, the type of masa, and the quality of the metate. My estimate is that it takes about five hours a day to make enough masa for a family of five. This may seem incredible but it is in line with other estimates for contemporary Mexico and Guatemala collected by Michael Searcy⁠, with Arnold Bauer’s estimate for Mexico⁠, and experimental estimates for Europe collected in David Peacock’s in The Stone of Life (2013), pg127. Since five hours is about as much as anyone can grind, the labor of one in five adults has to be devoted to making the staple bread.

    …Why then were young women like Margarita still grinding at the end of the twentieth century? Why are women in India still grinding flour and women in Africa still pounding maize? Why did what seems like a clear case of technological progress, of dramatic improvement in labor productivity fail to take hold? Culture is often invoked. Grinding and pounding was women’s work. In Mexico, husbands grumbled that tortillas made with mill-ground masa, let alone masa harina, did not taste as good. They did not want their wives gossiping at the mill, nor paying the miller’s fees. The very identity of women, many insisted, lay in their provision of the family tortillas…Until affordable and locally-appropriate improvements in grinding technology were introduced, women had no option but unchosen, mind-numbing, physically exhausting labor. And the locking up of so much of the talent and energy of these women in pounding and grinding grains surely impeded the betterment of society as a whole.

  46. ⁠, Gideon Nave, Jason Rentfrow, Sudeep Bhatia (2020-11-26):

    The proliferation of media streaming services has increased the volume of personalized video consumption, allowing marketers to reach massive audiences and deliver a range of customized content at scale. However, relatively little is known about how consumers’ psychological makeup is manifested in their media preferences. The present paper addresses this gap in a preregistered study of the relationship between movie plots, quantified via user-generated keywords, and the aggregate personality profiles of those who “like” them on social media. We find that movie plots can be used to accurately predict aggregate fans’ personalities, above and beyond the demographic characteristics of fans, and general film characteristics such as quality, popularity, and genre. Further analysis reveals various associations between the movies’ psychological themes and their fans’ personalities, indicating congruence between the two. For example, films with keywords related to anxiety are liked more among people who are high in Neuroticism and low in ⁠. In contrast, angry and violent movies are liked more by people who are low in Agreeableness. Our findings provide a fine-grained mapping between personality dimensions and preferences for media content, and demonstrate how these links can be leveraged for assessing audience psychographics at scale.

    Supplementary Figure 1: Correlations between dimensions of the aggregate fan personality (AFPP), aggregate fan demographic profile (AFDP) and Metadata variables, across Movies
    Extremes of movie correlations: movies with the highest/​​​​lowest correlation to Openness to Experience
    Extremes: Extraversion
    Extremes: Agreeableness
  47. Movies#rheingold-siegfried-gotterdammerung

  48. Movies#singin-in-the-rain