newsletter/2020/10 (Link Bibliography)

“newsletter/​2020/​10” links:

  1. 10

  2. https://gwern.substack.com

  3. 09

  4. newsletter

  5. Changelog

  6. https://www.patreon.com/gwern

  7. Design

  8. ⁠, Edward A. Kmett (2012):

    Configurable Knuth-Liang : Uses the UTF8 encoded hyphenation patterns provided by hyph-utf8⁠.

    Usage:

    hyphenate english_US "supercalifragilisticexpialidocious"-- ["su","per","cal","ifrag","ilis","tic","ex","pi","al","ado","cious"] 
    hyphenate english_US "hyphenation"-- ["hy","phen","ation"]
  9. Books

  10. Movies

  11. Anime

  12. ⁠, Jonathan Gray, Adam Lerer, Anton Bakhtin, Noam Brown (2020-10-06):

    Prior AI breakthroughs in complex games have focused on either the purely adversarial or purely cooperative settings. In contrast, Diplomacy is a game of shifting alliances that involves both cooperation and competition. For this reason, Diplomacy has proven to be a formidable research challenge. In this paper we describe an agent for the no-press variant of Diplomacy that combines supervised learning on human data with one-step lookahead search via regret minimization. Regret minimization techniques have been behind previous AI successes in adversarial games, most notably poker, but have not previously been shown to be successful in large-scale games involving cooperation. We show that our agent greatly exceeds the performance of past no-press Diplomacy bots, is unexploitable by expert humans, and ranks in the top 2% of human players when playing anonymous games on a popular Diplomacy website.

  13. ⁠, Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba (2020-10-05):

    Intelligent agents need to generalize from past experience to achieve goals in complex environments. World models facilitate such generalization and allow learning behaviors from imagined outcomes to increase sample-efficiency. While learning world models from image inputs has recently become feasible for some tasks, modeling Atari games accurately enough to derive successful behaviors has remained an open challenge for many years. We introduce DreamerV2, a agent that learns behaviors purely from predictions in the compact space of a powerful world model. The world model uses discrete representations and is trained separately from the policy. DreamerV2 constitutes the first agent that achieves human-level performance on the Atari benchmark of 55 tasks by learning behaviors inside a separately trained world model. With the same computational budget and wall-clock time, Dreamer V2 reaches 200M frames and surpasses the final performance of the top single-GPU agents IQN and ⁠. DreamerV2 is also applicable to tasks with continuous actions, where it learns an accurate world model of a complex humanoid robot and solves stand-up and walking from only pixel inputs.

  14. ⁠, Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, Paolo Tonella (2019-11-07):

    The growing application of deep neural networks in safety-critical domains makes the analysis of faults that occur in such systems of enormous importance. In this paper we introduce a large taxonomy of faults in deep learning (DL) systems. We have manually analysed 1059 artefacts gathered from commits and issues of projects that use the most popular DL frameworks (TensorFlow, Keras and PyTorch) and from related Stack Overflow posts. Structured interviews with 20 researchers and practitioners describing the problems they have encountered in their experience have enriched our taxonomy with a variety of additional faults that did not emerge from the other two sources. Our final taxonomy was validated with a survey involving an additional set of 21 developers, confirming that almost all fault categories (13⁄15) were experienced by at least 50% of the survey participants.

    Figure 1: Taxonomy of Real Faults in Deep Learning Systems
    • Model:

      • Layers:

        • Activation Function: wrong type; missing softmax; missing RELU
        • Layer Properties: wrong input sample size; wrong defined input shape; wrong defined output shape; both wrong; wrong filter size in convolution; missing bias; wrong number of neurons in layer; wrong amount & type of pooling in convolutional layer; layer dimension mismatch
      • Model Type & Properties:

        • wrong model initialization
        • wrong weight initialization
        • multiple initializations of CNN
        • wrong selection of model
        • wrong network architecture
        • suboptimal network structure
    • GPU Usage:

      • missing destination device
      • incorrect state sharing
      • wrong reference to GPU device
      • wrong data parallelism on GPUs
      • calling unsupported operations on CUDA tensors
      • conversion to CUDA tensor inside the training/​​​​​​​test loop
      • wrongly implemented data transfer function (CPU-GPU)
      • missing transfer of data to GPU
      • wrong tensor transfer to GPU
      • GPU tensor is used instead of CPU tensor
    • API:

      • deprecated API
      • wrong use of image decoding API
      • wrong position of data shuffle operation
      • missing global variables initialization
      • wrong API usage
      • missing API call
      • wrong reference to operational graph
      • wrong usage of placeholder restoration API
      • missing argument scoping
    • Training:

      • Training Data Quality:

        • wrong labels for training data
        • wrong selection of features
        • unbalanced training data
        • not enough training data
        • low quality training data
        • overlapping output classes in training data
        • too many output categories
        • small range of values for a feature
        • discarding important features
      • Training Process: wrong management of memory resources; reference for non-existing checkpoint; model too big to fit into available memory; missing data augmentation; redundant data augmentation

      • Optimizer: wrong optimization function; epsilon for Adam optimizer too low

      • Loss Function: wrong loss function calculation; missing masking of invalid values to zero; wrong selection of ⁠; missing loss function

      • Validation/​​​​​​​Testing: missing validation set; wrong performance metric; incorrect train/​​​​​​​test data split

      • Hyperparameters:

        • suboptimal hyperparameter tuning
        • suboptimal learning rate
        • data batching required
        • suboptimal number of epochs
        • suboptimal batch size
        • wrongly implemented data batching
        • missing regularization (loss & weight)
      • Preprocessing of Training data:

        • Missing Preprocessing: missing preprocessing step (subsampling, normalization, input scaling, resize of the images, oversampling, encoding of categorical data, padding…data shuffling, interpolation)
        • Wrong Preprocessing: wrong preprocessing step (pixel encoding, padding, text segmentation, normalization…positional encoding, character encoding)
  15. ⁠, Steven Kapturowski, Georg Ostrovski, John Quan, Remi Munos, Will Dabney (2018-09-27; reinforcement-learning):

    Building on the recent successes of distributed training of RL agents, in this paper we investigate the training of -based RL agents from distributed prioritized experience replay. We study the effects of parameter lag resulting in representational drift and recurrent state staleness and empirically derive an improved training strategy. Using a single network architecture and fixed set of hyper-parameters, the resulting agent, Recurrent Replay Distributed (R2D2), quadruples the previous state of the art on Atari-57, and matches the state of the art on ⁠. It is the first agent to exceed human-level performance in 52 of the 57 Atari games.

    [Keywords: RNN, ⁠, experience replay, distributed training, reinforcement learning]

    TL;DR: Investigation on combining recurrent neural networks and experience replay leading to state-of-the-art agent on both Atari-57 and DMLab-30 using single set of hyper-parameters.

  16. ⁠, Nils Köbis, Luca Mossink (2020-09-08):

    The release of openly available, robust natural language generation algorithms (NLG) has spurred much public attention and debate. One reason lies in the algorithms’ purported ability to generate human-like text across various domains. Empirical evidence using incentivized tasks to assess whether people (a) can distinguish and (b) prefer algorithm-generated versus human-written text is lacking. We conducted two experiments assessing behavioral reactions to the state-of-the-art Natural Language Generation algorithm (Ntotal = 830). Using the identical starting lines of human poems, GPT-2 produced samples of poems. From these samples, either a random poem was chosen (Human-out-of-the-loop) or the best one was selected (Human-in-the-loop) and in turn matched with a human-written poem. In a new incentivized version of the Turing Test, participants failed to reliably detect the algorithmically-generated poems in the Human-in-the-loop treatment, yet succeeded in the Human-out-of-the-loop treatment. Further, people reveal a slight aversion to algorithm-generated poetry, independent on whether participants were informed about the algorithmic origin of the poem (Transparency) or not (Opacity). We discuss what these results convey about the performance of NLG algorithms to produce human-like text and propose methodologies to study such learning algorithms in human-agent experimental settings.

  17. ⁠, Daniel Smilkov, Shan Carter (2016-11-05):

    [(Github) An in-browser implementation using TypeScript of a simple feedforward MLP, whose architecture, LR, activation, regularization, and task can be varied and the NN retrained with the intermediate function of each neuron visualized and the decision boundary on the data plotted. One can see how different hyperparameters lead to different learned units and boundaries of varying smoothness and shapes, and how updates it each iteration.

    Available settings:

    • Show test data
    • Discretize output
    • Play button
    • Step button
    • Reset button
    • Learning rate
    • Activation
    • Regularization
    • Regularization rate
    • Problem type
    • Which dataset
    • Ratio train data
    • Noise level
    • Batch size
    • number of hidden layers]
  18. ⁠, Gwern Branwen (2020-10-30):

    Subreddit for discussing AI, machine learning, or deep learning approaches involving big numbers: billions of parameters, millions of n, petaflops, etc. eg ⁠. Most research is conducted at much smaller scale; this subreddit is for research analogous to ‘high energy physics’, requiring specialized approaches, large investments, consortium, etc.

    Topics: How? Who? Why do they work? What are they good for? What resources are available? Who will pay & how? What is the future of such approaches? What global consequences will there be?

  19. ⁠, Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, Sam McCandlish (2020-10-28):

    We identify empirical scaling laws for the loss in four domains: generative image modeling, video modeling, multimodal image ↔︎ text models, and mathematical problem solving. In all cases autoregressive smoothly improve in performance as model size and compute budgets increase, following a plus constant scaling law. The optimal model size also depends on the compute budget through a power-law, with exponents that are nearly universal across all data domains.

    The cross-entropy loss has an information theoretic interpretation as S(True)+DKL(True||Model), and the empirical scaling laws suggest a prediction for both the true data distribution’s entropy and the KL divergence between the true and model distributions. With this interpretation, billion-parameter Transformers are nearly perfect models of the YFCC100M image distribution downsampled to an 8×8 resolution, and we can forecast the model size needed to achieve any given reducible loss (ie DKL) in nats/​​​​image for other resolutions.

    We find a number of additional scaling laws in specific domains: (a) we identify a scaling relation for the mutual information between captions and images in multimodal models, and show how to answer the question “Is a picture worth a thousand words?”; (b) in the case of mathematical problem solving, we identify scaling laws for model performance when extrapolating beyond the training distribution; (c) we finetune generative image models for classification and find smooth scaling of the classification loss and error rate, even as the generative loss levels off. Taken together, these results strengthen the case that scaling laws have important implications for neural network performance, including on downstream tasks.

    …As we increase model and dataset sizes, optimization becomes increasingly efficient, until eventually learning curves begin to merge with the L(D) trend, so that there are no benefits to be gained from training for more than a single epoch [].

    …We have argued that a single neural architecture, the Transformer, can be applied to the generative modeling of images, videos, multimodal data, and math, along with language [⁠, ]. We identified common scaling laws for the loss achieved on all data modalities as a function of both model size and compute budget. As in the case of language, these results imply that larger models become more sample-efficient. Furthermore, we found that in some important cases, fine-tuned performance on downstream tasks also follows similar scaling laws. This suggests that trends in the generative modeling loss translate into advantages in practical capabilities.

    A greater surprise was the approximately universal trend (figure 2) for optimal model size as a function of the training compute budget—we did not anticipate that the exponent NoptC0.7 would be largely independent of the data distribution. This trend implies a dual trend for the number of tokens elapsed during optimized training, as a function of C or N, and leads to the conclusion that larger compute budgets should be “spent” mostly on larger models, rather than much longer training runs. So this lesson from language modeling [Kaplan et al 2020] generalizes. These empirical regularities beg for theoretical explanation—why do these scaling relations hold? The scaling laws also suggest a shift in perspective away from the particularities of neural architectures, loss functions, and training algorithms and towards the broader commonalities that appear when machine learning is studied across a large hierarchy of model, data, and compute scales. Work in ML often involves identifying specific deficiencies in current capabilities and remedying them through the alteration of models and algorithms. Perhaps many capabilities simply lie on a spectrum that can be continuously unlocked through increasing scale, as might be suggested by the meta-learning capabilities of the GPT-3 model [Brown et al 2020].

    Figure 1: Smooth scaling of reducible loss across domains—We show power-law scaling laws for the reducible loss L−L∞ as a function of compute, where the irreducible loss L∞ is a fitted domain-dependent constant. Under plausible assumptions concerning the infinite data and compute limits, the irreducible loss estimates the entropy of the underlying data distribution, while the reducible loss approximates the KL divergence between the data and model distributions. In the case of language we use results from [BMR+20], and only show the full loss L.
    Table 1: Summary of scaling laws—In this table we summarize the model size and compute scaling fits to equation (1.1) along with Nopt(C), with the loss in nats/​​​​token, and compute measured in petaflop-days. In most cases the irreducible losses match quite well between model size and compute scaling laws. The math compute scaling law may be affected by the use of weight decay, which typically hurts performance early in training and improves performance late in training. The compute scaling results and data for language are from [BMR+20], while_N_opt(C)comes from [KMH+20]. Unfortunately, even with data from the largest language models we cannot yet obtain a meaningful estimate for the entropy of natural language.
    Figure 2: Optimal model size is consistent across domains—We display the optimal model size Nopt as a function of the training compute budget C. Not only does Nopt(C) behave as a power-law, but the behavior is remarkably similar for all data modalities.
    Figure 31: Q&A—We show the progression of simple Q&A capabilities of GPT-3 family models as we increase the parameter count [BMR+20]. We ask the model who the first and second president of the United States was. · Tiny models appear to have trouble understanding the question, and don’t place any substantial probability on the correct answer. Larger models understand that we’re requesting a US president, but fail to understand that the “second president” and “first president” are different requests, placing most of their weight for both questions on “George Washington”. Only larger models understand both aspects of the questions, answering both correctly.

    [See also: Figure 3 & Figure 11⁠.]

  20. https://www.youtube.com/QMqPAM_knrE?t=2380

  21. ⁠, Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby (2020-09-28):

    One-sentence Summary: Transformers applied directly to image patches and pre-trained on large datasets work really well on image classification.

    While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer can perform very well on image classification tasks when applied directly to sequences of image patches. When pre-trained on large amounts of data [JFT-300M] and transferred to multiple recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc), Vision Transformer attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train…Our Vision Transformer, pre-trained on the JFT-300M dataset, approaches or beats state of the art on multiple image recognition benchmarks, reaching accuracy of 88.36% on ImageNet, 90.77% on ImageNet-ReaL, 94.55% on CIFAR-100, and 77.16% on the VTAB suite of 19 tasks…Interestingly, our models took substantially less compute to pre-train than state of the art, however, we note that pre-training efficiency may be affected not only by the architecture choice, but also other parameters, such as training schedule, optimizer, ⁠, etc. We provide a controlled study of performance vs. compute for different architectures in Section 4.4…Finally, [we plan] to further scale ViT, given that the performance does not seem yet to be saturating with the increased model size.

    [Keywords: computer vision, image recognition, self-attention, transformer, large-scale training]

    [Blog⁠. See also ⁠.]

  22. https://github.com/lucidrains/vit-pytorch

  23. ⁠, Vladimir Mikulik, Grégoire Delétang, Tom McGrath, Tim Genewein, Miljan Martic, Shane Legg, Pedro A. Ortega (2020-10-21):

    Memory-based meta-learning is a powerful technique to build agents that adapt fast to any task within a target distribution. A previous theoretical study has argued that this remarkable performance is because the meta-training protocol incentivises agents to behave Bayes-optimally. We empirically investigate this claim on a number of prediction and bandit tasks. Inspired by ideas from theoretical computer science, we show that meta-learned and Bayes-optimal agents not only behave alike, but they even share a similar computational structure, in the sense that one agent system can approximately simulate the other. Furthermore, we show that Bayes-optimal agents are fixed points of the meta-learning dynamics. Our results suggest that memory-based meta-learning might serve as a general technique for numerically approximating Bayes-optimal agents—that is, even for task distributions for which we currently don’t possess tractable models.

  24. ⁠, Aran Komatsuzaki (2019-06-19):

    In unsupervised learning, collecting more data is not always a costly process unlike the training. For example, it is not hard to enlarge the 40GB WebText used for training by modifying its sampling methodology considering how many webpages there are in the Internet. On the other hand, given that training on this dataset already costs tens of thousands of dollars, training on a larger dataset naively is not cost-wise feasible. In this paper, we suggest to train on a larger dataset for only one epoch unlike the current practice, in which the unsupervised models are trained for from tens to hundreds of epochs. Furthermore, we suggest to adjust the model size and the number of iterations to be performed appropriately. We show that the performance of Transformer language model becomes dramatically improved in this way, especially if the original number of epochs is greater. For example, by replacing the training for 10 epochs with the one epoch training, this translates to 1.9–3.3× speedup in wall-clock time in our settings and more if the original number of epochs is greater. Under one epoch training, no overfitting occurs, and regularization method does nothing but slows down the training. Also, the curve of test loss over iterations follows power-law extensively. We compare the wall-clock time of the training of models with different parameter budget under one epoch training, and we show that size/​​​​iteration adjustment based on our proposed heuristics leads to 1–2.7× speedup in our cases. With the two methods combined, we achieve 3.3–5.1× speedup. Finally, we speculate various implications of one epoch training and size/​​​​iteration adjustment. In particular, based on our analysis we believe that we can reduce the cost to train the state-of-the-art models as and GPT-2 dramatically, maybe even by the factor of 10.

  25. Backstop#deep-bayes

  26. ⁠, Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel (2020-10-22):

    The recent “Text-to-Text Transfer Transformer” () leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 that was pre-trained on a new -based dataset covering 101 languages. We detail the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. We also describe a simple technique to prevent “accidental translation” in the zero-shot setting, where a generative model chooses to (partially) translate its prediction into the wrong language. All of the code and model checkpoints used in this work are publicly available.

  27. ⁠, Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, Armand Joulin (2020-10-21):

    Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages. However, much of this work is English-centric by training only on data which was translated from or to English. While this is supported by large sources of training data, it does not reflect translation needs worldwide. In this work, we create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Then, we explore how to effectively increase model capacity through a combination of dense scaling and language-specific sparse parameters to create high quality models. Our focus on non-English-centric models brings gains of more than 10 when directly translating between non-English directions while performing competitively to the best single systems of WMT. We open-source our scripts so that others may reproduce the data, evaluation, and final M2M-100 model.

  28. {#linkBibliography-(fb)-2020 .docMetadata}, Angela Fan (FB) (2020-10-19):

    • Facebook AI is introducing, the first multilingual machine translation (MMT) model that translates between any pair of 100 languages without relying on English data. It’s open sourced here⁠.
    • When translating, say, Chinese to French, previous best multilingual models train on Chinese to English and English to French, because English training data is the most widely available. Our model directly trains on Chinese to French data to better preserve meaning. It outperforms English-centric systems by 10 points on the widely used BLEU metric for evaluating machine translations.
    • M2M-100 is trained on a total of 2,200 language directions—or 10× more than previous best, English-centric multilingual models. Deploying M2M-100 will improve the quality of translations for billions of people, especially those who speak low-resource languages.
    • This milestone is a culmination of years of Facebook AI’s foundational work in machine translation. Today, we’re sharing details on how we built a more diverse MMT training data set and model for 100 languages. We’re also releasing the model, training, and evaluation setup to help other researchers reproduce and further advance multilingual models.

    …In a culmination of many years of MT research at Facebook, we’re excited to announce a major milestone: the first single massive MMT model that can directly translate 100×100 languages in any direction without relying on only English-centric data. Our single multilingual model performs as well as traditional bilingual models and achieved a 10 BLEU point improvement over English-centric multilingual models. Using novel mining strategies to create translation data, we built the first truly “many-to-many” data set with 7.5 billion sentences for 100 languages. We used several scaling techniques to build a universal model with 15 billion parameters, which captures information from related languages and reflects a more diverse script of languages and morphology.

    …It’s a lot easier to find translations for Chinese to English and English to French, than, say, French to Chinese. What’s more, the volume of data required for training grows quadratically with the number of languages that we support. For instance, if we need 10M sentence pairs for each direction, then we need to mine 1B sentence pairs for 10 languages and 100B sentence pairs for 100 languages.

    We took on this ambitious challenge of building the most diverse many-to-many MMT data set to date: 7.5 billion sentence pairs across 100 languages. This was possible by combining complementary data mining resources that have been years in the making, including ⁠, ⁠, and LASER⁠. As part of this effort, we created a new LASER 2.0 and improved fastText language identification, which improves the quality of mining and includes open sourced training and evaluation scripts. All of our data mining resources leverage publicly available data and are open sourced.

  29. ⁠, Yu Zhang, James Qin, Daniel S. Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V. Le, Yonghui Wu (2020-10-20):

    We employ a combination of recent developments in semi-supervised learning for automatic speech recognition to obtain state-of-the-art results on LibriSpeech utilizing the unlabeled audio of the dataset. More precisely, we carry out noisy student training with SpecAugment using giant Conformer models pre-trained using pre-training. By doing so, we are able to achieve word-error-rates (WERs) of 1.4%/​​​​2.6% on the LibriSpeech test/​​​​test-other sets against the current state-of-the-art WERs 1.7%/​​​​3.3%.

  30. Attention

  31. ⁠, Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, Ruoming Pang (2020-05-16):

    Recently Transformer and Convolution neural network () based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer substantially outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1%/​​​​4.3% without using a language model and 1.9%/​​​​3.9% with an external language model on test/​​​​test~other~. We also observe competitive performance of 2.7%/​​​​6.3% with a small model of only 10M parameters.

  32. ⁠, Richard Karlsson Linnér, Travis T. Mallard, Peter B. Barr, Sandra Sanchez-Roige, James W. Madole, Morgan N. Driver, Holly E. Poore, Andrew D. Grotzinger, Jorim J. Tielbeek, Emma C. Johnson, Mengzhen Liu, Hang Zhou, Rachel L. Kember, Joëlle A. Pasman, Karin J. H. Verweij, Dajiang J. Liu, Scott Vrieze, COGA Collaborators, Henry R. Kranzler, Joel Gelernter, Kathleen Mullan Harris, Elliot M. Tucker-Drob, Irwin Waldman, Abraham A. Palmer, K. Paige Harden, Philipp D. Koellinger, Danielle M. Dick (2020-10-16):

    Behaviors and disorders related to self-regulation, such as substance use, antisocial conduct, and ⁠, are collectively referred to as externalizing and have a shared genetic liability. We applied a multivariate approach that leverages genetic correlations among externalizing traits for genome-wide association analyses. By pooling data from ~1.5 million people, our approach is statistically more powerful than single-trait analyses and identifies more than 500 genetic loci. The identified loci were enriched for genes expressed in the brain and related to nervous system development. A constructed from our results captures variation in a broad range of behavioral and medical outcomes that were not part of our genome-wide analyses, including traits that until now lacked well-performing polygenic scores, such as use disorder, suicide, HIV infections, criminal convictions, and unemployment. Our findings are consistent with the idea that persistent difficulties in self-regulation can be conceptualized as a neurodevelopmental condition.

  33. ⁠, Margot Van de Weijer, Lianne de Vries, Meike Bartels (2020-10-07):

    In light of major global trends (eg., rise of ageing populations, increasing longevity, decreasing birth rates), maintaining, facilitating, and building well-being (WB) is crucial, but also becomes increasingly complex and demanding. Over the past decade, twin studies have helped us get better insight into the extent to which genes and environments contribute to individual differences in well-being. Our knowledge about these genetic and environmental factors is continuously growing with studies on well-being related phenotypes, extensions of twin studies, molecular genetic studies, and environmental studies. In this chapter, we provide an overview of past, present, and future directions of behavioural genetic research on well-being, happiness, and related phenotypes.

    [Keywords: Well-being; happiness; twin studies; genetics; behaviour genetics; positive psychology]

  34. ⁠, Allison Meisner, Prosenjit Kundu, Yan Dora Zhang, Lauren V. Lan, Sungwon Kim, Disha Ghandwani, Parichoy Pal Choudhury, Sonja I. Berndt, Neal D. Freedman, Montserrat Garcia-Closas, Nilanjan Chatterjee (2020-06-16):

    While have identified susceptibility variants for numerous traits, their combined utility for predicting broad measures of health, such as mortality, remains poorly understood. We used data from the UK Biobank to combine polygenic risk scores (PRS) for 13 diseases and 12 mortality risk factors into sex-specific composite PRS (cPRS). These cPRS were moderately associated with all-cause mortality in independent data: the estimated hazard ratios per standard deviation were 1.10 (95% 1.05, 1.16) and 1.15 (1.10, 1.19) for women and men, respectively. Differences in life expectancy between the top and bottom 5% of the cPRS were estimated to be 4.79 (1.76, 7.81) years and 6.75 (4.16, 9.35) years for women and men, respectively. These associations were substantially attenuated after adjusting for non-genetic mortality risk factors measured at study entry. The cPRS may be useful in counseling younger individuals at higher genetic risk of mortality on modification of non-genetic factors.

  35. ⁠, Cristopher V. Van Hout, Ioanna Tachmazidou, Joshua D. Backman, Joshua D. Hoffman, Daren Liu, Ashutosh K. Pandey, Claudia Gonzaga-Jauregui, Shareef Khalid, Bin Ye, Nilanjana Banerjee, Alexander H. Li, Colm O’Dushlaine, Anthony Marcketta, Jeffrey Staples, Claudia Schurmann, Alicia Hawes, Evan Maxwell, Leland Barnard, Alexander Lopez, John Penn, Lukas Habegger, Andrew L. Blumenfeld, Xiaodong Bai, Sean O’Keeffe, Ashish Yadav, Kavita Praveen, Marcus Jones, William J. Salerno, Wendy K. Chung, Ida Surakka, Cristen J. Willer, Kristian Hveem, Joseph B. Leader, David J. Carey, David H. Ledbetter, Geisinger-Regeneron DiscovEHR Collaboration, Lon Cardon, George D. Yancopoulos, Aris Economides, Giovanni Coppola, Alan R. Shuldiner, Suganthi Balasubramanian, Michael Cantor, Regeneron Genetics Center, Matthew R. Nelson, John Whittaker, Jeffrey G. Reid, Jonathan Marchini, John D. Overton, Robert A. Scott, Gonçalo R. Abecasis, Laura Yerges-Armstrong, Aris Baras (2020-10-21):

    The is a prospective study of 502,543 individuals, combining extensive phenotypic and genotypic data with streamlined access for researchers around the world. Here we describe the release of data for the first 49,960 study participants, revealing approximately 4 million coding variants (of which around 98.6% have a frequency of less than 1%). The data include 198,269 autosomal predicted loss-of-function (LOF) variants, a more than 14-fold increase compared to the imputed sequence. Nearly all genes (more than 97%) had at least one carrier with a LOF variant, and most genes (more than 69%) had at least 10 carriers with a LOF variant. We illustrate the power of characterizing LOF variants in this population through association analyses across 1,730 phenotypes. In addition to replicating established associations, we found novel LOF variants with large effects on disease traits, including PIEZO1 on varicose veins, COL6A1 on corneal resistance, MEPE on bone density, and IQGAP2 and GMPR on blood cell traits. We further demonstrate the value of exome sequencing by surveying the prevalence of pathogenic variants of clinical importance, and show that 2% of this population has a medically actionable variant. Furthermore, we characterize the penetrance of cancer in carriers of pathogenic BRCA1 and BRCA2 variants. Exome sequences from the first 49,960 participants highlight the promise of genome sequencing in large population-based studies and are now accessible to the scientific community.

  36. ⁠, Thomas Battram, Tom R. Gaunt, Doug Speed, Nicholas J. Timpson, Gibran Hemani (2020-10-10):

    Following years of epigenome-wide association studies (EWAS), traits analysed to date tend to yield few associations. Reinforcing this observation, we conducted EWAS on 400 traits and 16 yielded at least one association at the conventional significance threshold (p < 1×10−7). To investigate why EWAS yield is low, we formally estimated the proportion of phenotypic variation captured by 421,693 blood derived DNA methylation markers (h2EWAS) across all 400 traits. The mean h2EWAS was zero, with evidence for regular cigarette smoking exhibiting the largest association with all markers (h2EWAS = 0.42) and the only one surpassing a false discovery rate < 0.1. Though underpowered to determine the h2EWAS value for any one trait, h2EWAS was predictive of the number of EWAS hits across the traits analysed (AUC = 0.7). Modelling the contributions of the methylome on a per-site versus a per-region basis gave varied h2EWAS estimates (r = 0.47) but neither approach obtained substantially higher model fits across all traits. Our analysis indicates that most complex traits do not heavily associate with markers commonly measured in EWAS within blood. However, it is likely DNA methylation does capture variation in some traits and h2EWAS may be a reasonable way to prioritise traits that are likely to yield associations.

  37. ⁠, Fankang Meng, Tom Ellis (2020-10-14):

    Synthetic biology is among the most hyped research topics this century, and in 2010 it entered its teenage years. But rather than these being a problematic time, we’ve seen synthetic biology blossom and deliver many new technologies and landmark achievements.

    1. …Looking back at 2010, the biggest synthetic biology story of the year was the complete synthesis of a working bacterial genome by a team at the J. Craig Venter Institute (JCVI)
    2. …Could hard biological problems such as context, noise, burden and cross-reactivity really be solved to allow us to engineer cells like we wire-up electronic circuits? Well, thanks to a lot of challenging technical biology and biological engineering work undertaken by many in the field, but especially MIT’s Chris Voigt, the answer to this was yes.
    3. …It’s no surprise therefore, that synthetic biology groups were the first to pounce on gene editing technologies like CRISPR as they appeared in 2011 and 2012.
    4. …While there’s no doubt that was the breakthrough of the decade in biosciences, it’s perhaps its forerunner TALENs (TAL-Effector Nucleases) that deserve more credit in revolutionizing how synthetic biology changed in the past 10 years.
    5. …The drop in cost for gene synthesis can mostly be attributed to new methods for printing thousands of oligonucleotides in parallel on chips to make ‘oligo pools’ and teaming this with next generation sequencing (NGS) as a much more cost-effective method for validating assembled DNA.
    6. …High-power computation also opened up new frontiers in what can be modelled and predicted in the last 10 years…This helped inform JCVI’s project towards a minimal genome, which delivered a further landmark in 2016 with the impressive engineering of a bacteria with a minimized synthetic genome
    7. …Synthetic genomics also moved into eukaryotes with the international Sc2.0 consortium constructing highly-modified, yet fully-functional synthetic versions of Baker’s yeast chromosomes
    8. …DNA also became a way to store data, initially just in vitro via chemical synthesis, but then also in cells via ‘molecular recorder’ genetic systems that use recombinases or CRISPR to modify DNA as cells grow, divide and change their gene expression
    9. …Academic achievements include engineering cells to fix CO2 and nitrogen, and getting yeast to make opioids and cannabinoids.

    …A multibillion dollar industry now exists that makes chemicals, drugs, proteins, probiotics, sensors, fertilisers, textiles, food and many other things from engineered cells.

  38. ⁠, The Royal Swedish Academy of Sciences (2020-10-07):

    Emmanuelle Charpentier and Jennifer A. Doudna have discovered one of gene technology’s sharpest tools: the genetic scissors. Using these, researchers can change the DNA of animals, plants and microorganisms with extremely high precision. This technology has had a revolutionary impact on the life sciences, is contributing to new cancer therapies and may make the dream of curing inherited diseases come true.

    Researchers need to modify genes in cells if they are to find out about life’s inner workings. This used to be time-consuming, difficult and sometimes impossible work. Using the CRISPR/​​​​Cas9 genetic scissors, it is now possible to change the code of life over the course of a few weeks.

    “There is enormous power in this genetic tool, which affects us all. It has not only revolutionised basic science, but also resulted in innovative crops and will lead to ground-breaking new medical treatments”, says Claes Gustafsson, chair of the Nobel Committee for Chemistry.

    As so often in science, the discovery of these genetic scissors was unexpected. During Emmanuelle Charpentier’s studies of Streptococcus pyogenes, one of the bacteria that cause the most harm to humanity, she discovered a previously unknown molecule, tracrRNA. Her work showed that tracrRNA is part of bacteria’s ancient immune system, CRISPR/​​​​Cas, that disarms viruses by cleaving their DNA.

    Charpentier published her discovery in 2011. The same year, she initiated a collaboration with Jennifer Doudna, an experienced biochemist with vast knowledge of RNA. Together, they succeeded in recreating the bacteria’s genetic scissors in a test tube and simplifying the scissors’ molecular components so they were easier to use.

    In an epoch-making experiment, they then reprogrammed the genetic scissors. In their natural form, the scissors recognise DNA from viruses, but Charpentier and Doudna proved that they could be controlled so that they can cut any DNA molecule at a predetermined site. Where the DNA is cut it is then easy to rewrite the code of life.

    Since Charpentier and Doudna discovered the CRISPR/​​​​Cas9 genetic scissors in 2012 their use has exploded. This tool has contributed to many important discoveries in basic research, and plant researchers have been able to develop crops that withstand mould, pests and drought. In medicine, clinical trials of new cancer therapies are underway, and the dream of being able to cure inherited diseases is about to come true. These genetic scissors have taken the life sciences into a new epoch and, in many ways, are bringing the greatest benefit to humankind.

  39. ⁠, Mira van der Naald, Steven Wenker, Pieter A. Doevendans, Kimberley E. Wever, Steven A. J. Chamuleau (2020-08-27):

    Objectives: The ultimate goal of biomedical research is the development of new treatment options for patients. Animal models are used if questions cannot be addressed otherwise. Currently, it is widely believed that a large fraction of performed studies are never published, but there are no data that directly address this question.

    Methods: We have tracked a selection of animal study protocols approved in the University Medical Center Utrecht in the Netherlands, to assess whether these have led to a publication with a follow-up period of 7 years.

    Results: We found that 60% of all animal study protocols led to at least one publication (full text or abstract). A total of 5590 animals were used in these studies, of which 26% was reported in the resulting publications.

    Conclusions: The data presented here underline the need for preclinical ⁠, in view of the risk of reporting and publication bias in preclinical research. We plea that all animal study protocols should be prospectively registered on an online, accessible platform to increase transparency and data sharing. To facilitate this, we have developed a platform dedicated to animal study protocol registration: www.preclinicaltrials.eu⁠.

    Strengths and limitations of this study:

    • This study directly traces animal study protocols to potential publications and is the first study to assess the number of animals used and the number of animals published.
    • We had full access to all documents submitted to the animal experiment committee of the University Medical Center Utrecht from the selected protocols.
    • There is a sufficient follow-up period for researchers to publish their animal study.
    • Due to privacy reasons, we are not able to publish the exact search terms used.
    • A delay has occurred between the start of this project and time of publishing, this is related to the political sensitivity of this subject.
  40. {#linkBibliography-(science)-2020 .docMetadata doi=“10.1126/​​science.abf2669”}, Dalmeet Singh Chawla (Science) (2020-10-14):

    Most animals used in biomedical experiments are not accounted for in published papers, a first-of-its-kind study suggests. The analysis found that only one-quarter of more than 5500 lab animals used over a 2-year period at one university in the Netherlands ended up being mentioned in a scientific paper afterward. The researchers believe the pattern could be similar at institutions around the world, resulting in potentially millions of animals disappearing from scientific studies.

    “I think it’s just outrageous that we have such a low rate of results published for the number of animals used”, says Michael Schlüssel, a medical statistician at the University of Oxford who was not involved in the study. “If we only look for groundbreaking research, the evidence base won’t be solid”, he adds. And that could impact studies that may confirm or refute the benefits of certain drugs or medical interventions.

    …For the new study, researchers asked scientists at three University Medical Center Utrecht (UMCU) departments for permission to review the study protocols they had filed with an animal ethics committee in 2008 and 2009. (They picked those years in part to be completely sure that the scientists had plenty of time to conduct and report the studies.) Then the team—led by Mira van der Naald, a doctoral student at UMCU—searched the medical literature for papers resulting from the work.

    Of the approved studies, 46% were published as a full-text paper; if conference abstracts—short summaries of a talk or poster presented at a scientific meeting—were counted as well, 60% ended up being published. Yet out of the 5590 animals used in the studies, only 1471 were acknowledged in published papers and abstracts, the team reports in BMJ Open Science. Small animals, including mice, rats, and rabbits—which made up 90% of the total—were most often missing in action: Only 23% of them showed up in publications, versus 52% of sheep, dogs, and pigs.

    The researchers also surveyed the scientists involved to find out why so many animals were missing. The most common reasons they gave were that the studies didn’t achieve statistical-significance, a controversial but commonly used threshold for publication; that the data were part of a pilot project; and that there were technical issues with the animal models. But none of these is a valid excuse to not publish your findings in the scientific record, says study co-author Kimberley Wever, a metascientist at Radboud University Medical Center. “All animal studies should be published, and all studies are valuable for the research community.” Not publishing all research means other scientists may waste time, effort, and money redoing studies that have previously failed, Wever says. She adds that the trend likely holds up at institutions around the world and hopes other researchers will conduct similar studies.

  41. ⁠, Stephen Wolfram (2020-09-28):

    [Consideration of Euclid’s Elements as a network or directed acyclic graph of axioms, proofs, and theorems. Starting with the axioms, each proof relies on previously defined/​​​​proven elements; as such they form a network. What does this network look like? What does it reveal? What ‘nodes’ are most central, most reused elsewhere? Could the network be simplified or made more compact, by adding missing theorems or more complex ‘super-axioms’?

    Wolfram uses Mathematica to parse the Elements and generate graph theory statistics about the 13 books, 131 definitions, and 465 theorems. There is clearly structure, and ‘popular’ theorems which are reused often, while some theorems are obscure and don’t come up again. Similarly, the books differ greatly in importance, and some books form groupings by themselves as relatively independent topics from the rest. As the books develop, they use more and more theorems, and fewer and fewer axioms: the overall structure implies that, after a slow start ⁠, a few critical theorems ‘unlock’ many of the later theorems. Interestingly, the ‘climax’, the theorem which requires the most earlier theorems, is Euclid’s proof that there are 5 Platonic solids (a concept that fascinated many Greco-Roman mathematicians and philosophers).

    Similar questions could be asked of modern mathematics after formalization, and might yield an ‘empirical metamathematics’ showing how human mathematicians structure their theories and go about proving the theorems they choose to.]

  42. ⁠, Viktor Blåsjö (2020-06-21):

    Greek geometry is written in a style adapted to oral teaching. Mathematicians memorised theorems the way bards memorised poems. Several oddities about how Euclid’s Elements is written can be explained this way. Greek geometry is oral geometry. Mathematicians memorised theorems the way bards memorised poems. Euclid’s Elements was almost like a song book or the script of a play: it was something the connoisseur was meant to memorise and internalise word for word. Actually we can see this most clearly in purely technical texts, believe it or not. It is the mathematical details of Euclid’s proofs that testify to this cultural practice.

    …Here’s an example of this, which I have taken from Reviel Netz’s book The Shaping of Deduction in Greek Mathematics. Consider the equation A + B = C + D. Here’s how the Greeks expressed this in writing: THEAANDTHEBTAKENTOGETHERAREEQUALTOTHECANDTHED. This is written as one single string of all-caps letters. No punctuation, no spacing, no indication of where one word stops and the next one begins. A Greek text is basically a tape recording. It records the sounds being spoken…Modern editions of Euclid’s Elements are full of cross-references. Each step of a proof is justified by a parenthetical reference to a previous theorem or definition or postulate. But that’s inserted by later editors. There is no such thing in the original text. Because it’s a tape recording of a spoken explanation. Referring back to “Theorem 8” is only useful if the audience has a written document in front of them

    …Consider for example Proposition 4 of Euclid’s Elements…“The triangle will be equal to the triangle”, says Euclid: this is his way of saying that they have equal area. After Euclid has stated this, he goes on to re-state the same thing, but now in terms the diagram…This is exactly the same thing that he just said in words. But now he’s saying it with reference to the diagram. He always does this. He always has these two version of every proposition: the purely verbal one, and the one full of letters referring to the diagram…That’s something of a puzzle in itself, but here’s the real kicker though. Not only does Euclid insist on including the abstruse verbal formulation of every theorem, he actually includes it twice! This is because, at the end of the proof, his last sentence is always “therefore…” and then he literally repeats the entire verbal statement of the theorem. It is literally the exact same statement, word for word, repeated verbatim. You say the exact same thing when you state the proposition and then again when you conclude the proof. Copy-paste. The exact same text just a few paragraphs apart.

    …So what was the value of this very expensive business of repeating the statement of the proposition? The oral tradition explains it. The verbal statement of the proposition is like the chorus of a song. It’s the key part, the key message, the most important part to memorise. It is repeated for the same reason the chorus of a song is repeated. It’s the sing-along part. In a written culture you can refer back to propositions and expect the reader to have the text in front of them. Not so in an oral culture. You need to evoke the memory of the proposition to an audience who do not have a text in front of them but who have learned the propositions by heart, word by word, exactly as it was stated, the way you memorise a poem or song. This is why, anytime Euclid uses a particular theorem at a particular point in a proof, he doesn’t says “this follows by Theorem 8” or anything like that. He doesn’t refer to earlier theorems by number or name. Instead he evokes the earlier theorem by mimicking its exact wording. Just as you just have to hear a few words of your favourite chorus and you can immediately fill in the rest. So also the reader, or listener, of a Euclidean proof would immediately recognise certain phrasings as corresponding word for word to particular earlier propositions…In one case it is even irrelevant that the remain sides are equal as well, but Euclid still needlessly remarks on this pointless information in the course of the proof of Proposition 5 even though it has no logical bearing on the proof. Go look up Euclid’s proof if you want to see this nonsense for yourself.

    …It’s pretty fascinating, I think, how textual aspects that appear to be purely technical and mathematical, such as a few barely noticeable superfluous bits of information in the proof of Proposition 5, can open a window like this into an entire cultural practice.

  43. ⁠, Alvaro de Menard (2020-01-17):

    [Summary of the that gripped Western classical literary scholarship for centuries: who wrote the Iliad/​​​​Odyssey, when, and how? They appear in Greek history out of nowhere: 2 enormously lengthy, sophisticated, beautiful, canonical, unified works that would dominate Western literature for millennia, and yet, appeared to draw on no earlier tradition nor did Homer have any earlier (non-spurious) works. How was this possible?

    The iconoclastic Analysts proposed it was a fraud, and the works were pieced together later out of scraps from many earlier poets. The Unitarians pointed to the overall quality; the complex (apparently planned) structure; the disagreements of Analysts on what parts were what pieces; and the Analysts’ inability to explain many anomalies in Homer: there are passages splicing together Greek dialects, passages which were metrical only given long-obsolete Greek letters/​​​​pronunciations, and even individual words which mixed up Greek dialects! (Not that these anomalies were all that much easier to explain by the Unitarian hypothesis of a single author).

    The eventual resolution relied an old hypothesis: that Homer was in fact the product of a lost ⁠. There was, unfortunately, no particular evidence for it, and so it never made any headway against the Analysts or Unitarians—until Milman Parry found a living oral tradition of epic poetry in the Balkans, and discovered in it all the signs of the Homeric poems, from repetitive epithets to a patchwork of dialects, and thus empirical examples of how long oral traditions could produce a work like Homer if one of them happened to get written down at some point.]

  44. 2020-dorling.pdf: ⁠, James L. Dorling, Stephan van Vliet, Kim M. Huffman, William E. Kraus, Manjushri Bhapkar, Carl F. Pieper, Tiffany Stewart, Sai Krupa Das, Susan B. Racette, Susan B. Roberts, Eric Ravussin, Leanne M. Redman, Corby K. Martin [for the CALERIE Study Group] (2020-09-17; longevity):

    Caloric restriction (CR) is a strategy that attenuates aging in multiple nonhuman species. The Comprehensive Assessment of Long-term Effects of Reducing Intake of Energy (CALERIE) trials are part of a research program aiming to test the effects of CR on aging and longevity biomarkers in humans. Building on CALERIE phase 1, CALERIE phase 2 (CALERIE 2) was the largest study to date to assess sustained CR in healthy humans without obesity. In a 24-month comprising 218 participants at baseline, CALERIE 2 showed that moderate CR, 11.9% on average, induced improvements in aging-related biomarkers without adversely affecting psychological or behavioral outcomes. The objectives of this report are to summarize and review the highlights of CALERIE 2 and report previously unpublished results on eating disorder symptoms and cognitive function. This article specifically summarizes the physiological, psychological, aging, behavioral, and safety results of the trial. Also provided are research directions beyond CALERIE 2 that highlight important opportunities to investigate the role of CR in aging, longevity, and health span in humans.

  45. ⁠, Brian Hayes (2020-09-04):

    [Series of simulations exploring explanations for the “paradox of the plankton”: why, in apparently homogenous environments, such as open sea water, are there countless thousands of species of plankton all doing the same task of photosynthesis & competing for the same resources? Why isn’t there just a few, or one, species which is optimal in that niche and outcompetes all the others rapidly? Similarly, in a forest: why is there such a dense mix of tree species, rather than relatively continuous stands of species as local conditions vary?

    The initial simulations demonstrate that even a tiny fitness difference will result in a loss of diversion; in fact, with no difference, simple random fluctuations, ‘drift’, will eventually irreversibly drive species to extinction (even assuming occasional ‘immigrants’). This can be fended off by assuming a specialty, like metabolizing a particular chemical well, but can this really explain forest stands with 200+ species? More plausible is predatory-prey dynamics like the Lotka-Volterra model: a species which becomes too common gets preyed on by diseases and parasites and predators, stopping it from spreading further. The dynamics of it are chaotic, but preserve diversity. To some extent, probably all of these explanations are true.]

  46. ⁠, Alex Armstrong, Chi Chung Lam, Shievanie Sabesan, Nicholas A. Lesica (2020-10-04):

    Hearing aids are the only available treatment for mild-to-moderate sensorineural hearing loss, but often fail to improve perception in difficult listening conditions. To identify the reasons for this failure, we studied the underlying neural code using large-scale single-neuron recordings in gerbils, a common animal model of human hearing. We found that a hearing aid restored the sensitivity of neural responses, but failed to restore their selectivity. The low selectivity of aided responses was not a direct effect of hearing loss per se, but rather a consequence of the strategies used by hearing aids to restore sensitivity: compression, which decreases the spectral and temporal contrast of incoming sounds, and amplification, which produces high intensities that distort the neural code even with normal hearing. To improve future hearing aids, new processing strategies that avoid this tradeoff between neural sensitivity and selectivity must be developed.

  47. ⁠, Tasmin Humphrey, Leanne Proops, Jemma Forman, Rebecca Spooner, Karen McComb (2020-10-05):

    Domestic animals are sensitive to human cues that facilitate inter-specific communication, including cues to emotional state. The eyes are important in signalling emotions, with the act of narrowing the eyes appearing to be associated with positive emotional communication in a range of species. This study examines the communicatory importance of a widely reported behaviour that involves eye narrowing, referred to as the slow blink sequence. Slow blink sequences typically involve a series of half-blinks followed by either a prolonged eye narrow or an eye closure. Our first experiment revealed that cat half-blinks and eye narrowing occurred more frequently in response to owners’ slow blink stimuli towards their cats (compared to no owner-cat interaction). In a second experiment, this time where an experimenter provided the slow blink stimulus, cats had a higher propensity to approach the experimenter after a slow blink interaction than when they had adopted a neutral expression. Collectively, our results suggest that slow blink sequences may function as a form of positive emotional communication between cats and humans.

  48. ⁠, Paul Foley (2011-10-12):

    But the illness provoked a flood of publications throughout the 1920s and 1930s, as its kaleidoscopic combination of neurologic and psychiatric phenomena provided insights into brain function that had previously been the subject of speculation. These insights have had an enduring impact upon both neurology and psychiatry.

    …The psychiatric facets of this phase were no less important. A peculiar lack of internal drive separating the patient from their world was typical. Despite normal intelligence, these patients could not summon the will power to execute their wishes. More insightful sufferers described how neither their perceptions nor their own thoughts were associated with the required emotional content that permitted exercise of their will. Patients could appreciate that a pianist played with great technical skill, for instance, but no longer sensed the beauty of the music…The only consolation was that this same apathy often meant the sufferers were not overly depressed by their illness or by the prospect of a life in an institution (remembering that these young patients might live for another half century or more).

    …The second phase was marked by a general loss of concentration and interest in life, giving a vague sensation that the patient was not the person they had once been. But this period, which resembled chronic fatigue syndrome, was the calm before the storm. Unbeknownst to the victim, localized neurodegeneration proceeded apace through the first phase, and after an interval—lasting between a few days and 30 years—post-encephalitic parkinsonism (PEP) emerged. Unmistakable and irreversible, PEP consigned the young sufferers (mostly between 15 and 35 years of age) to decades of disability. For those who had not yet passed adolescence, the second period was marked by pathologic changes of character that approached the ⁠.

    Younger children, between 5 and 10 years old, might merely irritate with their clinginess; their impaired concentration; their incessant restlessness and need for noise; and their lack of consideration for others—not unlike current attention deficit disorders. But as they grew in strength, their incorrigible impulsiveness escalated in violence and they posed a danger to themselves and others. Errant behaviours included cruelty to anyone who crossed them; destructiveness; lying; and self-mutilation including, in one example, removal of eyes. When they reached adolescence, these patients manifested inappropriate and excessive sexuality, including sexual assault without regard for age or gender. Bizarrely, these children were driven by impulsiveness, not self-interest. Thefts, for example, were not undertaken for personal benefit and stolen goods were often immediately forgotten, or given away. Patients often expressed genuine remorse for their actions, explaining they recognized their wrongdoing but had been compelled to act as they did.

    Some children improved after adolescence, but in many the only brake on their bad behavior was the parkinsonism that developed as they entered adulthood. Those not confined to hospital with parkinsonism often proceeded to a life of habitual criminality—mostly theft in men, prostitution in women, but also ranging up to rape and murder.

    This phenomenon encouraged many countries to re-examine laws regarding legal responsibility in those whose actions were curtailed neither by encouragement nor prison, but who nonetheless maintained a sense of what was socially appropriate.

  49. 2020-stojanoski.pdf: ⁠, Bobby Stojanoski, Conor J. Wild, Michael E. Battista, Emily S. Nichols, Adrian M. Owen (2020-09-24; dual-n-back):

    The foundational tenet of brain training is that general cognitive functioning can be enhanced by completing computerized games, a notion that is both intuitive and appealing. Moreover, there is strong incentive to improve our cognitive abilities, so much so that it has driven a billion-dollar industry. However, whether brain training can really produce these desired outcomes continues to be debated. This is, in part, because the literature is replete with studies that use ill-defined criteria for establishing transferable improvements to cognition, often using single training and outcome measures with small samples. To overcome these limitations, we conducted a large-scale online study to examine whether practices and beliefs about brain training are associated with better cognition. We recruited a diverse sample of over 1000 participants, who had been using an assortment of brain training programs for up to 5 years. Cognition was assessed using multiple tests that measure attention, reasoning, working memory and planning. We found no association between any measure of cognitive functioning and whether participants were currently ‘brain training’ or not, even for the most committed brain trainers. Duration of brain training also showed no relationship with any cognitive performance measure. This result was the same regardless of participant age, which brain training program they used, or whether they expected brain training to work. Our results pose a substantial challenge for ‘brain training’ programs that purport to improve general cognitive functioning among the general population.

  50. 2020-grohn.pdf: ⁠, Kristopher J. Grohn, Brandon S. Moyer, Danique C. Wortel, Cheyanne M. Fisher, Ellie Lumen, Anthony H. Bianchi, Kathleen Kelly, Paul S. Campbell, Douglas E. Hagrman, Roger G. Bagg, James Clement, Aaron J. Wolfe, Andrea Basso, Cristina Nicoletti, Giovanni Lai, Mauro Provinciali, Marco Malavolta, Kelsey J. Moody (2020-10-29; longevity):

    C60 is a potent antioxidant that has been reported to substantially extend the lifespan of rodents when formulated in olive oil (C60-OO) or extra virgin olive oil (C60-EVOO). Despite there being no regulated form of C60-OO, people have begun obtaining it from online sources and dosing it to themselves or their pets, presumably with the assumption of safety and efficacy. In this study, we obtain C60-OO from a sample of online vendors, and find marked discrepancies in appearance, impurity profile, concentration, and activity relative to pristine C60-OO formulated in-house. We additionally find that pristine C60-OO causes no acute toxicity in a rodent model but does form toxic species that can cause morbidity and mortality in mice in under 2 weeks when exposed to light levels consistent with ambient light. Intraperitoneal injections of C60-OO did not affect the lifespan of CB6F1 female mice. Finally, we conduct a lifespan and health span study in males and females C57BL/​​​​6 J mice comparing oral treatment with pristine C60-EVOO and EVOO alone versus untreated controls. We failed to observe statistically-significant lifespan and health span benefits of C60-EVOO or EVOO supplementation compared to untreated controls, both starting the treatment in adult or old age. Our results call into question the biological benefit of C60-OO in aging.

  51. Replication#animal-models

  52. https://b-ok.cc/fulltext/

  53. 2015-kanev.pdf#google: ⁠, Svilen Kanev, Juan Pablo Darago, Kim M. Hazelwood, Parthasarathy Ranganathan, Tipp J. Moseley, Gu-Yeon Wei, David Michael Brooks (2015-06-01; cs):

    With the increasing prevalence of warehouse-scale (WSC) and cloud computing, understanding the interactions of server applications with the underlying microarchitecture becomes ever more important in order to extract maximum performance out of server hardware. To aid such understanding, this paper presents a detailed microarchitectural analysis of live datacenter jobs, measured on more than 20,000 Google machines over a three year period, and comprising thousands of different applications.

    We first find that WSC workloads are extremely diverse, breeding the need for architectures that can tolerate application variability without performance loss. However, some patterns emerge, offering opportunities for co-optimization of hardware and software. For example, we identify common building blocks in the lower levels of the software stack. This “datacenter tax” can comprise nearly 30% of cycles across jobs running in the fleet, which makes its constituents prime candidates for hardware specialization in future server systems-on-chips. We also uncover opportunities for classic microarchitectural optimizations for server processors, especially in the cache hierarchy. Typical workloads place substantial stress on instruction caches and prefer memory latency over bandwidth. They also stall cores often, but compute heavily in bursts. These observations motivate several interesting directions for future warehouse-scale computers.

  54. 2010-ren.pdf#google: ⁠, Gang Ren; Tune, E.; Moseley, T.; Yixin Shi; Rus, S.; Hundt, R. (2010-08-19; cs):

    Google-Wide Profiling (GWP), a continuous profiling infrastructure for data centers, provides performance insights for cloud applications. With negligible overhead, GWP provides stable, accurate profiles and a datacenter-scale tool for traditional performance analyses. Furthermore, GWP introduces novel applications of its profiles, such as application-platform affinity measurements and identification of platform-specific, microarchitectural peculiarities.

  55. 1993-trauth.pdf: ⁠, Kathleen M. Trauth, Stephen C. Hora, Robert V. Guzowski (1993-11; technology):

    (SNL) convened an expert panel to develop design characteristics for and to judge the efficacy of the markers in deterring inadvertent human intrusion in the (WIPP). The WIPP, located in southeastern New Mexico, is designed to demonstrate the safe disposal of transuranic (TRU) radioactive wastes generated by the United States Department of Energy (DOE) defense programs. The DOE must evaluate WIPP compliance with the Environmental Protection Agency (EPA) regulation “Environmental Standards for the Management and Disposal of Spent Nuclear Fuel, High-Level and Transuranic Radioactive Wastes (40 CFR Part 191, Subpart E)”; this EPA regulation requires: “Disposal sites shall be designated by the most permanent markers, records, and other passive institutional controls practicable to indicate the dangers of the wastes and their location” (Federal Register 50; 38086). The period of regulatory concern is 10,000 years.

    The expert panel identified basic principles to guide current and future marker development efforts: (1) the site must be marked, (2) message(s) must be truthful and informative, (3) multiple components within a marker system, (4) multiple means of communication (eg., language, pictographs, scientific diagrams), (5) multiple levels of complexity within individual messages on individual marker system elements, (6) use of materials with little recycle value, and (7) international effort to maintain knowledge of the locations and contents of nuclear waste repositories. The efficacy of the markers in deterring inadvertent human intrusion was estimated to decrease with time, with the probability function varying with the mode of intrusion (who is intruding and for what purpose) and the level of technological development of the society. The development of a permanent, passive marker system capable of surviving and remaining interpretable for 10,000 years will require further study prior to implementation.

    [Keywords: management of radioactive and non-radioactive wastes from nuclear facilities, nuclear fuel cycle and fuel materials, WIPP, human intrusion, alpha-bearing wastes, underground disposal, radiation hazards, communications, safety, recommendations, design, waste disposal and storage, health and safety]

  56. {#linkBibliography-yorker)-2020 .docMetadata}, Raffi Khatchadourian () (2020-09-28):

    [Profile of the problem of space debris and near-misses of collisions with rockets, astronauts, and space stations. While few major accidents have happened yet, the amount of debris is only going to increase. The problem was first identified by Don Kessler, an astrophysicist investigating the threat of asteroids in Earth-Mars orbits for the expected upcoming Martian flights through the asteroid belt; thinking about the debris from asteroids colliding, he began to wonder about satellites closer to home. With proposals for Skylab and large power satellites beaming down microwaves, the problem had become more concerning, and Kessler’s projections indicated that it would soon be a lethal threat and would eventually become self-sustaining as debris fragments create more fragments—“Kessler syndrome”. NASA leadership angrily denied his modeling until incidents like Kosmos 954 made the problem undeniable. Since then, orbital debris has been tracked in ever more detail, and the problem of Kessler syndrome has not gone away, but become more acute. With the militarization of space, anti-satellite missile tests, and large commercial fleets like SpaceX’s Starlink building up, the threat has motivated research into how to actively clean up orbit: harpoons, nets, electrodynamic tethers and more are contemplated and being actively tested. They will be needed.]

  57. {#linkBibliography-(ars)-2020 .docMetadata}, Richard C. Moss (Ars) (2020-07-07):

    [Retrospective profile of Adobe Flash, with interviews from creator Jonathan Gay about its founding in 1992: it began as a vector drawing program for now-forgotten tablet / PDA devices, a project that was killed, and they pivoted to porting it to desktop. This too flopped, as customers suggested that cel-shading and rotoscoping animation would be more useful; with the Web emerging, they decided to retarget Java applets. Their prototype ran at 2FPS, and Adobe was unimpressed. Microsoft & Disney, however, saw promise in it, and made it a highlight of their new websites like MSN and The Daily Blast, despite Flash being on the brink of death. Macromedia heard of it through them, and acquired Flash, as a bridge from their fading multimedia CD-ROM to the hot new Internet. The highly expanded Flash was the most interactive and versatile web development tool in an era when JavaScript barely existed, and easy to use. Soon, games were being written in it (to the creators’ surprise, considering how weak a programming language it was, with barely-working conditionals or variables), and an online ecosystem springing up around sites like Newgrounds with literally millions of players. Flash soon became used for video, and animating. Adobe, a decade after declining Flash, bought it for billions. But at its zenith in the mid-2000s, Flash was about to fall, as Adobe was distracted by corporate uses rather than video/​​​​games/​​​​general web, open web standards/​​​​browsers gradually accreted its capabilities natively; finally, a major blow was declaring Flash dead—slow & power-hungry, proprietary (ie. ‘not Apple-owned’), insecure, and ill-designed for the mobile-first future. Flash entered a death spiral, and quickly was abandoned by even Adobe. Its legacy is now primarily opening up creative Internet uses worldwide and getting countless people involved in media.]

  58. ⁠, Jonas Richner (2020-07-22):

    …This Flash game is called Canabalt. A businessman crashes out of a window and starts running to escape the destruction of his city. Canabalt sparked the entire endless runner genre of gameplay, which is now one of the most popular genres on mobile. The game has since been included in the New York Museum of Modern Art, alongside Pac-Man and Tetris. Escape room games, now a popular genre, originally came from Flash games. They even made the jump into real life, with many physical escape rooms all over the world. There were many more Flash games. Millions more. Played billions of times on thousands of different gaming websites. It was creative chaos. Flash games were the gateway for many developers in the games industry, and served as an experimental playground for distilling games down to their most pure and engaging elements. The end-of-life of Flash in December 2020 marks the end of one of the most creative periods in the history of gaming. It all started in 1996, when the Flash player was first released. Originally it was intended for Web graphics and animations, but when it got its own programming language in 2000, developers started to use it to make games. That was the same year we saw the rise of the first automated Flash games website, Newgrounds. Anyone could upload their games and they were published immediately…

    [Followed by timeline of Flash games; >20 testimonials from ex-Flash developers and game industry figures.]

  59. https://www.alexirpan.com/2021/02/18/flash-games.html

  60. Bakewell

  61. ⁠, Leon Bambrick (2019-06-04):

    [(Github) A ‘brutalist’ website which shows the raw material (source code) it is made of, and so represents a kind of : a clever use of CSS to simultaneously render the literal HTML tags and source code, while also styling them appropriately.]

  62. ⁠, Samuel Reed (2014-04):

    [Tech demo: a website which “codes itself” line by line, CSS/​​​​HTML by CSS/​​​​HTML, gradually enhancing into a regular-looking website (serving as a resume/​​​​portfolio for its author, better known for co-founding the cryptocurrency exchange ).]

  63. ⁠, Chris Offutt (2015-02-05):

    [Excerpt from memoir/​​​​investigation of SF author Andrew Jefferson Offutt V by his son, charged posthumously with sorting through an enormous horde of writing, notes, art, novels, and everything pornographic, as Andrew Offutt was more prolific as an author of SF/​​​​fantasy & regular pornography than mere SF. Initially a side-gig to make money, it became his passion and secret life, and he saw himself as elevating pornographic fiction, a task to which he brought all his organizational skills in order to publish one novel per month across his many pseudonyms.

    How? He kept extensive notes, cross-classified by every fetish & sexual act & plot device & physical description, then developed a draft outline; used items from the notes would be struck out so as to avoid the artistic sin of repetition. The outline would be filled out in 20–40 page hand-writing stints, and the completed novel edited while transcribed on a typewriter, for his wife to make a final edited copy. At his fastest, he could write a book in 3 days.

    This was only the visible part of an even more intense and secretive fantasy life, where he made 120 books (4000 pages) of comics using a collage technique, taking art from countless magazines/​​​​catalogues, and lightly redrawing or editing. The comics were an outlet for his bondage and fetishes, and he credited them (his son disagrees) as a safety valve stopping him from becoming a serial killer.]

    The commercial popularity of American erotic novels peaked during the 1970s, coinciding with my father’s most prolific and energetic period. Dad combined porn with all manner of genre fiction. He wrote pirate porn, ghost porn, science-fiction porn, vampire porn, historical porn, time-travel porn, secret-agent porn, thriller porn, zombie porn and Atlantis porn. An unpublished Old West novel opens with sex in a barn, featuring a gunslinger called Quiet Smith, without doubt Dad’s greatest character name. By the end of the decade, Dad claimed to have single-handedly raised the quality of American pornography. He believed future scholars would refer to him as the “king of 20th-century written pornography.” He considered himself the “class operator in the field.”

    …Dad’s writing process was simple—he’d get an idea, brainstorm a few notes, then write the first chapter. Next he’d develop an outline from one to 10 pages. He followed the outline carefully, relying on it to dictate the narrative. He composed his first drafts longhand, wearing rubber thimbles on finger and thumb. Writing with a felt-tip pen, he produced 20 to 40 pages in a sitting. Upon completion of a full draft, he transcribed the material to his typewriter, revising as he went. Most writers get more words per page as they go from longhand to a typed manuscript, but not Dad. His handwriting was small, and he used ampersands and abbreviations. His first drafts were often the same length as the final ones. Manuscripts of science fiction and fantasy received multiple revisions, but he had to work much faster on porn. After a longhand first chapter, he typed the rest swiftly, made editorial changes and passed that draft to my mother. She retyped it for final submission. At times, Mom would be typing the beginning of the book while Dad was still writing the end. His goal was a minimum of a book a month. To achieve that, he refined his methods further, inventing a way that enabled him to maintain a supply of raw material with a minimum of effort. He created batches in advance—phrases, sentences, descriptions and entire scenes on hundreds of pages organized in three-ring binders. Tabbed index dividers separated the sections into topics. 80% of the notebooks described sexual aspects of women. The longest section focused on their bosoms. Another binder listed descriptions of individual actions, separated by labeling tabs that included: Mouth. Tongue. Face. Legs. Kiss. The heading of Orgasm had subdivisions of Before, During and After. The thickest notebook was designed strictly for BDSM novels with a list of 150 synonyms for “pain.” Sections included Spanking, Whipping, Degradation, Predegradation, Distress, Screams, Restraints and Tortures. These were further subdivided into specific categories followed by brief descriptions of each. Dad was like Henry Ford applying principles of assembly-line production with pre-made parts. The methodical technique proved highly efficient. Surrounded by his tabulated notebooks, he could quickly find the appropriate section and transcribe lines directly into his manuscript. Afterward, he blacked them out to prevent plagiarizing himself. Ford hired a team of workers to manufacture a Model-T in hours. Working alone, Dad could write a book in three days.