“Agent57: Outperforming the Atari Human Benchmark”, Badia et al 2020 (blog; Agent57 reaches the median human level across ALE—including Pitfall!/Montezuma’s Revenge. It is impressive but still sample-inefficient & uncomfortably baroque in combining what seems like every DM model-free DRL technique in one place: DDQN, Impala, R2D2, Memory Networks, Transformers, Neural Episodic Control, RND, NGU, PBT, MABs… Is model-free DRL a dead end if this is what it takes? I would have preferred to see ALE solved by better exploration in the enormously simpler MuZero.)
Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/changes/additions.
Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular /r/Gwern subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).
A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.
Given a sample of n pairs of 2 normal variables A & B which are correlated r, what is the probability pmax that the maximum on the first variable A is also the maximum on the second variable B? This is analogous to many testing or screening situations, such as employee hiring (“what is the probability the top-scoring applicant on the first exam is the top-scorer on the second as well?”) or athletic contests (“what is the probability the current world champ will win the next championship?”).
Order statistics has long proven that asymptotically, pmax approaches . Exact answers are hard to find, but confirm the asymptotics; the closest that exists is for an approximation & special-case of the Ali-Mikhail-Haq copula: which roughly indicates that r functions as a constant factor boost in pmax, and the boost from r fades out as n increases.
As long as r≠1, “the tails will come apart”. n increases the difficult too fast for any fixed r to overcome. This has implications for interpreting extremes and test metrics.
Because many users do not have access to a browser/OS which explicitly supports dark mode, cannot modify the browser/OS setting without undesired side-effects, wish to opt in only for specific websites, or simply forget that they turned on dark mode & dislike it, we make dark mode controllable by providing a widget at the top of the page.
“The genetic architecture of the human cerebral cortex”, Katrina L. Grasby, Neda Jahanshad, Jodie N. Painter, Lucía Colodro-Conde, Janita Bralten, Derrek P. Hibar, Penelope A. Lind, Fabrizio Pizzagalli, Christopher R. K. Ching, Mary Agnes B. McMahon, Natalia Shatokhina, Leo C. P. Zsembik, Sophia I. Thomopoulos, Alyssa H. Zhu, Lachlan T. Strike, Ingrid Agartz, Saud Alhusaini, Marcio A. A. Almeida, Dag Alnæs, Inge K. Amlien, Micael Andersson, Tyler Ard, Nicola J. Armstrong, Allison Ashley-Koch, Joshua R. Atkins, Manon Bernard, Rachel M. Brouwer, Elizabeth E. L. Buimer, Robin Bülow, Christian Bürger, Dara M. Cannon, Mallar Chakravarty, Qiang Chen, Joshua W. Cheung, Baptiste Couvy-Duchesne, Anders M. Dale, Shareefa Dalvie, Tânia K. de Araujo, Greig I. de Zubicaray, Sonja M. C. de Zwarte, Anouk den Braber, Nhat Trung Doan, Katharina Dohm, Stefan Ehrlich, Hannah-Ruth Engelbrecht, Susanne Erk, Chun Chieh Fan, Iryna O. Fedko, Sonya F. Foley, Judith M. Ford, Masaki Fukunaga, Melanie E. Garrett, Tian Ge, Sudheer Giddaluru, Aaron L. Goldman, Melissa J. Green, Nynke A. Groenewold, Dominik Grotegerd, Tiril P. Gurholt, Boris A. Gutman, Narelle K. Hansell, Mathew A. Harris, Marc B. Harrison, Courtney C. Haswell, Michael Hauser, Stefan Herms, Dirk J. Heslenfeld, New Fei Ho, David Hoehn, Per Hoffmann, Laurena Holleran, Martine Hoogman, Jouke-Jan Hottenga, Masashi Ikeda, Deborah Janowitz, Iris E. Jansen, Tianye Jia, Christiane Jockwitz, Ryota Kanai, Sherif Karama, Dalia Kasperaviciute, Tobias Kaufmann, Sinead Kelly, Masataka Kikuchi, Marieke Klein, Michael Knapp, Annchen R. Knodt, Bernd Krämer, Max Lam, Thomas M. Lancaster, Phil H. Lee, Tristram A. Lett, Lindsay B. Lewis, Iscia Lopes-Cendes, Michelle Luciano, Fabio Macciardi, Andre F. Marquand, Samuel R. Mathias, Tracy R. Melzer, Yuri Milaneschi, Nazanin Mirza-Schreiber, Jose C. V. Moreira, Thomas W. Mühleisen, Bertram Müller-Myhsok, Pablo Najt, Soichiro Nakahara, Kwangsik Nho, Loes M. Olde Loohuis, Dimitri Papadopoulos Orfanos, John F. Pearson, Toni L. Pitcher, Benno Pütz, Yann Quidé, Anjanibhargavi Ragothaman, Faisal M. Rashid, William R. Reay, Ronny Redlich, Céline S. Reinbold, Jonathan Repple, Geneviève Richard, Brandalyn C. Riedel, Shannon L. Risacher, Cristiane S. Rocha, Nina Roth Mota, Lauren Salminen, Arvin Saremi, Andrew J. Saykin, Fenja Schlag, Lianne Schmaal, Peter R. Schofield, Rodrigo Secolin, Chin Yang Shapland, Li Shen, Jean Shin, Elena Shumskaya, Ida E. Sønderby, Emma Sprooten, Katherine E. Tansey, Alexander Teumer, Anbupalam Thalamuthu, Diana Tordesillas-Gutiérrez, Jessica A. Turner, Anne Uhlmann, Costanza Ludovica Vallerga, Dennis van der Meer, Marjolein M. J. van Donkelaar, Liza van Eijk, Theo G. M. van Erp, Neeltje E. M. van Haren, Daan van Rooij, Marie-José van Tol, Jan H. Veldink, Ellen Verhoef, Esther Walton, Mingyuan Wang, Yunpeng Wang, Joanna M. Wardlaw, Wei Wen, Lars T. Westlye, Christopher D. Whelan, Stephanie H. Witt, Katharina Wittfeld, Christiane Wolf, Thomas Wolfers, Jing Qin Wu, Clarissa L. Yasuda, Dario Zaremba, Zuo Zhang, Marcel P. Zwiers, Eric Artiges, Amelia A. Assareh, Rosa Ayesa-Arriola, Aysenil Belger, Christine L. Brandt, Gregory G. Brown, Sven Cichon, Joanne E. Curran, Gareth E. Davies, Franziska Degenhardt, Michelle F. Dennis, Bruno Dietsche, Srdjan Djurovic, Colin P. Doherty, Ryan Espiritu, Daniel Garijo, Yolanda Gil, Penny A. Gowland, Robert C. Green, Alexander N. Häusler, Walter Heindel, Beng-Choon Ho, Wolfgang U. Hoffmann, Florian Holsboer, Georg Homuth, Norbert Hosten, Clifford R. Jack Jr., MiHyun Jang, Andreas Jansen, Nathan A. Kimbrel, Knut Kolskår, Sanne Koops, Axel Krug, Kelvin O. Lim, Jurjen J. Luykx, Daniel H. Mathalon, Karen A. Mather, Venkata S. Mattay, Sarah Matthews, Jaqueline Mayoral Van Son, Sarah C. McEwen, Ingrid Melle, Derek W. Morris, Bryon A. Mueller, Matthias Nauck, Jan E. Nordvik, Markus M. Nöthen, Daniel S. O’Leary, Nils Opel, Marie-Laure Paillère Martinot, G. Bruce Pike, Adrian Preda, Erin B. Quinlan, Paul E. Rasser, Varun Ratnakar, Simone Reppermund, Vidar M. Steen, Paul A. Tooney, Fábio R. Torres, Dick J. Veltman, James T. Voyvodic, Robert Whelan, Tonya White, Hidenaga Yamamori, Hieab H. H. Adams, Joshua C. Bis, Stephanie Debette, Charles Decarli, Myriam Fornage, Vilmundur Gudnason, Edith Hofer, M. Arfan Ikram, Lenore Launer, W. T. Longstreth, Oscar L. Lopez, Bernard Mazoyer, Thomas H. Mosley, Gennady V. Roshchupkin, Claudia L. Satizabal, Reinhold Schmidt, Sudha Seshadri, Qiong Yang, Alzheimer’s Disease Neuroimaging Initiative, CHARGE Consortium, EPIGEN Consortium, IMAGEN Consortium, SYS Consortium, Parkinson’s Progression Markers Initiative, Marina K. M. Alvim, David Ames, Tim J. Anderson, Ole A. Andreassen, Alejandro Arias-Vasquez, Mark E. Bastin, Bernhard T. Baune, Jean C. Beckham, John Blangero, Dorret I. Boomsma, Henry Brodaty, Han G. Brunner, Randy L. Buckner, Jan K. Buitelaar, Juan R. Bustillo, Wiepke Cahn, Murray J. Cairns, Vince Calhoun, Vaughan J. Carr, Xavier Caseras, Svenja Caspers, Gianpiero L. Cavalleri, Fernando Cendes, Aiden Corvin, Benedicto Crespo-Facorro, John C. Dalrymple-Alford, Udo Dannlowski, Eco J. C. de Geus, Ian J. Deary, Norman Delanty, Chantal Depondt, Sylvane Desrivières, Gary Donohoe, Thomas Espeseth, Guillén Fernández, Simon E. Fisher, Herta Flor, Andreas J. Forstner, Clyde Francks, Barbara Franke, David C. Glahn, Randy L. Gollub, Hans J. Grabe, Oliver Gruber, Asta K. Håberg, Ahmad R. Hariri, Catharina A. Hartman, Ryota Hashimoto, Andreas Heinz, Frans A. Henskens, Manon H. J. Hillegers, Pieter J. Hoekstra, Avram J. Holmes, L. Elliot Hong, William D. Hopkins, Hilleke E. Hulshoff Pol, Terry L. Jernigan, Erik G. Jönsson, René S. Kahn, Martin A. Kennedy, Tilo T. J. Kircher, Peter Kochunov, John B. J. Kwok, Stephanie Le Hellard, Carmel M. Loughland, Nicholas G. Martin, Jean-Luc Martinot, Colm McDonald, Katie L. McMahon, Andreas Meyer-Lindenberg, Patricia T. Michie, Rajendra A. Morey, Bryan Mowry, Lars Nyberg, Jaap Oosterlaan, Roel A. Ophoff, Christos Pantelis, Tomas Paus, Zdenka Pausova, Brenda W. J. H. Penninx, Tinca J. C. Polderman, Danielle Posthuma, Marcella Rietschel, Joshua L. Roffman, Laura M. Rowland, Perminder S. Sachdev, Philipp G. Sämann, Ulrich Schall, Gunter Schumann, Rodney J. Scott, Kang Sim, Sanjay M. Sisodiya, Jordan W. Smoller, Iris E. Sommer, Beate St Pourcain, Dan J. Stein, Arthur W. Toga, Julian N. Trollor, Nic J. A. Van der Wee, Dennis van ’t Ent, Henry Völzke, Henrik Walter, Bernd Weber, Daniel R. Weinberger, Margaret J. Wright, Juan Zhou, Jason L. Stein, Paul M. Thompson, Sarah E. Medland, Enhancing NeuroImaging Genetics through Meta-Analysis Consortium (ENIGMA)—Genetics working group (2020-03-20):
The human cerebral cortex is important for cognition, and it is of interest to see how genetic variants affect its structure. Grasby et al. combined genetic data with brain magnetic resonance imaging from more than 50,000 people to generate a genome-wide analysis of how human genetic variation influences human cortical surface area and thickness. From this analysis, they identified variants associated with cortical structure, some of which affect signaling and gene expression. They observed overlap between genetic loci affecting cortical structure, brain development, and neuropsychiatric disease, and the correlation between these phenotypes is of interest for further study.
Introduction: The cerebral cortex underlies our complex cognitive capabilities. Variations in human cortical surface area and thickness are associated with neurological, psychological, and behavioral traits and can be measured in vivo by magnetic resonance imaging (MRI). Studies in model organisms have identified genes that influence cortical structure, but little is known about common genetic variants that affect human cortical structure.
Rationale: To identify genetic variants associated with human cortical structure at both global and regional levels, we conducted a genome-wide association meta-analysis of brain MRI data from 51,665 individuals across 60 cohorts. We analyzed the surface area and average thickness of the whole cortex and 34 cortical regions with known functional specializations.
Results: We identified 306 nominally genome-wide significant loci (p < 5 × 10−8) associated with cortical structure in a discovery sample of 33,992 participants of European ancestry. Of the 299 loci for which replication data were available, 241 loci influencing surface area and 14 influencing thickness remained significant after replication, with 199 loci passing multiple testing correction (p < 8.3 × 10−10; 187 influencing surface area and 12 influencing thickness).
Common genetic variants explained 34% (SE = 3%) of the variation in total surface area and 26% (SE = 2%) in average thickness; surface area and thickness showed a negative genetic correlation (rg = −0.32, SE = 0.05, p = 6.5 × 10−12), which suggests that genetic influences have opposing effects on surface area and thickness. Bioinformatic analyses showed that total surface area is influenced by genetic variants that alter gene regulatory activity in neural progenitor cells during fetal development. By contrast, average thickness is influenced by active regulatory elements in adult brain samples, which may reflect processes that occur after mid-fetal development, such as myelination, branching, or pruning. When considered together, these results support the radial unit hypothesis that different developmental mechanisms promote surface area expansion and increases in thickness.
To identify specific genetic influences on individual cortical regions, we controlled for global measures (total surface area or average thickness) in the regional analyses. After multiple testing correction, we identified 175 loci that influence regional surface area and 10 that influence regional thickness. Loci that affect regional surface area cluster near genes involved in the Wnt signaling pathway, which is known to influence areal identity.
We observed significant positive genetic correlations and evidence of bidirectional causation of total surface area with both general cognitive functioning and educational attainment. We found additional positive genetic correlations between total surface area and Parkinson’s disease but did not find evidence of causation. Negative genetic correlations were evident between total surface area and insomnia, attention deficit hyperactivity disorder, depressive symptoms, major depressive disorder, and neuroticism.
Conclusion: This large-scale collaborative work enhances our understanding of the genetic architecture of the human cerebral cortex and its regional patterning. The highly polygenic architecture of the cortex suggests that distinct genes are involved in the development of specific cortical areas. Moreover, we find evidence that brain structure is a key phenotype along the causal pathway that leads from genetic variation to differences in general cognitive function.
Genomic prediction of complex human traits (e.g., height, cognitive ability, bone density) and disease risks (e.g., breast cancer, diabetes, heart disease, atrial fibrillation) has advanced considerably in recent years. Predictors have been constructed using penalized algorithms that favor sparsity: i.e., which use as few genetic variants as possible. We analyze the specific genetic variants (SNPs) utilized in these predictors, which can vary from dozens to as many as thirty thousand. We find that the fraction of SNPs in or near genic regions varies widely by phenotype. For the majority of disease conditions studied, a large amount of the variance is accounted for by SNPs outside of coding regions. The state of these SNPs cannot be determined from exome-sequencing data. This suggests that exome data alone will miss much of the heritability for these traits – i.e., existing PRS cannot be computed from exome data alone. We also study the fraction of SNPs and of variance that is in common between pairs of predictors. The DNA regions used in disease risk predictors so far constructed seem to be largely disjoint (with a few interesting exceptions), suggesting that individual genetic disease risks are largely uncorrelated. It seems possible in theory for an individual to be a low-risk outlier in all conditions simultaneously.
Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing outstandingly well on many games of the set, but very poorly in several of the most challenging games. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games. To achieve this result, we train a neural network which parameterizes a family of policies ranging from very exploratory to purely exploitative. We propose an adaptive mechanism to choose which policy to prioritize throughout the training process. Additionally, we utilize a novel parameterization of the architecture that allows for more consistent and stable learning.
The Atari57 suite of games is a long-standing benchmark to gauge agent performance across a wide range of tasks. We’ve developed Agent57, the first deep reinforcement learning agent to obtain a score that is above the human baseline on all 57 Atari 2600 games. Agent57 combines an algorithm for efficient exploration with a meta-controller that adapts the exploration and long vs. short-term behaviour of the agent.
…In 2012, the Arcade Learning environment—a suite of 57 Atari 2600 games (dubbed Atari57)—was proposed as a benchmark set of tasks: these canonical Atari games pose a broad range of challenges for an agent to master….Unfortunately, the average performance can fail to capture how many tasks an agent is doing well on, and so is not a good statistic for determining how general an agent is: it captures that an agent is doing sufficiently well, but not that it is doing sufficiently well on a sufficiently wide set of tasks. So although average scores have increased, until now, the number of above human games has not.
…Back in 2012, DeepMind developed the Deep Q-network agent (DQN) to tackle the Atari57 suite. Since then, the research community has developed many extensions and alternatives to DQN. Despite these advancements, however, all deep reinforcement learning agents have consistently failed to score in four games: Montezuma’s Revenge, Pitfall, Solaris and Skiing. For Agent57 to tackle these four challenging games in addition to the other Atari57 games, several changes to DQN were necessary.
Intrinsic motivation methods to encourage directed exploration
Seeking novelty over long time scales
Seeking novelty over short time scales
Meta-controller: learning to balance exploration with exploitation
Agent57: putting it all together
…With Agent57, we have succeeded in building a more generally intelligent agent that has above-human performance on all tasks in the Atari57 benchmark. It builds on our previous agent Never Give Up, and instantiates an adaptive meta-controller that helps the agent to know when to explore and when to exploit, as well as what time-horizon it would be useful to learn with. A wide range of tasks will naturally require different choices of both of these trade-offs, therefore the meta-controller provides a way to dynamically adapt such choices.
Agent57 was able to scale with increasing amounts of computation: the longer it trained, the higher its score got. While this enabled Agent57 to achieve strong general performance, it takes a lot of computation and time; the data efficiency can certainly be improved. Additionally, this agent shows better 5th percentile performance on the set of Atari57 games. This by no means marks the end of Atari research, not only in terms of data efficiency, but also in terms of general performance. We offer two views on this: firstly, analyzing the performance among percentiles gives us new insights on how general algorithms are. While Agent57 achieves strong results on the first percentiles of the 57 games and holds better mean and median performance than NGU or R2D2, as illustrated by MuZero, it could still obtain a higher average performance. Secondly, all current algorithms are far from achieving optimal performance in some games. To that end, key improvements to use might be enhancements in the representations that Agent57 uses for exploration, planning, and credit assignment.
We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memory-based intrinsic reward using k-nearest neighbors over the agent’s recent experience to train the directed exploratory policies, thereby encouraging the agent to repeatedly revisit all states in its environment. A self-supervised inverse dynamics model is used to train the embeddings of the nearest neighbour lookup, biasing the novelty signal towards what the agent can control. We employ the framework of Universal Value Function Approximators (UVFA) to simultaneously learn many directed exploration policies with the same neural network, with different trade-offs between exploration and exploitation. By using the same neural network for different degrees of exploration/exploitation, transfer is demonstrated from predominantly exploratory policies yielding effective exploitative policies. The proposed method can be incorporated to run with modern distributed RL agents that collect large amounts of experience from many actors running in parallel on separate environment instances. Our method doubles the performance of the base agent in all hard exploration in the Atari-57 suite while maintaining a very high score across the remaining games, obtaining a median human normalised score of 1344.0 achieve non-zero rewards (with a mean score of 8,400) in the game of Pitfall! without using demonstrations or hand-crafted features.
Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess and Go, where a perfect simulator is available. However, in real-world problems the dynamics governing the environment are often complex and unknown. In this work we present the MuZero algorithm which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. MuZero learns a model that, when applied iteratively, predicts the quantities most directly relevant to planning: the reward, the action-selection policy, and the value function. When evaluated on 57 different Atari games—the canonical video game environment for testing AI techniques, in which model-based planning approaches have historically struggled—our new algorithm achieved a new state of the art. When evaluated on Go, chess and shogi, without any knowledge of the game rules, MuZero matched the superhuman performance of the AlphaZero algorithm that was supplied with the game rules.
Machine learning research has advanced in multiple aspects, including model structures and learning methods. The effort to automate such research, known as AutoML, has also made significant progress. However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks—or similarly restrictive search spaces. Our goal is to show that AutoML can go further: it is possible today to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. Despite the vastness of this space, evolutionary search can still discover two-layer neural networks trained by backpropagation. These simple neural networks can then be surpassed by evolving directly on tasks of interest, e.g. CIFAR-10 variants, where modern techniques emerge in the top algorithms, such as bilinear interactions, normalized gradients, and weight averaging. Moreover, evolution adapts algorithms to different task types: e.g., dropout-like techniques appear when little data is available. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction for the field.
AutoML-Zero aims to automatically discover computer programs that can solve machine learning tasks, starting from empty or random programs and using only basic math operations. The goal is to simultaneously search for all aspects of an ML algorithm—including the model structure and the learning strategy—while employing minimal human bias.
Despite AutoML-Zero’s challenging search space, evolutionary search shows promising results by discovering linear regression with gradient descent, 2-layer neural networks with backpropagation, and even algorithms that surpass hand designed baselines of comparable complexity. The figure above shows an example sequence of discoveries from one of our experiments, evolving algorithms to solve binary classification tasks. Notably, the evolved algorithms can be interpreted. Below is an analysis of the best evolved algorithm: the search process “invented” techniques like bilinear interactions, weight averaging, normalized gradient, and data augmentation (by adding noise to the inputs).
More examples, analysis, and details can be found in the paper.
One defense of free markets notes the inability of non-market mechanisms to solve planning & optimization problems. This has difficulty with Coase’s paradox of the firm, and I note that the difficulty is increased by the fact that with improvements in computers, algorithms, and data, ever larger planning problems are solved. Expanding on some Cosma Shalizi comments, I suggest interpreting phenomenon as multi-level nested optimization paradigm: many systems can be usefully described as having two (or more) levels where a slow sample-inefficient but ground-truth ‘outer’ loss such as death, bankruptcy, or reproductive fitness, trains & constrains a fast sample-efficient but possibly misguided ‘inner’ loss which is used by learned mechanisms such as neural networks or linear programming group selection perspective. So, one reason for free-market or evolutionary or Bayesian methods in general is that while poorer at planning/optimization in the short run, they have the advantage of simplicity and operating on ground-truth values, and serve as a constraint on the more sophisticated non-market mechanisms. I illustrate by discussing corporations, multicellular life, reinforcement learning & meta-learning in AI, and pain in humans. This view suggests that are inherent balances between market/non-market mechanisms which reflect the relative advantages between a slow unbiased method and faster but potentially arbitrarily biased methods.
“15.ai”, 15 & the Pony Preservation Project (2020-03-06):
[NN TTS service demonstrating results from custom DL research project by 15 for generating natural high-quality voices of characters with minimal data/few-shot learning; available voices include GLaDOS from Portal and especially high-quality My Little Pony: Friendship Is Magic voices (currently: Fluttershy & Twilight Sparkle); demos: 1/2.
The MLP:FiM voices are trained on a large dataset constructed by the 4chan crowdsourced project “Pony Preservation Project”, begun ~2019. PPP has crowdsourced parsed audio and hand-written transcriptions of all dialogue for all character from all 9 MLP:FiM seasons, the movie, the spinoffs, and various other things voiced by the same voice actresses in case that might help, while processing to remove noise or using ‘leaked’ original data from Hasbro for higher quality still.]
This is a text-to-speech tool that you can use to generate 44.1 kHz voices of various characters. The voices are generated in real time using multiple audio synthesis algorithms and customized deep neural networks trained on very little available data (between 30 and 120 minutes of clean dialogue for each character). This project demonstrates a significant reduction in the amount of audio required to realistically clone voices while retaining their affective prosodies.
I plan to keep this tool up gratis and ad-free indefinitely. This website is intended for strictly non-commercial use.
Thanks to the MIT Computer Science & Artificial Intelligence Laboratory (CSAIL) for providing the initial funding that kickstarted this project two years ago. Further thanks to the Julia Lab, Lincoln Lab, and the Media Lab.
Special shoutouts go to 4chan’s /mlp/ and the anons who have collectively spent hundreds of hours collecting, cleaning, and organizing clips of dialogue taken from the show My Little Pony: Friendship Is Magic. Honorable mention to /g/ for some entertaining speculations.
And of course, nothing but the utmost respect to the voice actors who originally voiced the characters.
[Compilation of 29 videos & ~25 audio files created using a new neural network service for voice synthesis of various characters, particularly My Little Pony characters; scripts include everything from every Star Wars opening to F1 car racing commentary to the “Who’s on First?” Abbott & Costello comedy dialogue to 1 hour recitation of π to the Dune Litany Against Fear & Blade Runner Tears in the Rain monologue.]
Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference. We study the impact of model size in this setting, focusing on Transformer models for NLP tasks that are limited by compute: self-supervised pretraining and high-resource machine translation. We first show that even though smaller Transformer models execute faster per iteration, wider and deeper models converge in significantly fewer steps. Moreover, this acceleration in convergence typically outpaces the additional computational overhead of using larger models. Therefore, the most compute-efficient training strategy is to counterintuitively train extremely large models but stop after a small number of iterations.
This leads to an apparent trade-off between the training efficiency of large Transformer models and the inference efficiency of small Transformer models. However, we show that large models are more robust to compression techniques such as quantization and pruning than small models. Consequently, one can get the best of both worlds: heavily compressed, large models achieve higher accuracy than lightly compressed, small models.
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth have minimal effects within a wide range. Simple equations govern the dependence of overfitting on model/dataset size and the dependence of training speed on model size. These relationships allow us to determine the optimal allocation of a fixed compute budget. Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
Neural networks appear to have mysterious generalization properties when using parameter counting as a proxy for complexity. Indeed, neural networks often have many more parameters than there are data points, yet still provide good generalization performance. Moreover, when we measure generalization as a function of parameters, we see double descent behaviour, where the test error decreases, increases, and then again decreases. We show that many of these properties become understandable when viewed through the lens of effective dimensionality, which measures the dimensionality of the parameter space determined by the data. We relate effective dimensionality to posterior contraction in Bayesian deep learning, model selection, width-depth tradeoffs, double descent, and functional diversity in loss surfaces, leading to a richer understanding of the interplay between parameters and functions in deep models. We also show that effective dimensionality compares favourably to alternative norm- and flatness- based generalization measures.
The key distinguishing property of a Bayesian approach is marginalization, rather than using a single setting of weights. Bayesian marginalization can particularly improve the accuracy and calibration of modern deep neural networks, which are typically underspecified by the data, and can represent many compelling but different solutions. We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization, and propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction, without significant overhead. We also investigate the prior over functions implied by a vague distribution over neural network weights, explaining the generalization properties of such models from a probabilistic perspective. From this perspective, we explain results that have been presented as mysterious and distinct to neural network generalization, such as the ability to fit images with random labels, and show that these results can be reproduced with Gaussian processes. We also show that Bayesian model averaging alleviates double descent, resulting in monotonic performance improvements with increased flexibility. Finally, we provide a Bayesian perspective on tempering for calibrating predictive distributions.
Identifiability is a desirable property of a statistical model: it implies that the true model parameters may be estimated to any desired precision, given sufficient computational resources and data. We study identifiability in the context of representation learning: discovering nonlinear data representations that are optimal with respect to some downstream task. When parameterized as deep neural networks, such representation functions typically lack identifiability in parameter space, because they are overparameterized by design. In this paper, building on recent advances in nonlinear ICA, we aim to rehabilitate identifiability by showing that a large family of discriminative models are in fact identifiable in function space, up to a linear indeterminacy. Many models for representation learning in a wide variety of domains have been identifiable in this sense, including text, images and audio, state-of-the-art at time of publication. We derive sufficient conditions for linear identifiability and provide empirical support for the result on both simulated and real-world data.
In 2016, the US Defense Advanced Research Projects Agency (DARPA) told eight research groups that their proposals had made it through the review gauntlet and would soon get a few million dollars from its Biological Technologies Office (BTO). Along with congratulations, the teams received a reminder that their award came with an unusual requirement—an independent shadow team of scientists tasked with reproducing their results. Thus began an intense, multi-year controlled trial in reproducibility. Each shadow team consists of three to five researchers, who visit the ‘performer’ team’s laboratory and often host visits themselves. Between 3% and 8% of the programme’s total funds go to this independent validation and verification (IV&V) work…Awardees were told from the outset that they would be paired with an IV&V team consisting of unbiased, third-party scientists hired by and accountable to DARPA. In this programme, we relied on US Department of Defense laboratories, with specific teams selected for their technical competence and ability to solve problems creatively.
…Results so far show a high degree of experimental reproducibility. The technologies investigated include using chemical triggers to control how cells migrate1; introducing synthetic circuits that control other cell functions2; intricate protein switches that can be programmed to respond to various cellular conditions3; and timed bacterial expression that works even in the variable environment of the mammalian gut4…getting to this point was more difficult than we expected. It demanded intense coordination, communication and attention to detail…Our effort needed capable research groups that could dedicate much more time (in one case, 20 months) and that could flexibly follow evolving research…A key component of the IV&V teams’ effort has been to spend a day or more working with the performer teams in their laboratories. Often, members of a performer laboratory travel to the IV&V laboratory as well. These interactions lead to a better grasp of methodology than reading a paper, frequently revealing person-to-person differences that can affect results…Still, our IV&V efforts have been derailed for weeks at a time for trivial reasons (see ‘Hard lessons’), such as a typo that meant an ingredient in cell media was off by an order of magnitude. We lost more than a year after discovering that commonly used biochemicals that were thought to be interchangeable are not.
Document Reagents:…We lost weeks of work and performed useless experiments when we assumed that identically named reagents (for example, polyethylene glycol or fetal bovine serum) from different vendors could be used interchangeably. · See It Live:…In our hands, washing cells too vigorously or using the wrong-size pipette tip changed results unpredictably. · State a range: …Knowing whether 21 ° C means 20.5–21.5 ° C or 20–22 ° C can tell you whether cells will thrive or wither, and whether you’ll need to buy an incubator to make an experiment work. · Test, then ship: …Incorrect, outdated or otherwise diminished products were sent to the IV&V team for verification many times. · Double check: ….A typo in one protocol cost us four weeks of failed experiments, and in general, vague descriptions of formulation protocols (for example, for expressing genes and making proteins without cells) caused months of delay and cost thousands of dollars in wasted reagents. · Pick a person: …The projects that lacked a dedicated and stable point of contact were the same ones that took the longest to reproduce. That is not coincidence. · Keep in silico analysis up to date: …Teams had to visit each others’ labs more than once to understand and fully implement computational-analysis pipelines for large microscopy data sets.
…We have learnt to note the flow rates used when washing cells from culture dishes, to optimize salt concentration in each batch of medium and to describe temperature and other conditions with a range rather than a single number. This last practice came about after we realized that diminished slime-mould viability in our Washington DC facility was due to lab temperatures that could fluctuate by 2 °C on warm summer days, versus the more tightly controlled temperature of the performer lab in Baltimore 63 kilometres away. Such observations can be written up in a protocol paper…As one of our scientists said, “IV&V forces performers to think more critically about what qualifies as a successful system, and facilitates candid discussion about system performance and limitations.”
The diverse migratory modes displayed by different cell types are generally believed to be idiosyncratic. Here we show that the migratory behaviour of Dictyostelium was switched from amoeboid to keratocyte-like and oscillatory modes by synthetically decreasing phosphatidylinositol-4,5-bisphosphate levels or increasing Ras/Rap-related activities. The perturbations at these key nodes of an excitable signal transduction network initiated a causal chain of events: the threshold for network activation was lowered, the speed and range of propagating waves of signal transduction activity increased, actin-driven cellular protrusions expanded and, consequently, the cell migratory mode transitions ensued. Conversely, innately keratocyte-like and oscillatory cells were promptly converted to amoeboid by inhibition of Ras effectors with restoration of directed migration. We use computational analysis to explain how thresholds control cell migration and discuss the architecture of the signal transduction network that gives rise to excitability.
De novo-designed proteins1,2,3 hold great promise as building blocks for synthetic circuits, and can complement the use of engineered variants of natural proteins4,5,6,7. One such designer protein—degronLOCKR, which is based on ‘latching orthogonal cage-key proteins’ (LOCKR) technology8—is a switch that degrades a protein of interest in vivo upon induction by a genetically encoded small peptide. Here we leverage the plug-and-play nature of degronLOCKR to implement feedback control of endogenous signalling pathways and synthetic gene circuits. We first generate synthetic negative and positive feedback in the yeast mating pathway by fusing degronLOCKR to endogenous signalling molecules, illustrating the ease with which this strategy can be used to rewire complex endogenous pathways. We next evaluate feedback control mediated by degronLOCKR on a synthetic gene circuit9, to quantify the feedback capabilities and operational range of the feedback control circuit. The designed nature of degronLOCKR proteins enables simple and rational modifications to tune feedback behaviour in both the synthetic circuit and the mating pathway. The ability to engineer feedback control into living cells represents an important milestone in achieving the full potential of synthetic biology10,11,12. More broadly, this work demonstrates the large and untapped potential of de novo design of proteins for generating tools that implement complex synthetic functionalities in cells for biotechnological and therapeutic applications.
“De novo design of bioactive protein switches”, Robert A. Langan, Scott E. Boyken, Andrew H. Ng, Jennifer A. Samson, Galen Dods, Alexandra M. Westbrook, Taylor H. Nguyen, Marc J. Lajoie, Zibo Chen, Stephanie Berger, Vikram Khipple Mulligan, John E. Dueber, Walter R. P. Novak, Hana El-Samad, David Baker (2019-07-24):
Allosteric regulation of protein function is widespread in biology, but is challenging for de novo protein design as it requires the explicit design of multiple states with comparable free energies. Here we explore the possibility of designing switchable protein systems de novo, through the modulation of competing intermolecular and intramolecular interactions. We design a static, five-helix ‘cage’ with a single interface that can interact either intramolecularly with a terminal ‘latch’ helix or intermolecularly with a peptide ‘key’. Encoded on the latch are functional motifs for binding, degradation or nuclear export that function only when the key displaces the latch from the cage. We describe orthogonal cage-key systems that function in vitro, in yeast and in mammalian cells with up to 40-fold activation of function by key. The ability to design switchable protein functions that are controlled by induced conformational change is a milestone for de novo protein design, and opens up new avenues for synthetic biology and cell engineering.
Synthetic gene oscillators have the potential to control timed functions and periodic gene expression in engineered cells. Such oscillators have been refined in bacteria in vitro, however, these systems have lacked the robustness and precision necessary for applications in complex in vivo environments, such as the mammalian gut. Here, we demonstrate the implementation of a synthetic oscillator capable of keeping robust time in the mouse gut over periods of days. The oscillations provide a marker of bacterial growth at a single-cell level enabling quantification of bacterial dynamics in response to inflammation and underlying variations in the gut microbiota. Our work directly detects increased bacterial growth heterogeneity during disease and differences between spatial niches in the gut, demonstrating the deployment of a precise engineered genetic oscillator in real-life settings.
Snorkelers in mangrove forest waters inhabited by the upside-down jellyfish Cassiopea xamachana report discomfort due to a sensation known as stinging water, the cause of which is unknown. Using a combination of histology, microscopy, microfluidics, videography, molecular biology, and mass spectrometry-based proteomics, we describe C. xamachana stinging-cell structures that we term cassiosomes. These structures are released within C. xamachana mucus and are capable of killing prey. Cassiosomes consist of an outer epithelial layer mainly composed of nematocytes surrounding a core filled by endosymbiotic dinoflagellates hosted within amoebocytes and presumptive mesoglea. Furthermore, we report cassiosome structures in four additional jellyfish species in the same taxonomic group as C. xamachana (Class Scyphozoa; Order Rhizostomeae), categorized as either motile (ciliated) or nonmotile types. This inaugural study provides a qualitative assessment of the stinging contents of C. xamachana mucus and implicates mucus containing cassiosomes and free intact nematocytes as the cause of stinging water.
About 15 years ago, one of us (G.J.L.) got an uncomfortable phone call from a colleague and collaborator. After nearly a year of frustrating experiments, this colleague was about to publish a paper1 chronicling his team’s inability to reproduce the results of our high-profile paper2 in a mainstream journal. Our study was the first to show clearly that a drug-like molecule could extend an animal’s lifespan. We had found over and over again that the treatment lengthened the life of a roundworm by as much as 67%. Numerous phone calls and e-mails failed to identify why this apparently simple experiment produced different results between the labs. Then another lab failed to replicate our study. Despite more experiments and additional publications, we couldn’t work out why the labs were getting different lifespan results. To this day, we still don’t know. A few years later, the same scenario played out with different compounds in other labs….In another, now-famous example, two cancer labs spent more than a year trying to understand inconsistencies6. It took scientists working side by side on the same tumour biopsy to reveal that small differences in how they isolated cells—vigorous stirring versus prolonged gentle rocking—produced different results. Subtle tinkering has long been important in getting biology experiments to work. Before researchers purchased kits of reagents for common experiments, it wasn’t unheard of for a team to cart distilled water from one institution when it moved to another. Lab members would spend months tweaking conditions until experiments with the new institution’s water worked as well as before. Sources of variation include the quality and purity of reagents, daily fluctuations in microenvironment and the idiosyncratic techniques of investigators7. With so many ways of getting it wrong, perhaps we should be surprised at how often experimental findings are reproducible.
…Nonetheless, scores of publications continued to appear with claims about compounds that slow ageing. There was little effort at replication. In 2013, the three of us were charged with that unglamorous task…Our first task, to develop a protocol, seemed straightforward.
But subtle disparities were endless. In one particularly painful teleconference, we spent an hour debating the proper procedure for picking up worms and placing them on new agar plates. Some batches of worms lived a full day longer with gentler technicians. Because a worm’s lifespan is only about 20 days, this is a big deal. Hundreds of e-mails and many teleconferences later, we converged on a technique but still had a stupendous three-day difference in lifespan between labs. The problem, it turned out, was notation—one lab determined age on the basis of when an egg hatched, others on when it was laid. We decided to buy shared batches of reagents from the start. Coordination was a nightmare; we arranged with suppliers to give us the same lot numbers and elected to change lots at the same time. We grew worms and their food from a common stock and had strict rules for handling. We established protocols that included precise positions of flasks in autoclave runs. We purchased worm incubators at the same time, from the same vendor. We also needed to cope with a large amount of data going from each lab to a single database. We wrote an iPad app so that measurements were entered directly into the system and not jotted on paper to be entered later. The app prompted us to include full descriptors for each plate of worms, and ensured that data and metadata for each experiment were proofread (the strain names MY16 and my16 are not the same). This simple technology removed small recording errors that could disproportionately affect statistical analyses.
Once this system was in place, variability between labs decreased. After more than a year of pilot experiments and discussion of methods in excruciating detail, we almost completely eliminated systematic differences in worm survival across our labs9 (see ‘Worm wonders’)…Even in a single lab performing apparently identical experiments, we could not eliminate run-to-run differences.
…We have found one compound that lengthens lifespan across all strains and species. Most do so in only two or three strains, and often show detrimental effects in others.
“Impact of genetic background and experimental reproducibility on identifying chemical compounds with robust longevity effects”, Mark Lucanic, W. Todd Plummer, Esteban Chen, Jailynn Harke, Anna C. Foulger, Brian Onken, Anna L. Coleman-Hulbert, Kathleen J. Dumas, Suzhen Guo, Erik Johnson, Dipa Bhaumik, Jian Xue, Anna B. Crist, Michael P. Presley, Girish Harinath, Christine A. Sedore, Manish Chamoli, Shaunak Kamat, Michelle K. Chen, Suzanne Angeli, Christina Chang, John H. Willis, Daniel Edgar, Mary Anne Royal, Elizabeth A. Chao, Shobhna Patel, Theo Garrett, Carolina Ibanez-Ventoso, June Hope, Jason L. Kish, Max Guo, Gordon J. Lithgow, Monica Driscoll, Patrick C. Phillips (2017-02-21):
Limiting the debilitating consequences of ageing is a major medical challenge of our time. Robust pharmacological interventions that promote healthy ageing across diverse genetic backgrounds may engage conserved longevity pathways. Here we report results from the Caenorhabditis Intervention Testing Program in assessing longevity variation across 22 Caenorhabditis strains spanning 3 species, using multiple replicates collected across three independent laboratories. Reproducibility between test sites is high, whereas individual trial reproducibility is relatively low. Of ten pro-longevity chemicals tested, six significantly extend lifespan in at least one strain. Three reported dietary restriction mimetics are mainly effective across C. elegans strains, indicating species and strain-specific responses. In contrast, the amyloid dye ThioflavinT is both potent and robust across the strains. Our results highlight promising pharmacological leads and demonstrate the importance of assessing lifespans of discrete cohorts across repeat studies to capture biological variation in the search for reproducible ageing interventions.
[Brutal, lengthy memoir of 6 years as a computer science/software engineering grad student at Stanford University. As positively as the author regards his experience, it comes off as a nightmarish publish-or-perish dystopia where professors burn through naive idealistic grad students doing grunt-work in an endless death-march towards conference deadlines and where marketing is far more important than merit (“sell, sell, sell”), peer reviewers are sadistic rolls of dice and reject papers for superficial problems like not using the exact jargon of a subfield; the software used is filled with endless bugs and takes months to be hacked into shape, never to be used in the real world, and even the original authors can’t get it to work a second time. Many students pursue a promising idea only for it to not work out, and wash out of the field—with so many people chasing so few academic positions, anything short of enormous success is a fatal failure. The notes added in 2015 as a followup, recounting the fate of various grad students or assistant professors, reinforce the daunting odds against a intellectually-satisfying career in academia. It is unsurprising that so many grad students appear to have minor mental breakdowns like him. Strikingly, his by far most successful year was the one spent outside academia, at Microsoft Research. Guo provides these lessons:
[Memoir of an ex-theoretical-physics grad student at the University of Rochester with Sarada Rajeev who gradually became disillusioned with physics research, burned out, and left to work in finance and is now a writer. Henderson was attracted by the life of the mind and the grandeur of uncovering the mysteries of the universe, only to discover that, after the endless triumphs of the 20th century and predicting enormous swathes of empirical experimental data, theoretical physics has drifted and become a branch of abstract mathematics, exploring ever more recondite, simplified, and implausible models in the hopes of obtaining any insight into physics’ intractable problems; one must be brilliant to even understand the questions being asked by the math and incredibly hardworking to make any progress which hasn’t already been tried by even more brilliant physicists of the past (while living in ignominious poverty and terror of not getting a grant or tenure), but one’s entire career may be spent chasing a useless dead end without one having any clue.]
The next thing I knew I was crouched in a chair in Rajeev’s little office, with a notebook on my knee and focused with everything I had on an impromptu lecture he was giving me on an esoteric aspect of some mathematical subject I’d never heard of before. Zeta functions, or elliptic functions, or something like that. I’d barely introduced myself when he’d started banging out equations on his board. Trying to follow was like learning a new game, with strangely shaped pieces and arbitrary rules. It was a challenge, but I was excited to be talking to a real physicist about his real research, even though there was one big question nagging me that I didn’t dare to ask: What does any of this have to do with physics?
…Even a Theory of Everything, I started to realize, might suffer the same fate of multiple interpretations. The Grail could just be a hall of mirrors, with no clear answer to the “What?” or the “How?”—let alone the “Why?” Plus physics had changed since Big Al bestrode it. Mathematical as opposed to physical intuition had become more central, partly because quantum mechanics was such a strange multi-headed beast that it diminished the role that everyday, or even Einstein-level, intuition could play. So much for my dreams of staring out windows and into the secrets of the universe.
…If I did lose my marbles for a while, this is how it started. With cutting my time outside of Bausch and Lomb down to nine hours a day—just enough to pedal my mountain bike back to my bat cave of an apartment each night, sleep, shower, and pedal back in. With filling my file cabinet with boxes and cans of food, and carting in a coffee maker, mini-fridge, and microwave so that I could maximize the time spent at my desk. With feeling guilty after any day that I didn’t make my 15-hour quota. And with exceeding that quota frequently enough that I regularly circumnavigated the clock: staying later and later each night until I was going home in the morning, then in the afternoon, and finally at night again.
…The longer and harder I worked, the more I realized I didn’t know. Papers that took days or weeks to work through cited dozens more that seemed just as essential to digest; the piles on my desk grew rather than shrunk. I discovered the stark difference between classes and research: With no syllabus to guide me I didn’t know how to keep on a path of profitable inquiry. Getting “wonderfully lost” sounded nice, but the reality of being lost, and of re-living, again and again, that first night in the old woman’s house, with all of its doubts and dead-ends and that horrible hissing voice was … something else. At some point, flipping the lights on in the library no longer filled me with excitement but with dread.
…My mental model building was hitting its limits. I’d sit there in Rajeev’s office with him and his other students, or in a seminar given by some visiting luminary, listening and putting each piece in place, and try to fix in memory what I’d built so far. But at some point I’d lose track of how the green stick connected to the red wheel, or whatever, and I’d realize my picture had diverged from reality. Then I’d try toggling between tracing my steps back in memory to repair my mistake and catching all the new pieces still flying in from the talk. Stray pieces would fall to the ground. My model would start falling down. And I would fall hopelessly behind. A year or so of research with Rajeev, and I found myself frustrated and in a fog, sinking deeper into the quicksand but not knowing why. Was it my lack of mathematical background? My grandiose goals? Was I just not intelligent enough?
…I turned 30 during this time and the milestone hit me hard. I was nearly four years into the Ph.D. program, and while my classmates seemed to be systematically marching toward their degrees, collecting data and writing papers, I had no thesis topic and no clear path to graduation. My engineering friends were becoming managers, getting married, buying houses. And there I was entering my fourth decade of life feeling like a pitiful and penniless mole, aimlessly wandering dark empty tunnels at night, coming home to a creepy crypt each morning with nothing to show for it, and checking my bed for bugs before turning out the lights…As I put the final touches on my thesis, I weighed my options. I was broke, burned out, and doubted my ability to go any further in theoretical physics. But mostly, with The Grail now gone and the physics landscape grown so immense, I thought back to Rajeev’s comment about knowing which problems to solve and realized that I still didn’t know what, for me, they were.
But lately, Gates has been obsessing over a dark question: what’s likeliest to kill more than 10 million human beings in the next 20 years? He ticks off the disaster movie stuff—“big volcanic explosion, gigantic earthquake, asteroid”—but says the more he learns about them, the more he realizes the probability is “very low.” Then there’s war, of course. But Gates isn’t that worried about war because the entire human race worries about war pretty much all the time, and the most dangerous kind of war, nuclear war, seems pretty contained, at least for now.
But there’s something out there that’s as bad as war, something that kills as many people as war, and Gates doesn’t think we’re ready for it. “Look at the death chart of the 20th century,” he says, because he’s the kind of guy that looks at death charts. “I think everybody would say there must be a spike for World War I. Sure enough, there it is, like 25 million. And there must be a big spike for World War II, and there it is, it’s like 65 million. But then you’ll see this other spike that is as large as World War II right after World War I, and most people, would say, ‘What was that?’” “Well, that was the Spanish flu.”
No one can say we weren’t warned. And warned. And warned. A pandemic disease is the most predictable catastrophe in the history of the human race, if only because it has happened to the human race so many, many times before…“You can’t use the word lucky or fortunate about something like Ebola that killed 10,000 people,” Klain says. “But it was the most favorable scenario for the world to face one of these things. Ebola is very difficult to transmit. Everyone who is contagious has a visible symptom. It broke out in three relatively small countries that don’t send many travelers to the US. And those three countries have good relationships with America and were welcoming of Western aid.” “With a pandemic flu, the disease would be much more contagious than Ebola,” Klain continues. “The people who are contagious may not have visible symptoms. It could break out in a highly populous country that sends thousands of travelers a day to the US. It could be a country with megacities with tens of millions of people. And it could be a country where sending in the 101st Airborne isn’t possible.”
…Behind Gates’s fear of pandemic disease is an algorithmic model of how disease moves through the modern world. He funded that model to help with his foundation’s work eradicating polio. But then he used it to look into how a disease that acted like the Spanish flu of 1918 would work in today’s world. The results were shocking, even to Gates. “Within 60 days it’s basically in all urban centers around the entire globe,” he says. “That didn’t happen with the Spanish flu.”
Dr. Helen Y. Chu, an infectious disease expert in Seattle, knew that the United States did not have much time…As luck would have it, Dr. Chu had a way to monitor the region. For months, as part of a research project into the flu, she and a team of researchers had been collecting nasal swabs from residents experiencing symptoms throughout the Puget Sound region. To repurpose the tests for monitoring the coronavirus, they would need the support of state and federal officials. But nearly everywhere Dr. Chu turned, officials repeatedly rejected the idea, interviews and emails show, even as weeks crawled by and outbreaks emerged in countries outside of China, where the infection began.
By Feb. 25, Dr. Chu and her colleagues could not bear to wait any longer. They began performing coronavirus tests, without government approval. What came back confirmed their worst fear…In fact, officials would later discover through testing, the virus had already contributed to the deaths of two people, and it would go on to kill 20 more in the Seattle region over the following days.
Federal and state officials said the flu study could not be repurposed because it did not have explicit permission from research subjects; the labs were also not certified for clinical work. While acknowledging the ethical questions, Dr. Chu and others argued there should be more flexibility in an emergency during which so many lives could be lost. On Monday night, state regulators told them to stop testing altogether…Later that day, the investigators and Seattle health officials gathered with representatives of the C.D.C. and the F.D.A. to discuss what happened. The message from the federal government was blunt. “What they said on that phone call very clearly was cease and desist to Helen Chu,” Dr. Lindquist remembered. “Stop testing.”
…Even now, after weeks of mounting frustration toward federal agencies over flawed test kits and burdensome rules, states with growing cases such as New York and California are struggling to test widely for the coronavirus. The continued delays have made it impossible for officials to get a true picture of the scale of the growing outbreak, which has now spread to at least 36 states and Washington, D.C…But the Seattle Flu Study illustrates how existing regulations and red tape—sometimes designed to protect privacy and health—have impeded the rapid rollout of testing nationally, while other countries ramped up much earlier and faster.
…The flu project primarily used research laboratories, not clinical ones, and its coronavirus test was not approved by the Food and Drug Administration. And so the group was not certified to provide test results to anyone outside of their own investigators. They began discussions with state, C.D.C. and F.D.A. officials to figure out a solution, according to emails and interviews…the F.D.A. could not offer the approval because the lab was not certified as a clinical laboratory under regulations established by the Centers for Medicare & Medicaid Services, a process that could take months. Dr. Chu and Dr. Lindquist tried repeatedly to wrangle approval to use the Seattle Flu Study. The answers were always no. “We felt like we were sitting, waiting for the pandemic to emerge,” Dr. Chu said. “We could help. We couldn’t do anything.”…“This virus is faster than the F.D.A.,” he said, adding that at one point the agency required him to submit materials through the mail in addition to over email.
…On a phone call the day after the C.D.C. and F.D.A. had told Dr. Chu to stop, officials relented, but only partially, the researchers recalled. They would allow the study’s laboratories to test cases and report the results only in future samples. They would need to use a new consent form that explicitly mentioned that results of the coronavirus tests might be shared with the local health department. They were not to test the thousands of samples that had already been collected.
In politics, a noble lie is a myth or untruth, often, but not invariably, of a religious nature, knowingly propagated by an elite to maintain social harmony or to advance an agenda. The noble lie is a concept originated by Plato as described in the Republic.
The tragedy of the anticommons is a type of coordination breakdown, in which a single resource has numerous rightsholders who prevent others from using it, frustrating what would be a socially desirable outcome. It is a mirror-image of the older concept of tragedy of the commons, in which numerous rights holders' combined use exceeds the capacity of a resource and depletes or destroys it. The "tragedy of the anticommons" covers a range of coordination failures including patent thickets, and submarine patents. Overcoming these breakdowns can be difficult, but there are assorted means, including eminent domain, laches, patent pools, or other licensing organizations.
Extensive paraphrase summary of Herbert Hoover: while remembered solely as one of the worse American presidents because of the Great Depression, Hoover had a remarkable life: he rose from grinding poverty to the first student at Stanford University (later a trustee) to becoming a mining magnate after revamping Australia & China (the latter in the midst of the Boxer Rebellion) and penning a definitive mining textbook. Along the way, he invented a popular CrossFit medicine ball exercise, relieved the worst flood disaster in American history, organized the evacuation of Americans trapped by the outbreak of WWI and then reorganized American agriculture for WWI…
Hoover, in the service of the highest goods, ruthlessly crushes all opposition, shamelessly exploits PR tactics to the maximum extent, lies and deceives his negotiating partners, and bankrupts himself—and he succeeds, becoming arguably one of the greatest philanthropists in history for organizing repeated famine reliefs in Europe and Communist Russia after.
A shockingly competent technocrat and now regarded as one of the greatest men in the world, he succeeds Coolidge and attempts to forestall the looming Great Depression, and then takes unprecedented action to stop it; while he ultimately fails, he initially seemed like he was succeeding, and it may be bad luck plus the deliberate sabotage of his efforts by President-elect Franklin Roosevelt which prolonged the Great Depression. Embittered, he spends the rest of his life inveighing against FDR and the New Deal, founding modern conservatism.
Alexander ponders why Hoover, who was so unarguably competent at everything he turned his hand to, achieving impossible feats of management and logistics, appears to have failed when he became President at stopping the Great Depression or being re-elected, and what we can learn about philanthropy from him.
Herbert Clark Hoover was an American politician, businessman, and engineer, who served as the 31st president of the United States from 1929 to 1933. A member of the Republican Party, he held office during the onset of the Great Depression. Before serving as president, Hoover led the Commission for Relief in Belgium, served as the director of the U.S. Food Administration, and served as the third U.S. Secretary of Commerce.
Do voters effectively hold elected officials accountable for policy decisions? Using data on natural disasters, government spending, and election returns, we show that voters reward the incumbent presidential party for delivering disaster relief spending, but not for investing in disaster preparedness spending. These inconsistencies distort the incentives of public officials, leading the government to underinvest in disaster preparedness, thereby causing substantial public welfare losses. We estimate that $1 spent on preparedness is worth about $15 in terms of the future damage it mitigates. By estimating both the determinants of policy decisions and the consequences of those policies, we provide more complete evidence about citizen competence and government accountability.
In the aftermath of many natural and man-made disasters, people often wonder why those affected were underprepared, especially when the disaster was the result of known or regularly occurring hazards (e.g., hurricanes). We study one contributing factor: prior near-miss experiences. Near misses are events that have some nontrivial expectation of ending in disaster but, by chance, do not. We demonstrate that when near misses are interpreted as disasters that did not occur, people illegitimately underestimate the danger of subsequent hazardous situations and make riskier decisions (e.g., choosing not to engage in mitigation activities for the potential hazard). On the other hand, if near misses can be recognized and interpreted as disasters that almost happened, this will counter the basic “near-miss” effect and encourage more mitigation. We illustrate the robustness of this pattern across populations with varying levels of real expertise with hazards and different hazard contexts (household evacuation for a hurricane, Caribbean cruises during hurricane season, and deep-water oil drilling). We conclude with ideas to help people manage and communicate about risk. [Keywords: near miss; risk; decision making; natural disasters; organizational hazards; hurricanes; oil spills.]
Although redditors didn’t yet know it, Huffman could edit any part of the site. He wrote a script that would automatically replace his username with those of The_Donald’s most prominent members, directing the insults back at the insulters in real time: in one comment, “Fuck u/Spez” became “Fuck u/Trumpshaker”; in another, “Fuck u/Spez” became “Fuck u/MAGAdocious.” The_Donald’s users saw what was happening, and they reacted by spinning a conspiracy theory that, in this case, turned out to be true. “Manipulating the words of your users is fucked,” a commenter wrote. “Even Facebook and Twitter haven’t stooped this low.” “Trust nothing.”
…In October, on the morning the new policy was rolled out, Ashooh sat at a long conference table with a dozen other employees. Before each of them was a laptop, a mug of coffee, and a few hours’ worth of snacks. “Welcome to the Policy Update War Room,” she said. “And, yes, I’m aware of the irony of calling it a war room when the point is to make Reddit less violent, but it’s too late to change the name.” The job of policing Reddit’s most pernicious content falls primarily to three groups of employees—the community team, the trust-and-safety team, and the anti-evil team—which are sometimes described, respectively, as good cop, bad cop, and RoboCop. Community stays in touch with a cross-section of redditors, asking them for feedback and encouraging them to be on their best behavior. When this fails and redditors break the rules, trust and safety punishes them. Anti-evil, a team of back-end engineers, makes software that flags dodgy-looking content and sends that content to humans, who decide what to do about it.
Ashooh went over the plan for the day. All at once, they would replace the old policy with the new policy, post an announcement explaining the new policy, warn a batch of subreddits that they were probably in violation of the new policy, and ban another batch of subreddits that were flagrantly, irredeemably in violation. I glanced at a spreadsheet with a list of the hundred and nine subreddits that were about to be banned (r/KKK, r/KillAllJews, r/KilltheJews, r/KilltheJoos), followed by the name of the employee who would carry out each deletion, and, if applicable, the reason for the ban (“mostly just swastikas?”). “Today we’re focusing on a lot of Nazi stuff and bestiality stuff,” Ashooh said. “Context matters, of course, and you shouldn’t get in trouble for posting a swastika if it’s a historical photo from the 1936 Olympics, or if you’re using it as a Hindu symbol. But, even so, there’s a lot that’s clear-cut.” I asked whether the same logic—that the Nazi flag was an inherently violent symbol—would apply to the Confederate flag, or the Soviet flag, or the flag under which King Richard fought the Crusades. “We can have those conversations in the future,” Ashooh said. “But we have to start somewhere.”
At 10 a.m., the trust-and-safety team posted the announcement and began the purge. “Thank you for letting me do DylannRoofInnocent,” one employee said. “That was one of the ones I really wanted.”
“What is ReallyWackyTicTacs?” another employee asked, looking down the list. “Trust me, you don’t want to know,” Ashooh said. “That was the most unpleasant shit I’ve ever seen, and I’ve spent a lot of time looking into Syrian war crimes.”
Some of the comments on the announcement were cynical. “They don’t actually want to change anything,” one redditor wrote, arguing that the bans were meant to appease advertisers. “It was, in fact, never about free speech, it was about money.” One trust-and-safety manager, a young woman wearing a leather jacket and a ship captain’s cap, was in charge of monitoring the comments and responding to the most relevant ones. “Everyone seems to be taking it pretty well so far,” she said. “There’s one guy, freespeechwarrior, who seems very pissed, but I guess that makes sense, given his username.” “People are making lists of all the Nazi subs getting banned, but nobody has noticed that we’re banning bestiality ones at the same time,” Ashooh said…“I’m going to get more cheese sticks,” the woman in the captain’s cap said, standing up. “How many cheese sticks is too many in one day? At what point am I encouraging or glorifying violence against my own body?” “It all depends on context,” Ashooh said.
I understood why other companies had been reluctant to let me see something like this. Never again would I be able to read a lofty phrase about a social-media company’s shift in policy—“open and connected,” or “encouraging meaningful interactions”—without imagining a group of people sitting around a conference room, eating free snacks and making fallible decisions. Social networks, no matter how big they get or how familiar they seem, are not ineluctable forces but experimental technologies built by human beings. We can tell ourselves that these human beings aren’t gatekeepers, or that they have cleansed themselves of all bias and emotion, but this would have no relation to reality. “I have biases, like everyone else,” Huffman told me once. “I just work really hard to make sure that they don’t prevent me from doing what’s right.”
When she inserts a key in the padlock, the door swings open to reveal thousands of books, paintings, engravings, photographs and films—all, in one way or another, connected to sex. It was the kinkiest secret in the Soviet Union: across from the Kremlin, the country’s main library held a pornographic treasure trove. Founded by the Bolsheviks as a repository for aristocrats’ erotica, the collection eventually grew to house 12,000 items from around the world, ranging from 18th-century Japanese engravings to Nixon-era romance novels. Off limits to the general public, the collection was always open to top party brass—some of whom are said to have enjoyed visiting. Today, the collection is still something of a secret: there is no complete compendium of its contents and many of them are still not listed in the catalogue.
…One of the most stunning items seized from an unknown owner is The Seven Deadly Sins, an oversized book of engravings self-published in 1918 by Vasily Masyutin, who also illustrated classics by Pushkin and Chekhov. Among its depictions of gluttony is a large woman masturbating with a ghoulish smile. Before the revolution, it was fashionable among the upper classes to assemble so-called knigi dlya dam (Ladies’ Books)—a kind of bawdy scrapbook. An ostentatious leather-bound album with Kniga Dlya Dam embossed in gold on the cover opens to reveal a Chinese silk drawing of an entwined couple. Further on, dozens of engravings show aristocratic duos fornicating in sumptuously upholstered settings…Among Skorodumov’s treasures was a portfolio of drawings and watercolours by the avant-garde titan Mikhail Larionov. Made in the 1910s, they are no less scandalous in today’s Russia. One pencil sketch features a happily panting dog standing in front of a human, who is engaged in much more than petting. A watercolor depicts two soldiers having an intimate encounter on a bench.
…How did Skorodumov amass such a collection when owning a foreign title could result in a Gulag sentence?…There is also a second theory. Stalin’s secret police chief Genrikh Yagoda, a pornography aficionado whose apartment reportedly held a dildo collection, is said to have enjoyed viewing Skorodumov’s holdings. Librarians believe that he personally ensured the latter’s safety….Safely ensconced in the spetskhran, the erotica collection became available for viewing by top Stalinist henchmen. According to legend, they included the mustachioed cavalry officer and civil war hero Semyon Budyonny and grandfatherly Mikhail Kalinin, the longtime figurehead of the Soviet state. “They were supposedly interested in the visual stuff—postcards, photos,” Chestnykh said. A Politburo member did not need a pass: “No one could refuse them.”
He and his wife live in an apartment not far from mine that was originally occupied by his grandfather, who was the Soviet Union’s chief literary censor under Stalin. The most striking thing about the building was, and is, its history. In the nineteen-thirties, during Stalin’s purges, the House of Government earned the ghoulish reputation of having the highest per-capita number of arrests and executions of any apartment building in Moscow. No other address in the city offers such a compelling portal into the world of Soviet-era bureaucratic privilege, and the horror and murder to which this privilege often led….“Why does this house have such a heavy, difficult aura?” he said. “This is why: on the one hand, its residents lived like a new class of nobility, and on the other they knew that at any second they could get their guts ripped out.”
…This is the opening argument of a magisterial new book by Yuri Slezkine, a Soviet-born historian who immigrated to the United States in 1983, and has been a professor at the University of California, Berkeley, for many years. His book, The House of Government, is a 1200-page epic that recounts the multigenerational story of the famed building and its inhabitants—and, at least as interesting, the rise and fall of Bolshevist faith. In Slezkine’s telling, the Bolsheviks were essentially a millenarian cult, a small tribe radically opposed to a corrupt world. With Lenin’s urging, they sought to bring about the promised revolution, or revelation, which would give rise to a more noble and just era. Of course, that didn’t happen. Slezkine’s book is a tale of “failed prophecy,” and the building itself—my home for the past several years—is “a place where revolutionaries came home and the revolution went to die.”…The Soviet Union had experienced two revolutions, Lenin’s and Stalin’s, and yet, in the lofty imagery of Slezkine, the “world does not end, the blue bird does not return, love does not reveal itself in all of its profound tenderness and charity, and death and mourning and crying and pain do not disappear.” What to do then? The answer was human sacrifice, “one of history’s oldest locomotives,” Slezkine writes. The “more intense the expectation, the more implacable the enemies; the more implacable the enemies, the greater the need for internal cohesion; the greater the need for internal cohesion, the more urgent the search for scapegoats.” Soon, in Stalin’s Soviet Union, the purges began.
…N.K.V.D. agents would sometimes use the garbage chutes that ran like large tubes through many apartments, popping out inside a suspect’s home without having to knock on the door. After a perfunctory trial, which could last all of three to five minutes, prisoners were taken to the left or to the right: imprisonment or execution. “Most House of Government leaseholders were taken to the right,” Slezkine writes…eight hundred residents of the House of Government were arrested or evicted during the purges, thirty per cent of the building’s population. Three hundred and forty-four were shot…Before long, the arrests spread from the tenants to their nannies, guards, laundresses, and stairwell cleaners. The commandant of the house was arrested as an enemy of the people, and so was the head of the Communist Party’s housekeeping department…“He felt a premonition,” she said. “He was always waiting, never sleeping at night.” One evening, Malyshev heard footsteps coming up the corridor—and dropped dead of a heart attack. In a way, his death saved the family: there was no arrest, and thus no reason to kick his relatives out of the apartment.
…One of Volin’s brothers was…called back, arrested, and shot. One of Volin’s sisters was married to an N.K.V.D. officer, and they lived in the House of Government, in a nearby apartment. When the husband’s colleagues came to arrest him, he jumped out of the apartment window to his death. Volin, I learned, kept a suitcase packed with warm clothes behind the couch, ready in case of arrest and sentence to the Gulag…They gave their daughter, Tolya’s mother, a peculiar set of instructions. Every day after school, she was to take the elevator to the ninth floor—not the eighth, where the family lived—and look down the stairwell. If she saw an N.K.V.D. agent outside the apartment, she was supposed to get back on the elevator, go downstairs, and run to a friend’s house.
Amidst the social and political turmoil of the 1970s, a handful of women—among them a onetime Barnard student, a Texas sorority sister, the daughter of a former communist journalist—joined and became leaders of the May 19th Communist Organization. Named to honor the shared birthday of civil rights icon Malcolm X and Vietnamese leader Ho Chi Minh, M19 took its belief in “revolutionary anti-imperialism” to violent extremes: It is “the first and only women-created and women-led terrorist group,” says national security expert and historian William Rosenau.
M19’s status as an “incredible outlier” from male-led terrorist organizations prompted Rosenau, an international security fellow at the think tank New America, to excavate the inner workings of the secretive and short-lived militant group. The resulting book, Tonight We Bombed the Capitol, pieces together the unfamiliar story of “a group of essentially middle-class, well educated, white people who made a journey essentially from anti-war and civil rights protest to terrorism,” he says.
…Eventually, M19 turned to building explosives themselves. Just before 11 p.m. on November 7, 1983, they called the U.S. Capitol switchboard and warned them to evacuate the building. Ten minutes later, a bomb detonated in the building’s north wing, harming no one but blasting a 15-foot gash in a wall and causing $1 million in damage. Over the course of a 20-month span in 1983 and 1984, M19 also bombed an FBI office, the Israel Aircraft Industries building, and the South African consulate in New York, D.C.’s Fort McNair and Navy Yard (which they hit twice.) The attacks tended to follow a similar pattern: a warning call to clear the area, an explosion, a pre-recorded message to media railing against U.S. imperialism or the war machine under various organizational aliases (never using the name M19)…As M19’s spree turned more and more violent, M19’s members became evermore insular and paranoid, nearly cultish, living communally and rotating through aliases and disguises until, in 1985, law enforcement captured the group’s most devoted lieutenants. After that, Rosenau writes, “The far-left terrorist project that began with the Weathermen…and continued into the mid-1980s with May 19th ended in abject failure.”
…People talk about polarization now, but just look at the early 1970s where literally thousands of bombs were set off per year. The important thing is just to realize that there are some similarities, but these are very different periods in time and each period of time is unique.
The May 19th Communist Organization, was a US-based terrorist organization formed by members of the Weather Underground Organization. The group was originally known as the New York chapter of the Prairie Fire Organizing Committee (PFOC), an organization devoted to legally promoting the causes of the Weather Underground. This was part of the Prairie Fire Manifesto's change in Weather Underground Organization strategy, which demanded both aboveground mass movements and clandestine organizations. The role of the clandestine organization would be to build the "consciousness of action" and prepare the way for the development of a people's militia. Concurrently, the role of the mass movement would include support for, and encouragement of, armed action. Such an alliance would, according to Weather, "help create the 'sea' for the guerrillas to swim in." The M19CO name was derived from the birthdays of Ho Chi Minh and Malcolm X. The May 19 Communist Organization was active from 1978 to 1985. M19CO was a combination of the Black Liberation Army and the Weather Underground. It also included members of the Black Panthers and the Republic of New Africa (RNA).
‘On the short life and violent death of French mathematical prodigy Évariste Galois, who, “when he wasn’t trying to overthrow the government, was reinventing algebra”. He mastered the entirety of contemporary mathematics while still at school, made fundamental advances in group theory at the age of 17—then took to drink, insulted his examiners, joined the National Guard, declared his desire to kill the king, spent eight months in jail, fell in love, lost a duel, and died in 1832 at the age of twenty.’
Évariste Galois was a French mathematician and political activist. While still in his teens, he was able to determine a necessary and sufficient condition for a polynomial to be solvable by radicals, thereby solving a problem standing for 350 years. His work laid the foundations for Galois theory and group theory, two major branches of abstract algebra, and the subfield of Galois connections. He died at age 20 from wounds suffered in a duel.
If you’ve spent any time thinking about complex systems, you surely understand the importance of networks. Networks rule our world. From the chemical reaction pathways inside a cell, to the web of relationships in an ecosystem, to the trade and political networks that shape the course of history. Or consider this very post you’re reading. You probably found it on a social network, downloaded it from a computer network, and are currently deciphering it with your neural network.
But as much as I’ve thought about networks over the years, I didn’t appreciate (until very recently) the importance of simple diffusion. This is our topic for today: the way things move and spread, somewhat chaotically, across a network. Some examples to whet the appetite:
Infectious diseases jumping from host to host within a population
Memes spreading across a follower graph on social media
A wildfire breaking out across a landscape
Ideas and practices diffusing through a culture
Neutrons cascading through a hunk of enriched uranium
A quick note about form. Unlike all my previous work, this essay is interactive. There will be sliders to pull, buttons to push, and things that dance around on the screen. I’m pretty excited about this, and I hope you are too.
An explorable explanation is a form of informative media where an interactive computer simulation of a given concept is presented, along with some form of guidance that suggests ways that the audience can learn from the simulation. Explorable explanations encourage users to discover things about the concept for themselves, and test their expectations of its behaviour against its actual behaviour, promoting a more active form of learning than reading or listening.
Nonpharmaceutical interventions (NPIs) intended to reduce infectious contacts between persons form an integral part of plans to mitigate the impact of the next influenza pandemic. Although the potential benefits of NPIs are supported by mathematical models, the historical evidence for the impact of such interventions in past pandemics has not been systematically examined. We obtained data on the timing of 19 classes of NPI in 17 U.S. cities during the 1918 pandemic and tested the hypothesis that early implementation of multiple interventions was associated with reduced disease transmission. Consistent with this hypothesis, cities in which multiple interventions were implemented at an early phase of the epidemic had peak death rates ≈50% lower than those that did not and had less-steep epidemic curves. Cities in which multiple interventions were implemented at an early phase of the epidemic also showed a trend toward lower cumulative excess mortality, but the difference was smaller (≈20%) and less statistically significant than that for peak death rates. This finding was not unexpected, given that few cities maintained NPIs longer than 6 weeks in 1918. Early implementation of certain interventions, including closure of schools, churches, and theaters, was associated with lower peak death rates, but no single intervention showed an association with improved aggregate outcomes for the 1918 phase of the pandemic. These findings support the hypothesis that rapid implementation of multiple NPIs can significantly reduce influenza transmission, but that viral spread will be renewed upon relaxation of such measures.
…In comparisons across cities (Fig. 2 a, Table 2), we found that aggressive early intervention was significantly associated with a lower peak of excess mortality (Spearman ρ = −0.49 to −0.68, p = 0.002–0.047; see Table 2, Number of interventions before, for the number of NPIs before a given CEPID cutoff vs. peak mortality). Cities that implemented three or fewer NPIs before 20/100,000 CEPID had a median peak weekly death rate of 146/100,000, compared with 65/100,000 in those implementing four or more NPIs by that time (Fig. 2 a, p = 0.005). The relationship was similar for normalized peak death rates and for a range of possible cutoffs (see Table 2, CEPID at time of intervention), although the relationship became weaker as later interventions were included. Cities with more early NPIs also had fewer total excess deaths during the study period (Fig. 2 b, Table 2, 1918 total), but this association was weaker: cities with three or fewer NPIs before CEPID = 20/100,000 experienced a median total excess death rate of 551/100,000, compared with a median rate of 405/100,000 in cities with four or more NPIs (p = 0.03).
When the Spanish flu reached the United States in the summer of 1918, it seemed to confine itself to military camps. But when it arrived in Philadelphia in September, it struck with a vengeance. By the time officials there grasped the threat of the virus, it was too late. The disease was rampaging through the population, partly because the city had allowed large public gatherings, including a citywide parade in support of a World War I loan drive, to go on as planned. In four months, more than 12,000 Philadelphians died, an excess death rate of 719 people for every 100,000 inhabitants.
The story was quite different in St. Louis. Two weeks before Philadelphia officials began to react, doctors in St. Louis persuaded the city to require that influenza cases be registered with the health department. And two days after the first civilian cases, police officers helped the department enforce a shutdown of schools, churches and other gathering places. Infected people were quarantined in their homes.
Excess deaths in St. Louis were 347 per 100,000 people, less than half the rate in Philadelphia. Early action appeared to have saved thousands of lives.
…Dr. Hatchett, who is a researcher at the National Institutes of Health, said the findings might hold lessons for the 21st century. “When multiple interventions were introduced early, they were very effective in 1918,” he said, “and that certainly offers hope that they would be similarly useful in an epidemic today if we didn’t have an effective vaccine.”
…What these results mean for a future epidemic is not clear. “If avian flu became a pandemic tomorrow,” Dr. Ferguson said, “we would start a crash program to make a vaccine.” But he added that rigid preventive measures like quarantines, mandated mask wearing and widespread business closings would still need to be put in place. “What our study shows,” he continued, “is that interventions even without a vaccine can be effective in blocking transmission. What’s much less certain is whether society is prepared to bear the costs of implementing such intrusive and costly measures for the months that would be required to manufacture a vaccine.”
This paper uses the 1918 influenza pandemic as a natural experiment for testing the fetal origins hypothesis. The pandemic arrived unexpectedly in the fall of 1918 and had largely subsided by January 1919, generating sharp predictions for long-term effects. Data from the 1960–80 decennial U.S. Census indicate that cohorts in utero during the pandemic displayed reduced educational attainment, increased rates of physical disability, lower income, lower socioeconomic status, and higher transfer payments compared with other birth cohorts. These results indicate that investments in fetal health can increase human capital.
The 1918 influenza pandemic struck the United States with most ferocity in October of 1918 and then over the next four months killed more people than all the US combat deaths of the 20th century. The sudden nature of the pandemic meant that children born just months apart experienced very different conditions in utero. In particular, children born in 1919 were much more exposed to influenza in utero than children born in 1918 or 1920. The sudden differential to the 1918 flu lets Douglas Almond test for long-term effects in Is the 1918 Influenza Pandemic Over?
Almond finds large effects many decades after exposure.
Fetal health is found to affect nearly every socioeconomic outcome recorded in the 1960, 1970, and 1980 Censuses. Men and women show large and discontinuous reductions in educational attainment if they had been in utero during the pandemic. The children of infected mothers were up to 15% less likely to graduate from high school. Wages of men were 5–9% lower because of infection. Socioeconomic status…was substantially reduced, and the likelihood of being poor rose as much as 15% compared with other cohorts. Public entitlement spending was also increased.
…male disability rates in 1980, i.e. for males around the age of 60, by year and quarter of birth. Cohorts born between January and September of 1919 “were in utero at the height of the pandemic and are estimated to have 20% higher disability rates at age 61…”.
Figure 3 at right shows average years of schooling in 1960; once again the decline is clear for those born in 1918 and note that not all pregnant women contracted influenza so the actual effects of influenza exposure are larger, about a 5 month decline in education, mostly coming through lower graduate rates.
…a phenomenon recognized 2500 years ago by Hippocrates and Thucydides: Many infectious diseases are more common during specific seasons. “It’s a very old question, but it’s not very well studied,” Martinez says. It’s also a question that has suddenly become more pressing because of the emergence of COVID-19. With SARS-CoV-2, the virus that causes the disease, now infecting more than 135,000 around the globe, some hope it might mimic influenza and abate as summer arrives in temperate regions of the Northern Hemisphere, where about half of the world’s population lives….Different diseases have different patterns. Some peak in early or late winter, others in spring, summer, or fall…At least 68 infectious diseases are seasonal, according to a 2018 paper by Micaela Martinez of Columbia University…Some diseases have different seasonal peaks depending on latitude. And many have no seasonal cycle at all. Even for well-known seasonal diseases, it’s not clear why they wax and wane during the calendar year. “It’s an absolute swine of a field,” says Andrew Loudon, a chronobiologist at the University of Manchester. Investigating a hypothesis over several seasons can take 2 or 3 years. “Postdocs can only get one experiment done and it can be a career killer,” Loudon says. The field is also plagued by confounding variables. “All kinds of things are seasonal, like Christmas shopping,” says epidemiologist Scott Dowell, who heads vaccine development and surveillance at the Bill and Melinda Gates Foundation and in 2001 wrote a widely cited perspective that inspired Martinez’s current study. And it’s easy to be misled by spurious correlations, Dowell says.
Despite the obstacles, researchers are testing a multitude of theories. Many focus on the relationships between the pathogen, the environment, and human behavior. Influenza, for example, might do better in winter because of factors such as humidity, temperature, people being closer together, or changes in diets and vitamin D levels. Martinez is studying another theory, which Dowell’s paper posited but didn’t test: The human immune system may change with the seasons, becoming more resistant or more susceptible to different infections based on how much light our bodies experience.
…Except in the equatorial regions, respiratory syncytial virus (RSV) is a winter disease, Martinez wrote, but chickenpox favors the spring. Rotavirus peaks in December or January in the U.S. Southwest, but in April and May in the Northeast. Genital herpes surges all over the country in the spring and summer, whereas tetanus favors midsummer; gonorrhea takes off in the summer and fall, and pertussis has a higher incidence from June through October. Syphilis does well in winter in China, but typhoid fever spikes there in July. Hepatitis C peaks in winter in India but in spring or summer in Egypt, China, and Mexico. Dry seasons are linked to Guinea worm disease and Lassa fever in Nigeria and hepatitis A in Brazil.
Seasonality is easiest to understand for diseases spread by insects that thrive during rainy seasons, such as African sleeping sickness, chikungunya, dengue, and river blindness. For most other infections, there’s little rhyme or reason to the timing. “What’s really amazing to me is that you can find a virus that peaks in almost every month of the year in the same environment in the same location,” says Neal Nathanson, an emeritus virologist at the University of Pennsylvania Perelman School of Medicine. “That’s really crazy if you think about it.” To Nathanson, this variation suggests human activity—such as children returning to school or people huddling indoors in cold weather—doesn’t drive seasonality. “Most viruses get transmitted between kids, and under those circumstances, you’d expect most of the viruses to be in sync,” he says.
…A 2018 study in Scientific Reports supports the idea. Virologist Sandeep Ramalingam at the University of Edinburgh and his colleagues analyzed the presence and seasonality of nine viruses—some enveloped, some not—in more than 36,000 respiratory samples taken over 6.5 years from people who sought medical care in their region. “Enveloped viruses have a very, very definite seasonality,” Ramalingam says.
RSV and human metapneumovirus both have an envelope, like the flu, and peak during the winter months. None of the three are present for more than one-third of the year. Rhinoviruses, the best-known cause of the common cold, lack an envelope and—ironically—have no particular affinity for cold weather: The study found them in respiratory samples on 84.7% of the days of the year and showed that they peak when children return to school from summer and spring holidays. Adenoviruses, another set of cold viruses, also lack an envelope and had a similar pattern, circulating over half the year. Ramalingam’s team also studied the relationship between viral abundance and daily weather changes. Influenza and RSV both did best when the change in relative humidity over a 24-hour period was lower than the average (a 25% difference). “There’s something about the lipid envelope that’s more fragile” when the humidity changes sharply, Ramalingam concludes.
Seasonal cyclicity is a ubiquitous feature of acute infectious diseases  and may be a ubiquitous feature of human infectious diseases in general, as illustrated in Tables 11–4. Each acute infectious disease has its own seasonal window of occurrence, which, importantly, may vary among geographic locations and differ from other diseases within the same location. Here we explore the concept of an epidemic calendar, which is the idea that seasonality is a unifying feature of epidemic-prone diseases and, in the absence of control measures, the local calendar can be marked by epidemics (Fig 1). A well-known example of a calendar marked by epidemics is that of the Northern Hemisphere, where influenza outbreaks occur each winter [2, 3] (hence the colloquial reference to winter as “the flu season”). In contrast, chickenpox outbreaks peak each spring [4, 5], and polio transmission historically occurred each summer .
…In the broadest sense, seasonal drivers can be separated into four categories: (1) environmental factors, (2) host behavior, (3) host phenology, and (4) exogenous biotic factors. These seasonal drivers may enter into disease transmission dynamics by way of hosts, reservoirs, and/or vectors. In surveying the literature to gauge the breadth of seasonal drivers acting upon human infectious disease systems (Tables (Tables11–4), specific seasonal drivers were found to include (a) vector seasonality, (b) seasonality in nonhuman animal host (i.e., livestock, other domestic animals, or wildlife), (c) seasonal climate (e.g., temperature, precipitation, etc.), (d) seasonal nonclimatic abiotic environment (e.g., water salinity), (e) seasonal co-infection, (f) seasonal exposure and/or behavior and/or contact rate, (g) seasonal biotic environment (e.g., algal density in waterbodies), and (h) seasonal flare-ups/symptoms and/or remission/latency.
Numerous viruses can cause upper respiratory tract infections. They often precede serious lower respiratory tract infections. Each virus has a seasonal pattern, with peaks in activity in different seasons. We examined the effects of daily local meteorological data (temperature, relative humidity, “humidity-range” and dew point) from Edinburgh, Scotland on the seasonal variations in viral transmission. We identified the seasonality of rhinovirus, adenovirus, influenza A and B viruses, human parainfluenza viruses 1–3 (HPIV), respiratory syncytial virus (RSV) and human metapneumovirus (HMPV) from the 52060 respiratory samples tested between 2009 and 2015 and then confirmed the same by a generalised linear model. We also investigated the relationship between meteorological factors and viral seasonality. Non-enveloped viruses were present throughout the year. Following logistic regression adenovirus, influenza viruses A, B, RSV and HMPV preferred low temperatures; RSV and influenza A virus preferred a narrow “humidity-range” and HPIV type 3 preferred the season with lower humidity. A change (i.e. increase or decrease) in specific meteorological factors is associated with an increase in activity of specific viruses at certain times of the year.
Rationale: Is it possible to have a psychedelic experience from a placebo alone? Most psychedelic studies find few effects in the placebo control group, yet these effects may have been obscured by the study design, setting, or analysis decisions.
Objective: We examined individual variation in placebo effects in a naturalistic environment resembling a typical psychedelic party.
Methods: 33 students completed a single-arm study ostensibly examining how a psychedelic drug affects creativity. The 4-h study took place in a group setting with music, paintings, coloured lights, and visual projections. Participants consumed a placebo that we described as a drug resembling psilocybin, which is found in psychedelic mushrooms. To boost expectations, confederates subtly acted out the stated effects of the drug and participants were led to believe that there was no placebo control group. The participants later completed the 5-Dimensional Altered States of Consciousness Rating Scale, which measures changes in conscious experience.
Results: There was considerable individual variation in the placebo effects; many participants reported no changes while others showed effects with magnitudes typically associated with moderate or high doses of psilocybin. In addition, the majority (61%) of participants verbally reported some effect of the drug. Several stated that they saw the paintings on the walls “move” or “reshape” themselves, others felt “heavy…as if gravity [had] a stronger hold”, and one had a “come down” before another “wave” hit her.
Conclusion: Understanding how context and expectations promote psychedelic-like effects, even without the drug, will help researchers to isolate drug effects and clinicians to maximise their therapeutic potential.
…In the second sample, before the debriefing, we asked participants to guess whether they had taken a psychedelic, a placebo, or whether they were uncertain. Overall, 35% reported being certain they had taken a placebo, 12% were certain that they had taken a psychedelic, and the rest (53%) were uncertain. In the first sample, we did not ask this question, but the same number of people spontaneously reported being certain that they had taken a psychedelic drug. During the debriefing, when we revealed the placebo nature of the study, many participants appeared shocked. Several gasped and started laughing. One stated, “It’s very funny!”, and another replied, “It’s sad!” One of the participants who had sat with a group near the paintings throughout the study asked, “So we were all sober and just watching these paintings for 45 minutes‽”
[“This is a remarkable study, and probably the most elaborate placebo ever reported. But how well did the trick work? The authors say that after they revealed the truth, some of the participants expressed shock. However, 35% of them said they were “certain” they had taken a placebo when quizzed just before the debriefing. Only 12% were “certain” that they’d taken a real psychedelic drug, which suggests that the deception was only partially successful.
Some of the participants did report very strong effects on a questionnaire of ‘psychedelic effects’. However, I noticed that the effects reported tended to be the more abstract kind, such as “insight” and “bliss”. In terms of actual hallucinogenic effects like ‘complex imagery’ and ‘elementary imagery’ (i.e. seeing things), no participants reported effects equal to even a low dose of LSD, let alone a stronger dose. See the rather confusing Figure 2 for details." —Neuroskeptic]
Research in educational psychology consistently finds a relationship between intelligence and academic performance. However, in recent decades, educational fields, including gifted education, have resisted intelligence research, and there are some experts who argue that intelligence tests should not be used in identifying giftedness. Hoping to better understand this resistance to intelligence research, we created a survey of beliefs about intelligence and administered it online to a sample of the general public and a sample of teachers. We found that there are conflicts between currently accepted intelligence theory and beliefs from the American public and teachers, which has important consequences on gifted education, educational policy, and the effectiveness of interventions.
[On the underappreciated cunning and escape artistry of orangutans. Despite seeming harmless and less of a reputation for intelligence than chimpanzees, they are just as dangerous (often deceptively calm until the instant they attack) and baffle their zookeepers with their escapes.
Orangutans must be captured as infants because adults are too uncooperative. Captive orangutans nevertheless will unscrew bolts and nuts, throw rocks to break glass windows, will trick people into waving to grab their hand and climb out, avoid any escape attempts when zookeepers are watching (even when they are ‘undercover’ as visitors) unless they can take advantage of the zookeepers watching another orangutan, construct ladders out of branches or steal workers’ tools & hide them for later, and cooperate in using them to escape (eg a pair using a stolen mop handle, one steadying it). Skilled climbers, they can find the most invisible holds, climb up edges using purely finger pressure, and can even shimmy up parallel walls like a human climber; when bringing in expert climbers to find and remove possible routes, the orangutans must be kept out of sight, lest they learn new routes. If a nylon net bars them, they will spend months patiently unraveling it. If electrified wires are added, they will learn to test the wires regularly and wait for an opportunity. One orangutan learned to defeat the wires by grounding it using wood sticks (others used rubber tires), and climbing over on the porcelain insulators. “Fu Manchu” hide a strip of metal in his mouth to pick open the lock on his door, while “Jonathan” used “a slab of cardboard in order to release himself through a complex guillotine door.”
The San Diego Zoo in 1989 spent $45k crafting an orangutan exhibit with all this in mind to make it inescapable. An orangutan escaped 4 years later.]
Orangutans are great apes native to Indonesia and Malaysia. They are found in the rainforests of Borneo and Sumatra, but during the Pleistocene they ranged throughout Southeast Asia and South China. Classified in the genus Pongo, orangutans were originally considered to be one species. From 1996, they were divided into two species: the Bornean orangutan and the Sumatran orangutan. In 2017, a third species, the Tapanuli orangutan, was identified. The orangutans are the only surviving species of the subfamily Ponginae, who split from humans, chimpanzees and gorillas 19.3 to 15.7 million years ago (mya).
Ken Allen was a Bornean orangutan at the San Diego Zoo. He became one of the most popular animals in the history of the zoo because of his many successful escapes from his enclosures. He was nicknamed "the Hairy Houdini".
[Tracing the history of Internet domain names and WWW URLs from ARPAnet’s need for emails to the present, and explaining how we got our confusing mishmash of Unix-style paths & URIs, and why URLs like google.com. are valid, with digressions into hacks like Punycode for representing non-English domains and query strings for turning a system for serving HTML documents into a system for arbitrary APIs/RPCs.]
[Why is web programming so screwed up? A highly-opinionated history of how worse-is-better played out online from 1995 to now, by a programmer who started writing HTML ~1996 and has seen the evolution of it all up close: HTML was never designed to support even 1% of the things it is expected to do, requiring gruesome workarounds like tables for positioning anything or using images for rounded corners, and has been constantly extended with ad hoc and poorly-thought-through capabilities, sabotaged further by the exigencies of history like the ‘browser wars’ between Netscape & Microsoft, and then Microsoft simply killing Internet Explorer (IE) development for several years after achieving a near-total global monopoly. With a vast amount of work, HTML/CSS can now support many desirable web pages, but the historical legacy continues to live on, in the use of now-obsolete workarounds, features which no one uses, strange inconsistencies & limitations, etc.]
I have been curious about data compression and the Zip file format in particular for a long time. At some point I decided to address that by learning how it works and writing my own Zip program. The implementation turned into an exciting programming exercise; there is great pleasure to be had from creating a well oiled machine that takes data apart, jumbles its bits into a more efficient representation, and puts it all back together again. Hopefully it is interesting to read about too.
This article explains how the Zip file format and its compression scheme work in great detail: LZ77 compression, Huffman coding, Deflate and all. It tells some of the history, and provides a reasonably efficient example implementation written from scratch in C…It is fascinating how the evolution of technology is both fast and slow. The Zip format was created 30 years ago based on technology from the fifties and seventies, and while much has changed since then, Zip files are essentially the same and more prevalent than ever. I think it is useful to have a good understanding of how they work.
Google-Wide Profiling (GWP), a continuous profiling infrastructure for data centers, provides performance insights for cloud applications. With negligible overhead, GWP provides stable, accurate profiles and a datacenter-scale tool for traditional performance analyses. Furthermore, GWP introduces novel applications of its profiles, such as application-platform affinity measurements and identification of platform-specific, microarchitectural peculiarities.
We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone. In cases where scene surface has a strong mirror-like material component, we generate highly detailed environment images, revealing room composition, objects, people, buildings, and trees visible through windows. Our approach yields state of the art view synthesis techniques, operates on low dynamic range imagery, and is robust to geometric and calibration errors.
Technically speaking, the researchers didn’t actually use chips; they reconstructed a room using a Korean brand of chocolate-dipped corn puffs called Corn Cho. But whether it’s corn puffs or potato chips, the snack bag acts like a bad, warped mirror. A heavily-distorted reflection of the room is contained in the glint of light that bounces off the bag, and the team developed an algorithm that unwarps that glint into a blurry but recognizable image. In one instance, the researchers were able to resolve the silhouette of a man standing in front of a window. In another, the bag reflections allowed them to see through a window to the house across the street clearly enough to count how many stories it had. The algorithm works on a variety of glossy objects—the shinier, the better. Using the sheen of a porcelain cat, for example, they could also reconstruct the layout of the surrounding ceiling lights.
…To reconstruct the environment, the researchers used a handheld color video camera with a depth sensor that roughly detects the shape and distance of the shiny objects. They filmed these objects for about a minute, capturing their reflections from a variety of perspectives. Then, they used a machine learning algorithm to reconstruct the surroundings, which took on the order of two hours per object. Their reconstructions are remarkably accurate considering the relatively small amount of data that they used to train the algorithm, says computer scientist Abe Davis of Cornell University, who was not involved with the work.
The researchers could achieve this accuracy with so little training data, in part, because they incorporate some physics concepts in their reconstruction algorithm—the difference between how light bounces off shiny surfaces versus matte surfaces, for example. This differs from typical online image recognition tools in use today, which simply look for patterns in images without any extra scientific information. However, researchers have also found that too much physics in an algorithm can cause the machine to make more mistakes, as its processing strategies become too rigid. “They do a good job of balancing physical insights with modern machine learning tools,” says Davis.
…However, some experts caution that future versions of the technology are ripe for abuse. For example, it could enable stalkers or child abusers, says ethicist Jacob Metcalf of Data & Society, a nonprofit research center that focuses on the social implications of emerging technologies. A stalker could download images off of Instagram without the creators’ consent, and if those images contained shiny surfaces, they could deploy the algorithm to try to reconstruct their surroundings and infer private information about that person. “You better believe that there are a lot of people who will use a Python package to scrape photos off Instagram,” says Metcalf. “They could find a photo of a celebrity or of a kid that has a reflective surface and try to do something.”
These formulas have turned an obscure idea that Galanis and his college buddies had a few years ago about making more money for second rate celebs into a thriving two-sided marketplace that has caught the attention of VCs, Hollywood, and professional sports. In June, Cameo raised $50 million in Series B funding, led by Kleiner Perkins (which recently began funding more early stage startups) to boost marketing, expand into international markets, and staff up to meet the growing demand. In the past 15 months, Cameo has gone from 20 to 125 employees, and moved from an 825-square-foot home base in the 1871 technology incubator into its current 6,000-square-foot digs in Chicago’s popping West Loop. Cameo customers have purchased more than 560,000 videos from some 20,000 celebs and counting, including ’80s star Steve Guttenberg and sports legend Kareem Abdul-Jabbar. And now, when the masses find themselves in quarantined isolation—looking for levity, distractions, and any semblance of the human touch—sending each other personalized videograms from the semi-famous has never seemed like a more pitch-perfect offering.
The product itself is as simple as it is improbable. For a price the celeb sets —anywhere from $5 to $2,500—famous people record video shout-outs, aka “Cameos,” that run for a couple of minutes, and then are delivered via text or email. Most Cameo videos are booked as private birthday or anniversary gifts, but a few have gone viral on social media. Even if you don’t know Cameo by name, there’s a good chance you caught Bam Margera of MTV’s Jackass delivering an “I quit” message on behalf of a disgruntled employee, or Sugar Ray’s Mark McGrath dumping some poor dude on behalf of the guy’s girlfriend. (Don’t feel too bad for the dumpee, the whole thing was a joke.)
…Back at the whiteboard, Galanis takes a marker and sketches out a graph of how fame works on his platform. “Imagine the grid represents all the celebrity talent in the world,” he says, “which by our definition, we peg at 5 million people.” The X-axis is willingness; the Y-axis is fame. “Say LeBron is at the top of the X-axis, and I’m at the bottom,” he says. On the willingness side, Galanis puts notoriously media-averse Seattle Seahawks running back Marshawn Lynch on the far left end. At the opposite end, he slots chatty celebrity blogger-turned-Cameo-workhorse Perez Hilton, of whom Galanis says, “I promise if you booked him right now, the video would be done before we leave this room.”
…“The contrarian bet we made was that it would be way better for us to have people with small, loyal followings, often unknown to the general population, but who were willing to charge $5 to $10,” Galanis says. Cameo would employ a revenue-sharing model, getting a 25% cut of each video, while the rest went to the celeb. They wanted people like Galanis’ co-founder (and former Duke classmate) Devon Townsend, who had built a small following making silly Vine videos of his travels with pal Cody Ko, a popular YouTuber. “Devon isn’t Justin Bieber, but he had 25,000 Instagram followers from his days as a goofy Vine star,” explains Galanis. “He originally charged a couple bucks, and the people who love him responded, ‘Best money I ever spent!’”
…After a customer books a Cameo, the celeb films the video via the startup’s app within four to seven days. Most videos typically come in at under a minute, though some talent indulges in extensive riffs. (Inexplicably, “plant-based activist and health coach” Courtney Anne Feldman, wife of Corey, once went on for more than 20 minutes in a video for a customer.) Cameo handles the setup, technical infrastructure, marketing, and support, with white-glove service for the biggest earners with “whatever they need”—details like help pronouncing a customer’s name or just making sure they aren’t getting burned-out doing so many video shout-outs.
…For famous people of any caliber—the washed-up, the obscure micro-celebrity, the actual rock star—becoming part of the supply side of the Cameo marketplace is as low a barrier as it gets. Set a price and go. The videos are short—Instagram comedian Evan Breen has been known to knock out more than 100 at $25 a pop in a single sitting—and they don’t typically require any special preparation. Hair, makeup, wardrobe, or even handlers aren’t necessary. In fact, part of the oddball authenticity of Cameo videos is that they have a take-me-as-I-am familiarity—filmed at breakfast tables, lying in bed, on the golf course, running errands, at a stoplight, wherever it fits into the schedule.
Cameo is an American video-sharing website headquartered in Chicago. Cameo was created in 2016 by Steven Galanis, Martin Blencowe, and Devon Spinnler Townsend. The site allows celebrities to send personalized video messages to fans. As of May 2020, more than 30,000 celebrities have joined the platform.
[Meditation on what drives social networks like Instagram: status and signaling. A social network provides a way for monkeys to create and ascend status hierarchies, and a new social network can bootstrap and succeed by offering a new way to do that.]
Let’s begin with two principles:
People are status-seeking monkeys
People seek out the most efficient path to maximizing social capital
…we can start to demystify social networks if we also think of them as SaaS businesses, but instead of software, they provide status.
Almost every social network of note had an early signature proof of work hurdle. For Facebook it was posting some witty text-based status update. For Instagram, it was posting an interesting square photo. For Vine, an entertaining 6-second video. For Twitter, it was writing an amusing bit of text of 140 characters or fewer. Pinterest? Pinning a compelling photo. You can likely derive the proof of work for other networks like Quora and Reddit and Twitch and so on. Successful social networks don’t pose trick questions at the start, it’s usually clear what they want from you.
…Thirst for status is potential energy. It is the lifeblood of a Status as a Service business. To succeed at carving out unique space in the market, social networks offer their own unique form of status token, earned through some distinctive proof of work.
…Most of these near clones have and will fail. The reason that matching the basic proof of work hurdle of an Status as a Service incumbent fails is that it generally duplicates the status game that already exists. By definition, if the proof of work is the same, you’re not really creating a new status ladder game, and so there isn’t a real compelling reason to switch when the new network really has no one in it.
…Why do social network effects reverse? Utility, the other axis by which I judge social networks, tends to be uncapped in value. It’s rare to describe a product or service as having become too useful. That is, it’s hard to over-serve on utility. The more people that accept a form of payment, the more useful it is, like Visa or Mastercard or Alipay. People don’t stop using a service because it’s too useful.
…Social network effects are different. If you’ve lived in New York City, you’ve likely seen, over and over, night clubs which are so hot for months suddenly go out of business just a short while later. Many types of social capital have qualities which render them fragile. Status relies on coordinated consensus to define the scarcity that determines its value. Consensus can shift in an instant. Recall the friend in Swingers, who, at every crowded LA party, quips, “This place is dead anyway.” Or recall the wise words of noted sociologist Groucho Marx: “I don’t care to belong to any club that will have me as a member.”
The Chicago Tylenol Murders were a series of poisoning deaths resulting from drug tampering in the Chicago metropolitan area in 1982. The victims had all taken Tylenol-branded acetaminophen capsules that had been laced with potassium cyanide. A total of seven people died in the original poisonings, with several more deaths in subsequent copycat crimes.
[Chapter 6 of the first book of The Book of the New Sun, and is famous for being an extended homage to Jorge Luis Borges as the blind librarian Ultan who was gifted blindness right as he became librarian, and also has some of the most beautiful writing in the series.]
…“You are in close contact, then, with your opposite numbers in the city,” I said. The old man stroked his beard. “The closest, for we are they. This library is the city library, and the library of the House Absolute too, for that matter. And many others.” “Do you mean that the rabble of the city is permitted to enter the Citadel to use your library?” “No,” said Ultan. “I mean that the library itself extends beyond the walls of the Citadel. Nor, I think, is it the only institution here that does so. It is thus that the contents of our fortress are so much larger than their container.”
…His grip on my shoulder tightened. “We have books here bound in the hides of echidnes, krakens, and beasts so long extinct that those whose studies they are, are for the most part of the opinion that no trace of them survives unfossilized. We have books bound wholly in metals of unknown alloy, and books whose bindings are covered with thickset gems. We have books cased in perfumed woods shipped across the inconceivable gulf between creations—books doubly precious because no one on Urth can read them.”
“We have books whose papers are matted of plants from which spring curious alkaloids, so that the reader, in turning their pages, is taken unaware by bizarre fantasies and chimeric dreams. Books whose pages are not paper at all, but delicate wafers of white jade, ivory, and shell; books too whose leaves are the desiccated leaves of unknown plants. Books we have also that are not books at all to the eye: scrolls and tablets and recordings on a hundred different substances. There is a cube of crystal here—though I can no longer tell you where—no larger than the ball of your thumb that contains more books than the library itself does. Though a harlot might dangle it from one ear for an ornament, there are not volumes enough in the world to counterweight the other. All these I came to know and made safeguarding them my life’s devotion. For seven years I busied myself with that; and then, just when the pressing and superficial problems of preservation were disposed of, and we were on the point of beginning the first general survey of the library since its foundation, my eyes began to gutter in their sockets. He who had given all books into my keeping made me blind so that I should know in whose keeping the keepers stand.”
…“In every library, by ancient precept, is a room reserved for children. In it are kept bright picture books such as children delight in, and a few simple tales of wonder and adventure. Many children come to these rooms, and so long as they remain within their confines, no interest is taken in them.” He hesitated, and though I could discern no expression on his face, I received the impression that he feared what he was about to say might cause Cyby pain.
“From time to time, however, a librarian remarks a solitary child, still of tender years, who wanders from the children’s room and at last deserts it entirely. Such a child eventually discovers, on some low but obscure shelf, The Book of Gold. You have never seen this book, and you will never see it, being past the age at which it is met.”
“It must be very beautiful,” I said."It is indeed. Unless my memory betrays me, the cover is of black buckram,considerably faded at the spine. Several of the signatures are coming out, and certain of the plates have been taken. But it is a remarkably lovely book. I wish that I might find it again, though all books are shut to me now. The child, as I said, in time discovers The Book of Gold. Then the librarians come—like vampires, some say, but others say like the fairy godparents at a christening. They speak to the child, and the child joins them. Henceforth he is in the library wherever he may be, and soon his parents know him no more.
The Book of the New Sun (1980–1983) is a series of four science fantasy novels, a tetralogy or single four-volume novel written by the American author Gene Wolfe. It inaugurated the so-called "Solar Cycle" that Wolfe continued by setting other works in the same universe.
The Shadow of the Torturer is a science fiction novel by American writer Gene Wolfe, published by Simon & Schuster in May 1980. It is the first of four volumes in The Book of the New Sun which Wolfe had completed in draft before The Shadow of the Torturer was published. It relates the story of Severian, an apprentice Seeker for Truth and Penitence, from his youth through his expulsion from the guild and subsequent journey out of his home city of Nessus.
Gene Rodman Wolfe was an American science fiction and fantasy writer. He was noted for his dense, allusive prose as well as the strong influence of his Catholic faith. He was a prolific short story writer and novelist and won many science fiction and fantasy literary awards.
Jorge Francisco Isidoro Luis Borges Acevedo was an Argentine short-story writer, essayist, poet and translator, and a key figure in Spanish-language and universal literature. His best-known books, Ficciones (Fictions) and El Aleph, published in the 1940s, are compilations of short stories interconnected by common themes, including dreams, labyrinths, philosophers, libraries, mirrors, fictional writers, and mythology. Borges' works have contributed to philosophical literature and the fantasy genre, and have been considered by some critics to mark the beginning of the magic realist movement in 20th century Latin American literature. His late poems converse with such cultural figures as Spinoza, Camões, and Virgil.
Golden Kamuy is a Japanese manga series written and illustrated by Satoru Noda. The story follows a young Ainu girl named Asirpa and her quest to find a huge fortune of gold of the Ainu people, helped by Saichi Sugimoto, a veteran of the early twentieth century Russo-Japanese War. The Ainu language in the story is supervised by Hiroshi Nakagawa, an Ainu language linguist from Chiba University. The manga won the ninth Manga Taishō award in 2016.
Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.
We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
"Who's on First?" is a comedy routine made famous by American comedy duo Abbott and Costello. The premise of the sketch is that Abbott is identifying the players on a baseball team for Costello, but their names and nicknames can be interpreted as non-responsive answers to Costello's questions. For example, the first baseman is named "Who"; thus, the utterance "Who's on first" is ambiguous between the question and the answer.
Microsoft Research (MSR) is the research subsidiary of Microsoft. It was formed in 1991, with the intent to advance state-of-the-art computing and solve difficult world problems through technological innovation in collaboration with academic, government, and industry researchers. The Microsoft Research team employs more than 1,000 computer scientists, physicists, engineers, and mathematicians, including Turing Award winners, Fields Medal winners, MacArthur Fellows, and Dijkstra Prize winners.
Long-standing problems in standard scientific methodology have exploded as the “Replication Crisis”: the discovery that many results in fields as diverse as psychology, economics, medicine, biology, and sociology are in fact false or quantitatively highly inaccurately measured. I cover here a handful of the issues and publications on this large, important, and rapidly developing topic up to about 2013, at which point the Replication Crisis became too large a topic to cover more than cursorily.
The crisis is caused by methods & publishing procedures which interpret random noise as important results, far too small datasets, selective analysis by an analyst trying to reach expected/desired results, publication bias, poor implementation of existing best-practices, nontrivial levels of research fraud, software errors, philosophical beliefs among researchers that false positives are acceptable, neglect of known confounding like genetics, and skewed incentives (financial & professional) to publish ‘hot’ results.
Thus, any individual piece of research typically establishes little. Scientific validation comes not from small p-values, but from discovering a regular feature of the world which disinterested third parties can discover with straightforward research done independently on new data with new procedures—replication.
A Uniform Resource Identifier (URI) is a string that provides a unique address where a resource can be found. It looks and is similar to the more commonly known Uniform Resource Locator (URL) that is used to locate web sites and pages within those sites. A URL is a kind of URI—all URLs are URIs, but not all URIs are valid URLs. URLs typically identify web pages that are meant to be viewed in a browser. URIs identify any type of resource, often meant to be used by a computer but not always to be viewed in a browser. The term "URL" most often refers to URIs that name webpages accessed with HTTP, while the term "URI" is used to indicate more general resources like email addresses and local files. URIs often identify resources that are part of an ontology defined using the Web Ontology Language (OWL). For example, the class Agent or the property email in the Friend of a Friend vocabulary would each have an individual URI. URIs have mostly been replaced by Internationalized Resource Identifiers (IRI) which are identical to URIs except IRIs can handle character sets such as Kanji rather than being restricted to ASCII as URIs are.
Punycode is a representation of Unicode with the limited ASCII character subset used for Internet hostnames. Using Punycode, host names containing Unicode characters are transcoded to a subset of ASCII consisting of letters, digits, and hyphens, which is called the Letter-Digit-Hyphen (LDH) subset. For example, München is encoded as Mnchen-3ya.
A query string is a part of a uniform resource locator (URL) that assigns values to specified parameters. A query string commonly includes fields added to a base URL by a Web browser or other client application, for example as part of an HTML form.
LZ77 and LZ78 are the two lossless data compression algorithms published in papers by Abraham Lempel and Jacob Ziv in 1977 and 1978.They are also known as LZ1 and LZ2 respectively. These two algorithms form the basis for many variations including LZW, LZSS, LZMA and others. Besides their academic influence, these algorithms formed the basis of several ubiquitous compression schemes, including GIF and the DEFLATE algorithm used in PNG and ZIP.
In computer science and information theory, a Huffman code is a particular type of optimal prefix code that is commonly used for lossless data compression. The process of finding or using such a code proceeds by means of Huffman coding, an algorithm developed by David A. Huffman while he was a Sc.D. student at MIT, and published in the 1952 paper "A Method for the Construction of Minimum-Redundancy Codes".
The replication crisis is, as of 2020, an ongoing methodological crisis in which it has been found that many scientific studies are difficult or impossible to replicate or reproduce. The replication crisis affects the social sciences and medicine most severely. The crisis has long-standing roots; the phrase was coined in the early 2010s as part of a growing awareness of the problem. The replication crisis represents an important body of research in the field of metascience.