June 2019 gwern.net newsletter with 5 new essays; links on deep learning, history, technological/cultural evolution, & Scott Alexander; and 2 books & 1 movie review newsletter 2019-05-31–2021-01-04finishedcertainty: logimportance: 0
On Machine Intelligence (Second Edition), Michie 1986 (considerably less interesting than Donald Michie: On Machine Intelligence, Biology and More, and almost entirely obsolete; I continue to be mystified at how little interest Michie took in connectionism.)
Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/changes/additions.
Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular /r/Gwern subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).
A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.
Cloning is widely used in animal & plant breeding despite steep costs due to its advantages; more unusual recent applications include creating entire polo horse teams and reported trials of cloning in elite police/Special Forces war dogs. Given the cost of dog cloning, however, can this ever make more sense than standard screening methods for selecting from working dog breeds, or would the increase in successful dog training be too low under all reasonable models to turn a profit?
I model the question as one of expected cost per dog with the trait of successfully passing training, success in training being a dichotomous liability threshold with a polygenic genetic architecture; given the extreme level of selection possible in selecting the best among already-elite Special Forces dogs and a range of heritabilities, this predicts clones’ success probabilities. To approximate the relevant parameters, I look at some reported training costs and success rates for regular dog candidates, broad dog heritabilities, and the few current dog cloning case studies reported in the media.
Since none of the relevant parameters are known with confidence, I run the cost-benefit equation for many hypothetical scenarios, and find that in a large fraction of them covering most plausible values, dog cloning would improve training yields enough to be profitable (in addition to its other advantages).
As further illustration of the use-case of screening for an extreme outcome based on a partial predictor, I consider the question of whether height PGSes could be used to screen the US population for people of NBA height, which turns out to be reasonably doable with current & future PGSes.
Analogous to the dog cloning scenario, I consider the case of selecting for extremes on PGSes, motivated by a scenario of scouting tall men for the NBA.
Setting up the NBA selection problem as a liability threshold model with current height PGSes as a noisy predictor, height selection can be modeled as selecting for extremes on a PGS which is regressed back to the mean to yield expected adult height, and probability of being tall enough to consider a NBA career.
Filling in reasonable values, nontrivial numbers of tall people can be found by genomic screening with a current PGS, and as PGSes approach their predictive upper bound (derived from whole-genome-based heritability estimates of height), selection is capable of selecting almost all tall people by taking the top PGS percentile.
What is the ‘hacker mindset’ or ‘security mentality’? What do accidentally Turing-complete systems and weird machines have in common with heist movies or cons or stage magic? They all share a specific paradigm whose essence is about seeing through illusions to a truer more reduced reality.
What they/OP/security/speedrunning/hacking/social-engineering all have in common is that they show that the much-ballyhooed ‘hacker mindset’ is, fundamentally, a sort of reductionism run amok, where one ‘sees through’ abstractions to a manipulable reality. Like Neo in the Matrix—a deeply cliche analogy for hacking, but cliche because it resonates—one achieves enlightenment by seeing through the surface illusions of objects and can now see the endless lines of green code which make up the Matrix.
In each case, the fundamental principle is that the hacker asks: “here I have a system W, which pretends to be made out of a few Xs; however, it is really made out of many Y, which form an entirely different system, Z; I will now proceed to ignore the X and understand how Z works, so I may use the Y to thereby change W however I like”.
After running out of socks one day, I reflected on how ordinary tasks get neglected. Anecdotally and in 3 online surveys, people report often not having enough socks, a problem which correlates with rarity of sock purchases and demographic variables, consistent with a neglect/procrastination interpretation: because there is no specific time or triggering factor to replenish a shrinking sock stockpile, it is easy to run out.
This reminds me of akrasia on minor tasks, ‘yak shaving’, and the nature of disaster in complex systems: lack of hard rules lets errors accumulate, without any ‘global’ understanding of the drift into disaster (or at least inefficiency). Humans on a smaller scale also ‘drift’ when they engage in System I reactive thinking & action for too long, resulting in cognitive biases. An example of drift is the generalized human failure to explore/experiment adequately, resulting in overly greedy exploitative behavior of the current local optimum. Grocery shopping provides a case study: despite large gains, most people do not explore, perhaps because there is no established routine or practice involving experimentation. Fixes for these things can be seen as ensuring that System II deliberative cognition is periodically invoked to review things at a global level, such as developing a habit of maximum exploration at first purchase of a food product, or annually reviewing possessions to note problems like a lack of socks.
While socks may be small things, they may reflect big things.
I explore BigGAN, another recent GAN with SOTA results on the most complex image domain tackled by GANs so far, ImageNet. BigGAN’s capabilities come at a steep compute cost, however. I experiment with 128px ImageNet transfer learning (successful) with ~6 GPU-days, and from-scratch 256px anime portraits of 1000 characters on a 8×2080ti machine for a month (mixed results). My BigGAN results are good but compromised by practical problems with the released BigGAN code base. While BigGAN is not yet superior to StyleGAN for many purposes, BigGAN-like approaches may turn out to be necessary to scale to whole anime images.
Cat domestication and selective breeding have resulted in tens of breeds with major morphological differences. These breeds may also show distinctive behaviour differences; which, however, have been poorly studied. To improve the understanding of feline behaviour, we examined whether behavioural differences exist among cat breeds and whether behaviour is heritable. For these aims, we utilized our extensive health and behaviour questionnaire directed to cat owners and collected a survey data of 5726 cats. Firstly, for studying breed differences, we utilized logistic regression models with multiple environmental factors and discovered behaviour differences in 19 breeds and breed groups in ten different behaviour traits. Secondly, the studied cat breeds grouped into four clusters, with the Turkish Van and Angora cats alone forming one of them. These findings indicate that cat breeds have diverged not only morphologically but also behaviourally. Thirdly, we estimated heritability in three breeds and obtained moderate heritability estimates in seven studied traits, varying from 0.4 to 0.53, as well as phenotypic and genetic correlations for several trait pairs. Our results show that it is possible to partition the observed variation in behaviour traits into genetic and environmental components, and that substantial genetic variation exists within breed populations.
Evolutionary medicine uses evolutionary theory to help elucidate why humans are vulnerable to disease and disorders. I discuss two different types of evolutionary explanations that have been used to help understand human psychiatric disorders. First, a consistent finding is that psychiatric disorders are moderately to highly heritable, and many, such as schizophrenia, are also highly disabling and appear to decrease Darwinian fitness. Models used in evolutionary genetics to understand why genetic variation exists in fitness-related traits can be used to understand why risk alleles for psychiatric disorders persist in the population. The usual explanation for species-typical adaptations—natural selection—is less useful for understanding individual differences in genetic risk to disorders. Rather, two other types of models, mutation-selection-drift and balancing selection, offer frameworks for understanding why genetic variation in risk to psychiatric (and other) disorders exists, and each makes predictions that are now testable using whole-genome data. Second, species-typical capacities to mount reactions to negative events are likely to have been crafted by natural selection to minimize fitness loss. The pain reaction to tissue damage is almost certainly such an example, but it has been argued that the capacity to experience depressive symptoms such as sadness, anhedonia, crying, and fatigue in the face of adverse life situations may have been crafted by natural selection as well. I review the rationale and strength of evidence for this hypothesis. Evolutionary hypotheses of psychiatric disorders are important not only for offering explanations for why psychiatric disorders exist, but also for generating new, testable hypotheses and understanding how best to design studies and analyze data. [Keywords: evolution, psychiatric disorders, genetics, schizophrenia, depression]
A Russian scientist says he is planning to produce gene-edited babies, an act that would make him only the second person known to have done this. It would also fly in the face of the scientific consensus that such experiments should be banned until an international ethical framework has agreed on the circumstances and safety measures that would justify them.
Molecular biologist Denis Rebrikov has told Nature he is considering implanting gene-edited embryos into women, possibly before the end of the year if he can get approval by then. Chinese scientist He Jiankui prompted an international outcry when he announced last November that he had made the world's first gene-edited babies—twin girls.
…Rebrikov heads a genome-editing laboratory at Russia's largest fertility clinic, the Kulakov National Medical Research Center for Obstetrics, Gynecology and Perinatology in Moscow and is a researcher at the Pirogov Russian National Research Medical University, also in Moscow. According to Rebrikov he already has an agreement with an HIV centre in the city to recruit women infected with HIV who want to take part in the experiment…[he] plans to implant embryos only into a subset of HIV-positive mothers who do not respond to standard anti-HIV drugs. Their risk of transmitting the infection to the child is higher. If editing successfully disables the CCR5 gene, that risk would be greatly reduced, Rebrikov says. 'This is a clinical situation which calls for this type of therapy', he says.
Malaria control efforts require implementation of new technologies that manage insecticide resistance. Metarhizium pingshaense provides an effective, mosquito-specific delivery system for potent insect-selective toxins. A semifield trial in a MosquitoSphere (a contained, near-natural environment) in Soumousso, a region of Burkina Faso where malaria is endemic, confirmed that the expression of an insect-specific toxin (Hybrid) increased fungal lethality and the likelihood that insecticide-resistant mosquitoes would be eliminated from a site. Also, as Hybrid-expressing M. pingshaense is effective at very low spore doses, its efficacy lasted longer than that of the unmodified Metarhizium. Deployment of transgenic Metarhizium against mosquitoes could (subject to appropriate registration) be rapid, with products that could synergistically integrate with existing chemical control strategies to avert insecticide resistance.
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling. However, relying on corrupting the input with masks, BERT neglects dependency between the masked positions and suffers from a pretrain-finetune discrepancy. In light of these pros and cons, we propose XLNet, a generalized autoregressive pretraining method that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes the limitations of BERT thanks to its autoregressive formulation. Furthermore, XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pretraining. Empirically, under comparable experiment settings, XLNet outperforms BERT on 20 tasks, often by a large margin, including question answering, natural language inference, sentiment analysis, and document ranking.
The 2019 ICML edition of David Abel’s famous conference notes: he goes to as many presentations and talks as possible, jotting down opinionated summaries & equations, with a particular focus on DRL. Topics covered:
Tutorial: PAC-Bayes Theory (Part II) · PAC-Bayes Theory · PAC-Bayes and Task Awareness · Tutorial: Meta-Learning · Two Ways to View Meta-Learning · Meta-Learning Algorithms · Meta-Reinforcement Learning · Challenges and Frontiers in Meta Learning · Tuesday June: Main Conference Best Paper Talk: Challenging Assumptions in Learning Disentangled Representations Contributed Talks: Deep RL · DQN and Time Discretization · Nonlinear Distributional Gradient TD Learning · Composing Entropic Policies using Divergence Correction · TibGM: A Graphical Model Approach for RL · Multi-Agent Adversarial IRL · Policy Consolidation for Continual RL · Off-Policy Evaluation Deep RL w/o Exploration · Random Expert Distillation · Revisiting the Softmax Bellman Operator · Contributed Talks: RL Theory · Distributional RL for Efficient Exploration · Optimistic Policy Optimization via Importance Sampling · Neural Logic RL · Learning to Collaborate in MDPs · Predictor-Corrector Policy Optimization · Learning a Prior over Intent via Meta IRL · DeepMDP: Learning Late Space Models for RL · Importance Sampling Policy Evaluation · Learning from a Learner · Separating Value Functions Across Time-Scales · Learning Action Representations in RL · Bayesian Counterfactual Risk Minimization · Per-Decision Option Counting · Problem Dependent Regret Bounds in RL · A Theory of Regularized MDPs · Discovering Options for Exploration by Minimizing Cover Time · Policy Certificates: Towards Accountable RL · Action Robust RL · The Value Function Polytope · Wednesday June: Main Conference Contributed Talks: Multitask and Lifelong Learning · Domain Agnostic Learning with Disentangled Representations · Composing Value Functions in RL · CAVIA: Fast Context Adaptation via Meta Learning · Gradient Based Meta-Learning · Towards Understanding Knowledge Distillation · Transferable Adversarial Training · Contributed Talks: RL Theory · Provably Efficient Imitation Learning from Observation Alone · Dead Ends and Secure Exploration · Statistics and Samples in Distributional RL · Hessian Aided Policy Gradient · Maximum Entropy Exploration · Combining Multiple Models for Off-Policy Evaluation · Sample-Optimal ParametricQ-Learning Using Linear Features · Transfer of Samples in Policy Search · Exploration Conscious RL Revisited · Kernel Based RL in Robust MDPs · Thursday June: Main Conference Contributed Talks: RL · Batch Policy learning under Constraints · Quantifying Generalization in RL · Learning Latent Dynamics for Planning from Pixels · Projections for Approximate Policy Iteration · Learning Structured Decision Problems with Unawareness · Calibrated Model-Based Deep RL · RL in Configurable Continuous Environments · Target-Based Temporal-Difference Learning · Linearized Control: Stable Algorithms and Complexity Guarantees · Contributed Talks: Deep Learning Theory · Why do Larger Models Generalize Better? · On the Spectral Bias of Neural Nets · Recursive Sketches for Modular Deep Learning · Zero-Shot Knowledge Distillation in Deep Networks · Convergence Theory for Deep Learning via Over-Parameterization · Best Paper Award: Rates of Convergence for Sparse Gaussian Process Regression · Friday June: Workshops Workshop: AI for Climate Change · John Platt on What ML can do to help Climate Change · Jack Kelly: Why It’s Hard to Mitigate Climate Change, and How to Do Better, Andrew Ng: Tackling Climate Change with AI through Collaboration · Workshop: RL for Real Life · Panel Discussion · Workshop: Real World Sequential Decision Making · Emma Brunskill on Efficient RL When Data is Costly · Miro Dudik: Doubly Robust Off-Policy Evaluation via Shrinkage
[Review/discussion] Meta-RL is meta-learning on reinforcement learning tasks. After trained over a distribution of tasks, the agent is able to solve a new task by developing a new RL algorithm with its internal activity dynamics. This post starts with the origin of meta-RL and then dives into three key components of meta-RL…, a good meta-learning model is expected to generalize to new tasks or new environments that have never been encountered during training. The adaptation process, essentially a mini learning session, happens at test with limited exposure to the new configurations. Even without any explicit fine-tuning (no gradient backpropagation on trainable variables), the meta-learning model autonomously adjusts internal hidden states to learn.
It has been established that diverse behaviors spanning the controllable subspace of an Markov decision process can be trained by rewarding a policy for being distinguishable from other policies . However, one limitation of this formulation is generalizing behaviors beyond the finite set being explicitly learned, as is needed for use on subsequent tasks. Successor features provide an appealing solution to this generalization problem, but require defining the reward function as linear in some grounded feature space. In this paper, we show that these two techniques can be combined, and that each method solves the other’s primary limitation. To do so we introduce Variational Intrinsic Successor FeatuRes (VISR), a novel algorithm which learns controllable features that can be leveraged to provide enhanced generalization and fast task inference through the successor feature framework. We empirically validate VISR on the full Atari suite, in a novel setup wherein the rewards are only exposed briefly after a long unsupervised phase. Achieving human-level performance on 14 games and beating all baselines, we believe VISR represents a step towards agents that rapidly learn from limited feedback.
The history of learning for control has been an exciting back and forth between two broad classes of algorithms: planning and reinforcement learning. Planning algorithms effectively reason over long horizons, but assume access to a local policy and distance metric over collision-free paths. Reinforcement learning excels at learning policies and the relative values of states, but fails to plan over long horizons. Despite the successes of each method in various domains, tasks that require reasoning over long horizons with limited feedback and high-dimensional observations remain exceedingly challenging for both planning and reinforcement learning algorithms. Frustratingly, these sorts of tasks are potentially the most useful, as they are simple to design (a human only need to provide an example goal state) and avoid reward shaping, which can bias the agent towards finding a sub-optimal solution. We introduce a general control algorithm that combines the strengths of planning and reinforcement learning to effectively solve these tasks. Our aim is to decompose the task of reaching a distant goal state into a sequence of easier tasks, each of which corresponds to reaching a subgoal. Planning algorithms can automatically find these waypoints, but only if provided with suitable abstractions of the environment – namely, a graph consisting of nodes and edges. Our main insight is that this graph can be constructed via reinforcement learning, where a goal-conditioned value function provides edge weights, and nodes are taken to be previously seen observations in a replay buffer. Using graph search over our replay buffer, we can automatically generate this sequence of subgoals, even in image-based environments. Our algorithm, search on the replay buffer (SoRB), enables agents to solve sparse reward tasks over one hundred steps, and generalizes substantially better than standard RL algorithms.
Recent breakthroughs in AI for multi-agent games like Go, Poker, and Dota, have seen great strides in recent years. Yet none of these games address the real-life challenge of cooperation in the presence of unknown and uncertain teammates. This challenge is a key game mechanism in hidden role games. Here we develop the DeepRole algorithm, a multi-agent reinforcement learning agent that we test on The Resistance: Avalon, the most popular hidden role game. DeepRole combines counterfactual regret minimization (CFR) with deep value networks trained through self-play. Our algorithm integrates deductive reasoning into vector-form CFR to reason about joint beliefs and deduce partially observable actions. We augment deep value networks with constraints that yield interpretable representations of win probabilities. These innovations enable DeepRole to scale to the full Avalon game. Empirical game-theoretic methods show that DeepRole outperforms other hand-crafted and learned agents in five-player Avalon. DeepRole played with and against human players on the web in hybrid human-agent teams. We find that DeepRole outperforms human players as both a cooperator and a competitor.
The Resistance is a social role-playing card-based party game. The game's premise involves a war between government and resistance groups, and players are assigned various roles related to these groups. A King Arthur themed-variant with additional roles is marketed as Avalon. Like other deductive reasoning party games, The Resistance and Avalon rely on certain players attempting to disrupt the larger group working together, while the rest of the players work to reveal the spy working against them.
The ability to identify medical reversals and other low-value medical practices is an essential prerequisite for efforts to reduce spending on such practices. Through an analysis of more than 3000 randomized controlled trials (RCTs) published in three leading medical journals (the Journal of the American Medical Association, the Lancet, and the New England Journal of Medicine), we have identified 396 medical reversals. Most of the studies (92%) were conducted on populations in high-income countries, cardiovascular disease was the most common medical category (20%), and medication was the most common type of intervention (33%).
Figure 0: The “four main determinants of forecasting accuracy.” This graph can be found here, the GJP’s list of academic literature on this topic. The graph illustrates approximate relative effects. It will be discussed more in Section 2.
Experience and data from the Good Judgment Project (GJP) provide important evidence about how to make accurate predictions. For a concise summary of the evidence and what we learn from it, see this page. For a review of Superforecasting, the popular book written on the subject, see this blog.
This post explores the evidence in more detail, drawing from the book, the academic literature, the older Expert Political Judgment book, and an interview with a superforecaster.
…Tetlock describes how superforecasters go about making their predictions.56 Here is an attempt at a summary:
Sometimes a question can be answered more rigorously if it is first “Fermi-ized,” i.e. broken down into sub-questions for which more rigorous methods can be applied.
Next, use the outside view on the sub-questions (and/or the main question, if possible). You may then adjust your estimates using other considerations (‘the inside view’), but do this cautiously.
Seek out other perspectives, both on the sub-questions and on how to Fermi-ize the main question. You can also generate other perspectives yourself.
Repeat steps 1–3 until you hit diminishing returns.
Your final prediction should be based on an aggregation of various models, reference classes, other experts, etc.
When measures of individual differences are used to predict group performance, the reporting of correlations computed on samples of individuals invites misinterpretation and dismissal of the data. In contrast, if regression equations, in which the correlations required are computed on bivariate means, as are the distribution statistics, it is difficult to underappreciate or lightly dismiss the utility of psychological predictors. Given sufficient sample size and linearity of regression, this technique produces cross-validated regression equations that forecast criterion means with almost perfect accuracy. This level of accuracy is provided by correlations approaching unity between bivariate samples of predictor and criterion means, and this holds true regardless of the magnitude of the “simple” correlation (e.g., rxy = .20, or rxy = .80). We illustrate this technique empirically using a measure of general intelligence as the predictor and other measures of individual differences and socioeconomic status as criteria. In addition to theoretical applications pertaining to group trends, this methodology also has implications for applied problems aimed at developing policy in education, medical, and psychological clinics, business, industry, the military, and other domains of public welfare. Linkages between this approach and epidemiological research reinforce its utility as a tool for making decisions about policy.
[Book review of an anthropologist text arguing for imitation and extensive cultural group selection as the driving force of human civilization, with imitation of other humans being the unique human cognitive skill that gave us the edge over other primates and all animals, with any kind of raw intelligence being strictly minor. Further this extensive multi-level group selectionism implies that most knowledge is embodied in apparently-arbitrary cultural practices, such as traditional food preparation or divination or hunting rituals, which are effective despite lacking any observable rationale and the actual reasons for their efficacy are inaccessible to mere reason (except possibly by a far more advanced science).]
“Predicting History”, Joseph Risi, Amit Sharma, Rohan Shah, Matthew Connelly, Duncan J. Watts (2019-06-03):
Can events be accurately described as historic at the time they are happening? Claims of this sort are in effect predictions about the evaluations of future historians; that is, that they will regard the events in question as significant. Here we provide empirical evidence in support of earlier philosophical arguments1 that such claims are likely to be spurious and that, conversely, many events that will one day be viewed as historic attract little attention at the time. We introduce a conceptual and methodological framework for applying machine learning prediction models to large corpora of digitized historical archives. We find that although such models can correctly identify some historically important documents, they tend to over-predict historical significance while also failing to identify many documents that will later be deemed important, where both types of error increase monotonically with the number of documents under consideration. On balance, we conclude that historical significance is extremely difficult to predict, consistent with other recent work on intrinsic limits to predictability in complex social systems2,3. However, the results also indicate the feasibility of developing ‘artificial archivists’ to identify potentially historic documents in very large digital corpora.
[Profile of the US federal Senate. What has gone wrong with the Senate, which has descended into stasis, legal pettifogging, legislative tactics, pointless procedures, and a total loss of collegiality and bipartisan links? Senators spend as much time fundraising as legislating, bills are ghostwritten, committees chair empty meetings, and measures to increase transparency and public accountability appear to have done nothing, or rather, outright backfired, eliminating the Senate’s role as a body of statesmen detached from immediate political passions. Some trace it back to the Republican wave of the 1970s, eliminating many of the experienced senators and bringing in small-government ideologues, and to the simultaneous creation of C-SPAN to broadcast Senatorial proceedings to the watching public. The problem with the Senate and American democracy may be too little Senate and too much democracy.]
The Romanian Revolution was a period of civil unrest in Romania during December 1989 as a part of the Revolutions of 1989 that occurred in several countries. The Romanian Revolution started in the city of Timișoara and soon spread throughout the country, ultimately culminating in the show trial and execution of longtime Communist Party General Secretary Nicolae Ceaușescu and his wife Elena, and the end of 42 years of Communist rule in Romania. It was also the last removal of a Marxist-Leninist government in a Warsaw Pact country during the events of 1989, and the only one that violently overthrew a country's government and executed its leader.
The automatic assessment of psychological traits from digital footprints allows researchers to study psychological traits at unprecedented scale and in settings of high ecological validity. In this research, we investigated whether spending records—a ubiquitous and universal form of digital footprint—can be used to infer psychological traits. We applied an ensemble machine-learning technique ( random-forest modeling) to a data set combining two million spending records from bank accounts with survey responses from the account holders (n = 2,193). Our predictive accuracies were modest for the Big Five personality traits ( r = 0.15, corrected ρ = 0.21) but provided higher precision for specific traits, including materialism ( r = 0.33, corrected ρ = 0.42). We compared the predictive accuracy of these models with the predictive accuracy of alternative digital behaviors used in past research, including those observed on social media platforms, and we found that the predictive accuracies were relatively stable across socioeconomic groups and over time.
The understanding, quantification and evaluation of individual differences in behavior, feelings and thoughts have always been central topics in psychological science. An enormous amount of previous work on individual differences in behavior is exclusively based on data from self-report questionnaires. To date, little is known about how individuals actually differ in their objectively quantifiable behaviors and how differences in these behaviors relate to big five personality traits. Technological advances in mobile computer and sensing technology have now created the possibility to automatically record large amounts of data about humans’ natural behavior. The collection and analysis of these records makes it possible to analyze and quantify behavioral differences at unprecedented scale and efficiency. In this study, we analyzed behavioral data obtained from 743 participants in 30 consecutive days of smartphone sensing (25,347,089 logging-events). We computed variables (15,692) about individual behavior from five semantic categories (communication & social behavior, music listening behavior, app usage behavior, mobility, and general daytime & nighttime activity). Using a machine learning approach (random forest, elastic net), we show how these variables can be used to predict self-assessments of the big five personality traits at the factor and facet level. Our results reveal distinct behavioral patterns that proved to be differentially-predictive of big five personality traits. Overall, this paper shows how a combination of rich behavioral data obtained with smartphone sensing and the use of machine learning techniques can help to advance personality research and can inform both practitioners and researchers about the different behavioral patterns of personality.
The stereotype threat literature primarily comprises lab studies, many of which involve features that would not be present in high-stakes testing settings. We meta-analyze the effect of stereotype threat on cognitive ability tests, focusing on both laboratory and operational studies with features likely to be present in high stakes settings. First, we examine the features of cognitive ability test metric, stereotype threat cue activation strength, and type of non-threat control group, and conduct a focal analysis removing conditions that would not be present in high stakes settings. We also take into account a previously unrecognized methodological error in how data are analyzed in studies that control for scores on a prior cognitive ability test, which resulted in a biased estimate of stereotype threat. The focal sample, restricting the database to samples utilizing operational testing-relevant conditions, displayed a threat effect of d = −0.14 (k = 45, N = 3,532, SDδ = 0.31). Second, we present a comprehensive meta-analysis of stereotype threat. Third, we examine a small subset of studies in operational test settings and studies utilizing motivational incentives, which yielded d-values ranging from 0.00 to −0.14. Fourth, the meta-analytic database is subjected to tests of publication bias, finding nontrivial evidence for publication bias. Overall, results indicate that the size of the stereotype threat effect that can be experienced on tests of cognitive ability in operational scenarios such as college admissions tests and employment testing may range from negligible to small.
Some early experimental studies with LSD suggested that doses of LSD too small to cause any noticeable effects may improve mood and creativity. Prompted by recent discussion of this claim and the purely anecdotal subsequent evidence for it, I decided to run a well-powered randomized blind trial of 3-day LSD microdoses from September 2012 to March 2013. No beneficial effects reached statistical-significance and there were worrisome negative trends. LSD microdosing did not help me.
The mind-altering agents such as tobacco, cannabis, and opium have been widely used since the evolution of human being. These substances have been widely used for recreational purposes. However, derivatives from reptiles such as snakes, reptiles, and scorpions can also be used for recreational purposes and as a substitute for other substances. Their use is rare and related literature is very scanty. In this report, we present a case of snake venom abuse and review the existing literature.
A defining feature of modern economic growth is the systematic application of science to advance technology. However, despite sustained progress in scientific knowledge, recent productivity growth in the United States has been disappointing. We review major changes in the American innovation ecosystem over the past century. The past three decades have been marked by a growing division of labor between universities focusing on research and large corporations focusing on development. Knowledge produced by universities is not often in a form that can be readily digested and turned into new goods and services. Small firms and university technology transfer offices cannot fully substitute for corporate research, which had previously integrated multiple disciplines at the scale required to solve significant technical problems. Therefore, whereas the division of innovative labor may have raised the volume of science by universities, it has also slowed, at least for a period of time, the transformation of that knowledge into novel products and processes.
The concept of multiple discovery is the hypothesis that most scientific discoveries and inventions are made independently and more or less simultaneously by multiple scientists and inventors. The concept of multiple discovery opposes a traditional view—the "heroic theory" of invention and discovery.
Apocryphal letters claiming divine origin circulated for centuries in Europe. After 1900, shorter more secular letters appeared in the US that promised good luck if copies were distributed and bad luck if not. Billions of these “luck chain letters” circulated in the the next 100 years. As they replicated through the decades, some accumulated copying errors, offhand comments, and calculated innovations that helped them prevail in the competition with other chain letters. For example, complementary testimonials developed, one exploiting perceived good luck, another exploiting perceived bad luck. Twelve successive types of paper luck chain letters are identified which predominated US circulation at some time in the twentieth century. These types, and their major variations, are described and analyzed for their replicative advantage.
In the 1970’s a luck chain letter from South America that touted a lottery winner invaded the US and was combined on one page with an indigenous chain letter. This combination rapidly dominated circulation. In 1979 a postscript concluding with “It Works” was added to one of these combination letters, and within a few years the progeny of this single letter had replaced all the millions of similar letters in circulation without this postscript. These and other events in paper chain letter history are described, and hypotheses are offered to explain advances and declines in circulation, including the near extinction of luck chain letters in the new millennium.
Perhaps the most dramatic event in chain letter history was the advent of money chain letters. This was spawned by the infamous “Send-a-Dime” chain letter which flooded the world in 1935. The insight and methods of its anonymous author, likely a woman motivated by charity, are examined in detail in a separate article titled “The Origin of Money Chain Letters.” This constitutes Section 4.1 below, where its link is repeated. It can be read independently from this treatise.
The online Paper Chain Letter Archive contains the text and documentation of over 900 chain letters. Most of these texts have been transcribed from collected physical letters, but many come from published sources including daily newspapers present in online searchable archives. Some unusual items in the archive are: an anonymous 1917 chain letter giving advice on obtaining conscientious objector status; a 1920 Sinn Féin revolutionary communication; rare unpublished scatological parody letters from 1935; a bizarre chain letter invitation to a suicide from 1937; and a libelous Proctor and Gamble boycott alleging satanism from 1986. An annotated index provides easy access to all chain letters in the archive. An Annotated Bibliography on Chain Letters and Pyramid Schemes contains over 425 entries. A Glossary gives precise definitions for terms used here, facilitating the independent reading of sections.
Edmund Wade Davis is a Canadian cultural anthropologist, ethnobotanist, author, and photographer. Davis came to prominence with his 1985 best-selling book The Serpent and the Rainbow about the zombies of Haiti. He is professor of anthropology and the BC Leadership Chair in Cultures and Ecosystems at Risk at the University of British Columbia.
So I illustrate the relevance of labour relations to economic development through the contrasting fortunes of India’s and Japan’s cotton textile industries in the interwar period, with some glimpses of Lancashire, the USA, interwar Shanghai, etc.
TL;DR version: At the beginning of the 20th century, the Indian and the Japanese textile industries had similar levels of wages and productivity, and both were exporting to global markets. But by the 1930s, Japan had surpassed the UK to become the world’s dominant exporter of textiles; while the Indian industry withdrew behind the tariff protection of the British Raj. Technology, human capital, and industrial policy were minor determinants of this divergence, or at least they mattered conditional on labour relations.
Indian textile mills were obstructed by militant workers who defended employment levels, resisted productivity-enhancing measures, and demanded high wages relative to effort. But Japanese mills suppressed strikes and busted unions; extracted from workers much greater effort for a given increase in wages; and imposed technical & organisational changes at will. The bargaining position of workers was much weaker in Japan than in India, because Japan had a true “surplus labour” economy with a large number of workers ‘released’ from agriculture into industry. But late colonial India was rather ‘Gerschenkronian’, where employers’ options were more limited by a relatively inelastic supply of labour.
The state also mattered. The British Raj did little to restrain on behalf of Indian capitalists the exercise of monopoly power by Indian workers. Britain had neither the incentive, nor the stomach, nor the legitimacy to do much about it. But a key element of the industrial policy of the pre-war Japanese state was repression of the labour movement.
Note: By “labour repression” I do not mean coercing workers, or suppressing wage levels, but actions which restrain the effects of worker combinations.
Nor am I saying unions are bad! I’ve written before that unions in Germany are great.
Also, I do not claim this post has any relevance for today’s developed countries. It’s mainly about labour-intensive manufacturing in historical industrialisation or in today’s developing countries.
Fujiwara Sadaie (藤原定家), better-known as Fujiwara no Teika, was a Japanese poet, critic, calligrapher, novelist, anthologist, scribe, and scholar of the late Heian and early Kamakura periods. His influence was enormous, and he is counted as among the greatest of Japanese poets, and perhaps the greatest master of the waka form – an ancient poetic form consisting of five lines with a total of 31 syllables.
Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.
Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal. To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale. We find that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator’s input. Our modifications lead to models which set the new state of the art in class-conditional image synthesis. When trained on ImageNet at 128×128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4, improving over the previous best IS of 52.52 and FID of 18.6.
We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. To quantify interpolation quality and disentanglement, we propose two new, automated methods that are applicable to any generator architecture. Finally, we introduce a new, highly varied and high-quality dataset of human faces.
We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.