newsletter/2019/09 (Link Bibliography)

“newsletter/​2019/​09” links:

  1. https://gwern.substack.com

  2. 08

  3. newsletter

  4. Changelog

  5. https://www.patreon.com/gwern

  6. Embryo-selection#sperm-phenotype-selection

  7. Statistical-notes#selective-emigration-and-personality-trait-change

  8. Inflation.hs: ⁠, Gwern Branwen (2019-03-27):

    Experimental Pandoc module for implementing automatic inflation adjustment of nominal date-stamped dollar or amounts to provide real prices; Bitcoin’s exchange rate has moved by multiple orders of magnitude over its early years (rendering nominal amounts deeply unintuitive), and this is particularly critical in any economics or technology discussion where a nominal price from 1950 is 11× the 2019 real price!

    Years/​​​​dates are specified in a variant of my interwiki link syntax; for example: $50 or [₿0.5]​(₿2017-01-01), giving link adjustments which compile to something like like <span class="inflationAdjusted" data-originalYear="2017-01-01" data-originalAmount="50.50" data-currentYear="2019" data-currentAmount="50,500">₿50.50<span class="math inline"><sub>2017</sub><sup>$50,500</sup></span></span>.

    Dollar amounts use year, and Bitcoins use full dates, as the greater temporal resolution is necessary. Inflation rates/​​​​exchange rates are specified as constants and need to be manually updated every once in a while; if out of date, the last available rate is carried forward for future adjustments.

  9. ⁠, Javier de la Fuente, Gail Davies, Andrew D. Grotzinger, Elliot M. Tucker-Drob, Ian J. Deary (2019-09-12):

    It has been known for 115 years that, in humans, diverse cognitive traits are positively intercorrelated; this forms the basis for the general factor of intelligence (g). We directly test for a genetic basis for g using data from seven different cognitive tests (n = 11,263 to n = 331,679) and genome-wide autosomal single nucleotide polymorphisms. A genetic g factor accounts for 58.4% (SE = 4.8%) of the genetic in the cognitive traits, with trait-specific genetic factors accounting for the remaining 41.6%. We distill genetic loci broadly relevant for many cognitive traits (g) from loci associated with only individual cognitive traits. These results elucidate the etiological basis for a long-known yet poorly-understood phenomenon, revealing a fundamental dimension of genetic sharing across diverse cognitive traits.

  10. ⁠, Varun Warrier, Roberto Toro, Hyejung Won, Claire S. Leblond, Freddy Cliquet, Richard Delorme, Ward De Witte, Janita Bralten, Bhismadev Chakrabarti, Anders D. Børglum, Jakob Grove, Geert Poelmans, David A. Hinds, Thomas Bourgeron, Simon Baron-Cohen (2019-09-03):

    The core diagnostic criteria for autism comprise two symptom domains: social and communication difficulties, and unusually repetitive and restricted behaviour, interests and activities. There is some evidence to suggest that these two domains are dissociable, though this hypothesis has not yet been tested using molecular genetics. We test this using a genome-wide association study (n = 51,564) of a non-social trait related to autism, systemising, defined as the drive to analyse and build systems. We demonstrate that systemising is heritable and genetically correlated with autism. In contrast, we do not identify statistically-significant between social autistic traits and systemising. Supporting this, polygenic scores for systemising are statistically-significantly and positively associated with restricted and repetitive behaviour but not with social difficulties in autistic individuals. These findings strongly suggest that the two core domains of autism are genetically dissociable, and point at how to fractionate the genetics of autism.

  11. 2019-pavan.pdf: ⁠, William J. Pavan, Richard A. Sturm (2019-01-01; genetics  /​ ​​ ​heritable):

    Human skin and hair color are visible traits that can vary dramatically within and across ethnic populations. The genetic makeup of these traits—including polymorphisms in the enzymes and signaling proteins involved in melanogenesis, and the vital role of ion transport mechanisms operating during the maturation and distribution of the melanosome—has provided new insights into the regulation of pigmentation. A large number of novel loci involved in the process have been recently discovered through four large-scale in Europeans, two large genetic studies of skin color in Africans, one study in Latin Americans, and functional testing in animal models. The responsible polymorphisms within these pigmentation genes appear at different population frequencies, can be used as ancestry-informative markers, and provide insight into the evolutionary selective forces that have acted to create this human diversity.

  12. ⁠, Loic Yengo, Naomi R. Wray, Peter M. Visscher (2019-09-03):

    In most human societies, there are taboos and laws banning mating between first-degreee and second-degree relatives, but actual prevalence and effects on health and fitness are poorly quantified. Here, we leverage a large observational study of ~450,000 participants of European ancestry from the UK Biobank (UKB) to quantify extreme inbreeding (EI) and its consequences. We use genotyped SNPs to detect large runs of (ROH) and call EI when >10% of an individual’s genome comprise ROHs. We estimate a prevalence of EI of ~0.03%, ie., ~1⁄3652. EI cases have phenotypic means between 0.3 and 0.7 standard deviation below the population mean for 7 traits, including stature and cognitive ability, consistent with estimated from individuals with low levels of inbreeding. Our study provides DNA-based quantification of the prevalence of EI in a European ancestry sample from the UK and measures its effects on health and fitness traits.

    In most human societies, there are taboos and laws banning mating between first-degree and second-degree relatives, but actual prevalence and effects on health and fitness are poorly quantified. Here, we leverage a large observational study of ~450,000 participants of European ancestry from the UK Biobank (UKB) to quantify extreme inbreeding (EI) and its consequences. We use genotyped SNPs to detect large runs of homozygosity (ROH) and call EI when >10% of an individual’s genome comprise ROHs. We estimate a prevalence of EI of ~0.03%, ie., ~1⁄3652. EI cases have phenotypic means between 0.3 and 0.7 standard deviation below the population mean for 7 traits, including stature and cognitive ability, consistent with inbreeding depression estimated from individuals with low levels of inbreeding. Our study provides DNA-based quantification of the prevalence of EI in a European ancestry sample from the UK and measures its effects on health and fitness traits.

  13. https://www.ukbiobank.ac.uk/learn-more-about-uk-biobank/news/uk-biobank-leads-the-way-in-genetics-research-to-tackle-chronic-diseases-1

  14. ⁠, Kevin M. Wright, Kristin A. Rand, Amir Kermany, Keith Noto, Don Curtis, Daniel Garrigan, Dmitri Slinkov, Ilya Dorfman, Julie M. Granka, Jake Byrnes, Natalie Myres, Catherine A. Ball, J. Graham Ruby (2019-09):

    We present a massive investigation into the genetic basis of human lifespan. Beginning with a genome-wide association (GWAS) study using a de-identified snapshot of the unique AncestryDNA database—more than 300,000 genotyped individuals linked to pedigrees of over 400,000,000 people—we mapped six genome-wide statistically-significant loci associated with parental lifespan. We compared these results to a GWA analysis of the traditional lifespan proxy trait, age, and found only one locus, APOE, to be associated with both age and lifespan. By combining the AncestryDNA results with those of an independent UK Biobank dataset, we conducted a of more than 650,000 individuals and identified fifteen parental lifespan-associated loci. Beyond just those loci, our genome-wide set of polymorphisms accounts for up to 8% of the variance in human lifespan; this value represents a large fraction of the heritability estimated from phenotypic correlations between relatives.

  15. 2019-zheng.pdf: ⁠, Yi Zheng, Xufeng Xue, Yue Shao, Sicong Wang, Sajedeh Nasr Esfahani, Zida Li, Jonathon M. Muncie, Johnathon N. Lakins, Valerie M. Weaver, Deborah L. Gumucio, Jianping Fu (2019-09-11; genetics  /​ ​​ ​editing):

    Early human embryonic development involves extensive lineage diversification, cell-fate specification and tissue patterning1. Despite its basic and clinical importance, early human embryonic development remains relatively unexplained owing to interspecies divergence2,3 and limited accessibility to human embryo samples. Here we report that human pluripotent stem cells (hPSCs) in a microfluidic device recapitulate, in a highly controllable and scalable fashion, landmarks of the development of the epiblast and amniotic ectoderm parts of the conceptus, including lumenogenesis of the epiblast and the resultant pro-amniotic cavity, formation of a bipolar embryonic sac, and specification of primordial germ cells and primitive streak cells. We further show that amniotic ectoderm-like cells function as a signalling centre to trigger the onset of gastrulation-like events in hPSCs. Given its controllability and scalability, the microfluidic model provides a powerful experimental system to advance knowledge of human embryology and reproduction. This model could assist in the rational design of differentiation protocols of hPSCs for disease modelling and cell therapy, and in high-throughput drug and toxicity screens to prevent pregnancy failure and birth defects.

  16. https://www.technologyreview.com/s/614292/meet-the-artificial-embryos-being-called-uncanny-and-spectacular/

  17. https://www.nature.com/articles/d41586-019-02563-y

  18. {#linkBibliography-atlantic)-2012 .docMetadata}, Alexis C. Madrigal () (2012-05-01):

    …Already, Badger-Bluff Fanny Freddie has 346 daughters who are on the books and thousands more that will be added to his progeny count when they start producing milk. This is quite a career for a young animal: He was only born in 2004.

    There is a reason, of course, that the semen that Badger-Bluff Fanny Freddie produces has become such a hot commodity in what one artificial-insemination company calls “today’s fast paced cattle semen market.” In January of 2009, before he had a single daughter producing milk, the United States Department of Agriculture took a look at his lineage and more than 50,000 markers on his genome and declared him the best bull in the land. And, three years and 346 milk-providing and data-providing daughters later, it turns out that they were right. “When Freddie [as he is known] had no daughter records our equations predicted from his DNA that he would be the best bull”, USDA research geneticist Paul VanRaden emailed me with a detectable hint of pride. “Now he is the best progeny tested bull (as predicted).”

    Data-driven predictions are responsible for a massive transformation of America’s dairy cows. While other industries are just catching on to this whole “big data” thing, the animal sciences—and dairy breeding in particular—have been using large amounts of data since long before VanRaden was calculating the outsized genetic impact of the most sought-after bulls with a pencil and paper in the 1980s. Dairy breeding is perfect for quantitative analysis. records have been assiduously kept; relatively easy artificial insemination has helped centralized genetic information in a small number of key bulls since the 1960s; there are a relatively small and easily measurable number of traits—milk production, fat in the milk, protein in the milk, longevity, udder quality—that breeders want to optimize; each cow works for three or four years, which means that farmers invest thousands of dollars into each animal, so it’s worth it to get the best semen money can buy. The economics push breeders to use the genetics.

    The bull market (heh) can be reduced to one key statistic, lifetime net merit, though there are many nuances that the single number cannot capture. Net merit denotes the likely additive value of a bull’s genetics. The number is actually denominated in dollars because it is an estimate of how much a bull’s genetic material will likely improve the revenue from a given cow. A very complicated equation weights all of the factors that go into dairy breeding and—voila—you come out with this single number. For example, a bull that could help a cow make an extra 1000 pounds of milk over her lifetime only gets an increase of $1$12012 in net merit while a bull who will help that same cow produce a pound more protein will get $4.32$3.412012 more in net merit. An increase of a single month of predicted productive life yields $44$352012 more.

    …In 1942, when my father was born, the average dairy cow produced less than 5,000 pounds of milk in its lifetime. Now, the average cow produces over 21,000 pounds of milk. At the same time, the number of dairy cows has decreased from a high of 25 million around the end of World War II to fewer than nine million today…a mere 70 years of quantitative breeding optimized to suit corporate imperatives quadrupled what all previous civilization had accomplished.

    …John Cole, yet another USDA animal improvement scientist, by choosing the optimal observed genetic sequences and hypothetically combining them. He found that the optimal bull would have a net merit value of $9,702$7,5152011, which absolutely blows any current bull out of the water. In other words, we’re nowhere near creating the perfect milk machine.

  19. ⁠, J. B. Cole, P. M. VanRaden (2011):

    Limits to selection and Mendelian sampling (MS) terms can be calculated using haplotypes by summing the individual additive effects on each chromosome. Haplotypes were imputed for 43 382 single nucleotide polymorphisms () in 1455 Brown Swiss, 40 351 Holstein and4064 Jersey bulls and cows using the Fortran program findhap.f90, which combines population and pedigree haplotyping methods. Lower and upper bounds of MS variance were calculated for daughter pregnancy rate (a measure of fertility), milk yield, lifetime net merit (a measure of profitability) and protein yield assuming either no or complete linkage among SNP on the same chromosome. Calculated selection limits were greater than the largest direct genomic values observed in all breeds studied. The best chromosomal genotypes generally consisted of two copies of the same haplotype even after adjustment for inbreeding. Selection of animals rather than chromosomes may result in slower progress, but limits may be the same because most chromosomes will become homozygous with either strategy. Selection on functions of MS could be used to change variances in later generations.

    Lifetime net merit: Lower selection limits for NM\$ with no adjustment for inbreeding were $4,980$38572011 (BS), $9,702$75152011 (HO) and $6,040$46782011 (JE). Adjusted values were slightly smaller and were $4,928$38172011 (BS), $9,675$74942011 (HO) and $5,947$46062011 (JE). Upper bounds had values of $11,800$91402011 (BS), $30,454$235882011 (HO) and $14,869$115172011 (JE) and were not adjusted for inbreeding because they were calculated from individual loci rather than complete haplotypes. The largest DGV among all genotyped animals in each breed were $1,423$11022011 (BS), $3,264$25282011 (HO) and $2,009$15562011 (JE). The top active bulls (AI and foreign bulls with semen distributed in the US that are in or above the 80th percentile, based on NM) in each breed following the August 2010 genetic evaluation had GEBV (Genomic estimated breeding value) for NM\$ of +$1,412$10942011 (BS: 054BS00374), +$2,050$15882011 (HO: 001HO08784) and +$1,668$12922011 (JE: 236JE00146).

    …If two copies of each of the 30 best haplotypes in the US Holstein population were combined in a single animal (Lower bounds of selection limit/​​​​SLC for NM\$), it would have a GEBV for NM\$ of +$9,702$75152011 (Figure 5), approximately five times larger than that of the current best Holstein bull in the US, whose GEBV for NM\$ are +$2,050$15882011.

  20. ⁠, Alphey, Nina Bonsall, Michael B (2018):

    The sterile insect technique is an area-wide pest control method that reduces agricultural pest populations by releasing mass-reared sterile insects, which then compete for mates with wild insects. Contemporary genetics-based technologies use insects that are homozygous for a repressible dominant lethal genetic construct rather than being sterilized by irradiation.Engineered strains of agricultural pest species, including moths such as the diamondback moth Plutella xylostella and fruit flies such as the Mediterranean fruit fly Ceratitis capitata, have been developed with lethality that only operates on females.Transgenic crops expressing insecticidal toxins are widely used; the economic benefits of these crops would be lost if toxin resistance spread through the pest population. The primary resistance management method is a high-dose/​​​​refuge strategy, requiring toxin-free crops as refuges near the insecticidal crops, as well as toxin doses sufficiently high to kill wild-type insects and insects heterozygous for a resistance allele.Mass-release of toxin-sensitive engineered males (carrying female-lethal genes), as well as suppressing populations, could substantially delay or reverse the spread of resistance. These transgenic insect technologies could form an effective resistance management strategy.We outline some policy considerations for taking genetic insect control systems through to field implementation.

  21. ⁠, Jeff Clune (2019-05-27):

    Perhaps the most ambitious scientific quest in human history is the creation of general artificial intelligence, which roughly means AI that is as smart or smarter than humans. The dominant approach in the machine learning community is to attempt to discover each of the pieces required for intelligence, with the implicit assumption that some future group will complete the Herculean task of figuring out how to combine all of those pieces into a complex thinking machine. I call this the “manual AI approach”. This paper describes another exciting path that ultimately may be more successful at producing general AI. It is based on the clear trend in machine learning that hand-designed solutions eventually are replaced by more effective, learned solutions. The idea is to create an AI-generating algorithm (AI-GA), which automatically learns how to produce general AI. Three Pillars are essential for the approach: (1) meta-learning architectures, (2) meta-learning the learning algorithms themselves, and (3) generating effective learning environments. I argue that either approach could produce general AI first, and both are scientifically worthwhile irrespective of which is the fastest path. Because both are promising, yet the ML community is currently committed to the manual approach, I argue that our community should increase its research investment in the AI-GA approach. To encourage such research, I describe promising work in each of the Three Pillars. I also discuss AI-GA-specific safety and ethical considerations. Because it it may be the fastest path to general AI and because it is inherently scientifically interesting to understand the conditions in which a simple algorithm can produce general AI (as happened on Earth where Darwinian evolution produced human intelligence), I argue that the pursuit of AI-GAs should be considered a new grand challenge of computer science research.

  22. ⁠, Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, Geoffrey Irving (2019-09-18):

    Reward learning enables the application of reinforcement learning (RL) to tasks where reward is defined by human judgment, building a model of reward by asking humans questions. Most work on reward learning has used simulated environments, but complex information about values is often expressed in natural language, and we believe reward learning for language is a key to making RL practical and safe for real-world tasks. In this paper, we build on advances in generative pretraining of language models to apply reward learning to four natural language tasks: continuing text with positive sentiment or physically descriptive language, and summarization tasks on the TL;DR and ⁠/​​​​Daily Mail datasets. For stylistic continuation we achieve good results with only 5,000 comparisons evaluated by humans. For summarization, models trained with 60,000 comparisons copy whole sentences from the input but skip irrelevant preamble; this leads to reasonable scores and very good performance according to our human labelers, but may be exploiting the fact that labelers rely on simple heuristics.

  23. ⁠, Daniel Ziegler, Nisan Stiennon, Jeffrey Wu, Tom Brown, Dario Amodei, Alec Radford, Paul Christiano, Geoffrey Irving () (2019-09-19):

    We’ve fine-tuned the 774M parameter language model using human feedback for various tasks, successfully matching the preferences of the external human labelers, though those preferences did not always match our own. Specifically, for summarization tasks the labelers preferred sentences copied wholesale from the input (we’d only asked them to ensure accuracy), so our models learned to copy. Summarization required 60k human labels; simpler tasks which continue text in various styles required only 5k. Our motivation is to move safety techniques closer to the general task of “machines talking to humans”, which we believe is key to extracting information about human values.

    This work applies human preference learning to several natural language tasks: continuing text with positive sentiment or physically descriptive language using the BookCorpus, and summarizing content from the TL;DR and CNN/​​​​Daily Mail datasets. Each of these tasks can be viewed as a text completion problem: starting with some text X, we ask what text Y should follow. [For summarization, the text is the article plus the string “TL;DR:”.]

    We start with a pretrained language model (the 774M parameter version of GPT-2) and fine-tune the model by asking human labelers which of four samples is best. Fine-tuning for the stylistic continuation tasks is sample efficient: 5,000 human samples suffice for strong performance according to humans. For summarization, models trained with 60,000 comparisons learn to copy whole sentences from the input while skipping irrelevant preamble; this copying is an easy way to ensure accurate summaries, but may exploit the fact that labelers rely on simple heuristics.

    Bugs can optimize for bad behavior

    One of our code refactors introduced a bug which flipped the sign of the reward. Flipping the reward would usually produce incoherent text, but the same bug also flipped the sign of the KL penalty. The result was a model which optimized for negative sentiment while preserving natural language. Since our instructions told humans to give very low ratings to continuations with sexually explicit text, the model quickly learned to output only content of this form. This bug was remarkable since the result was not gibberish but maximally bad output. The authors were asleep during the training process, so the problem was noticed only once training had finished. A mechanism such as Toyota’s Andon cord could have prevented this, by allowing any labeler to stop a problematic training process.

    Looking forward

    We’ve demonstrated reward learning from human preferences on two kinds of natural language tasks, stylistic continuation and summarization. Our results are mixed: for continuation we achieve good results with very few samples, but our summarization models are only “smart copiers”: they copy from the input text but skip over irrelevant preamble. The advantage of smart copying is truthfulness: the zero-shot and supervised models produce natural, plausible-looking summaries that are often lies. We believe the limiting factor in our experiments is data quality exacerbated by the online data collection setting, and plan to use batched data collection in the future.

    We believe the application of reward learning to language is important both from a capability and safety perspective. On the capability side, lets us correct mistakes that supervised learning would not catch, but RL with programmatic reward functions “can be detrimental to model quality.” On the safety side, reward learning for language allows important criteria like “don’t lie” to be represented during training, and is a step towards scalable safety methods such as a debate and amplification. [Followup: ⁠, Stiennon et al 2020.]

  24. ⁠, Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, Geoffrey Irving (OpenAI) (2019-09-14):

    Code for the paper ‘Fine-Tuning Language Models from Human Preferences’. Status: Archive (code is provided as-is, no updates expected). We provide code for:

    • Training reward models from human labels
    • Fine-tuning language models using those reward models

    It does not contain code for generating labels. However, we have released human labels collected for our experiments, at gs://lm-human-preferences/labels. For those interested, the question and label schemas are simple and documented in label_types.py.

    The code has only been tested using the smallest model (124M parameters). This code has only been tested using Python 3.7.3. Training has been tested on GCE machines with 8 s, running Ubuntu 16.04, but development also works on Mac OS X.

  25. ⁠, Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro (2019-09-17):

    Recent work in language modeling demonstrates that training large transformer models advances the state of the art in Natural Language Processing applications. However, very large models can be quite difficult to train due to memory constraints. In this work, we present our techniques for training very large transformer models and implement a simple, efficient intra-layer model parallel approach that enables training transformer models with billions of parameters. Our approach does not require a new compiler or library changes, is orthogonal and complimentary to pipeline model parallelism, and can be fully implemented with the insertion of a few communication operations in native PyTorch. We illustrate this approach by converging transformer based models up to 8.3 billion parameters using 512 GPUs. We sustain 15.1 PetaFLOPs across the entire application with 76% scaling efficiency when compared to a strong single baseline that sustains 39 TeraFLOPs, which is 30% of peak FLOPs. To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8.3 billion parameter transformer language model similar to GPT-2 and a 3.9 billion parameter model similar to BERT. We show that careful attention to the placement of layer normalization in BERT-like models is critical to achieving increased performance as the model size grows. Using the GPT-2 model we achieve SOTA results on the WikiText103 (10.8 compared to SOTA perplexity of 15.8) and (66.5% compared to SOTA accuracy of 63.2%) datasets. Our BERT model achieves SOTA results on the RACE dataset (90.9% compared to SOTA accuracy of 89.4%).

  26. https://arxiv.org/pdf/1909.08053.pdf#page=13

  27. ⁠, NVIDIA ADLR (2019-08-13):

    Larger language models are dramatically more useful for NLP tasks such as article completion, question answering, and dialog systems. Training the largest neural language model has recently been the best way to advance the state of the art in NLP applications. Two recent papers, and ⁠, demonstrate the benefits of large scale language modeling. Both papers leverage advances in compute and available text corpora to substantially surpass state of the art performance in natural language understanding, modeling, and generation. Training these models requires hundreds of exaflops of compute and to trade recomputation for a reduced memory footprint. However, for very large models beyond a billion parameters, the memory on a single GPU is not enough to fit the model along with the parameters needed for training, requiring model parallelism to split the parameters across multiple GPUs. Several approaches to model parallelism exist, but they are difficult to use, either because they rely on custom compilers, or because they scale poorly or require changes to the optimizer.

    In this work, we implement a simple and efficient model parallel approach by making only a few targeted modifications to existing PyTorch transformer implementations. Our code is written in native Python, leverages mixed precision training, and utilizes the NCCL library for communication between GPUs. We showcase this approach by training an 8.3 billion parameter transformer language model with 8-way model parallelism and 64-way data parallelism on 512 GPUs, making it the largest transformer based language model ever trained at 24× the size of BERT and 5.6× the size of GPT-2. We have published the code that implements this approach at our GitHub repository⁠.

    Our experiments are conducted on NVIDIA’s DGX SuperPOD⁠. Without model parallelism, we can fit a baseline model of 1.2B parameters on a single 32GB GPU, and sustain 39 TeraFLOPS during the overall training process, which is 30% of the theoretical peak FLOPS for a single GPU in a -H server. Scaling the model to 8.3 billion parameters on 512 GPUs with 8-way model parallelism, we achieved up to 15.1 PetaFLOPS sustained performance over the entire application and reached 76% scaling efficiency compared to the single GPU case.

  28. ⁠, Nouamane Laanait, Joshua Romero, Junqi Yin, M. Todd Young, Sean Treichler, Vitalii Starchenko, Albina Borisevich, Alex Sergeev, Michael Matheson (2019-09-24):

    We introduce novel communication strategies in synchronous distributed Deep Learning consisting of decentralized gradient reduction orchestration and computational graph-aware grouping of gradient tensors.

    These new techniques produce an optimal overlap between computation and communication and result in near-linear scaling (0.93) of distributed training up to 27,600 NVIDIA GPUs on the Summit Supercomputer. We demonstrate our gradient reduction techniques in the context of training a Fully Convolutional Neural Network to approximate the solution of a longstanding scientific inverse problem in materials imaging.

    The efficient distributed training on a dataset size of 0.5 PB, produces a model capable of an atomically-accurate reconstruction of materials, and in the process reaching a peak performance of 2.15(4) EFLOPS16.

  29. ⁠, Yuxian Meng, Xiangyuan Ren, Zijun Sun, Xiaoya Li, Arianna Yuan, Fei Wu, Jiwei Li (2019-09-26):

    In this paper, we investigate the problem of training neural machine translation (NMT) systems with a dataset of more than 40 billion bilingual sentence pairs, which is larger than the largest dataset to date by orders of magnitude. Unprecedented challenges emerge in this situation compared to previous NMT work, including severe noise in the data and prohibitively long training time. We propose practical solutions to handle these issues and demonstrate that large-scale pretraining significantly improves NMT performance. We are able to push the score of WMT17 Chinese-English dataset to 32.3, with a significant performance boost of +3.2 over existing state-of-the-art results.

  30. ⁠, Bardienus P. Duisterhof, Srivatsan Krishnan, Jonathan J. Cruz, Colby R. Banbury, William Fu, Aleksandra Faust, Guido C. H. E. de Croon, Vijay Janapa Reddi (2019-09-25):

    We present fully autonomous source seeking onboard a highly constrained nano ⁠, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter. Our deep-RL algorithm finds a high-performance solution to a challenging problem, even in presence of high noise levels and generalizes across real and simulation environments with different obstacle configurations. We verify our approach with simulation and in-field testing on a Bitcraze CrazyFlie using only the cheap and ubiquitous Cortex-M4 microcontroller unit. The results show that by end-to-end application-specific system design, our contribution consumes almost three times less additional power, as compared to competing learning-based navigation approach onboard a nano quadcopter. Thanks to our observation space, which we carefully design within the resource constraints, our solution achieves a 94% success rate in cluttered and randomized test environments, as compared to the previously achieved 80%. We also compare our strategy to a simple finite state machine (FSM), geared towards efficient exploration, and demonstrate that our policy is more robust and resilient at obstacle avoidance as well as up to 70% more efficient in source seeking. To this end, we contribute a cheap and lightweight end-to-end tiny robot learning (tinyRL) solution, running onboard a nano quadcopter, that proves to be robust and efficient in a challenging task using limited sensory input.

  31. http://manikvarma.org/pubs/kumar17.pdf

  32. ⁠, Richard Wiseman, Caroline Watt, Diana Kornbrot (2019-01-16):

    The recent ‘replication crisis’ in psychology has focused attention on ways of increasing methodological rigor within the behavioral sciences. Part of this work has involved promoting ‘Registered Reports’, wherein journals peer review papers to data collection and publication. Although this approach is usually seen as a relatively recent development, we note that a prototype of this publishing model was initiated in the mid-1970s by parapsychologist Martin Johnson in the European Journal of Parapsychology (EJP). A retrospective and observational comparison of Registered and non- published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices. This paper aims both to bring Johnson’s pioneering work to a wider audience, and to investigate the positive role that Registered Reports may play in helping to promote higher methodological and statistical standards.

    …The final dataset contained 60 papers: 25 RRs and 35 non-RRs. The RRs described 31 experiments that tested 131 hypotheses, and the non-RRs described 60 experiments that tested 232 hypotheses.

    28.4% of the statistical tests reported in non-RRs were statistically-significant (66⁄232: 95% [21.5%–36.4%]); compared to 8.4% of those in the RRs (11⁄131: 95% CI [4.0%–16.8%]). A simple 2 × 2 contingency analysis showed that this difference is highly statistically-significant (Fisher’s exact test: p < 0.0005, Pearson chi-square = 20.1, Cohen’s d = 0.48).

    …Parapsychologists investigate the possible existence of phenomena that, for many, have a low a priori likelihood of being genuine (see, eg., Wagenmakers et al 2011). This has often resulted in their work being subjected to a considerable amount of critical attention (from both within and outwith the field) that has led to them pioneering several methodological advances prior to their use within mainstream psychology, including the development of randomisation in experimental design (Hacking, 1988), the use of blinds (Kaptchuk, 1998), explorations into randomisation and statistical inference (Fisher, 1924), advances in replication issues (Rosenthal, 1986), the need for pre-specification in meta-analysis (Akers, 1985; Milton, 1999; Kennedy, 2004), and the creation of a formal study registry (Watt, 2012; Watt & Kennedy, 2015). Johnson’s work on RRs provides another striking illustration of this principle at work.

  33. 1975-johnson.pdf: ⁠, Martin U. Johnson (1975; statistics  /​ ​​ ​peer-review):

    The author discusses how to increase the quality and reliability of the research and reporting process in experimental parapsychology. Three levels of bias and control of bias are discussed. The levels are referred to as Model 1, Model 2 and Model 3 respectively.

    1. Model 1 is characterized by its very low level of intersubjective control. The reliability of the results depends to a very great extent upon the reliability of the investigator and the editor.
    2. Model 2 is relevant to the case when the experimenter is aware of the potential risk of making both errors of observation and recording and tries to control this bias. However, this model of control does not make allowances for the case when data are intentionally manipulated.
    3. Model 3 depicts a rather sophisticated system of control. One feature of this model is, that selective reporting will become harder since the editor has to make his decision as regards the acceptance or rejection of an experimental article prior to the results being obtained, and subsequently based upon the quality of the outline of the experiment. However, it should be stressed, that not even this model provides a fool-proof guarantee against deliberate fraud.

    It is assumed that the models of bias and control of bias under discussion are relevant to most branches of the behavioral sciences.

  34. 1975-johnson-2.pdf: ⁠, Martin U. Johnson (1975; statistics  /​ ​​ ​peer-review):

    This copy represents our first ‘real’ issue of the European Journal of Parapsychology…As far as experimental articles are concerned, we would like to ask potential contributors to try and adhere to the publishing policy which we have outlined in the editorial of the demonstration copy, and which is also discussed at some length in the article: ‘Models of Bias and Control of Bias’ [Johnson 1975a], in this issue. In short we shall try to avoid selective reporting and yet at the same time we shall try to refrain from making our journal a graveyard for all those studies which did not ‘turn out’. These objectives may be fulfilled by the editorial rule of basing our judgment entirely on our impressions of the quality of the design and methodology of the planned study. The acceptance or rejection of a manuscript should if possible take place prior to the carrying out and the evaluation of the results of the study.

  35. 1976-johnson.pdf: ⁠, Martin U. Johnson (1976; statistics  /​ ​​ ​peer-review):

    …even the most proper use of statistics may lead to spurious correlations or conclusions if there are inadequacies regarding the research process itself. One of these sources of error in the research process is related to selective reporting; another to human limitations with regard to the ability to make reliable observations or evaluations. says:

    The most common variant is, of course, the tendency to bury negative results. I only recently became aware of the massive size of this great graveyard for dead studies when a colleague expressed gratification that only a third of his studies ‘turned out’—as he put it. Recently, a second variant of this secret game was discovered, quite inadvertently, by ⁠, when he wrote to 37 authors to ask for the raw-data on which they had based recent journal articles. Wolins found that of the 37 who replied, 21 reported their data to be either misplaced, lost, or inadvertently destroyed. Finally, after some negotiation, Wolins was able to complete 7 re-analyses on the data supplied from 5 authors. Of the 7, he found gross errors in 3—errors so great as to clearly change the outcome of the experiments already reported.

    It should also be stressed that Rosenthal and others have demonstrated that experimenters tend to arrive at results found to be in full agreement with their expectancies, or with the expectancies of those within the scientific establishment in charge of the rewards. Even if some of Rosenthal’s results have been questioned [especially the ‘Pygmalion effect’] the general tendency seems to be unaffected.

    I guess we can all agree upon the fact that selective reporting in studies on the reliability and validity, of for instance a personality test, is a bad thing. But what could be the reason for selective reporting? Why does a research worker manipulate his dead? Is it only because the research worker has a ‘weak’ mind or does there exist some kind of ‘steering field’ that exerts such an influence that improper behavior on the part of the research worker occurs?

    It seems rather reasonable to assume that the editors of professional journals or research leaders in general could exert a certain harmful influence in this connection…There is no doubt at all in my mind about the ‘filtering’ or ‘shaping’ effect an editor may exert upon the output of his journal…As I see it, the major risk of selective reporting is not primarily a statistical one, but rather the research climate which the underlying policy create (“you are ‘good’ if you obtain supporting results; you are”no-good" if you only arrive at chance results").

    …The analysis I carried out has had practical implications for the publication policy which we have stated as an ideal for our new journal: the European Journal of Parapsychology.

  36. 1966-dunnette.pdf: ⁠, Marvin D. Dunnette (1966; statistics  /​ ​​ ​bias):

    [Influential early critique of academic psychology: weak theories, no predictions, poor measurements, poor replicability, high levels of publication bias, non-progressive theory building, and constant churn; many of these criticisms would be taken up by the ‘Minnesota school’ of Bouchard/​​​​Meehl/​​​​Lykken/​​​​etc.]

    Fads include brain-storming, Q technique, level of aspiration, forced choice, critical incidents, semantic differential, role playing, and need theory. Fashions include theorizing and theory building, criterion ⁠, model building, null-hypothesis testing, and sensitivity training. Folderol includes tendencies to be fixated on theories, methods, and points of view, conducting “little” studies with great precision, attaching dramatic but unnecessary trappings to experiments, grantsmanship, coining new names for old concepts, fixation on methods and apparatus, etc.

  37. 1962-wolins.pdf: ⁠, Leroy Wolins (1962-09; statistics  /​ ​​ ​bias):

    Comments on a Iowa State University graduate student’s endeavor of requiring data of a particular kind in order to carry out a study for his master’s thesis. This student wrote to 37 authors whose journal articles appeared in APA journals between 1959 and 1961. Of these authors, 32 replied. 21 of those reported the data misplaced, lost, or inadvertently destroyed. 2 of the remaining 11 offered their data on the conditions that they be notified of our intended use of their data, and stated that they have control of anything that we would publish involving these data. Errors were found in some of the raw data that was obtained which caused a dilemma of either reporting the errors or not. The commentator states that if it were clearly set forth by the APA that the responsibility for retaining raw data and submitting them for scrutiny upon request lies with the author, this dilemma would not exist. The commentator suggests that a possibly more effective means of controlling quality of publication would be to institute a system of quality control whereby random samples of raw data from submitted journal articles would be requested by editors and scrutinized for accuracy and the appropriateness of the analysis performed.

  38. 1989-diaconis.pdf: ⁠, Persi Diaconis, Frederick Mosteller (1989-01-01; statistics  /​ ​​ ​bias):

    This article illustrates basic statistical techniques for studying coincidences. These include data-gathering methods (informal anecdotes, case studies, observational studies, and experiments) and methods of analysis (exploratory and confirmatory data analysis, special analytic techniques, and probabilistic modeling, both general and special purpose). We develop a version of the birthday problem general enough to include dependence, inhomogeneity, and almost and multiple matches. We review Fisher’s techniques for giving partial credit for close matches. We develop a model for studying coincidences involving newly learned words. Once we set aside coincidences having apparent causes, four principles account for large numbers of remaining coincidences: hidden cause; psychology, including memory and perception; multiplicity of endpoints, including the counting of “close” or nearly alike events as if they were identical; and the law of truly large numbers, which says that when enormous numbers of events and people and their interactions cumulate over time, almost any outrageous event is bound to occur. These sources account for much of the force of synchronicity.

    [Keywords: birthday problems, extrasensory perception, Jung, Kammerer, multiple endpoints, rare events, synchronicity]

    …Because of our different reading habits, we readers are exposed to the same words at different observed rates, even when the long-run rates are the same Some words will appear relatively early in your experience, some relatively late. More than half will appear before their expected time of appearance, probably more than 60% of them if we use the exponential model, so the appearance of new words is like a Poisson process. On the other hand, some words will take more than twice the average time to appear, about 1⁄7 of them (1⁄e2) in the exponential model. They will look rarer than they actually are. Furthermore, their average time to reappearance is less than half that of their observed first appearance, and about 10% of those that took at least twice as long as they should have to occur will appear in less than 1⁄20 of the time they originally took to appear. The model we are using supposes an exponential waiting time to first occurrence of events. The phenomenon that accounts for part of this variable behavior of the words is of course the regression effect.

    …We now extend the model. Suppose that we are somewhat more complicated creatures, that we require k exposures to notice a word for the first time, and that k is itself a Poisson random variable…Then, the mean time until the word is noticed is (𝜆 + 1)T, where T is the average time between actual occurrences of the word. The variance of the time is (2𝜆 + 1)T2. Suppose T = 1 year and 𝜆 = 4. Then, as an approximation, 5% of the words will take at least time [𝜆 + 1 + 1.65 (2𝜆 + 1)(1⁄2)]T or about 10 years to be detected the first time. Assume further that, now that you are sensitized, you will detect the word the next time it appears. On the average it will be a year, but about 3% of these words that were so slow to be detected the first time will appear within a month by natural variation alone. So what took 10 years to happen once happens again within a month. No wonder we are astonished. One of our graduate students learned the word on a Friday and read part of this manuscript the next Sunday, two days later, illustrating the effect and providing an anecdote. Here, sensitizing the individual, the regression effect, and the recall of notable events and the non-recall of humdrum events produce a situation where coincidences are noted with much higher than their expected frequency. This model can explain vast numbers of seeming coincidences.

  39. ⁠, Michael Mitzenmacher, Andrea W. Richa, Ramesh Sitaraman (2001):

    …we begin with a simple problem that demonstrates a powerful fundamental idea. Suppose that n balls are thrown into n bins, with each ball choosing a bin independently and uniformly at random. Then the maximum load, or the largest number of balls in any bins, is approximately log n / log log n with high probability. Now suppose instead that the balls are placed sequentially, and each ball is placed in the least loaded of d≥2 bins chosen independently and uniformly at random. Azar et al 1999 showed that in this case, the maximum load is log log n / log d + Θ(1) with high probability.

    The important implication of this result is that even a small amount of choice can lead to drastically different results in load balancing. Indeed, having just two random choices (ie d = 2) yields a large reduction in the maximum load by just a constant factor. Over the past several years, there has been a great deal of research investigating this phenomenon. The picture that has emerged from this research is that the power of two choices is not simply an artifact of the simple balls-and-bins model, but a general and robust phenomenon applicable to a wide variety of situations. Indeed, this two-choice paradigm continues to be applied and refined, and new results appear frequently. Applications of the two-choice paradigm:…Hashing, Shared memory emulations, load balancing, low-congestion circuit routing.

    [See also “The Power of Two Choices in Randomized Load Balancing”⁠, Mitzenmacher 1996; Nginx⁠/​​​​HAProxy⁠, Marc Brooker⁠.]

  40. ⁠, Edward Tufte (2004):

    [Originally the draft chapter of the (“Intense, Simple, Word-Sized Graphics”) chapter of Beautiful Evidence (2005), this page is a compilation of sparkline examples, links to sparkline software tools, and debates over how best to use sparklines to graph statistical data.]

  41. 2004-tushnet.pdf: ⁠, Mark Tushnet (2004; sociology):

    For the past several years I have been noticing a phenomenon that seems to me new in my lifetime as a scholar of constitutional law. I call the phenomenon constitutional hardball. This Essay develops the idea that there is such a practice, that there is a sense in which it is new, and that its emergence (or re-emergence) is interesting because it signals that political actors understand that they are in a position to put in place a new set of deep institutional arrangements of a sort I call a “constitutional order”. A shorthand sketch of constitutional hardball is this: it consists of political claims and practices-legislative and executive initiatives-that are without much question within the bounds of existing constitutional doctrine and practice but that are nonetheless in some tension with existing pre-constitutional understandings. It is hardball because its practitioners see themselves as playing for keeps in a special kind of way; they believe the stakes of the political controversy their actions provoke are quite high, and that their defeat and their opponents’ victory would be a serious, perhaps permanent setback to the political positions they hold.

  42. 2019-horowitz.pdf: ⁠, Mark Horowitz, William Yaworsky, Kenneth Kickham (2019-10; sociology⁠, sociology  /​ ​​ ​preference-falsification):

    In recent decades the field of anthropology has been characterized as sharply divided between pro-science and anti-science factions. The aim of this study is to empirically evaluate that characterization. We survey anthropologists in graduate programs in the United States regarding their views of science and advocacy, moral and epistemic relativism, and the merits of evolutionary biological explanations. We examine anthropologists’ views in concert with their varying appraisals of major controversies in the discipline (⁠, ⁠, and ). We find that disciplinary specialization and especially gender and political orientation are statistically-significant predictors of anthropologists’ views. We interpret our findings through the lens of an intuitionist social psychology that helps explain the dynamics of such controversies as well as ongoing ideological divisions in the field.

  43. https://public.tableau.com/profile/jurijfedorov#!/vizhome/AnthropologysScienceWars/Field

  44. ⁠, Gary Saul Morson (2019-10):

    [This re-appraisal of Lenin is just about as damning as any re-appraisal of anybody could possibly be. “He invented a form of government we have come to call totalitarian, which rejected in principle the idea of any private sphere outside of state control. He invented the one-party state, a term that would previously have seemed self-contradictory since a party was, by definition, a part. He believed that state power had to be based on sheer terror, and so he created the terrorist state. Violence was a goal in itself”]

  45. ⁠, Robin Hanson (2012-01-05):

    Weird folks are often tempted to give up on grand ambitions, thinking there is little chance the world will let them succeed. Turns out, however, it isn’t as bad as all that. Especially if your main weirdness is in the realm of ideas…I’ve known some very successful people with quite weird ideas. But these folks mostly keep regular schedules of sleep and bathing. Their dress and hairstyles are modest, they show up on time for meetings, and they finish assignments by deadline. They are willing to pay dues and work on what others think are important for a while, and they have many odd ideas they’d pursue if given a chance, instead of just one overwhelming obsession. They are willing to keep changing fields, careers, and jobs until they find one that works for them…if you are not overtly rebellious, you can get away with a lot of abstract idea rebellion—few folks will even notice such deviations, and fewer still will care. So, ask yourself, do you want to look like a rebel, or do you want to be a rebel?

  46. 2019-zeraatkar.pdf: ⁠, Dena Zeraatkar, Bradley C. Johnston, Jessica Bartoszko, Kevin Cheung, Malgorzata M. Bala, Claudia Valli, Montserrat Rabassa, Deagan Sit, Kirolos Milio, Behnam Sadeghirad, Arnav Agarwal, Adriana M. Zea, Yung Lee, Mi Ah Han, Robin W. M. Vernooij, Pablo Alonso-Coello, Gordon H. Guyatt, Regina El Dib (2019-10-01; longevity):

    Background: Few randomized trials have evaluated the effect of reducing red meat intake on clinically important outcomes.

    Purpose: To summarize the effect of lower versus higher red meat intake on the incidence of cardiometabolic and cancer outcomes in adults.

    Data Sources: EMBASE, CENTRAL, CINAHL, Web of Science, and ProQuest from inception to July 2018 and MEDLINE from inception to April 2019, without language restrictions.

    Study Selection: Randomized trials (published in any language) comparing diets lower in red meat with diets higher in red meat that differed by a gradient of at least 1 serving per week for 6 months or more.

    Data Extraction: Teams of 2 reviewers independently extracted data and assessed the risk of bias and the certainty of the evidence.

    Data Synthesis: Of 12 eligible trials, a single trial enrolling 48 835 women provided the most credible, though still low-certainty, evidence that diets lower in red meat may have little or no effect on all-cause mortality (hazard ratio [HR], 0.99 [95% CI, 0.95 to 1.03]), cardiovascular mortality (HR, 0.98 [CI, 0.91 to 1.06]), and cardiovascular disease (HR, 0.99 [CI, 0.94 to 1.05]). That trial also provided low-certainty to very-low-certainty evidence that diets lower in red meat may have little or no effect on total cancer mortality (HR, 0.95 [CI, 0.89 to 1.01]) and the incidence of cancer, including colorectal cancer (HR, 1.04 [CI, 0.90 to 1.20]) and breast cancer (HR, 0.97 [0.90 to 1.04]).

    Limitations: There were few trials, most addressing only surrogate outcomes, with heterogeneous comparators and small gradients in red meat consumption between lower versus higher intake groups.

    Conclusion: Low-certainty to very-low-certainty evidence suggests that diets restricted in red meat may have little or no effect on major cardiometabolic outcomes and cancer mortality and incidence.

  47. {#linkBibliography-(jama)-2020 .docMetadata doi=“10.1001/​​jama.2019.21441”}, Rita Rubin (JAMA) (2020-01-15):

    [Summary of vegetarian activist/​​​​researcher reaction to recent reviews & meta-analysis indicating that the correlation of meat-eating with bad health often does not appear in epidemiological datasets, the randomized experiments do not support the strong claims, and the overall evidence that eating meat = bad health is low quality & weak:

    After breaking the embargo, they began lobbying against it, spamming the journal editor, demanding the papers be retracted before publication, denouncing it in talks, and contacting the Federal Trade Commission & district attorneys demanding they investigate; they justify these activities by saying that since high-quality evidence can’t be easily obtained in nutrition, there is no need for it, and accusing the authors of financial conflicts of interest and comparing them to global warming deniers.

    However, the conflicts of interest represent very small percentages of funding, and the vegetarian activist/​​​​researchers themselves are heavily funded by anti-meat interests, such as olive research institutions, walnut industry bodies, the egg industry, snack companies, and alternative diet groups, with the list of funders of one member including but far from limited to “the Research Network, the Almond Board of California, the International Nut and Dried Fruit Council; Soy Foods Association of North America; the Peanut Institute; Kellogg’s Canada; and Quaker Oats Canada.”]

  48. 2019-vrij.pdf: ⁠, Aldert Vrij, Maria Hartwig, Pär Anders Granhag (2019-01-01; psychology):

    The relationship between nonverbal communication and deception continues to attract much interest, but there are many misconceptions about it. In this review, we present a scientific view on this relationship. We describe theories explaining why liars would behave differently from truth tellers, followed by research on how liars actually behave and individuals’ ability to detect lies. We show that the nonverbal cues to deceit discovered to date are faint and unreliable and that people are mediocre lie catchers when they pay attention to behavior. We also discuss why individuals hold misbeliefs about the relationship between nonverbal behavior and deception—beliefs that appear very hard to debunk. We further discuss the ways in which researchers could improve the state of affairs by examining nonverbal behaviors in different ways and in different settings than they currently do.

  49. ⁠, Carl L. Hart, Caroline B. Marvin, Rae Silver, Edward E. Smith (2011-11-16):

    The prevailing view is that recreational methamphetamine use causes a broad range of severe cognitive deficits, despite the fact that concerns have been raised about interpretations drawn from the published literature. This article addresses an important gap in our knowledge by providing a critical review of findings from recent research investigating the impact of recreational methamphetamine use on human cognition. Included in the discussion are findings from studies that have assessed the acute and long-term effects of methamphetamine on several domains of cognition, including visuospatial perception, attention, inhibition, ⁠, long-term memory, and learning. In addition, relevant neuroimaging data are reviewed in an effort to better understand neural mechanisms underlying methamphetamine-related effects on cognitive functioning. In general, the data on acute effects show that methamphetamine improves cognitive performance in selected domains, that is, visuospatial perception, attention, and inhibition. Regarding long-term effects on cognitive performance and brain-imaging measures, statistically-significant differences between methamphetamine users and control participants have been observed on a minority of measures. More importantly, however, the clinical-significance of these findings may be limited because cognitive functioning overwhelmingly falls within the normal range when compared against normative data. In spite of these observations, there seems to be a propensity to interpret any cognitive and/​​​​or brain difference(s) as a clinically-significant abnormality. The implications of this situation are multiple, with consequences for scientific research, substance-abuse treatment, and public policy.

  50. ⁠, Francisco Javier Navas González, Jordi Jordana Vidal, José Manuel León Jurado, Amy Katherine McLean, Juan Vicente Delgado Bermejo (2019-09):

    Scientific evidence for intelligence in donkeys could expose their historical unmerited cognitive derogatory status. Psychometric testing enables quantifying animal cognitive capabilities and their genetic background.

    Owing to the impossibility to use the language-dependent scales that are widely used to measure intelligence in humans, we used a nonverbal operant-conditioning problem-solving test to compute a human-analogous IQ, scoring the information of 13 cognitive processes from 300 genetically tested donkeys. Principal components and Bayesian analyses were used to compute the variation in cognitive capabilities explained by the cognitive processes tested and their genetic parameters, respectively.

    According to our results, IQ may explain over 62% of the cognitive variance, and 0.06 to 0.38 heritabilities suggest that we could ascribe a substantial proportion to interacting genes describing the same patterns previously reported for humans and other animal species.

    Our results address the existence of a human-analogous heritable component and mechanisms underneath intelligence and cognition in probably one of the most traditionally misunderstood species from a cognitive perspective.

    [Keywords: cognition, g, genetic parameters, asses, intelligence quotient]

  51. {#linkBibliography-yorker)-2010 .docMetadata}, Burkhard Bilger () (2010-11-22):

    [Discussion of food subcultures: dumpster divers, raw food enthusiasts, fermenters, roadkill, and ‘high’ (fully rotten meat) food advocates, with visits to gay commune Hickory Knoll and raw milk dairies. The author ultimately draws the line at trying high game, however.]

    When Torma unclamped his jar, a sickly-sweet miasma filled the air—an odor as natural as it was repellent. Decaying meat produces its own peculiar scent molecules, I later learned, with names like putrescine and cadaverine. I could still smell them on my clothes hours later. Torma stuck two fingers down the jar and fished out a long, wet sliver. “Want a taste?” he said.

    It was the end of a long day. I’d spent most of it consuming everything set before me: ants, acorns, raw milk, dumpster stew, and seven kinds of mead, among other delicacies. But even Katz took a pass on high meat. While Torma threw back his head and dropped in his portion, like a seal swallowing a mackerel, we quietly took our leave. “You have to trust your senses”, Katz said, as we were driving away. “To me, that smelled like death.”

  52. ⁠, Yoshiki Ohshima, Dan Amelang, Ted Kaehler, Bert Freudenberg, Aran Lunzer, ⁠, Ian Piumarta, Takashi Yamamiya, Alan Borning, Hesam Samimi, Bret Victor, Kim Rose (2012):

    [Technical report from a research project aiming at writing a GUI OS in 20k LoC; tricks include ASCII art networking DSLs & generic optimization for text layout⁠, which lets them implement a full OS, sound, GUI desktops, Internet networking & web browsers, a text/​​​​document editor etc, all in less lines of code that most OSes need for small parts of any of those.]

    …Many software systems today are made from millions to hundreds of millions of lines of program code that is too large, complex and fragile to be improved, fixed, or integrated. (One hundred million lines of code at 50 lines per page is 5000 books of 400 pages each! This is beyond human scale.) What if this could be made literally 1000 times smaller—or more? And made more powerful, clear, simple and robust?…The ’STEPS

    STEPS Aims At ‘Personal Computing’STEPS takes as its prime focus the dynamic modeling of ‘personal computing’ as most people think of it…word processor, spreadsheet, Internet browser, other productivity SW; User Interface and Command Listeners: windows, menus, alerts, scroll bars and other controls, etc.; Graphics and Sound Engine: physical display, sprites, fonts, compositing, rendering, sampling, playing; Systems Services: development system, database query languages, etc.; Systems Utilities: file copy, desk accessories, control panels, etc.; Logical Level of OS: eg. file management, Internet, and networking facilities, etc.; Hardware Level of OS: eg. memory manager, process manager, device drivers, etc.

  53. http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html

  54. 1946-walker.pdf: {#linkBibliography-magazine)-1946 .docMetadata doi=“economics/​​copyright”}, Charles Lester Walker (Harper's Magazine) (1946-10-01; technology):

    Someone wrote to Wright Field recently, saying he understood this country had got together quite a collection of enemy war secrets, that many were now on public sale, and could he, please, be sent everything on German jet engines. The Air Documents Division of the Army Air Forces answered: “Sorry—but that would be fifty tons”. Moreover, that fifty tons was just a small portion of what is today undoubtedly the biggest collection of captured enemy war secrets ever assembled. ..It is estimated that over a million separate items must be handled, and that they, very likely, practically all the scientific, industrial and military secrets of Nazi Germany. One Washington official has called it “the greatest single source of this type of material in the world, the first orderly exploitation of an entire country’s brain-power.”

    What did we find? You’d like some outstanding examples from the war secrets collection?

    …the tiniest vacuum tube I had ever seen. It was about half thumb-size. Notice it is heavy porcelain—not glass—and thus virtually indestructible. It is a thousand watt—one-tenth the size of similar American tubes…“That’s Magnetophone tape”, he said. “It’s plastic, metallized on one side with iron oxide. In Germany that supplanted phonograph recordings. A day’s Radio program can be magnetized on one reel. You can demagnetize it, wipe it off and put a new program on at any time. No needle; so absolutely no noise or record wear. An hour-long reel costs fifty cents.”…He showed me then what had been two of the most closely-guarded, technical secrets of the war: the infra-red device which the Germans invented for seeing at night, and the remarkable diminutive generator which operated it. German cars could drive at any, speed in a total blackout, seeing objects clear as day two hundred meters ahead. Tanks with this device could spot; targets two miles away. As a sniper scope it enabled German riflemen to pick off a man in total blackness…We got, in addition, among these prize secrets, the technique and the machine for making the world’s most remarkable electric condenser…The Kaiser Wilhelm Institute for Silicate Research had discovered how to make it and—something which had always eluded scientists—in large sheets. We know now, thanks to FIAT teams, that ingredients of natural mica were melted in crucibles of carbon capable of taking 2,350 degrees of heat, and then—this was the real secret—cooled in a special way…“This is done on a press in one operation. It is called the ‘cold extrusion’ process. We do it with some soft, splattery metals. But by this process the Germans do it with cold steel! Thousands of parts now made as castings or drop forgings or from malleable iron can now be made this way. The production speed increase is a little matter of one thousand%.” This one war secret alone, many American steel men believe, will revolutionize dozens of our metal fabrication industries.

    …In textiles the war secrets collection has produced so many revelations, that American textile men are a little dizzy. But of all the industrial secrets, perhaps, the biggest windfall came from the laboratories and plants of the great German cartel, I. G. Farbenindustrie. Never before, it is claimed, was there such a store-house of secret information. It covers liquid and solid fuels, metallurgy, synthetic rubber, textiles, chemicals, plastics. drugs, dyes. One American dye authority declares: “It includes the production know-how and the secret formulas for over fifty thousand dyes. Many of them are faster and better than ours. Many are colors we were never able to make. The American dye industry will be advanced at least ten years.”

    …Milk pasteurization by ultra-violet light…how to enrich the milk with vitamin D…cheese was being made—“good quality Hollander and Tilsiter”—by a new method at unheard-of speed…a continuous butter making machine…The finished product served as both animal and human food. Its caloric value is four times that of lean meat, and it contains twice as much protein. The Germans also had developed new methods of preserving food by plastics and new, advanced refrigeration techniques…German medical researchers had discovered a way to produce synthetic blood plasma.

    …When the war ended, we now know, they had 138 types of guided missiles in various stages of production or development, using every known kind of remote control and fuse: radio, radar, wire, continuous wave, acoustics, infra-red, light beams, and magnetics, to name some; and for power, all methods of jet propulsion for either subsonic or supersonic speeds. Jet propulsion had even been applied to helicopter flight…Army Air Force experts declare publicly that in rocket power and guided missiles the Nazis were ahead of us by at least ten years.

  55. {#linkBibliography-(esquire)-1971 .docMetadata}, Ron Rosenbaum (Esquire) (1971-10-01):

    [Early account of and their most famous hacking device, the ⁠, used to control the Bell Phone System and enable free long-distance calls (then exorbitantly expensive); the blue box was famously based on an AT&T research paper describing the tone frequencies and how they control the phone switching system. The author hangs out with phreaks such as to see how it all works.

    After reading Rosenbaum’s article, and his partner in founding Apple, Steve Wozniak, “collaborated on building and selling blue boxes, devices that were widely used for making free—and illegal—phone calls. They raised a total of $26,823$6,0001971 from the effort.”]

  56. {#linkBibliography-hitt-(esquire)-1990 .docMetadata}, Jack Hitt, Paul Tough (Esquire) (1990-12-01):

    [Gonzo-style account of hanging out with teenage hackers and phreakers in NYC, Phiber Optik and Acid Phreak, similar to ]

    “Sometimes”, says Kool, “it’s so simple. I used to have contests with my friends to see how few words we could use to get a password. Once I called up and said, ‘Hi, I’m from the social-engineering center and I need your password’, and they gave it to me! I swear, sometimes I think I could call up and say, ‘Hi, I’m in a diner, eating a banana split. Give me your password.’” Like its mechanical counterpart, is half business and half pleasure. It is a social game that allows the accomplished hacker to show off his knowledge of systems, his mastery of jargon, and especially his ability to manipulate people. It not only allows the hacker to get information; it also has the comic attractions of the old-fashioned prank phone call—fooling an adult, improvisation, cruelty. In the months we spent with the hackers, the best performance in a social-engineering role was by a hacker named Oddjob. With him and three other guys we pulled a hacking all-nighter in the financial district, visiting pay phones in the hallway of the World Trade Center, outside the bathrooms of the Vista Hotel, and in the lobby of the international headquarters of American Express.

    …Where we see only a machine’s function, they see its potential. This is, of course, the noble and essential trait of the inventor. But hackers warp it with teenage anarchic creativity: Edison with attitude. Consider the fax machine. We look at it; we see a document-delivery device. One hacker we met, Kaos, looked at the same machine and immediately saw the Black Loop of Death. Here’s how it works: Photocopy your middle finger displaying the international sign of obscene derision. Make two more copies. Tape these three pages together. Choose a target fax machine. Wait until nighttime, when you know it will be unattended, and dial it up. Begin to feed your long document into your fax machine. When the first page begins to emerge below, tape it to the end of the last page. Ecce. This three-page loop will continuously feed your image all night long. In the morning, your victim will find an empty fax machine, surrounded by two thousand copies of your finger, flipping the bird.

    …From a distance, a computer network looks like a fortress—impregnable, heavily guarded. As you get closer, though, the walls of the fortress look a little flimsy. You notice that the fortress has a thousand doors; that some are unguarded, the rest watched by unwary civilians. All the hacker has to do to get in is find an unguarded door, or borrow a key, or punch a hole in the wall. The question of whether he’s allowed in is made moot by the fact that it’s unbelievably simple to enter. Breaking into computer systems will always remain easy because the systems have to accommodate dolts like you and me. If computers were used only by brilliant programmers, no doubt they could maintain a nearly impenetrable security system. But computers aren’t built that way; they are “dumbed down” to allow those who must use them to do their jobs. So hackers will always be able to find a trusting soul to reveal a dialup, an account, and a password. And they will always get in.

  57. ⁠, Ken Silverstein (1998-11-01):

    Growing up in suburban Detroit, David Hahn was fascinated by science. While he was working on his Atomic Energy badge for the Boy Scouts, David’s obsessive attention turned to nuclear energy. Throwing caution to the wind, he plunged into a new project: building a model nuclear reactor in his backyard garden shed.

    Posing as a physics professor, David solicited information on reactor design from the U.S. government and from industry experts. Following blueprints he found in an outdated physics textbook, David cobbled together a crude device that threw off toxic levels of radiation.

    His wholly unsupervised project finally sparked an environmental emergency that put his town’s forty thousand suburbanites at risk. The EPA ended up burying his lab at a radioactive dumpsite in Utah.

    [Keywords: 20th century, David Hahn, experiments, Michigan, nuclear engineering, radiochemistry, recreation, teenage boys]

  58. ⁠, Sergey Slyusarev (2019-03-19):

    The most obvious option—to draw all the illustrations in Illustrator and compose the whole thing in InDesign—was promptly rejected. Geometrical constructions are not exactly the easiest thing to do in Illustrator, and no obvious way to automatically connect the main image to miniatures came to my mind. As for InDesign, although it’s very good at dealing with such visually rich layouts, it promised to scare the hell out of me by the overcrowded “Links” panel. So, without thinking twice, I decided to use other tools that I was familiar with—MetaPost, which made it relatively easy to deal with geometry, and LaTeX, which I knew could do the job. Due to some problems with MetaPost libs for LaTeX, I replaced the latter with that enjoys an out-of-the-box merry relationship with MetaPost.

    Converting a Bryne Euclid diagram to ConTeXt vector graphics

    … There are also initials and vignettes in the original edition. On one hand, they were reasonably easy to recreate (at least, it wouldn’t take a lot of thought to do this), but I decided to go with a more interesting (albeit hopeless) option—automatically generating the initials and vignettes with a random ornament. Not only is it fun, but also, the Russian translation would require adapting the style of the original initials to the Cyrillic script, which was not something I’d prefer to do. So, long story short, when you compile the book, a list of initial letters is written to the disk, and a separate MetaPost script can process it (very slowly) to produce the initials and vignettes. No two of them have the exact same ornament.

  59. ⁠, Nicholas Rougeux (2018-12-16):

    Creating a faithful online reproduction of a book considered one of the most beautiful and unusual publications ever published is a daunting task. Byrne’s Euclid is my tribute to Oliver Byrne’s most celebrated publication from 1847 that illustrated the geometric principles established in Euclid’s original Elements from 300 BC.

    In 1847, Irish mathematics professor Oliver Byrne worked closely with publisher William Pickering in London to publish his unique edition titled The First Six Books of the Elements of Euclid in which Coloured Diagrams and Symbols are Used Instead of Letters for the Greater Ease of Learners—or more simply, Byrne’s Euclid. Byrne’s edition was one of the first multicolor printed books and is known for its unique take on Euclid’s original work using colorful illustrations rather than letters when referring to diagrams. The precise use of colors and diagrams meant that the book was very challenging and expensive to reproduce. Little is known about why Byrne only designed 6 of the 13 books but it was could have been due to time and cost involved…I knew of other projects like Sergey Slyusarev’s ConTeXt rendition and Kronecker Wallis’ modern redesign but I hadn’t seen anyone reproduce the 1847 edition online in its entirety and with a design true to the original. This was my goal and I knew it was going to be a fun challenge.

    Diagrams from Book 1

    [Detailed discussion of how to use Adobe Illustrator to redraw the modernist art-like primary color diagrams from Bryne in scalable vector graphics (SVG) for use in interactive HTML pages, creation of a custom to replicate Bryne, his (questionable) efforts to use the for greater authenticity, rendering the math using MathJax, and creating posters demonstrating all diagrams from the project for offline viewing.]

  60. https://www.c82.net/euclid/

  61. ⁠, Bill Casselman (University of British Columbia) ():

    Online scanned edition; part of a set of Euclid editions.

  62. 1990-tufte-envisioninginformation-ch5-byrneseuclid.pdf: ⁠, Edward Tufte (1990; design):

    [Extracts from Tufte textbook on graphing information and visual design, where he revives & popularizes Oliver Bryne’s obscure Euclid edition, noting how effectively Bryne converts lengthy proofs into short sequences of cleanly-designed diagrams exploiting primary colors for legibility, and the curious anticipation of modernist design movements like ⁠.]

  63. ⁠, Kazutaka Kurihara, Koji Tsukada (2012-02-28):

    In this paper we report on a system, “SpeechJammer”, which can be used to disturb people’s speech. In general, human speech is jammed by giving back to the speakers their own utterances at a delay of a few hundred milliseconds. This effect can disturb people without any physical discomfort, and disappears immediately by stop speaking. Furthermore, this effect does not involve anyone but the speaker. We utilize this phenomenon and implemented two prototype versions by combining a direction-sensitive microphone and a direction-sensitive speaker, enabling the speech of a specific person to be disturbed. We discuss practical application scenarios of the system, such as facilitating and controlling discussions. Finally, we argue what system parameters should be examined in detail in future formal studies based on the lessons learned from our preliminary study.

  64. ⁠, Jonathan Allen (2009-02):

    [Card marking is a venerable and sophisticated art. Jonathan Allen on juiced cards, luminous readers, sunning the deck, and other sharpers’ tricks ()]

    The history of the marked playing card, perhaps as old as the playing card itself, is a miscellany of inventive guile. “The systems of card-marking are as numerous as they are ingenious”, wrote John Nevil Maskelyne in 1894. “Card doctoring”, to use Erdnase’s term, covers many forms of subterfuge, but in the brief survey that follows, we shall focus our attention upon what might more usefully be termed the “language” of the marked card.

    …“Luminous readers” are cards treated in such a way that pale green ink traces become clearly visible when viewed through red-filtered spectacles or contact lenses. The technology caused alarm upon its discovery but, due to its limited effectiveness and its reliance upon somewhat vampiric eye adornment, has remained more of a popular novelty than a serious subterfuge.11 “Juiced cards”, on the other hand, do not need lens-based viewing, instead requiring the reader to defocus his or her eyes and spot liminal fluid-residue marks on an opponent’s distant cards (juiced cards are also known as “distance readers”). To many players, juicing, and its recent high-tech offshoot, “video juicing”, are the most effective real-world card-marking system available, and the considerable price of the closely guarded fluid recipe and application technique reflects this growing reputation.

  65. 2014-lewis.pdf: ⁠, Tasha L. Lewis, Brittany Haas (2014-03; economics):

    The Hermès brand is synonymous with a wealthy global elite clientele and its products have maintained an enduring heritage of craftsmanship that has distinguished it among competing luxury brands in the global market. Hermès has remained a family business for generations and has successfully avoided recent acquisition attempts by luxury group LVMH. Almost half of the luxury firm’s revenue ($1.90$1.52012B in 2012) is derived from the sale of its leather goods and saddlery, which includes its handbags. A large contributor to sales is global demand for one of its leather accessories, the Birkin bag, ranging in price from $12,298$10,0002014 to $307,458$250,0002014. Increased demand for the bag in the United States since 2002 resulted in an extensive customer waitlist lasting from months to a few years. Hermès retired the famed waitlist (sometimes called the ‘dream list’) in the United States in 2010, and while the waitlist has been removed, demand for the Birkin bag has not diminished and making the bag available to luxury consumers requires extensive, careful distribution management. In addition to inventory constraints related to demand for the Birkin bag in the United States, Hermès must also manage a range of other factors in the US market. These factors include competition with ‘affordable’ luxury brands like Coach, monitoring of unsolicited brand endorsers as well as counterfeit goods and resellers. This article examines some of the allocation practices used to carefully manage the Hermès brand in the US market.

  66. https://threadreaderapp.com/thread/1142799627600584704.html

  67. ⁠, Brittanny Newsom (2016-12-19):

    History · Design · Craftsmanship & Quality · How To Buy A Birkin · Demand & Exclusivity · The Secondhand Market · Clientele · Why the Birkin Is A Safe Investment · Investment Factors · Investment Pricing Factors · Comparisons with Other Investments · Fake vs. Real · How the Birkin Remains Dominant · The Media · The Defaced Birkin · Conclusion

    Birkin bags are carefully handcrafted. The creation process for each bag can take over 18 hours. That number can double if working on a Birkin accessorized with diamonds. The artisans who craft these bags are carefully screened and require years of high quality experience even before being considered for the job. “Hermès has a reputation of hiring mostly artisans who have graduated from the École Grégoire Ferrandi; a school that specializes in working with luxurious leathers.” It also typically takes about 2 years to train an Hermès craftsman, with each one supervised by an existing craftsman.

    Preparing the leather is the first step towards crafting the bag. The leather is examined for any defects an animal skin may have mosquito bites or wounds that must be repaired before the skin’s tanning. Leathers are obtained from different tanners in France, resulting in various smell sand textures. The stitching of the bag is also very precise. The bag is held together using wooden clamp, while the artisan applies each individual stitch on the bag. The linen that is used during the stitching process is waterproof and has a beeswax coating for rot prevention. Most Birkin bags are created with same color threads, but some rare bags have white threads even if the bag is not white. “More than 90% of the bag is hand stitched because it allows more freedom to shape the bag and makes it more resilient.” That’s when the hardware process begins. Unlike other bags, the hardware is attached using the unique Hermès process called “pearling” rather than by using screws. Artisans put a “small nail through a corner hole on the back of the clasp, the leather and the front clasp, take an awl with a concave tip and tap the bit of nail with a hammer gently in a circle until it is round like a tiny pearl.” This process ensures that the pearls will hold the two pieces of metal together forever. The bag is then turned right side out and ironed into shape.

    …As secondhand market sales have grown, interest from first time buyers has also increased. This shows the Birkin bag is an important sales channel for an expanding global luxury product market. Such growth has propelled the Birkin to near legendary status in a very demanding market. According to Bag Hunter, “Birkin bags have climbed in value by 500% over the past 35 years, and an increase expected to double over the next 10 years.”

    …Simply stated, it appears that the bag’s success hinges on this prestigious perception. A Birkin, terribly difficult to get is therefore highly coveted. In our global economy, that’s all the brand needs to pack the infinite waiting list. It is fashion’s version of Darwinism. We always want what we can’t have, so we will do whatever we can to get it. For instance, Victoria Beckham, the posh clothing designer, and wife of David Beckham reportedly owns about 100 Birkins, collectively valued at $2 million. It includes a pink Ostrich leather Birkin worth $150,000. Despite the fact that she has introduced her own line of handbags, she’s been spotted by the paparazzi wearing a Birkin bag. Kris Jenner also has a massive Birkin collection that she flaunts via social media and the willing participation of paparazzi. Her collection includes an Electric Blue 35cm which is supposedly worth $19,000. Actress Katie Holmes has gained attention for a bold red Birkin, while Julianne Moore has been seen wearing a hunter green 40cm with gold hardware. Julia Roberts and Eva Longoria all have even been seen with the bag. Even B-listed personalities such as reality star, Nicole Richie, with a black Birkin workout bag, is famously noted as frequently asking the paparazzi, “Did you get my bag?”. The Birkin has looked extra special on the arms of models, Alessandra Ambrosio and Kate Moss. Singers such as Jennifer Lopez and Courtney Love ironically show off their Birkins, and even world leaders such as Princess Mary of Denmark, with her black crocodile Birkin worth $44,500, is aware of its meaning and status.

  68. Bakker

  69. Story-Of-Your-Life

  70. ⁠, Steven R. Weisman (1990-12-29):

    December in Japan is a festive season, filled with gift-giving, prayers for the new year, bamboo and pine branches in front of houses, office parties and Beethoven’s Ninth.

    Beethoven’s Ninth? No one is sure how it happened, but indeed, Ludwig van Beethoven’s Choral Symphony is as much a staple of the season as dry weather and maddeningly short days. The symphony is being performed at least 170 times this month by professional and amateur groups throughout the country. Some orchestras play it several times in a row. The Symphony Orchestra has performed what the Japanese call the Daiku, or Big Nine, five times this month, the Tokyo Symphony Orchestra 13 times and the Japan Philharmonic Symphony Orchestra 11 times.

    “For Japanese, listening to Beethoven’s Ninth at the end of the year is a semi-religious experience”, said Naoyuki Miura, the artistic director of Music from Japan, which sponsors concerts abroad. “People feel they have not completed the year spiritually until they hear it.” Like the Christmastime sing-alongs of Handel’s Messiah in the West, Beethoven’s Ninth also draws audiences to sing-along performances at which the audiences lustily join in the choruses of Schiller’s “Ode to Joy”, singing German words they barely understand.

  71. Movies#mandy

  72. Movies#weiner

  73. Anime#redline

  74. Anime#concurrency

  75. https://old.reddit.com/r/TOUHOUMUSIC/search?q=author%3Agwern&sort=new&restrict_sr=on&t=all

  76. https://www.youtube.com/watch?v=C4aROrTmtnc

  77. https://enuplus.wixsite.com/enuplus

  78. https://www.youtube.com/watch?v=SNOqTzguLj4

  79. http://e-ns.net/discography/ens0063.html

  80. https://www.youtube.com/watch?v=6pp0kdfFzM0

  81. https://creberbrown.bandcamp.com/album/Getting-Stronger

  82. https://www.youtube.com/watch?v=u5wI6QVCjUU

  83. https://www.youtube.com/watch?v=XDD_ZAeFWOI

  84. https://blackgryph0n.bandcamp.com/album/immortal

  85. https://www.youtube.com/watch?v=8JJg9114bgs

  86. https://eternal-lives.booth.pm/items/1096764

  87. https://soundcloud.com/falmaki