1. 03

2. https://gwern.substack.com

3. 02

4. newsletter

5. Changelog

6. https://www.patreon.com/gwern

7. Faces

8. GPT-2

9. Embryo-selection#glue-robbers-sequencing-nobelists-using-collectible-letters

10. Batman

11. Sidenotes#tufte-css

12. sidenotes.js

13. https://joa.sh/posts/2015-09-14-prerender-mathjax.html

14. https://github.com/pkra/mathjax-node-page/

15. Embryo-selection

16. https://www.1001fonts.com/goudy-initialen-font.html

17. https://www.tug.org/TUGboat/tb12-1/tb31hara.pdf#page=8

18. https://www.typografie.info/3/Schriften/fonts.html/deutsche-zierschrift-r250/

19. https://wiki.obormot.net/Main/BonusFontsDemo?demo_font_one=Cheshire+Initials

20. https://wiki.obormot.net/Main/BonusFontsDemo?demo_font_one=Kanzlei+Initialen

21. https://tug.org/FontCatalogue/otherfonts.html#initials

22. Ads

23. Inflation.hs: ⁠, (2019-03-27):

Experimental Pandoc module for implementing automatic inflation adjustment of nominal date-stamped dollar or amounts to provide real prices; Bitcoin’s exchange rate has moved by multiple orders of magnitude over its early years (rendering nominal amounts deeply unintuitive), and this is particularly critical in any economics or technology discussion where a nominal price from 1950 is 11× the 2019 real price!

Years/​​​​dates are specified in a variant of my interwiki link syntax; for example: $50 or [₿0.5]​(₿2017-01-01), giving link adjustments which compile to something like like <span class="inflationAdjusted" data-originalYear="2017-01-01" data-originalAmount="50.50" data-currentYear="2019" data-currentAmount="50,500">₿50.50<span class="math inline"><sub>2017</sub><sup>$50,500</sup></span></span>.

Dollar amounts use year, and Bitcoins use full dates, as the greater temporal resolution is necessary. Inflation rates/​​​​exchange rates are specified as constants and need to be manually updated every once in a while; if out of date, the last available rate is carried forward for future adjustments.

24. https://github.com/mozilla/mozjpeg

25. 2019-lee.pdf: ⁠, (2019-07; iq):

There exists a moderate correlation between MRI-measured brain size and the general factor of IQ performance (g), but the question of whether the association reflects a theoretically important causal relationship or spurious remains somewhat open. Previous small studies (n < 100) looking for the persistence of this correlation within families failed to find a tendency for the sibling with the larger brain to obtain a higher test score. We studied the within-family relationship between brain volume and intelligence in the much larger sample provided by the Human Connectome Project (n = 1022) and found a highly statistically-significant correlation (disattenuated ρ = 0.18, p < 0.001). We replicated this result in the Minnesota Center for Twin and Family Research (n = 2698), finding a highly statistically-significant within-family correlation between head circumference and intelligence (disattenuated ρ = 0.19, p < 0.001). We also employed novel methods of causal inference relying on summary statistics from (GWAS) of head size (n ≈ 10,000) and measures of cognition (257,000 < n < 767,000). Using bivariate Score regression, we found a genetic correlation between intracranial volume (ICV) and years of education (EduYears) of 0.41 (p < 0.001). Using the (LCV) method, we found a genetic causality proportion of 0.72 (p < 0.001); thus the arises from an asymmetric pattern, extending to sub-significant loci, of genetic variants associated with ICV also being associated with EduYears but many genetic variants associated with EduYears not being associated with ICV. This is the pattern of genetic results expected from a causal effect of brain size on intelligence. These findings give reason to take up the hypothesis that the dramatic increase in brain volume over the course of human evolution has been the result of favoring general intelligence.

26. ⁠, (2019-03-25):

Heritability, the proportion of phenotypic explained by genetic factors, can be estimated from data 1, but such estimates are uninformative with respect to the underlying genetic architecture. Analyses of data from genome-wide association studies (GWAS) on unrelated individuals have shown that for human traits and disease, approximately one-third to two-thirds of heritability is captured by common SNPs 2–5. It is not known whether the remaining heritability is due to the imperfect tagging of causal variants by common SNPs, in particular if the causal variants are rare, or other reasons such as over-estimation of heritability from pedigree data. Here we show that pedigree heritability for height and (BMI) appears to be fully recovered from whole-genome sequence (WGS) data on 21,620 unrelated individuals of European ancestry. We assigned 47.1 million genetic variants to groups based upon their minor allele frequencies (MAF) and linkage disequilibrium (LD) with variants nearby, and estimated and partitioned variation accordingly. The estimated heritability was 0.79 (SE 0.09) for height and 0.40 (SE 0.09) for BMI, consistent with pedigree estimates. Low-MAF variants in low LD with neighbouring variants were enriched for heritability, to a greater extent for protein altering variants, consistent with thereon. Cumulatively variants in the MAF range of 0.0001 to 0.1 explained 0.54 (SE 0.05) and 0.51 (SE 0.11) of heritability for height and BMI, respectively. Our results imply that the still missing heritability of complex traits and disease is accounted for by rare variants, in particular those in regions of low LD.

27. ⁠, (2019-03-12):

Socio-economic position (SEP) is a multi-dimensional construct reflecting (and influencing) multiple socio-cultural, physical, and environmental factors. Previous genome-wide association studies (GWAS) using household income as a marker of SEP have shown that common genetic variants account for 11% of its variation. Here, in a sample of 286,301 participants from UK Biobank, we identified 30 independent genome-wide statistically-significant loci, 29 novel, that are associated with household income. Using a recently-developed method to meta-analyze data that leverages power from genetically-correlated traits, we identified an additional 120 income-associated loci. These loci showed clear evidence of functional enrichment, with transcriptional differences identified across multiple cortical tissues, in addition to links with GABAergic and serotonergic neurotransmission. We identified neurogenesis and the components of the synapse as candidate biological systems that are linked with income. By combining our GWAS on income with data from eQTL studies and chromatin interactions, 24 genes were prioritized for follow up, 18 of which were previously associated with cognitive ability. Using ⁠, we identified cognitive ability as one of the causal, partly-heritable phenotypes that bridges the gap between molecular genetic inheritance and phenotypic consequence in terms of income differences. Significant differences between genetic correlations indicated that, the genetic variants associated with income are related to better mental health than those linked to educational attainment (another commonly-used marker of SEP). Finally, we were able to predict 2.5% of income differences using genetic data alone in an independent sample. These results are important for understanding the observed socioeconomic inequalities in Great Britain today.

28. https://www.theatlantic.com/science/archive/2016/02/the-unexplored-marvels-locked-away-in-our-natural-history-museums/459306/

29. https://www.wired.com/story/a-new-method-of-dna-testing-could-solve-more-shootings/

30. https://www.theatlantic.com/science/archive/2019/03/dna-tests-for-envelopes-have-a-price/583636/

31. #gwern-embryo-selection-glue-robbers-sequencing-nobelists-using-collectible-letters

32. https://elifesciences.org/articles/43599

33. https://www.theatlantic.com/science/archive/2019/03/the-revolutionary-discovery-of-a-distributed-virus/584884/

34. ⁠, (2019-03-15):

To extend the frontier of genome editing and enable the radical redesign of mammalian genomes, we developed a set of dead-Cas9 base editor (dBEs) variants that allow editing at tens of thousands of loci per cell by overcoming the cell death associated with DNA double-strand breaks (DSBs) and single-strand breaks (SSBs). We used a set of gRNAs targeting repetitive elements—ranging in target copy number from about 31 to 124,000 per cell. dBEs enabled survival after large-scale base editing, allowing targeted mutations at up to ~13,200 and ~2610 loci in 293T and human induced pluripotent stem cells (hiPSCs), respectively, three orders of magnitude greater than previously recorded. These dBEs can overcome current on-target mutation and toxicity barriers that prevent cell survival after large-scale genome engineering.

One Sentence Summary

Base editing with reduced DNA nicking allows for the simultaneous editing of >10,000 loci in human cells.

35. https://www.newyorker.com/magazine/2014/12/08/ride-lives

36. ⁠, (2019-03-04):

Collective behaviors are widespread in nature are usually assumed to be strongly shaped by natural selection. However, the degree to which variation in collective behaviors is heritable and has fitness consequences—the two prerequisites for evolution by natural selection—is largely unknown. We used a new pharaoh ant (Monomorium pharaonis) mapping population to estimate the heritability, genetic correlations, and fitness consequences of three collective behaviors (foraging, aggression, and exploration) as well as body size, sex ratio, and caste ratio. Heritability estimates for the collective behaviors were moderate, ranging from 0.22 to 0.40, but lower than our estimates for the heritability of caste ratio, sex ratio, and the body size of new workers, queens, and males. Moreover, the collective behaviors were phenotypically correlated and in some cases genetically correlated, suggesting that they form a suite of correlated traits. Finally, we found evidence for directional, stabilizing, and disruptive selection that was similar in strength to estimates of selection in natural populations. Disruptive selection was very common and may act to maintain behavioral variation. Altogether, our study begins to elucidate the genetic architecture of collective behavior and is one of the first studies to demonstrate that it is shaped by selection.

37. ⁠, (2019-03-13):

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore’s law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.

…In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess…A similar pattern of research progress was seen in computer Go, only delayed by a further 20 years. Enormous initial efforts went into avoiding search by taking advantage of human knowledge, or of the special features of the game, but all those efforts proved irrelevant, or worse, once search was applied effectively at scale…In speech recognition, there was an early competition, sponsored by ⁠, in the 1970s. Entrants included a host of special methods that took advantage of human knowledge—knowledge of words, of phonemes, of the human vocal tract, etc. On the other side were newer methods that were more statistical in nature and did much more computation, based on hidden Markov models (HMMs). Again, the statistical methods won out over the human-knowledge-based methods…In computer vision…Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.

…We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that (1) AI researchers have often tried to build knowledge into their agents, (2) this always helps in the short term, and is personally satisfying to the researcher, but (3) in the long run it plateaus and even inhibits further progress, and (4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.

[My meme summary:

38. https://distill.pub/2019/activation-atlas/

39. Recent research used machine learning methods to predict a person’s sexual orientation from their photograph (Wang & Kosinski 2017). To verify this result, two of these models are replicated, one based on a deep neural network (DNN) and one on facial morphology (FM). Using a new dataset of 20,910 photographs from dating websites, the ability to predict sexual orientation is confirmed (DNN accuracy male 68%, female 77%, FM male 62%, female 72%). To investigate whether facial features such as brightness or predominant colours are predictive of sexual orientation, a new model based on highly blurred facial images was created. This model was also able to predict sexual orientation (male 63%, female 72%). The tested models are invariant to intentional changes to a subject’s makeup, eyewear, facial hair and head pose (angle that the photograph is taken at). It is shown that the head pose is not correlated with sexual orientation. While demonstrating that dating profile images carry rich information about sexual orientation these results leave open the question of how much is determined by facial morphology and how much by differences in grooming, presentation and lifestyle. The advent of new technology that is able to detect sexual orientation in this way may have serious implications for the privacy and safety of gay men and women.

40. https://slatestarcodex.com/2019/03/14/gwerns-ai-generated-poetry/

41. #gwern-gpt-2

42. https://www.gwern.net/docs/statistics/bias/2011-sainani.pdf

43. https://gen.medium.com/why-fears-of-fake-news-are-overhyped-2ed9ca0a52c9

44. https://dominiccummings.com/2019/03/01/on-the-referendum-31-project-maven-procurement-lollapalooza-results-nuclear-agi-safety/

45. https://www.rollingstone.com/culture/culture-news/in-the-valley-of-the-shadow-of-death-guyana-after-the-jonestown-massacre-242259/?print=true

46. https://slatestarcodex.com/2015/08/11/book-review-chronicles-of-wasted-time/

47. https://archive.org/details/MuggeridgeMalcolmChroniclesOfWastedTime/page/n2

48. https://www.thedailybeast.com/inside-the-secret-facebook-war-for-mormon-hearts-and-minds

49. 2011-kampfe.pdf: ⁠, (2010-11-08; music-distraction):

Background music has been found to have beneficial, detrimental, or no effect on a variety of behavioral and psychological outcome measures.

This article reports a that attempts to summarize the impact of background music. A global analysis shows a null effect, but a detailed examination of the studies that allow the calculation of effects sizes reveals that this null effect is most probably due to averaging out specific effects. In our analysis, the probability of detecting such specific effects was not very high as a result of the scarcity of studies that allowed the calculation of respective ⁠.

Nonetheless, we could identify several such cases: a comparison of studies that examined background music compared to no music indicates that background music disturbs the reading process, has some small detrimental effects on memory, but has a positive impact on emotional reactions and improves achievements in sports. A comparison of different types of background music reveals that the tempo of the music influences the tempo of activities that are performed while being exposed to background music.

It is suggested that effort should be made to develop more specific theories about the impact of background music and to increase the methodological quality of relevant studies.

[Keywords: background music, effects of music, healthy adults, meta-analysis, methodological problems]

50. https://www.imf.org/~/media/Files/Publications/WP/2018/wp18268.ashx#pdf

51. SMPY

52. ⁠, (2019-02-19):

[Meditation on what drives social networks like Instagram: status and signaling. A social network provides a way for monkeys to create and ascend status hierarchies, and a new social network can and succeed by offering a new way to do that.]

Let’s begin with two principles:

1. People are status-seeking monkeys
2. People seek out the most efficient path to maximizing social capital

…we can start to demystify social networks if we also think of them as SaaS businesses, but instead of software, they provide status.

Almost every social network of note had an early signature proof of work hurdle. For Facebook it was posting some witty text-based status update. For Instagram, it was posting an interesting square photo. For Vine, an entertaining 6-second video. For Twitter, it was writing an amusing bit of text of 140 characters or fewer. Pinterest? Pinning a compelling photo. You can likely derive the proof of work for other networks like Quora and Reddit and Twitch and so on. Successful social networks don’t pose trick questions at the start, it’s usually clear what they want from you.

…Thirst for status is potential energy. It is the lifeblood of a Status as a Service business. To succeed at carving out unique space in the market, social networks offer their own unique form of status token, earned through some distinctive proof of work.

…Most of these near clones have and will fail. The reason that matching the basic proof of work hurdle of an Status as a Service incumbent fails is that it generally duplicates the status game that already exists. By definition, if the proof of work is the same, you’re not really creating a new status ladder game, and so there isn’t a real compelling reason to switch when the new network really has no one in it.

…Why do social network effects reverse? Utility, the other axis by which I judge social networks, tends to be uncapped in value. It’s rare to describe a product or service as having become too useful. That is, it’s hard to over-serve on utility. The more people that accept a form of payment, the more useful it is, like Visa or Mastercard or Alipay. People don’t stop using a service because it’s too useful.

…Social network effects are different. If you’ve lived in New York City, you’ve likely seen, over and over, night clubs which are so hot for months suddenly go out of business just a short while later. Many types of social capital have qualities which render them fragile. Status relies on coordinated consensus to define the scarcity that determines its value. Consensus can shift in an instant. Recall the friend in Swingers, who, at every crowded LA party, quips, “This place is dead anyway.” Or recall the wise words of noted sociologist Groucho Marx: “I don’t care to belong to any club that will have me as a member.”

53. https://jeffhuang.com/Final_HaloLearning_CHI13.pdf

54. Spaced-repetition

55. https://academic.oup.com/jcem/article/84/12/4324/2864451

56. https://pubmed.ncbi.nlm.nih.gov/7825135/

57. ⁠, (2019-01-29):

58. Ads#replication

59. ⁠, (2010-10):

[Slideshow presentation on the “cosmic ladder”: how to calculate the distances between planets and stars by using geometry, brightness, radar, and progressively estimating further and further, solving one unknown at a time, from the Ancient Greeks to today.]

60. https://www.usenix.org/system/files/login/articles/login_fall16_08_beyer.pdf

61. https://kk.org/thetechnium/progression-of/

62. https://www.amazon.com/What-Technology-Wants-Kevin-Kelly/dp/0143120174

63. https://hapgood.us/2019/03/28/network-heuristics/

64. https://www.gq.com/story/secrets-of-the-worlds-greatest-art-thief

65. https://cseweb.ucsd.edu/~voelker/pubs/cloaking-ccs11.pdf

66. https://codegolf.stackexchange.com/questions/11880/build-a-working-game-of-tetris-in-conways-game-of-life/142673

67. https://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/

68. http://www.douglas-self.com/MUSEUM/COMMS/airclock/airclock.htm

69. https://news.ycombinator.com/item?id=19441300

70. https://archive.fo/MGuvf

71. https://khn.org/news/death-by-a-thousand-clicks/

72. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers

73. https://www.theguardian.com/business/2019/mar/05/long-read-aldi-discount-supermarket-changed-britain-shopping

74. 1997-mikkelson.pdf: ⁠, (1997-07; philosophy):

Once Ejo asked: “What is meant by the expression: ‘Cause and effect are not clouded’?” Dogen said: “Cause and effect are immovable.” Ejo asked: “If this is so, how can we escape?” Dogen replied: “Cause and effect emerge clearly at the same time.” Ejo asked: “If this is so, does cause prompt the next effect, or does effect bring about the next cause?” Dogen said: “If everything were like that, it would be like Nan-ch’uan cutting the ⁠. Because the assembly was unable to say anything, Nan-ch’uan cut the cat in two. Later, when Nan-ch’uan told this story to Chao-chou, the latter put his straw sandal on his head and went out, an excellent performance. If I had been Nan-ch’uan, I would have said: ‘Even if you can speak, I will cut the cat, and even if you cannot speak, I will still cut it. Who is arguing about the cat? Who can save the cat?’”

—Dogen, Shobogenzo Zuimonki, 1.61

…“One day a student asked me, ‘Does a man of enlightenment fall under the yoke of causation or not?’ I answered, ‘No, he does not.’ Since then I have been doomed to undergo five hundred rebirths as a fox. I beg you now to give the turning word to release me from my life as a fox. Tell me, does a man of enlightenment fall under the yoke of causation or not?” Hyakujo answered, “He does not ignore [cloud] causation [cause and effect].” No sooner had the old man heard these words than he was enlightened.2

“Causation” in this passage refers to “moral causation.” The Buddhist concept of karma acknowledges that good/​​​​bad deeds, thoughts, and so forth result in good/​​​​bad effects. Thus the import of the question posed by the “fox” is whether or not the enlightened person is subject to karma. Hyakujo’s answer, in effect, affirms that the enlightened person is subject to moral causation. Katsuki Sekida offers a common Zen interpretation of this passage in his comment: “Thus to ignore causation only compounds one’s malady. To recognize causation constitutes the remedy for it.”4

Dōgen’s employment of this story in the “Daishugyo” chapter of the implies that, on one level, he thinks Hyakujo’s answer indeed provides a “remedy” for the old man’s predicament.5 Yet Dogen was rarely content with merely citing traditional Zen interpretations of passages; typically, he sought to push his students to a further understanding by a creative reinterpretation of a passage. Lest his disciple therefore think this not-ignoring/​​​​recognition of causation is de facto a release from it in an ultimate sense, Dogen answers that the passage means “cause and effect are immovable.” In other words, moral causation, for Dogen, is an inexorable fact of human existence.

Given this fact, Ejo then asks how we can ever “escape” moral causation. Dogen’s response is enigmatic: “Cause and effect arise at the same time.” Nowhere in the Shōbōgenzō Zuimonki does he further clarify this passage. However, the key to understanding this statement can be gleaned from his discussion of causation in the “Shoakumakusa” chapter of the Shōbōgenzō, wherein he observes that “cause is not before and effect is not after.”6 As Hee-Jin Kim explains, Dogen saw cause and effect as absolutely discontinuous moments that, in any given action, arise simultaneously from “thusness.” Therefore,

no sooner does one choose and act according to a particular course of action than are the results thereof (heavens, hells, or otherwise) realized in it… Man lives in the midst of causation from which he cannot escape even for a moment; nevertheless, he can live from moment to moment in such a way that these moments are the fulfilled moments of moral and spiritual freedom and purity in thusness.7

…Dogen’s own proposed response helps us to see the point he is trying to make via the words of the old Master: “In expressing full function, there are no fixed methods.” In other words, there is no fixed formula for expressing and eliciting without-thinking. Nan-ch’uan, in Dogen’s view, betrayed an attachment to only two positions—to kill or not kill the cat. He was “fixated”, we might say, by these two possibilities. This is evidenced by the fact that he does indeed carry out one of them precisely as he said he would.

75. 1974-lem-cyberiad-trurlselectronicbard.pdf: “The First Sally (A), or, Trurl's Electronic Bard”⁠, Stanisflaw Lem, Michael Kandel

76. Anime#made-in-abyss

77. Movies#stalker

78. Movies#die-walkure

79. https://www.youtube.com/watch?v=BeSbmS06k1k

80. https://www.amazon.com/Abyss-Original-Soundtrack-KEVIN-PENKIN/dp/B073QWKGNS

81. https://www.youtube.com/watch?v=FSV3EfuPxmA

82. https://www.youtube.com/watch?v=KPxSS1zHWwQ

83. https://www.youtube.com/watch?v=CMpOe-qYS_4

84. https://www.youtube.com/watch?v=oaxWILI6vbA

85. https://www.youtube.com/watch?v=HQdD-OgFKxQ

86. https://www.youtube.com/watch?v=-L6z4oYm9lU

87. https://www.youtube.com/watch?v=0-4-vzXYr0E

88. https://lycandesebeats.bandcamp.com/album/beats-me-3