January 2020 gwern.net newsletter with 5 writeups, and links on AI scaling, videos-for-cats, and art; 1 book and 1 opera review.
topics:
source; created: 26 Dec 2019; modified: 31 Jan 2020; status: finished; confidence: log; importance: 0
This is the January 2020 edition of the gwern.net newsletter; previous, December 2019/2019 round-up (archives). This is a summary of the revision-history RSS feed, overlapping with my Changelog & /r/gwern; brought to you by my donors on Patreon.
Writings
“Danbooru2019: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”
- Preference Learning GPT-2 Music: Null Result
- This Waifu Does Not Existv3: 100k StyleGAN 2 anime portrait samples
- Subreddit Simulator: GPT-2-1.5b upgrade
- 14 Internet Search Case Studies
gwern.net: margin notes are now inlined on mobile
Media
Links
Genetics:
Everything Is Heritable:
Recent Evolution:
“The Exposome in Human Evolution: From Dust to Diesel”
, Trumble & Finch 2019 (media; previously, Hubbard et al 2016)
Engineering:
“Utility and First Clinical Application of Screening Embryos for Polygenic Disease Risk Reduction”
, Treff et al 2019 (Genomic Prediction)“CC, world’s first cloned cat, turns 18 years old”
, Katz
AI:
“2019 AI Alignment Literature Review and Charity Comparison”
, LarksMatters Of Scale:
“Scaling Laws for Neural Language Models”
, Kaplan et al 2020 (NN LMs appear to be nowhere near saturation nor training infeasibility as larger models show predictable gains and are both more compute-efficient & data-efficient (!)—onwards to GPT-3?)“Large Scale Learning of General Visual Representations for Transfer”
, Kolesnikov et al 2019 (using JFT-300M for transfer learning)“DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames”
, Wijmans et al 2019 (blog;“Mishkin et al 2019 benchmarked classical (mapping + planning) and learning-based methods…and showed that classical methods outperform learning-based. However, they trained for ‘only’ 5 million steps…Savva et al 2019 then scaled this training to 75 million steps and found that this trend reverses…Fig. 1 shows an agent does not saturate before 1 billion steps, suggesting that previous studies were incomplete by 1–2 orders of magnitude.”
)“Meena: Towards a Human-like Open-Domain Chatbot”
, Adiwardana et al 2020 (blog; 2.6b parameters trained on 341GB text, although Kaplan et al 2020 suggests they’d’ve done better to go much bigger & trained much less than 164 epoches; likelihood loss near identical with human-rated performance? Likelihood loss seems ultimately a flawed metric, but maybe we’re still far from hitting its limits…)
“A Very Unlikely Chess Game”
(GPT-2-1.5b shenanigans; rival implementation)
Statistics/meta-science/mathematics:
“Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study”
, DeVito et al 2020;“FDA and NIH let clinical trial sponsors keep results secret and break the law”
, Science“A national experiment reveals where a growth mindset improves achievement”
, Yeager et al 2019 (the incredible shrinking ‘growth mindset’ effect & the Stainless Steel Law)“Backlash Over Meat Dietary Recommendations Raises Questions About Corporate Ties to Nutrition Scientists”
, Rubin 2019 (criticizing the critics of Carroll & Doherty 2019 / Zeraatkar et al 2019a / Han et al 2019 / Vernooij et al 2019 / Zeraatkar et al 2019b / Valli et al 2019 / Johnston et al 2019—“what’s sauce for the goose is sauce for the gander”)- “Follow-up: I found two identical packs of Skittles, among 468 packs with a total of 27,740 Skittles” (empirically verifying the birthday paradox in Skittles bags)
Politics/religion:
“The rape of men: the darkest secret of war”
(case study: Libya)“Under the Weather: As Psychiatrists And Philosophers Begin To Define A Pervasive Mental Health Crisis Triggered By Climate Change, They Ask Who Is Really Sick: The Individual Or Society?”
“Statistical Reliability Analysis For A Most Dangerous Occupation: Roman Emperor”
, Saleh 2019“Parachuting For Charity: Is It Worth The Money? A 5-Year Audit Of Parachute Injuries In Tayside And The Cost To The NHS”
, Lee et al 1999“What’s in a Font?: Ideological Perceptions of Typography”
, Haenschen & Tamul 2019 (‘Sunrise is the future liberals want’)
Psychology/biology:
“A Meta-Analysis of Procedures to Change Implicit Measures”
, Forscher et al 2019“Attention And Awareness In Stage Magic: Turning Tricks Into Research”
, Macknik et al 2008“A World Without Pain: Does hurting make us human?”
(cf my essay on pain)“What Intellectual Progress Did I Make In The 2010s?”
, Scott Alexander (a look back on how his ideas/beliefs evolved over the past decade of psychiatry blogging)“Three cases of giant panda attack on humans at Beijing Zoo”
, Zhang et al 2014 (graphic images)
Technology:
“Cats, Once YouTube Stars, Are Now an ‘Emerging Audience’: They’re addicted to channels like Little Kitty & Family, Handsome Nature, and Videos for Your Cat—provided their owners switch on the iPad first”
(After reading this, I gave videos-for-cats another try with my cat, since he gets stir-crazy in winter. I full-screened it, in landscape mode; he continued resolutely ignoring the screen as always, until I left my earphones out and he heard the birds chirping—he went nuts, convinced a bird had gotten into the apartment, until he finally noticed the screen, and I could see the instant the lightbulb went on and he was hooked. He’ll watch videos like Paul Dinning’s“8 Hour Bird Bonanza”
for as many hours as I’ll leave it on, hunting behind the monitor for the birds, and makes a nuisance of himself sitting in front of it, waiting for the birds to come back—which he is doing as I try to write this! These are truly superstimuli: cats would never see these many birds this close up so unsuspecting in the wild.)
Economics:
“Clustering of health, crime and social-welfare inequality in 4 million citizens from two nations”
, Richmond-Rakerd et al 2020 (everything is correlated:“Figure 4: Aggregation of poor health, crime and social-welfare dependency”
; previously, Belsky et al 2016 & Caspi et al 2016)“The Exquisitely English (and Amazingly Lucrative) World of London Clerks”
Philosophy:
Fiction:
“Behind the Sensationalism: Images of a Decaying Corpse in Japanese Buddhist Art”
, Kanda 2005 (on kusozu, cf Maraṇasati; graphic images; famous examples: Body of a Courtesan in Nine Stages, Kobayashi Eitaku c. 1870s, and The Death Of A Noble Lady And The Decay Of Her Body, c. 1700s)“Having Had No Predecessor to Imitate, He Had No Successor Capable of Imitating Him”
, Alvaro de Menard (summary of the Homeric Question; also worth reading: Borges on the literary merits of different translations of Homer & The Thousand and One Nights)“Choose Your Own Adventure: One Book, Many Readings”
, Christian Swinehart 2009 (visualizing paths through the classic CYOA gamebooks; see also:“These Maps Reveal the Hidden Structures of Choose Your Own Adventure Books: If you decide to see more, click on this story”
, Atlas Obscura 2017)“Master of Orion”
, Jimmy Maher (review of the seminal 4X strategy game)
Misc:
Books
Nonfiction:
- An Introduction to Japanese Court Poetry, Miner 1968 (a short textbook I used years ago in writing Fujiwara no Teika & Shōtetsu; a whirlwind tour of waka court poetry from the Man’yoshu to Shōtetsu. It is not remotely as thorough as his main textbook with Brower, Japanese Court Poetry, but is not intended to be: Miner profiles the major poets and offers commentary on a few key poems, trying to bring out the esthetics, and convey to the reader, assumed to be a student, what there is to appreciate about traditional Japanese court poetry—a genre so easy to bounce off & write off as painfully plain prose sentences. He does a reasonable job in this, although I increasingly find his translations to be a little too wordy & explain too much.)
Film/TV
Live-action:
Entertainingly ironic backfire. They Live (1988; John Carpenter film)
I enjoyed The Thing, and They Live was the next-most famous Carpenter movie.
TL expresses the American paranoid style in a package justly made iconic by its thrifty but effective use of special effects: the protagonist flips between social consensus and a monochrome Art Deco-esque reality revealing 1984-like slogans painted everywhere by the secret alien masters of the world, which brainwash everyone (even though such priming ads don’t work, it at least makes a great metaphor). The pace is perhaps unnecessarily slow, and I had to wonder why a fist fight implausibly takes up several minutes—it’s a great fight, but it has little to do with the rest of the movie and requires the characters to act stupidly. The overall plot is reasonably straightforward and doesn’t need to invoke too much plot armor to explain how the aliens are defeated. I would not say it was as good as The Thing, but few movies are, and this was reasonably entertaining. TL did give me some food for thought, however.
TL takes pains to make clear its liberal credentials: if you somehow missed how Reaganism was responsible for everything bad in America and growing slums and homelessness, it shows an alien on TV giving Reaganesque speeches. (Ironically for Carpenter’s hamartiology, it puts heavy stress on homelessness as criticism, and yet, where is homelessness the worst now in the USA? Those places Reagan is most hated, like the Bay Area. Another irony is that in depicting the 1980s, it reminded me chiefly of how poor 1980s America was in comparison to now, which can be seen in how crude and limited are many of the things then we now take for granted: it’s not just the aliens sporting advanced wristwatches which are little more than two-way radios, but also the shabbiness of cars, the terrible TVs everywhere, the limited selection in the grocery store he confronts the aliens in…)
But there’s something about this that began to bug me. Consider this 100% accurate description of TL’s world-building:
“America, and the world as you know it, is not controlled by people like you—but by an alien race of invaders, parasites from far away, who have secretly wormed their way into our society and taken it over relatively recently. They hunger only for money, and have little genuine culture of their own, assimilating into yours to pass as one of us, despite their distinctly different (and often repulsive) facial appearance. They are few, but they are well-coordinated, highly intelligent, & technically adept and they occupy the heights of business, finance, politics, and media, from which they constantly beam out propaganda to delude the masses that threaten them, and which allows the parasites to execute their globalist free-trade agenda: to accelerate economic growth, homogenize the world under one government, drain us dry, discard the empty husk, and move on. Given enough strength of mind, some individuals can overcome the brainwashing, or they can use advanced new technology to learn the truth and see the world with moral clarity in black and white, for what it really is, and the coded commands from the aliens. Unfortunately, those of us who discover the truth, alerted by a black preacher, are either bought off by money & power (the aliens assume we are just as craven as they are, and are all too often right), suppressed as evil crazy ‘conspiracy theorists’ when our late-night broadcasts sometimes get through uncensored, or if they take action and try to defend us against the invaders, executed as ‘terrorists’. Organizations which resist are crushed, and infiltrated with traitors in the pay of the aliens. Their weakness is, however, they are cowardly, physically weak compared to our strapping working-class soldiers, and vastly outnumbered by the rest of us. If we can recruit enough ‘strong men’ and awaken the masses, we work together to defeat them and restore America to its former glory, and send the aliens back whence they came—the planet Zion!”
OK, OK, I made one change there: Carpenter doesn’t name any alien planets. But everything else sounds straight out of far-right fantasy: there’s even black sunglasses as the initiation instead of red pills. (Maybe the sequel can use fedoras?) I thought perhaps I was being silly, until I looked at the Wikipedia article and found that this is such a common interpretation of TL & popular among neo-Nazis that Carpenter has angrily denied it!
Now, of course, I believe Carpenter when he says he didn’t have that in mind and only intended a critique of Reaganism. But the more interesting questions here would be: how could Carpenter make a film which is so naturally and so easily misread in neo-Nazi tropes to the point of making one wonder if Carpenter dictated the screenplay while clutching a copy of The Protocols of the Elders of Zion in one hand & Mein Kampf in the other, without ever realizing it? And what does this blindness mean?
It looks to me like an example of ‘horseshoe theory’: the reason Carpenter’s TL can be so misread is because extremists on both ends of the spectrum are more alike than they are different—embracing a paranoid conspiracy theory explanation of the world, merely playing Mad Libs with the labels. They Live, accidentally rather than deliberately, demonstrates the same thing as Foucault’s Pendulum or Unsong: the flexibility of the paranoid style in enabling extremists to accommodate both anti-Reaganism & anti-Semitism is not a merit but discredit (much as Rosenthal’s ability to find large effects everywhere discredits him).
Extremists are like tribesmen out of an anthropology ethnography: everything bad that happens is due to “witchcraft”; people never get sick because of chance or because some pork went bad, and if some are healthier or sick, richer or poorer, it definitely has nothing to do with individual differences, but malign trafficking with the ruinous powers. Once you postulate that all existing social ills can be explained by witchcraft, you will go looking for witches, preferably fellow tribals who aren’t as equal as others and should be taken down a notch in the interests of hardwired egalitarianism (pace Graeber), and whether those witches are Jews or capitalists or cishet white men, witches must be found and found witches will be. To fill the hole in the extremist worldview, by working backwards to ‘save the appearances’, they must have certain powers, they must be numerically minorities, they must be motivated by lurid impure things like money (surely we have more sacred values), and so on. And the result is that you try to create a critique of Reaganism, by depicting your paranoid worldview where Reaganites are the witches, but your witches’ allegorical coating happen to superficially resemble a different set of witches and hey presto, you accidentally created neo-Nazis’ favorite allegorical movie. Oops.
The problem here, such as it is, comes well before any specific choices by Carpenter to portray the aliens as ugly or as rich corporate executives…
Failed anti-war opera. Wozzeck (Alban Berg; NYT, New Yorker).
A relentless crashing bore and a third-rate Carmen being crammed into an anti-war mold. I was left wishing it was either much shorter or much longer. The production absolutely hammers in the WWI kitsch theme, and the reviews praise its ‘searching criticism of militarism’ or whatever in driving the titular Wozzeck to madness and murder—except the text and events don’t support that in the least. It’s unclear if Wozzeck has so much as even been to a war, much less it had anything to do with his problems; the ‘sadistic’ (in Wikipedia’s description) townspeople act quite normally, Wozzeck’s captain comes off as a quite nice chap, and even the mad doctor running medical experiments on Wozzeck wants to do nothing worse than diet experiments which entail stuffing him full of beans & mutton. Marie is hardly threatened by starvation as she shows off her new gold earrings (shades of Manon), Wozzeck himself seems well off, with so few official duties he can do all these part-time jobs, and as he lives in the barracks and presumably the Army feeds him, he is hardly in any danger of starvation or homelessness. Wozzeck doesn’t seem tragic or noble so much as a rather dimwitted Charlie Brown unable to understand his problems, such as what looks like schizophrenia, but still trying to live up to various obligations he (entirely unnecessarily) took on.
The production relies heavily on gimmicks. Dressing everyone up as cripples or in gas masks is cute the first time, as are the eccentric Monty Python-style clipshows—except they are done again and again and again, without any rhyme or reason. The video clipshow is beamed onto the stage endlessly, and could be useful, similar to the projections used in The Ring, except it never seems to connect with the action! What does any of this have to do with militarism, or WWI, or anything? A similar point can be made for the choice to close with Wozzeck’s bastard being played by a puppet with a gas mask head, much like the bastard in Madama Butterfly, except while there using a puppet instead of a child actor was interesting and cool for how well the puppeteers interacted with Butterfly, here it is just pointless. The production seems particularly dumb when, checking Wikipedia’s plot summary, I see that it just hacked out various connective tissues, like why he drowned himself (paranoia in trying to retrieve the murder weapon), or that the captain/doctor were supposed to see him drowning while the production just has them wander by wondering about an odd sound and anti-climatically leaving.
Relentlessly crashingly dumb, with no good parts, and the worst Met opera I’ve seen so far—this was the first Met HD broadcast I was seriously tempted to get up and walk out early, even after telling myself it was only about an hour and a half. The Magic Flute, Turandot, and Dialogues des Carmélites all had some weaknesses, but also had their strengths, and I never thought of leaving early. I don’t know if Wozzeck is normally this bad, but this production certainly was bad in its crudity and illogic. On the bright side, the 2020 operas can only go up from here!