August 2020 Gwern.net newsletter with an essay on sidenotes; links on human competence, efficient computing, and hardware overhangs; no reviews.
2019-12-26–2021-02-15
finished
certainty: log
importance: 0
August 2020’s Gwern.net newsletter is now out; previous, July 2020 (archives). This is a summary of the revision-history RSS feed, overlapping with my Changelog & /
Writings
- Sidenotes For Web Design
- Highly Potent Drugs As Psychological Warfare Weapons (chemistry trivia)
- Notes on The Lizardman Constant
Media
Links
AI:
Matters Of Scale:
- The Bitter Lesson: “Accuracy and Performance Comparison of Video Action Recognition Approaches”, Hutchinson et al 2020 (simple large 2D CNNs performed best when compared under controlled conditions); “A Metric Learning Reality Check”, Musgrave et al 2020 (the doubling of metric learning performance since 2006 appears almost entirely due to larger neural nets + more compute used in training, not fancy loss functions/
theory) - “Self-supervised learning through the eyes of a child”, Orhan et al 2020 (dataset; it’d be fun to see this used for rats or dogs)
- “The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost”, Herculano-Houzel 2012
- “The Node Is Nonsense: There are better ways to measure progress than the old Moore’s law metric”, Moore 2020
- “Measuring hardware overhang”, hippke (“with today’s algorithms, computers would have beat the world chess champion already in 1994 on a contemporary desk computer”)
- The Bitter Lesson: “Accuracy and Performance Comparison of Video Action Recognition Approaches”, Hutchinson et al 2020 (simple large 2D CNNs performed best when compared under controlled conditions); “A Metric Learning Reality Check”, Musgrave et al 2020 (the doubling of metric learning performance since 2006 appears almost entirely due to larger neural nets + more compute used in training, not fancy loss functions/
Genetics:
Everything Is Heritable:
- “Sibling validation of polygenic risk scores and complex trait prediction”, Lello et al 2020
- “Polygenic Scores for Cognitive Abilities and their Association with Different Aspects of General Intelligence—a Deep Phenotyping Approach”, Genç et al 2020
- “Genetic testing of dogs predicts problem behaviors in clinical and nonclinical samples”, Zapata et al 2020 (previously: MacLean et al 2019)
Statistics/
- “Objecting to experiments even while approving of the policies or treatments they compare”, Heck et al 2020
Psychology/
“Epigenetic clocks: A review”, José Luis 2020; “TRIIM: Reversal of epigenetic aging and immunosenescent trends in humans”, Fahy et al 2019; “Reversal of Epigenetic Age with Diet and Lifestyle in a Pilot Randomized Clinical Trial”, Fitzgerald et al 2020
“The quest to slow ageing through drug discovery”, Partridge et al 2020
“Atribacteria reproducing over millions of years in the Atlantic abyssal subseafloor”, Vuillemin et al 2020
“Exploring the persome: The power of the item in understanding personality structure”, Revelle et al 2020 (“everything is correlated”)
How Competent Are Humans?
- “Better All the Time: How the “performance revolution” came to athletics—and beyond"; “Personal Best: Top athletes and singers have coaches. Should you?”
- “Does Management Matter? Evidence from India”, Bloom et al 2012; “The Impact of Consulting Services on Small and Medium Enterprises: Evidence from a Randomized Trial in Mexico”, Bruhn et al 2018
- “95%-ile isn’t that good”, Dan Luu (persistent incompetence in Overwatch players)
- “Worker Rush: Descent to Bronze”, Gheed 2011 (and in Starcraft players)
- “Playing to Win”, Sirlin 2006
“The Scent of the Nile: Jean-Claude Ellena creates a new perfume”, Chandler Burr 2005
Technology:
Your Computer Is Really Fast:
- “There’s plenty of room at the Top: What will drive computer performance after Moore’s law?”, Leiserson et al 2020 (gains from end-to-end systems design; AI scaling can continue even if semiconductors do not, particularly with self-optimizing stacks; John Carmack notes, apropos of Seymour Cray, that “Hyperscale data centers and even national supercomputers are loosely coupled things today, but if challenges demanded it, there is a world with a zetta[flops] scale, tightly integrated, low latency matrix dissipating a gigawatt in a swimming pool of circulating fluorinert.”)
- “Scalability! But at what COST?”, McSherry et al 2015 (when laptops > clusters: the importance of good baselines)
- “You’re Doing It Wrong: Think you’ve mastered the art of server performance? Think again.”, Poul-Henning Kamp 2010 (on using cache-oblivious B-heaps to optimize Varnish performance 10×)
- “Sample Factory: Egocentric 3D Control from Pixels at 100,000 FPS with Asynchronous Reinforcement Learning”, Petrenko et al 2020
- “What Every Programmer Should Know About Memory”, Drepper 2007
- “What it takes to run Stack Overflow”, Nick Craver
- see previously on latency/
UI : “It’s the Latency, Stupid.”/“Computer latency: 1977–2017”/ “Keyboard Latency”/ “Terminal Latency”/ “Local-first software: You own your data, in spite of the cloud”/ Kleppmann et al 2019/ “Web Bloat”
“Fälschungserschwerende Schrift (forgery-impeding typeface)”
“Inukshuk: Caribou Drive Lanes on Southern Victoria Island, Nunavut, Canada”, Brink 2005
“The case of the barnacled crystal”, Kohman 1950 (on disappearing polymorphs)
Economics:
- “Moving off the Map: How Knowledge of Organizational Operations Empowers and Alienates”, Huising 2019 (“The CEO, after being walked through the map, sat down, put his head on the table, and said, ‘This is even more fucked up than I imagined.’”)
- “The Age of Reason: Financial Decisions over the Life-Cycle with Implications for Regulation”, Agarwal et al 2009
Fiction:
- “An Autobiographical Essay”, Jorge Luis Borges 1971
- “Singular: Possible futures of the singularity”, James Yu & GPT-3
Misc:
Music
- “Moonbow” (Kirin feat. Chata; Sprout Intention {C88}) [trance]
- “Harvester’s Dance” (Ganemes; Sister’s 3 {TK12}) [folk]