AI

• “Defining and Characterizing Reward Hacking”, Skalse et al 2022

• “Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles”, Kusano et al 2023

• RL IRL: on Google Search use of ranking & preference-learning 2015–2019

• “Explaining grokking through circuit efficiency”, Varma et al 2023

• “PIGEON: Predicting Image Geolocations”, Haas et al 2023 (a much better PlaNet—it beat Rainbolt!)

• “Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero”, Schut et al 2023 (human GMs can learn from AZ… but remain far inferior)

• “Large-scale statistical forecasting models reassess the unpredictability of chaotic systems”, Gilpin2023

• “Random Search Wired Into Animals May Help Them Hunt: The nervous systems of foraging and predatory animals may prompt them to move along a special kind of random path called a Lévy walk to find food efficiently when no clues are available”

• “Computer Backgammon”, Hans J. Berliner1980 (“BKG 9.8 is the first computer program to defeat a world champion at any board or card game”); “Marvin Minsky’s Vision of the Future”, Bernstein1981

• “The Manga Whisperer: Automatically Generating Transcriptions for Comics”, Sachdeva & Zisserman2024

• “Making Amulets with LLaMA-2”, Blackle Mori

Matters Of Scale:

• “Thousands of AI Authors on the Future of AI”, Grace et al 2024 (dropping timelines)

• “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training”, Hubinger et al 2024 (larger models better at hiding backdoors from safety training)

• “Generative Multimodal Models are In-Context Learners”, Sun et al 2023

• “Large Language Models Can Teach Themselves to Use Tools”, Schick et al 2023 (API/tool-use emerges ~0.8b-parameters)

• Phi-2: The surprising power of small language models

• “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, Gu & Dao2023: Attention-free models scale poorly at in-context recall/induction (Arora et al 2023), which is mostly why Transformers beat them

• “ByteDance is secretly using OpenAI’s tech to build a competitor” (why is it so hard for competing LLMs to surpass GPT-3.5/4? maybe because they are lying about being based on it)

• “CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images”, Gokaslan et al 2023

• “Character-Aware Models Improve Visual Text Rendering”, Liu et al 2022 (ByT5 vs T5 vs PaLM demonstrates BPEs are responsible for screwed-up text in images; PaLM’s scale can solve common spelling by memorization, but not generalize—creating the illusion of a solution)

• “What’s Hidden in a Randomly Weighted Neural Network?”, Ramanujan et al 2019 (even random nets contain, with increasing probability in size, an accurate sub-net)

Genetics

Everything Is Heritable:

• “Shared genetic architectures of educational attainment in East Asian and European populations”, Chen et al 2024

Recent Evolution:

• “Genetic variants underlying human bisexual behavior are reproductively advantageous”, Song & Zhang2024

• “Elevated genetic risk for multiple sclerosis emerged in steppe pastoralist populations”, Barrie et al 2024

Engineering:

• “Concordance of whole-genome amplified embryonic DNA with the subsequently born child”, Li et al 2024

• “Cloned rhesus monkey lives to adulthood for first time”

• “AAV1-hOTOF gene therapy for autosomal recessive deafness 9: a single-arm trial”, Lv et al 2024; “Gene Therapy Allows an 11-Year-Old Boy to Hear for the First Time: The genetic treatment targeted a particular kind of congenital deafness and will soon be tried in children who are younger”

Statistics/Meta-Science

• “The Usefulness of Useless Knowledge”, Flexner1939

• “Laboratory-acquired infections and pathogen escapes worldwide 2000–2021: a scoping review”, Blacksell et al 2023

• “Making the Micropipette”, Metacelsus

• “Cracking the Code of Linear B”, Theodore Nash

• “Projecting complete redaction of clinical trial protocols (RAPTURE): redacted cross sectional study”, ███████ et al 20██

• “Behind The Scenes: The Struggle For Each Paper”, Jeff Huang (2 decades of CS/HCI research)

• “Possible Bubbles of Spacetime Curvature in the South Pacific”, Tippett2012

• “Solar Eclipses and the Origins of Critical Thinking and Complexity”, Litina & Fernández2023 (astronomy as the training wheels of Science?)

Politics/Religion

• “Liberal Visions and Boring Machines: The Early History of the Channel Tunnel”

• “The Cultural Power of Elite Immigrants: A Slaveowner, a Nazi, a Lutheran, a Jew, and a Forty-Eighter Walk Into a Voting Booth”, Cremieux

• “Economic Consequences of Kinship: Evidence From U.S. Bans on Cousin Marriage”, Ghosh et al 2023

• “Notes on the Ivory Coast”, Matt Lakeman

• Fires in Edo

• “Review: Demons, by Fyodor Dostoevsky”

• “Is white always the standard? Using replication to revisit and extend what we know about the leadership prototype”, Obenauer & Kalsher2023 (a pre-registered failure to replicate pro-white-leader bias & evidence of anti-white-leader bias)

Psychology/Biology

• “Efficacy and safety of oral orforglipron in patients with type 2 diabetes: a multicenter, randomized, dose-response, phase 2 study”, Frias et al 2023

• “A vibrating ingestible bioelectronic stimulator modulates gastric stretch receptors for illusory satiety”, Srinivasan et al 2023

• “Randomized Trial of Ketamine Masked by Surgical Anesthesia in Depressed Patients”, Lii et al 2023

• “An energy costly architecture of neuromodulators for human brain evolution and cognition”, Castrillon et al 2023; “Gene purging and the evolution of Neoave [bird] metabolism and longevity”, Ng et al 2023; “Tempo and Pattern of Avian Brain Size Evolution”, Ksepka et al 2020

• “The value of information gathering in phage-bacteria warfare”, Dahan et al 2024

• “Fetching felines: a survey of cat owners on the diversity of cat (Felis catus) fetching behavior”, Forman et al 2023

• “How This Marine Worm Can Tell Moonglow From Sunbeams” (a clever solution: have a light-sensitive protein which is extremely fragile)

• The discovery of mesopelagic fish

• “The Woman Who Spent Five Hundred Days in a Cave: Beatriz Flamini liked to be alone so much that she decided to live underground—and pursue a world record. The experience was gruelling and surreal.” (contrary to the initial press conference, she suffered severe mental & physical health issues)

• Thomas Cochrane, 10^th Earl of Dundonald

• “Nail Growth: 35 Years of Observation”, Bean1980; “Effects of Sexual Activity on Beard Growth in Man”, Anonymous1970

Technology

• “Ultraviolet light can kill almost all the viruses in a room. Why isn’t it everywhere?”

• “Building Apollo: A review of Angle of Attack: Harrison Storms and the Race to the Moon” (project management)

• “Programming as Theory Building”, Naur1985

• “Artificial Molecular Machines”, Erbas-Cakmak et al 2015

• “Plants Find Light Using Gaps Between Their Cells” (unexpected way to compute phototropism)

• “Thanks to This Man, Airplanes Don’t Crash Into Mountains Anymore: Don Bateman’s terrain mapping device has nearly eliminated the largest cause of death in jetliner accidents” (initially using USSR global terrain maps bought on the Russian blackmarket)

• “The World’s Fastest Road Cars—and the People Who Drive Them”

• “Chaffing and winnowing”: cryptography without encryption

• “Non-detection of the Tooth Fairy at Optical Wavelengths”, Armstrong2012

Economics

• “Lay economic reasoning: An integrative review and call to action”, Bhattacharjee & Dana2024

• “Does alleviating poverty increase cognitive performance? Short- and long-term evidence from a randomized controlled trial”, Szaszi et al 2023 (no); “Cognitive Biases: Mistakes or Missing Stakes?”, Enke et al 2023

• “Can Rock Dust Soak Up Carbon Emissions? A Giant Experiment Is Set to Find Out”

• “The Tremendous Yet Troubled State of Gaming in 2024”, Matthew Ball

Philosophy

Fiction

• “The Names of the Hare”, Anonymous (translated Heaney1982)

• “SAURON, Mount Doom, and Elvish Moths: The Influence of Tolkien on Modern Science”, Larsen2007

• The first successful AMV: Steamboat Willie (“Turkey in the Straw”+click tracks)

• “Cannibalism in the Cars”, Mark Twain1868

• “I Found David Lynch’s Lost Dune II Script” in the Frank Herbert university archives, Andy Greenberg