AI

• “ReBeL: Com­bin­ing Deep Rein­force­ment Learn­ing and Search for Imper­fec­t-In­for­ma­tion Games”, Brown et al 2020 (head­s-up no-limit Texas hold’em poker close to solved with a gen­er­al­iz­able Alp­haZe­ro-like approach?)
• “Assess­ing Game Bal­ance with Alp­haZe­ro: Explor­ing Alter­na­tive Rule Sets in Chess”, Tomašev et al 2020
• “A dig­i­tal bio­marker of dia­betes from smart­phone-based vas­cu­lar sig­nals”, Avram et al 2020 (time-series of blood pulses can diag­nose dia­betes? weird, but not as weird as pre­dict­ing gen­der from reti­nas)
• “Tasks, sta­bil­i­ty, archi­tec­ture, and com­pute: Train­ing more effec­tive learned opti­miz­ers, and using them to train them­selves”, Metz et al 2020; “Gra­di­ent Descent: The Ulti­mate Opti­mizer”, Chan­dra et al 2019 (deep­er—we need to go deep­er… now scales to VGG-Net)
• “X-LXMERT/X-UNITER: Paint, Cap­tion and Answer Ques­tions with Mul­ti­-Modal Trans­form­ers”, Cho et al 2020 (online demo; final­ly, some new work on text → image)
• “StyleGAN net­work blend­ing”, Justin Pinkney/“Res­o­lu­tion Depen­dent GAN Inter­po­la­tion for Con­trol­lable Image Syn­the­sis Between Domains”, Pinkney & Adler 2020 (in­ter­po­lat­ing between mod­els to trans­fer images between domains, eg FFHQ ↔︎ foxes ↔︎ anime ↔︎ MLP ↔︎ fur­ries)
• “Opti­mal Peanut But­ter and Banana Sand­wiches”, Ethan Rosen­thal (bin pack­ing banana slices)

Mat­ters Of Scale:

• “Deep­Speed: Extreme-s­cale model train­ing for every­one” (demon­strates train­ing of GPT-3-180b & 1t-pa­ra­me­ter mod­els (“The tril­lion-pa­ra­me­ter model has 298 lay­ers of Trans­form­ers with a hid­den dimen­sion of 17,408 and is trained with sequence length 2,048 and batch size 2,048”), w/open-source code; able to use CPU+GPU RAM simul­ta­ne­ously for 13b-pa­ra­me­ter mod­els on each node per Pudipeddi et al 2020; sparse atten­tion for sav­ing RAM; approx­i­mated Adam gra­di­ents for sav­ing band­width)
• “GPT-f: Gen­er­a­tive Lan­guage Mod­el­ing for Auto­mated The­o­rem Prov­ing”, Polu & Sutskever 2020b (GPT-2 for Meta­math scales & can boot­strap its the­o­rem-prov­ing abil­i­ty—onward to IMO!)
• “Mea­sur­ing Mas­sive Mul­ti­task Lan­guage Under­stand­ing”, Hendrycks et al 2020 (in­tro­duc­ing new much harder bench­marks to stress-test mod­el­s—n­ev­er­the­less, big­ger=­bet­ter: GPT-3 model size vs Q&A, with GPT-3-173b hit­ting almost 50% out of 4-choice ques­tions vs base­line mod­els like T5 at the ran­dom base­line of only 25%)
• “Dynam­icEm­bed­ding: Extend­ing Ten­sor­Flow for Colos­sal-S­cale Appli­ca­tions”, Zeng et al 2020 (em­bed­dings-as-a-ser­vice Google-wide; tool AIs want to be agent AIs…)
• “A Time Leap Chal­lenge for SAT Solv­ing”, Fichte et al 2020 (hard­ware over­hang: ~2.5× per­for­mance increase in SAT solv­ing since 2000, about equally due to soft­ware & hard­ware gains, although slightly more soft­ware—new soft­ware on old hard­ware beats old on new. See pre­vi­ously Grace 2013.)
• “Effi­cient Trans­form­ers: A Sur­vey”, Tay et al 2020 (more on effi­cient atten­tion approach­es)

Genetics

Every­thing Is Her­i­ta­ble:

• “A gen­eral dimen­sion of genetic shar­ing across diverse cog­ni­tive traits inferred from mol­e­c­u­lar data”, de la Fuente et al 2020
• “Map­ping genomic loci pri­ori­tises genes and impli­cates synap­tic biol­ogy in schiz­o­phre­nia”, PGC et al 2020 (PGC SCZ3: 248 genome-wide hits; 7.7% PGS; OR = 44 com­par­ing top vs bot­tom per­centile risk); “Exome sequenc­ing iden­ti­fies rare cod­ing vari­ants in 10 genes which con­fer sub­stan­tial risk for schiz­o­phre­nia”, Singh et al 2020 (fig­ure 6a is a strik­ing demon­stra­tion of negative/purifying selec­tion: more harm­ful muta­tions are rar­er)
• “Genome-wide asso­ci­a­tion study of over 40,000 bipo­lar dis­or­der cases pro­vides novel bio­log­i­cal insights”, Mullins et al 2020 (PGC3 BPD; PGS: 4.75%)
• “Genetic For­tune: Win­ning or Los­ing Edu­ca­tion, Income, and Health”, Kweon et al 2020 (with­in-fam­ily education/income/health PGSes)
• “GWAS of Over 427,000 Indi­vid­u­als Estab­lishes GABAergic and Synap­tic Mol­e­c­u­lar Path­ways as Key for Cog­ni­tive Exec­u­tive Func­tions”, Hatoum et al 2020
• “Shared her­i­tabil­ity of face and brain shape dis­tinct from cog­ni­tive traits”, Naqvi et al 2020

Recent Evo­lu­tion:

• “Rapid Evo­lu­tion of Plas­tic-de­grad­ing Enzymes Preva­lent in the Global Ocean”, Alam et al 2020

Engi­neer­ing:

• The first suc­cess­ful cloning of the endan­gered Prze­wal­ski’s horse
• “Human Embryo Gene Edit­ing Gets a Road Map—Not a Green Light” (Na­tional Acad­emy of Sci­ence’s Her­i­ta­ble Human Genome Edit­ing 2020 report)
• “CRISPR-engineered human brown-like adipocytes pre­vent diet-in­duced obe­sity and ame­lio­rate meta­bolic syn­drome in mice”, Wang et al 2020; “CRISPR-enhanced human adipocyte ‘brown­ing’ as cell ther­apy for meta­bolic dis­ease”, Tsagkaraki et al 2020

Statistics/Meta-Science/Mathematics

• “Blue­berry Earth”, Sand­berg 2018

Politics/Religion

• “Amer­ica in Decay: The Sources of Polit­i­cal Dys­func­tion”, Fukuyama 2014 (see also: con­sti­tu­tional hard­ball/“The Empty Cham­ber”)
• “‘Dwarf Pride’ Was Hard Won. Will a Growth Drug Under­mine It? An exper­i­men­tal med­ica­tion that increases height in chil­dren with the most com­mon form of dwarfism has raised hope that it can help them lead eas­ier lives. But some say the con­di­tion is not a prob­lem in need of a cure.” (Savari­rayan et al 2020; amus­ing triv­ia: the rea­son why cross­ing any dog breed with Cor­gis gives you a Corgi­fied ver­sion of that dog breed is because Cor­gis are homozy­gous dom­i­nant on an FGF4 muta­tion caus­ing sim­i­lar achon­dropla­sia as in humans)
• “Laws of Xmas”, Akiva & Ilene Miller 1998
• Gru­inard Island
• Lon­don Necrop­o­lis rail­way sta­tion

Psychology/Biology

• “Self­-Su­per­vised Nat­ural Image Recon­struc­tion and Rich Seman­tic Clas­si­fi­ca­tion from Brain Activ­ity”, Gaziv et al 2020 (to­wards brain imi­ta­tion learn­ing)
• “Catch­ing Cow­pox: The Early Spread of Small­pox Vac­ci­na­tion, 1798–1810”, Rus­nock 2009
• “Cre­ativ­ity and Intel­li­gence: An Inves­ti­ga­tion of the Thresh­old Hypoth­e­sis”, Weiss et al 2020
• Dis­cov­ery of more water on Mars and first extra-galac­tic planet
• The Col­lyer Broth­ers; The Yuba County Five

Technology

• “Unrav­el­ing The JPEG”, Omar She­hata 2019 (ex­plorable: edit­ing JPEGs to under­stand their encod­ing)
• “The Recur­sive Uni­verse”, Amanda Ghas­saei (how to imple­ment Con­way’s Game of Life inside itself as a sin­gle giant ‘metapixel’ pat­tern)
• “The First Roman Fonts”, John Board­ley
• Nau­machia

Economics

• “Salary Nego­ti­a­tion: Make More Mon­ey, Be More Val­ued”, patio11 (pro­gram­mers: ask for more mon­ey!)
• “A Deep Dive into K-pop”, Dormin
• “Wikipedia Mat­ters”, Hin­nosaar et al 2019 (ran­dom­iza­tion of edits to for­eign lan­guages’ Wikipedia entries about an obscure Span­ish city increases tourism to the city from those for­eign coun­tries by 9%)

Miscellaneous

• 1955 Le Mans dis­as­ter (the past is a Third World coun­try)