AI

• “Re­BeL: Com­bin­ing Deep Re­in­force­ment Learn­ing and Search for Im­per­fec­t-In­for­ma­tion Games”, Brown et al 2020 (head­s-up no-limit Texas hold’em poker close to solved with a gen­er­al­iz­able Al­p­haZe­ro-like ap­proach?)
• “As­sess­ing Game Bal­ance with Al­p­haZe­ro: Ex­plor­ing Al­ter­na­tive Rule Sets in Chess”, Tomašev et al 2020
• “A dig­i­tal bio­marker of di­a­betes from smart­phone-based vas­cu­lar sig­nals”, Avram et al 2020 (time-series of blood pulses can di­ag­nose di­a­betes? weird, but not as weird as pre­dict­ing gen­der from reti­nas)
• “Tasks, sta­bil­i­ty, ar­chi­tec­ture, and com­pute: Train­ing more effec­tive learned op­ti­miz­ers, and us­ing them to train them­selves”, Metz et al 2020; “Gra­di­ent De­scent: The Ul­ti­mate Op­ti­mizer”, Chan­dra et al 2019 (deep­er—we need to go deep­er… now scales to VGG-Net)
• “X-LXMERT/X-UNITER: Paint, Cap­tion and An­swer Ques­tions with Mul­ti­-Modal Trans­form­ers”, Cho et al 2020 (on­line demo; fi­nal­ly, some new work on text → im­age)
• “StyleGAN net­work blend­ing”, Justin Pinkney/“Res­o­lu­tion De­pen­dent GAN In­ter­po­la­tion for Con­trol­lable Im­age Syn­the­sis Be­tween Do­mains”, Pinkney & Adler 2020 (in­ter­po­lat­ing be­tween mod­els to trans­fer im­ages be­tween do­mains, eg FFHQ ↔︎ foxes ↔︎ anime ↔︎ MLP ↔︎ fur­ries)
• “Op­ti­mal Peanut But­ter and Ba­nana Sand­wiches”, Ethan Rosen­thal (bin pack­ing ba­nana slices)

Mat­ters Of Scale:

• “Deep­Speed: Ex­treme-s­cale model train­ing for every­one” (demon­strates train­ing of GPT-3-180b & 1t-pa­ra­me­ter mod­els (“The tril­lion-pa­ra­me­ter model has 298 lay­ers of Trans­form­ers with a hid­den di­men­sion of 17,408 and is trained with se­quence length 2,048 and batch size 2,048”), w/open-source code; able to use CPU+GPU RAM si­mul­ta­ne­ously for 13b-pa­ra­me­ter mod­els on each node per Pudipeddi et al 2020; sparse at­ten­tion for sav­ing RAM; ap­prox­i­mated Adam gra­di­ents for sav­ing band­width)
• “GPT-f: Gen­er­a­tive Lan­guage Mod­el­ing for Au­to­mated The­o­rem Prov­ing”, Polu & Sutskever 2020b (GPT-2 for Meta­math scales & can boot­strap its the­o­rem-prov­ing abil­i­ty—on­ward to IMO!)
• “Mea­sur­ing Mas­sive Mul­ti­task Lan­guage Un­der­stand­ing”, Hendrycks et al 2020 (in­tro­duc­ing new much harder bench­marks to stress-test mod­el­s—n­ev­er­the­less, big­ger=­bet­ter: GPT-3 model size vs Q&A, with GPT-3-173b hit­ting al­most 50% out of 4-choice ques­tions vs base­line mod­els like T5 at the ran­dom base­line of only 25%)
• “Dy­nam­icEm­bed­ding: Ex­tend­ing Ten­sor­Flow for Colos­sal-S­cale Ap­pli­ca­tions”, Zeng et al 2020 (em­bed­dings-as-a-ser­vice Google-wide; tool AIs want to be agent AIs…)
• “A Time Leap Chal­lenge for SAT Solv­ing”, Fichte et al 2020 (hard­ware over­hang: ~2.5× per­for­mance in­crease in SAT solv­ing since 2000, about equally due to soft­ware & hard­ware gains, al­though slightly more soft­ware—new soft­ware on old hard­ware beats old on new. See pre­vi­ously Grace 2013.)
• “Effi­cient Trans­form­ers: A Sur­vey”, Tay et al 2020 (more on effi­cient at­ten­tion ap­proach­es)

Genetics

Every­thing Is Her­i­ta­ble:

• “A gen­eral di­men­sion of ge­netic shar­ing across di­verse cog­ni­tive traits in­ferred from mol­e­c­u­lar data”, de la Fuente et al 2020
• “Map­ping ge­nomic loci pri­ori­tises genes and im­pli­cates synap­tic bi­ol­ogy in schiz­o­phre­nia”, PGC et al 2020 (PGC SCZ3: 248 genome-wide hits; 7.7% PGS; OR = 44 com­par­ing top vs bot­tom per­centile risk); “Ex­ome se­quenc­ing iden­ti­fies rare cod­ing vari­ants in 10 genes which con­fer sub­stan­tial risk for schiz­o­phre­nia”, Singh et al 2020 (fig­ure 6a is a strik­ing demon­stra­tion of neg­a­tive/pu­ri­fy­ing se­lec­tion: more harm­ful mu­ta­tions are rar­er)
• “Genome-wide as­so­ci­a­tion study of over 40,000 bipo­lar dis­or­der cases pro­vides novel bi­o­log­i­cal in­sights”, Mullins et al 2020 (PGC3 BPD; PGS: 4.75%)
• “Ge­netic For­tune: Win­ning or Los­ing Ed­u­ca­tion, In­come, and Health”, Kweon et al 2020 (with­in-fam­ily ed­u­ca­tion/in­come/health PGSes)
• “GWAS of Over 427,000 In­di­vid­u­als Es­tab­lishes GABAergic and Synap­tic Mol­e­c­u­lar Path­ways as Key for Cog­ni­tive Ex­ec­u­tive Func­tions”, Ha­toum et al 2020
• “Shared her­i­tabil­ity of face and brain shape dis­tinct from cog­ni­tive traits”, Naqvi et al 2020 (set­ting lim­its to fa­cial pre­dic­tion of some phe­no­types)

Re­cent Evo­lu­tion:

• “Rapid Evo­lu­tion of Plas­tic-de­grad­ing En­zymes Preva­lent in the Global Ocean”, Alam et al 2020

En­gi­neer­ing:

• The first suc­cess­ful cloning of the en­dan­gered Prze­wal­ski’s horse
• “Hu­man Em­bryo Gene Edit­ing Gets a Road Map—Not a Green Light” (Na­tional Acad­emy of Sci­ence’s Her­i­ta­ble Hu­man Genome Edit­ing 2020 re­port)
• “CRISPR-engineered hu­man brown-like adipocytes pre­vent di­et-in­duced obe­sity and ame­lio­rate meta­bolic syn­drome in mice”, Wang et al 2020; “CRISPR-enhanced hu­man adipocyte ‘brown­ing’ as cell ther­apy for meta­bolic dis­ease”, Tsagkaraki et al 2020

Statistics/Meta-Science/Mathematics

• “Blue­berry Earth”, Sand­berg 2018

Politics/Religion

• “Amer­ica in De­cay: The Sources of Po­lit­i­cal Dys­func­tion”, Fukuyama 2014 (see al­so: con­sti­tu­tional hard­ball/“The Empty Cham­ber”)
• “‘Dwarf Pride’ Was Hard Won. Will a Growth Drug Un­der­mine It? An ex­per­i­men­tal med­ica­tion that in­creases height in chil­dren with the most com­mon form of dwarfism has raised hope that it can help them lead eas­ier lives. But some say the con­di­tion is not a prob­lem in need of a cure.” (Savari­rayan et al 2020; amus­ing triv­ia: the rea­son why cross­ing any dog breed with Cor­gis gives you a Corgi­fied ver­sion of that dog breed is be­cause Cor­gis are ho­mozy­gous dom­i­nant on an FGF4 mu­ta­tion caus­ing sim­i­lar achon­dropla­sia as in hu­mans)
• “Laws of Xmas”, Akiva & Ilene Miller 1998
• Gru­inard Is­land
• Lon­don Necrop­o­lis rail­way sta­tion

Psychology/Biology

• “Self­-Su­per­vised Nat­ural Im­age Re­con­struc­tion and Rich Se­man­tic Clas­si­fi­ca­tion from Brain Ac­tiv­ity”, Gaziv et al 2020 (to­wards brain im­i­ta­tion learn­ing)
• “Catch­ing Cow­pox: The Early Spread of Small­pox Vac­ci­na­tion, 1798–1810”, Rus­nock 2009
• “Cre­ativ­ity and In­tel­li­gence: An In­ves­ti­ga­tion of the Thresh­old Hy­poth­e­sis”, Weiss et al 2020
• Dis­cov­ery of more wa­ter on Mars and first ex­tra-galac­tic planet
• The Col­lyer Broth­ers; The Yuba County Five

Technology

• “Un­rav­el­ing The JPEG”, Omar She­hata 2019 (ex­plorable: edit­ing JPEGs to un­der­stand their en­cod­ing)
• “The Re­cur­sive Uni­verse”, Amanda Ghas­saei (how to im­ple­ment Con­way’s Game of Life in­side it­self as a sin­gle gi­ant ‘metapixel’ pat­tern)
• “The First Ro­man Fonts”, John Board­ley
• Nau­machia

Economics

• “Salary Ne­go­ti­a­tion: Make More Mon­ey, Be More Val­ued”, pa­tio11 (pro­gram­mers: ask for more mon­ey!)
• “A Deep Dive into K-pop”, Dormin
• “Wikipedia Mat­ters”, Hin­nosaar et al 2019 (ran­dom­iza­tion of ed­its to for­eign lan­guages’ Wikipedia en­tries about an ob­scure Span­ish city in­creases tourism to the city from those for­eign coun­tries by 9%)

Miscellaneous

• 1955 Le Mans dis­as­ter (the past is a Third World coun­try)