2017 News

Annual summary of 2017 gwern.net newsletters, selecting my best writings, the best 2017 links by topic, and the best books/movies/anime I saw in 2017.
2017-11-302021-01-04 finished certainty: log importance: 0

This is the 2017 sum­mary edi­tion of the Gw­ern.net newslet­ter (), sum­ma­riz­ing the best of the monthly 2017 newslet­ters:

Pre­vi­ous an­nual newslet­ters: , .



Site traffic (July 2017-Jan­u­ary 2018) was up: 326,852 page-views by 155,532 unique users.




AI: as I hoped in 2016, 2017 saw a re-e­mer­gence of mod­el-based RL with var­i­ous deep ap­proaches to learn­ing rea­son­ing, meta-RL, and en­vi­ron­ment mod­els. Us­ing re­la­tional log­ics and do­ing plan­ning over in­ter­nal mod­els and zero/few-shot learn­ing are no longer things “deep learn­ing can’t do”. My se­lec­tion for the sin­gle biggest break­through of the year was when Al­phaGo racked up a sec­ond ma­jor in­tel­lec­tual vic­tory with the demon­stra­tion by Zero that us­ing a sim­ple ex­pert it­er­a­tion al­go­rithm (with MCTS as the ex­pert) does not only solve the long-s­tand­ing prob­lem of NN self­-play be­ing wildly un­sta­ble (dat­ing back to at­tempts to failed at­tempts to ex­tend TD-Gam­mon to non-backgam­mon do­mains in the 1990s), but also al­lows su­pe­rior learn­ing to the com­pli­cated hu­man-ini­tial­ized AGs—in both wall­clock time & end strength, which is deeply hum­bling. 2000 years of study and tens of mil­lions of ac­tive play­ers, and that’s all it takes to sur­pass the best hu­man Go play­ers ever in the sup­pos­edly uniquely hu­man do­main of sub­tle global pat­tern recog­ni­tion. Not to men­tion chess. (, .) Ex­pert it­er­a­tion is an in­trigu­ingly gen­eral and un­der­used de­sign pat­tern, which I think may prove use­ful, es­pe­cially if peo­ple can re­mem­ber that it is not lim­ited to two-player games but is a gen­eral method for solv­ing any MDP. The sec­ond most no­table would be GAN work: Wasser­stein GAN losses () con­sid­er­ably ame­lio­rated the in­sta­bil­ity is­sues when us­ing GANs with var­i­ous ar­chi­tec­tures, and al­though WGANs can still di­verge or fail to learn, they are not so much of a black art as the orig­i­nal DCGANs tended to be. This prob­a­bly helped with later GAN work in 2017, such as the in­ven­tion of the CycleGAN ar­chi­tec­ture (Zhu et al 2017) which ac­com­plishes mag­i­cal & bizarre kinds of learn­ing such as learn­ing, us­ing horse and ze­bra im­ages, to turn an ar­bi­trary horse im­age into a ze­bra & vice-ver­sa, or your face into a car or a bowl of ra­men soup. “Who or­dered that?” I did­n’t, but it’s de­li­cious & hi­lar­i­ous any­way, and sug­gests that GANs re­ally will be im­por­tant in un­su­per­vised learn­ing be­cause they ap­pear to be learn­ing a lot about their do­mains. Ad­di­tional demon­stra­tions like be­ing able to trans­late be­tween hu­man lan­guages given only monolin­gual cor­puses merely em­pha­size that lurk­ing pow­er—I still feel that CycleGAN should not work, much less high­-qual­ity neural trans­la­tion with­out any trans­la­tion pairs, but it does. The path to larg­er-s­cale pho­to­re­al­is­tic GANs was dis­cov­ered by Nvidi­a’s ProGAN pa­per (): es­sen­tially StackGAN’s ap­proach of lay­er­ing sev­eral GANs trained in­cre­men­tally as up­scalers does work (as I ex­pect­ed), but you need much more GPU-compute to reach 1024x1024-size pho­tos and it helps if each new up­scal­ing GAN is only grad­u­ally blended in to avoid the ran­dom ini­tial­iza­tion de­stroy­ing every­thing pre­vi­ously learned (anal­o­gous to trans­fer learn­ing need­ing low learn­ing rates or to freeze lay­er­s). Time will tell if the ProGAN ap­proach is a one-trick pony for GANs lim­ited to pho­tos. Fi­nal­ly, GANs started turn­ing up as use­ful com­po­nents in semi­-su­per­vised learn­ing in the GAIL par­a­digm () for deep RL ro­bot­ics. I ex­pect GANs are still a while off from be­ing pro­duc­tized or truly crit­i­cal for any­thing—they re­main a so­lu­tion in search of a prob­lem, but less so than I com­mented last year. In­deed, from Al­phaGo to GANs, 2017 was the year of deep RL (sub­red­dit traffic oc­tu­pled). Pa­pers tum­bled out con­stant­ly, ac­com­pa­nied by am­bi­tious com­mer­cial moves: Jeff Dean laid out a vi­sion for us­ing NNs/deep RL es­sen­tially every­where in­side Google’s soft­ware stack, Google be­gan full self­-driv­ing ser­vices in Phoenix, while noted re­searchers like Pieter Abbeel founded ro­bot­ics star­tups bet­ting that deep RL has fi­nally cracked im­i­ta­tion & few-shot learn­ing. I can only briefly high­light, in deep RL, con­tin­ued work on meta-RL & neural net ar­chi­tec­ture search with fast weights, re­la­tional rea­son­ing & logic mod­ules, zero/few-shot learn­ing, deep en­vi­ron­ment mod­els (crit­i­cal for plan­ning), and ro­bot progress in sam­ple efficiency/im­i­ta­tion learn­ing/mod­el-based & off-pol­icy learn­ing, in ad­di­tion to the in­te­gra­tion of GANs a la GAIL. What will hap­pen if every year from now on sees as much progress in deep re­in­force­ment learn­ing as we saw in 2017? (Sup­pose deep learn­ing ul­ti­mately does lead to a Sin­gu­lar­i­ty; how would it look any differ­ent than it does now?) One thing miss­ing from 2017 for me was use of very large NNs us­ing ex­pert mix­tures, syn­thetic gra­di­ents, or other tech­niques; in ret­ro­spect, this may re­flect hard­ware lim­i­ta­tions as non-Googlers in­creas­ingly hit the lim­its of what can be it­er­ated on rea­son­ably quickly us­ing just 1080tis or P100s. So I am in­trigued by the in­creas­ing avail­abil­ity of Google’s sec­ond-gen­er­a­tion TPUs (which can do train­ing) and by dis­cus­sions of mul­ti­ple ma­tur­ing NN ac­cel­er­a­tor star­tups which might break Nvidi­a’s costly mo­nop­oly and offer 100s of ter­aflops or petaflops at non-A­m­a­Goog­Book­Soft researcher/hobbyist bud­gets.

Ge­net­ics in 2017 was a straight-line con­tin­u­a­tion of 2016: the UKBB dataset came on­line and is fully armed & op­er­a­tional, with ex­omes now fol­low­ing (and whole-genomes soon), re­sult­ing in the typ­i­cal flur­ries of pa­pers on every­thing which is her­i­ta­ble (which is every­thing). Ge­netic en­gi­neer­ing had a ban­ner year be­tween CRISPR and older meth­ods in the pipeline—it seemed like every week there was a new mouse or hu­man trial cur­ing some­thing or oth­er, to the point where I lost track and the NYT has be­gun re­port­ing on clin­i­cal tri­als be­ing de­layed by lack of virus man­u­fac­tur­ing ca­pac­i­ty. (A good prob­lem to have!) Genome syn­the­sis con­tin­ues to greatly con­cern me but noth­ing news­wor­thy hap­pened in 2017 other than, pre­sum­ably, con­tin­u­ing to get cheaper on sched­ule. In­tel­li­gence re­search did not de­liver any par­tic­u­larly amaz­ing re­sults as the SSGAC pa­per has ap­par­ently been de­layed to 2018 (with a glimpse in ), but we saw two crit­i­cal method­olog­i­cal im­prove­ments which I ex­pect to yield fruit in 2017–2018: first, as ge­netic cor­re­la­tion re­searchers have noted for years, ge­netic cor­re­la­tions should be able to boost power con­sid­er­ably by cor­rect­ing for mea­sure­ment er­ror & in­creas­ing effec­tive sam­ple size by ap­pro­pri­ate com­bi­na­tion of poly­genic scores, and MTAG demon­strates this works well for in­tel­li­gence ( in­creases PGS to ~7% & to ~10%); sec­ond, Hsu’s lasso pre­dic­tions were proven true by demon­strat­ing the cre­ation of a poly­genic score ex­plain­ing most SNP heritability/predicting 40% of height vari­ance. The use of these two si­mul­ta­ne­ously with SSGAC & other datasets ought to boost IQ PGSes to >10% and pos­si­bly much more. Per­haps the most no­table sin­gle de­vel­op­ment was the res­o­lu­tion of the long-s­tand­ing dys­gen­ics ques­tion us­ing mol­e­c­u­lar ge­net­ics: has the de­mo­graphic tran­si­tion in at least some West­ern coun­tries led to de­creases in the ge­netic po­ten­tial for in­tel­li­gence (mean poly­genic score), as sug­gested by most but not all phe­no­typic analy­ses of intelligence/education/fertility? Yes, in Iceland/USA/UK, dys­gen­ics has in­deed done that on a mean­ing­ful scale, as shown by straight­for­ward cal­cu­la­tions of mean poly­genic score by birth decade & ge­netic cor­re­la­tions. More in­ter­est­ing­ly, the in­creas­ing avail­abil­ity of an­cient DNA al­lows for pre­lim­i­nary analy­ses of how poly­genic scores change over time: over tens of thou­sands of years, hu­man in­tel­li­gence & dis­ease traits ap­pear to have been slowly se­lected against (con­sis­tent with most ge­netic vari­ants be­ing harm­ful & un­der pu­ri­fy­ing se­lec­tion) but that trend re­versed at some point rel­a­tively re­cent.

For 2016, I noted that the main story of VR was that it had­n’t failed & was mod­estly suc­cess­ful; 2017 saw the con­tin­u­a­tion of this trend as it climbs into its “trough of pro­duc­tiv­ity”—the me­dia hype has popped and for 2017, VR just kept suc­ceed­ing and build­ing up an in­creas­ingly large li­brary of games & ap­pli­ca­tions, while the price con­tin­ued to drop dra­mat­i­cally (as every­one should have re­al­ized but did­n’t) with the Ocu­lus now ~$300. So much for “mo­tion sick­ness will kill VR again” or “VR is too ex­pen­sive for gamers”. Per­haps the ma­jor sur­prise for me was that Sony’s quiet & non­com­mit­tal ap­proach to its head­set (which made me won­der if it would be launched at all) masked a huge suc­cess, as PSVR has sold into the mil­lions of units and is prob­a­bly the most pop­u­lar ‘real’ VR so­lu­tion de­spite its tech­ni­cal draw­backs com­pared to Vive/Oculus. There con­tin­ues to be no killer app, but the many up­com­ing hard­ware im­prove­ments like 4K dis­plays or wire­less head­sets or eye­track­ing+­foveat­ed-ren­der­ing will con­tinue in­creas­ing qual­ity while prices drop and li­braries con­tinue to build up; if there is any nat­ural limit to the VR mar­ket, I haven’t seen any sign of it yet. So for 2018–2019, I won­der if VR will sim­ply con­tinue to grow grad­u­ally with mo­bile smart­phone VR so­lu­tions eat­ing the lunch of full head­sets, or if there will be a break­out mo­ment where the price, qual­i­ty, li­brary, and a killer app hit a crit­i­cal com­bi­na­tion?

Bit­coin un­der­went one of its pe­ri­odic ‘bub­bles’, com­plete with the clas­sic ac­cu­sa­tions that this time Bit­coin will surely go to ze­ro, the fee spikes mean Bit­coin will never scale (“no­body goes there any­more, it’s too pop­u­lar”), peo­ple can’t use it to pay for any­thing, it’s a clear scam be­cause of var­i­ous peo­ples’ fool­ish­ness like tak­ing out mort­gages to gam­ble on fur­ther in­creas­es, Coin­base is run by fools & knaves, ran­dom other alt­coins have bub­bled too & will doubt­less re­place Bit­coin soon, Bit­coin has failed to achieve any lib­er­tar­ian goals and is now a play­thing of the rich, peo­ple who were wrong about Bit­coin every time from $1$12011 in 2011 to now will claim to be right moral­ly, the PoW se­cu­rity is waste­ful, etc—one could copy­-paste most ar­ti­cles or com­ments from the last bub­ble (or the one be­fore that, or be­fore that) into this one with no change other than the num­bers. As such, while I have ben­e­fited from it, there is lit­tle worth say­ing about it other than to note its ex­is­tence with be­muse­ment, and re­flect on how far Bit­coin & cryp­tocur­ren­cies have come since I first be­gan us­ing them in 2011: Even if Bit­coin goes to zero now, it’s un­leashed an in­cred­i­ble Cam­brian ex­plo­sion of cryp­tog­ra­phy ap­pli­ca­tions and eco­nom­ics crossovers. Cryp­toe­con­o­mists are go­ing to spend decades di­gest­ing proof-of-work, proof-of-s­take, slash­ing, Truthcoin/HiveMind/Augur, zk-SNARKs and zk-STARKs, Mim­blewim­ble, TrueBit, script­less scripts & other ap­pli­ca­tions of Schnorr sig­na­tures, Tur­ing-com­plete con­tracts, ob­served cryp­tomar­kets like the DNMs… You can go through and each sec­tion cor­re­sponds to a project made pos­si­ble only via Bit­coin’s in­flu­ence. Bit­coin had more in­flu­ence in its first 5 years than Chaum’s dig­i­tal cash has had in 30 years. Cryp­tog­ra­phy will never be the same. The fu­ture’s so bright I gotta wear mir­ror­shades.

A short note on pol­i­tics: Don­ald Trump’s pres­i­dency and its back­lash in the form of Gi­rar­dian scape­goat­ing (sex­ual ha­rass­ment scan­dals & so­cial-me­dia purges) have re­ceived truly dis­pro­por­tion­ate cov­er­age and have be­come al­most an ad­dic­tion. They have dis­tracted from im­por­tant is­sues and from im­por­tant facts like 2017 be­ing one of the best years in hu­man his­to­ry, many sci­en­tific & tech­no­log­i­cal im­prove­ments and break­throughs like ge­netic en­gi­neer­ing or AI, or global & US eco­nomic growth. Ob­jec­tive­ly, Trump’s first year has been largely a non-event; a few things were ac­com­plished like pack­ing fed­eral courts and a bizarre tax bill, but over­all not much hap­pened, and Trump has not lived up to the apoc­a­lyp­tic pre­dic­tions & hys­te­ria. If the next 3 years are sim­i­lar to 2017, one would have to ad­mit that Trump as pres­i­dent turned out bet­ter than George W. Bush!



  1. Se­lected Non-Fic­tions, (July se­lec­tion)
  2. Site Re­li­a­bil­ity En­gi­neer­ing: How Google Runs Pro­duc­tion Sys­tems (Jan­u­ary re­view)
  3. The Play­boy In­ter­view: Vol­ume II
  4. Ar­ti­fi­cial Life, Levy
  5. Pos­si­ble Worlds and Other Es­says, 1927
  6. An­nual Berk­shire Hath­away let­ters of
  7. The Grand Strat­egy of the Ro­man Em­pire: From the First Cen­tury CE to the Third, Luttwak 2016
  8. Moon Dust: In Search of the Men Who Fell to Earth, Smith 2005


  1. , Scott Alexan­der
  2. , Steven D. Carter
  3. , (Jan­u­ary re­view)
  4. Sun­set in a Spi­der­web: Sijo Po­etry of An­cient Ko­rea, Baron & Kim 1974


Non­fic­tion movies:

  1. (2015; June re­view)
  2. (No­vem­ber re­view)


  1. (May re­view)
  2. (Oc­to­ber re­view)
  3. (1995; Jan­u­ary re­view)
  4. (Feb­ru­ary re­view)
  5. (June re­view)
  6. (March re­view)


  1. (2013; re­view)
  2. (2016; Jan­u­ary re­view)
  3. (May re­view)