Meta-Learning

Learn­ing to learn. In May 2020, OA re­leased—to re­mark­ably lit­tle in­ter­est from re­searchers, no blog post, no me­dia blitz, and lit­tle pub­lic dis­cus­sion be­yond the snidely dis­mis­sive—the long-awaited fol­lowup to GPT-2, one model to rule them all: a 117× larger 175b-pa­ra­me­ter model with far more pow­er­ful lan­guage gen­er­a­tion, which lets it solve a wide va­ri­ety of prob­lems from arith­metic¹ to Eng­lish trans­la­tion to un­scram­bling ana­grams to SAT analo­gies—purely from be­ing prompted with text ex­am­ples, with­out any spe­cial­ized train­ing or fine­tun­ing what­so­ev­er, merely nex­t-word pre­dic­tion train­ing on a big In­ter­net text cor­pus. This im­plies GPT-3’s at­ten­tion mech­a­nisms serve as “fast weights” that have “learned to learn” by train­ing on suffi­ciently var­ied data², forc­ing it to do more than just learn or­di­nary tex­tual re­la­tion­ships. Like Ope­nAI’s Juke­box just weeks ago (it­self a re­mark­able demon­stra­tion of scal­ing in syn­the­siz­ing raw au­dio mu­sic com­plete with re­mark­ably re­al­is­tic voices/instruments), the an­nounce­ment of GPT-3 ap­pears to have sunk al­most with­out a trace, so I will go into more depth than usu­al.

Flexing GPT

“At­tacks only get bet­ter.” 2 years ago, GPT-1 was in­ter­est­ingly use­ful pre­train­ing and adorable with its “sen­ti­ment neu­ron”. 1 year ago, GPT-2 was im­pres­sive with its ex­cel­lent text gen­er­a­tion & fine­tun­ing ca­pa­bil­i­ties. This year, GPT-3 is scary be­cause it’s a mag­nifi­cently ob­so­lete ar­chi­tec­ture from early 2018 (used mostly for soft­ware en­gi­neer­ing con­ve­nience as the in­fra­struc­ture has been de­bugged), which is small & shal­low com­pared to what’s pos­si­ble³⁴, with a sim­ple uni­form ar­chi­tec­ture⁵ trained in the dumb­est way pos­si­ble (u­ni­di­rec­tional pre­dic­tion of next text to­ken) on a sin­gle im­pov­er­ished modal­ity (ran­dom In­ter­net HTML text dumps⁶) on tiny data (fits on a lap­top), sam­pled in a dumb way⁷, its bench­mark per­for­mance sab­o­taged by bad prompts & data en­cod­ing prob­lems (e­spe­cially arith­metic & com­mon­sense rea­son­ing), and yet, the first ver­sion al­ready man­i­fests crazy run­time meta-learn­ing—and the scal­ing curves still are not bend­ing! The sam­ples are also bet­ter than ever, whether it’s GPT-3 in­vent­ing new pe­nis jokes⁸ or writ­ing (mostly work­ing) JavaScript tu­to­ri­als about ro­tat­ing ar­rays.

It’s odd that this qual­i­ta­tive leap ap­pears to be largely missed by the stan­dard NLP bench­marks. Noth­ing in the raw met­rics re­ported on, say, Penn Tree Bank or LAMBADA or Wino­Grande would lead you to ex­pect all of this hi­lar­i­ous and cre­ative out­put; the meta-learn­ing re­sults might, but only if you thought meta-learn­ing was im­por­tant. This sug­gests to me that a use­ful post-GPT-3 con­tri­bu­tion would be fig­ur­ing out how to bench­mark these sorts of flex­i­ble text gen­er­a­tion ca­pa­bil­i­ties (pos­si­bly some­thing along the lines of Chol­let’s im­age-based Ab­strac­tion and Rea­son­ing Cor­pus (ARC)).

Baking The Cake

Not the whole pic­ture, but a big part. Does it set SOTA on every task? No, of course not. But the ques­tion is not whether we can lawyerly find any way in which it might not work, but whether there is any way which it might work. And there are many ways it might work bet­ter (see the “Lim­i­ta­tions” sec­tion for just a few). Does GPT-3 do any­thing like steer a ro­bot around SF shoot­ing lasers and rock­ets at hu­mans⸮ No, of course not. It is ‘just’ a text pre­dic­tion mod­el, an id­iot sa­vant of text; but an id­iot sa­vant, we should re­mem­ber, is only a ge­netic mu­ta­tion or bit of brain dam­age away from a nor­mal hu­man. If RL is the cherry on the top of the su­per­vised learn­ing frost­ing, and su­per­vised learn­ing is the frost­ing on top of the un­su­per­vised learn­ing cake, well, it looks like the cake lay­ers are fi­nally ris­ing.

Scal­ing still work­ing. I was sur­prised, as I had ex­pected closer to 100b pa­ra­me­ters, and I thought that the per­for­mance of CTRL/Meena/Mega­tronLM/T5/Turing-NLG/GPipe sug­gested that, the scal­ing pa­pers⁹ notwith­stand­ing, the scal­ing curves had started to bend and by 100b, it might be hard to jus­tify fur­ther scal­ing. How­ev­er, in the lat­est ver­sion of “the un­rea­son­able effec­tive­ness of data”, GPT-3 hits twice that with­out no­tice­able change in scal­ing fac­tors: its scal­ing con­tin­ues to be roughly logarithmic/power-law, as it was for much smaller mod­els & as fore­cast, and it has not hit a regime where gains effec­tively halt or start to re­quire in­creases vastly be­yond fea­si­bil­i­ty. That sug­gests that it would be both pos­si­ble and use­ful to head to tril­lions of pa­ra­me­ters (which are still well within avail­able com­pute & bud­gets, re­quir­ing merely thou­sands of GPUs & per­haps $10–$100m bud­gets as­sum­ing no im­prove­ments which of course there will be, see Her­nan­dez & Brown 2020 etc in this is­sue), and eye­balling the graphs, many bench­marks like the Wino­grad schema Wino­Grande would fall by 10t pa­ra­me­ters. The pre­dictabil­ity of scal­ing is strik­ing, and makes scal­ing mod­els more like sta­tis­tics than AI. (AI is sta­tis­tics which does what we want it to but does­n’t work; and sta­tis­tics is AI which works but does­n’t do what we wan­t.)

An­ti-s­cal­ing: pen­ny-wise, pound-fool­ish. GPT-3 is an ex­tra­or­di­nar­ily ex­pen­sive model by the stan­dards of ma­chine learn­ing: it is es­ti­mated that train­ing it may re­quire the an­nual cost of more ma­chine learn­ing re­searchers than you can count on one hand (~$5m¹⁰), up to $30 of hard drive space to store the model (500–800G­B), and mul­ti­ple pen­nies of elec­tric­ity per 100 pages of out­put (0.4 kWH). Re­searchers are con­cerned about the prospects for scal­ing: can ML afford to run projects which cost more than 0.1 mil­li-Man­hat­tan-Pro­jects⸮¹¹ Surely it would be too ex­pen­sive, even if it rep­re­sented an­other large leap in AI ca­pa­bil­i­ties, to spend up to 10 mil­li-Man­hat­tan-Pro­jects to scale GPT-3 100× to a triv­ial thing like hu­man-like per­for­mance in many do­mains⸮ Many re­searchers feel that such a sug­ges­tion is ab­surd and re­futes the en­tire idea of scal­ing ma­chine learn­ing re­search fur­ther, and that the field would be more pro­duc­tive if it in­stead fo­cused on re­search which can be con­ducted by an im­pov­er­ished goatherder on an old lap­top run­ning off so­lar pan­els.¹² Nonethe­less, I think we can ex­pect fur­ther scal­ing. (10×? No, 10× is­n’t cool. You know what’s cool? 100–1000×, trained on a fancy new su­per­com­puter.)

Scaling

How far will scal­ing go? The scal­ing pa­pers sug­gest that the leaps we have seen over the past few years are not even half way there in terms of ab­solute like­li­hood loss, never mind what re­al-world ca­pa­bil­i­ties each ad­di­tional decre­ment trans­lates in­to. The scal­ing curves are clean; from Ka­plan et al 2020:

GPT-3 rep­re­sents ~10³ on this chart, leav­ing plenty of room for fur­ther loss de­creas­es—e­spe­cially given the un­cer­tainty in ex­trap­o­la­tion:

Lo and be­hold, the scal­ing laws con­tinue for GPT-3 mod­els for sev­eral or­ders past Ka­plan et al 2020; from Brown et al 2020:

If we see such strik­ing gains in halv­ing the val­i­da­tion loss but with so far left to go, what is left to emerge as we third or halve again? How far does this go, ex­act­ly? How do we pre­dict what emerges when? Bueller? Bueller? (See also Meena’s per­plex­ity vs hu­man-ness chat­bot rat­ings, GPT-3-written news ar­ti­cles’ prob­a­bil­ity of fool­ing hu­mans by pa­ra­me­ter count, and GPT-3 model size vs Q&A from Hendrycks et al 2020.)

Blessings Of Scale

“Ex­trap­o­lat­ing the spec­tac­u­lar per­for­mance of GPT-3 into the fu­ture sug­gests that the an­swer to life, the uni­verse and every­thing is just 4.398 tril­lion pa­ra­me­ters.”

Ge­off Hin­ton

We don’t know how to train NNs. The bless­ings of scale is the ob­ser­va­tion that for deep learn­ing, hard prob­lems are eas­ier to solve than easy prob­lem­s—ev­ery­thing gets bet­ter as it gets larger (in con­trast to the usual out­come in re­search, where small things are hard and large things im­pos­si­ble). The big­ger the neural net/compute/data/problem, the faster it learns, the bet­ter it learns, the sta­bler it learns, and so on. A prob­lem we can’t solve at all at small n may sud­denly be­come straight­for­ward with mil­lions or bil­lions of n. “NNs are lazy”: they can do far more than we make them do when we push them be­yond easy an­swers & cheap short­cuts. The bit­ter les­son is the harder and big­ger, the bet­ter. (Be­sides GPT-3, one could men­tion re­cent progress in semi­-su­per­vised learn­ing & the mod­el-based DRL re­nais­sance.)

Bless­ings of scale: sta­bil­ity → gen­er­al­iza­tion → meta-learn­ing. GPT-3 is ham­strung by its train­ing & data, but DL en­joys an un­rea­son­ably effec­tive bless­ing of di­men­sion­al­ity—just sim­ply train­ing a big model on a lot of data in­duces bet­ter prop­er­ties like meta-learn­ing with­out even the slight­est bit of that ar­chi­tec­ture be­ing built in; and in gen­er­al, train­ing on more and harder tasks cre­ates ever more hu­man-like per­for­mance, gen­er­al­iza­tion, and ro­bust­ness. The GPT nat­u­ral-lan­guage & pro­gram­ming lan­guage mod­els, iGPT/Vi­sion Trans­former for im­ages (and to some de­gree GPT-f), show that sim­ply scal­ing up mod­els & datasets with­out any su­per­vi­sion pro­duces re­sults com­pet­i­tive with the best (and most com­plex) al­ter­na­tives, us­ing the same sim­ple ar­chi­tec­ture, grad­u­ally pass­ing from su­per­fi­cial sur­face cor­re­la­tions to more hu­man-like brain ac­tiv­ity (Schrimpf et al 2020) and lin­guis­tic bi­ases as data in­creases (eg Warstadt et al 2020) OA5 does not just scale to, but sta­bi­lizes at, mini­batches of mil­lions due to gra­di­ent noise. OA5-like, BigGAN sta­bi­lizes at large-s­cale im­age datasets like JFT-300M & ben­e­fits from un­usu­ally large mini­batches and VAEs (long an al­so-ran to GANs or au­tore­gres­sive mod­els in terms of sharp im­age gen­er­a­tion) catch up if you make them very deep (Child 2020, Vah­dat & Kautz 2020); while clas­si­fier CNNs like BiT¹³/Do­jo­longa et al 2020 or ResNeXt or Noisy Stu­dent trans­fer & ro­bus­tify with hu­man-like er­rors¹⁴, mul­ti­modal learn­ing pro­duces bet­ter rep­re­sen­ta­tions on fewer data (eg ViLBERT/VideoBERT, mo­ti­vat­ing OA’s in­ter­est in big mul­ti­modal mod­els), and RNNs can pre­dict videos. Al­phaS­tar reaches hu­man-level with hun­dreds of com­pet­ing self­-play­ers to cover pos­si­ble strate­gies. Im­i­ta­tion learn­ing DRL like MetaMimic gen­er­al­izes at hun­dreds of tasks to train a deep net. Dis­en­tan­gle­ment emerges in StyleGAN with suffi­ciently deep w em­bed­dings, with enough pa­ra­me­ters to train raw au­dio in the afore­men­tioned Juke­box, or in re­la­tional net­works/GQN/Trans­form­ers with enough sam­ples to force fac­tor­iza­tion. (See also Hill et al 2019/Chap­lot et al 2017/Yu et al 2018/Lake 2019/In­ter­ac­tive Agents Group 2020.) Train­ing Dactyl on mil­lions of do­main ran­dom­iza­tions in­duced sim­i­lar im­plicit meta-learn­ing where dur­ing each run­time in­vo­ca­tion, the RNN probes its en­vi­ron­ment and en­codes its un­der­stand­ing of ro­bot hand con­trol into its hid­den state; and DD-PPO out­per­forms clas­si­cal ro­bot plan­ners by scal­ing 2 or­ders. Or in Proc­gen or Coin­Run, train­ing on hun­dreds of lev­els trains agents to solve lev­els in­di­vid­u­al­ly, but at thou­sands of lev­els, they be­gin to gen­er­al­ize to un­seen lev­els. Al­p­haZero demon­strated truly su­per­hu­man Go with­out ‘delu­sions’ just by train­ing a big­ger model on a richer sig­nal & pro-level play with­out any search—and MuZero, for that mat­ter, demon­strated that just train­ing an RNN end-to-end to pre­dict a re­ward on enough data is enough to ob­so­lete even Al­p­haZero and learn tree search im­plic­itly (but bet­ter). And on and on. DM re­searcher Matthew Botvinick, dis­cussing their meta-re­in­force­ment learn­ing work where they were sur­prised to dis­cover meta-learn­ing emerg­ing, and that it did so re­gard­less of which spe­cific ar­chi­tec­ture they used:

“…it’s some­thing that just hap­pens. In a sense, you can’t avoid this hap­pen­ing. If you have a sys­tem that has mem­o­ry, and the func­tion of that mem­ory is shaped by re­in­force­ment learn­ing, and this sys­tem is trained on a se­ries of in­ter­re­lated tasks, this is go­ing to hap­pen. You can’t stop it.”

Pace Breiman, why? Why do they trans­fer and gen­er­al­ize? Why do these bless­ings of scale ex­ist? Why do we need to train large mod­els when small mod­els prov­ably ex­ist with the same per­for­mance? Why do larger mod­els not over­fit (though they can) and gen­er­al­ize bet­ter than smaller mod­els? What’s up with the whole ‘dou­ble de­scent’ any­way?

These are all, ahem, deep ques­tion about neural net­works and heav­ily de­bat­ed, but right now, I would sug­gest that the an­swer lies in some mix of the model compression/distillation, ‘lot­tery ticket hy­poth­e­sis’, Bayesian neural net­work, and learned rep­re­sen­ta­tion (like cir­cuits) lit­er­a­tures.

Big mod­els work be­cause they en­code a dizzy­ingly vast num­ber of sub­-mod­els in an ex­tremely high­-di­men­sional ab­stract space, rep­re­sent­ing count­less small sub­-mod­els (Orseau et al 2020) in­ter­po­lat­ing over data, one of which is likely to solve the prob­lem well, and so en­sures the prob­lem is sol­u­ble by the over­all mod­el. They func­tion as an en­sem­ble: even though there count­less over­fit sub­-mod­els in­side the sin­gle big mod­el, they all av­er­age out, lead­ing to a pref­er­ence for sim­ple so­lu­tions. This Oc­cam’s ra­zor bi­ases the model to­wards sim­ple so­lu­tions which are flex­i­ble enough to grad­u­ally ex­pand in com­plex­ity to match the da­ta.

How­ev­er, “neural nets are lazy”: sub­-mod­els which mem­o­rize pieces of the data, or latch onto su­per­fi­cial fea­tures, learn quick­est and are the eas­i­est to rep­re­sent in­ter­nal­ly. If the model & data & com­pute are not big or var­ied enough, the op­ti­miza­tion, by the end of the cur­sory train­ing, will have only led to a sub­-model which achieves a low loss but missed im­por­tant pieces of the de­sired so­lu­tion.

On the other hand, for a model like GPT-3, it is suffi­ciently pow­er­ful a model that its sub­-mod­els can do any­thing from po­etry to arith­metic, and it is trained on so much data that those su­per­fi­cial mod­els may do well early on, but grad­u­ally fall be­hind more ab­stract mod­els; a sub­-model which mem­o­rizes some of the data is in­deed much sim­pler than a sub­-model which en­codes gen­uine arith­metic (a NN can prob­a­bly mem­o­rize tens of thou­sands of lookup ta­ble en­tries stor­ing ex­am­ples of ad­di­tion in the space it would take to en­code an ab­stract al­go­rithm like ‘ad­di­tion’), but it can’t pos­si­bly mem­o­rize all the in­stances of arith­metic (im­plicit or ex­plic­it) in GPT-3’s In­ter­net-s­cale dataset. If a mem­o­riz­ing sub­-model tried to do so, it would be­come ex­tremely large and pe­nal­ized. Even­tu­al­ly, after enough ex­am­ples and enough up­dates, the sim­plest ‘arith­metic’ model which ac­cu­rately pre­dicts the data just is arith­metic. And then the meta-learn­ing, after see­ing enough in­stances of al­go­rithms which vary slightly within each sam­ple, mak­ing it hard to learn each task sep­a­rate­ly, just is learn­ing of more generic al­go­rithms, yield­ing sub­-mod­els which achieve lower loss than the ri­val sub­-mod­els, which ei­ther fail to pre­dict well or bloat un­ac­cept­ably. (GPT-2-1.5b ap­par­ently was too small or shal­low to en­sem­ble eas­ily over sub­-mod­els en­cod­ing meta-learn­ing al­go­rithms, or per­haps not trained long enough on enough data to lo­cate the meta-learner mod­els; GPT-3 was.)

So, the larger the mod­el, the bet­ter, if there is enough data & com­pute to push it past the easy con­ve­nient sub­-mod­els and into the sub­-mod­els which ex­press de­sir­able traits like gen­er­al­iz­ing, fac­tor­iz­ing per­cep­tion into mean­ing­ful la­tent di­men­sions, meta-learn­ing tasks based on de­scrip­tions, learn­ing causal rea­son­ing & log­ic, and so on. If the in­gre­di­ents are there, it’s go­ing to hap­pen.

Why Does Pretraining Work?

The pre­train­ing the­sis goes some­thing like this:

Hu­mans, one might say, are the cyanobac­te­ria of AI: we con­stantly emit large amounts of struc­tured data, which im­plic­itly rely on log­ic, causal­i­ty, ob­ject per­ma­nence, his­to­ry—all of that good stuff. All of that is im­plicit and en­coded into our writ­ings and videos and ‘data ex­haust’. A model learn­ing to pre­dict must learn to un­der­stand all of that to get the best per­for­mance; as it pre­dicts the easy things which are mere sta­tis­ti­cal pat­tern-match­ing, what’s left are the hard things.

Early on in train­ing, a model learns the crud­est lev­els: that some let­ters are more fre­quent than oth­ers, that every 5 char­ac­ters or so there is a space, and so on. It goes from pre­dicted uni­form­ly-dis­trib­uted bytes to what looks like Base-60 en­cod­ing—al­phanu­meric gib­ber­ish. As crude as this may be, it’s enough to make quite a bit of ab­solute pro­gress: a ran­dom pre­dic­tor needs 8 bits to ‘pre­dict’ a byte/character, but just by at least match­ing let­ter and space fre­quen­cies, it can al­most halve its er­ror to around 5 bits.¹⁵ Be­cause it is learn­ing so much from every char­ac­ter, and be­cause the learned fre­quen­cies are sim­ple, it can hap­pen so fast that if one is not log­ging sam­ples fre­quent­ly, one might not even ob­serve the im­prove­ment.

As train­ing pro­gress­es, the task be­comes more diffi­cult. Now it be­gins to learn what words ac­tu­ally ex­ist and do not ex­ist. It does­n’t know any­thing about mean­ing, but at least now when it’s asked to pre­dict the sec­ond half of a word, it can ac­tu­ally do that to some de­gree, sav­ing it a few more bits. This takes a while be­cause any spe­cific in­stance will show up only oc­ca­sion­al­ly: a word may not ap­pear in a dozen sam­ples, and there are many thou­sands of words to learn. With some more work, it has learned that punc­tu­a­tion, plu­ral­iza­tion, pos­ses­sives are all things that ex­ist. Put that to­geth­er, and it may have pro­gressed again, all the way down to 3–4 bits er­ror per char­ac­ter! (While the progress is grat­i­fy­ingly fast, it’s still all gib­ber­ish, though, makes no mis­take: a sam­ple may be spelled cor­rect­ly, but it does­n’t make even a bit of sense.)

But once a model has learned a good Eng­lish vo­cab­u­lary and cor­rect formatting/spelling, what’s next? There’s not much juice left in pre­dict­ing with­in-words. The next thing is pick­ing up as­so­ci­a­tions among words. What words tend to come first? What words ‘clus­ter’ and are often used nearby each oth­er? Nau­ti­cal terms tend to get used a lot with each other in sea sto­ries, and like­wise Bible pas­sages, or Amer­i­can his­tory Wikipedia ar­ti­cle, and so on. If the word “Jeffer­son” is the last word, then “Wash­ing­ton” may not be far away, and it should hedge its bets on pre­dict­ing that ‘W’ is the next char­ac­ter, and then if it shows up, go al­l-in on “ash­ing­ton”. Such bag-of-words ap­proaches still pre­dict bad­ly, but now we’re down to per­haps <3 bits per char­ac­ter.

What next? Does it stop there? Not if there is enough data and the ear­lier stuff like learn­ing Eng­lish vo­cab does­n’t hem the model in by us­ing up its learn­ing abil­i­ty. Grad­u­al­ly, other words like “Pres­i­dent” or “gen­eral” or “after” be­gin to show the model sub­tle cor­re­la­tions: “Jeffer­son was Pres­i­dent after…” With many such pas­sages, the word “after” be­gins to serve a use in pre­dict­ing the next word, and then the use can be broad­ened.

By this point, the loss is per­haps 2 bits: every ad­di­tional 0.1 bit de­crease comes at a steeper cost and takes more time. How­ev­er, now the sen­tences have started to make sense. A sen­tence like “Jeffer­son was Pres­i­dent after Wash­ing­ton” does in fact mean some­thing (and if oc­ca­sion­ally we sam­ple “Wash­ing­ton was Pres­i­dent after Jeffer­son”, well, what do you ex­pect from such an un-con­verged mod­el). Jar­ring er­rors will im­me­di­ately jos­tle us out of any il­lu­sion about the mod­el’s un­der­stand­ing, and so train­ing con­tin­ues. (Around here, Markov chain & n-gram mod­els start to fall be­hind; they can mem­o­rize in­creas­ingly large chunks of the train­ing cor­pus, but they can’t solve in­creas­ingly crit­i­cal syn­tac­tic tasks like bal­anc­ing paren­the­ses or quotes, much less start to as­cend from syn­tax to se­man­tic­s.)

Now train­ing is hard. Even sub­tler as­pects of lan­guage must be mod­eled, such as keep­ing pro­nouns con­sis­tent. This is hard in part be­cause the mod­el’s er­rors are be­com­ing rare, and be­cause the rel­e­vant pieces of text are in­creas­ingly dis­tant and ‘long-range’. As it makes pro­gress, the ab­solute size of er­rors shrinks dra­mat­i­cal­ly. Con­sider the case of as­so­ci­at­ing names with gen­der pro­nouns: the differ­ence be­tween “Janelle ate some ice cream, be­cause he likes sweet things like ice cream” and “Janelle ate some ice cream, be­cause she likes sweet things like ice cream” is one no hu­man could fail to no­tice, and yet, it is a differ­ence of a sin­gle let­ter. If we com­pared two mod­els, one of which did­n’t un­der­stand gen­der pro­nouns at all and guessed ‘he’/‘she’ purely at ran­dom, and one which un­der­stood them per­fectly and al­ways guessed ‘she’, the sec­ond model would at­tain a lower av­er­age er­ror of barely <0.02 bits per char­ac­ter!

Nev­er­the­less, as train­ing con­tin­ues, these prob­lems and more, like im­i­tat­ing gen­res, get solved, and even­tu­ally at a loss of 1–2 (where a small char-RNN might con­verge on a small cor­pus like Shake­speare or some Project Guten­berg ebook­s), we will fi­nally get sam­ples that sound hu­man—at least, for a few sen­tences. These fi­nal sam­ples may con­vince us briefly, but, aside from is­sues like rep­e­ti­tion loops, even with good sam­ples, the er­rors ac­cu­mu­late: a sam­ple will state that some­one is “alive” and then 10 sen­tences lat­er, use the word “dead”, or it will di­gress into an ir­rel­e­vant ar­gu­ment in­stead of the ex­pected next ar­gu­ment, or some­one will do some­thing phys­i­cally im­prob­a­ble, or it may just con­tinue for a while with­out seem­ing to get any­where.

All of these er­rors are far less than <0.02 bits per char­ac­ter; we are now talk­ing not hun­dredths of bits per char­ac­ters but less than ten-t­hou­sandths.

The pre­train­ing the­sis ar­gues that this can go even fur­ther: we can com­pare this per­for­mance di­rectly with hu­mans do­ing the same ob­jec­tive task, who can achieve closer to 0.7 bits per char­ac­ter. What is in that miss­ing >0.4?

Well—every­thing! Every­thing that the model miss­es. While just bab­bling ran­dom words was good enough at the be­gin­ning, at the end, it needs to be able to rea­son our way through the most diffi­cult tex­tual sce­nar­ios re­quir­ing causal­ity or com­mon­sense rea­son­ing. Every er­ror where the model pre­dicts that ice cream put in a freezer will “melt” rather than “freeze”, every case where the model can’t keep straight whether a per­son is alive or dead, every time that the model chooses a word that does­n’t help build some­how to­wards the ul­ti­mate con­clu­sion of an ‘es­say’, every time that it lacks the the­ory of mind to com­press novel scenes de­scrib­ing the Machi­avel­lian schem­ing of a dozen in­di­vid­u­als at din­ner jock­ey­ing for power as they talk, every use of logic or ab­strac­tion or in­struc­tions or Q&A where the model is be­fud­dled and needs more bits to cover up for its mis­take where a hu­man would think, un­der­stand, and pre­dict. Each of these cog­ni­tive break­throughs al­lows ever so slightly bet­ter pre­dic­tion of a few rel­e­vant texts; noth­ing less than true un­der­stand­ing will suffice for ideal pre­dic­tion.

If we trained a model which reached that loss of <0.7, which could pre­dict text in­dis­tin­guish­able from a hu­man, whether in a di­a­logue or quizzed about ice cream or be­ing tested on SAT analo­gies or tu­tored in math­e­mat­ics, if for every string the model did just as good a job of pre­dict­ing the next char­ac­ter as you could do, how could we say that it does­n’t truly un­der­stand every­thing? (If noth­ing else, we could, by de­fi­n­i­tion, re­place hu­mans in any kind of tex­t-writ­ing job!)

The last bits are deep­est. The im­pli­ca­tion here is that the fi­nal few bits are the most valu­able bits, which re­quire the most of what we think of as in­tel­li­gence. A help­ful anal­ogy here might be our ac­tions: for the most part, all hu­mans ex­e­cute ac­tions equally well. We all pick up a tea mug with­out drop­ping, and can lift our legs to walk down thou­sands of steps with­out falling even once. For every­day ac­tions, any­body, of any in­tel­li­gence, can get enough prac­tice & feed­back to do them quite well. Where in­di­vid­u­als differ is when they start run­ning into novel choic­es, rare choic­es, choices that take sec­onds but un­fold over a life­time, choices where we will never get any feed­back (like after our death). One only has to make a sin­gle bad de­ci­sion, out of a life­time of mil­lions of dis­crete de­ci­sions, to wind up in jail or dead. A small ab­solute av­er­age im­prove­ment in de­ci­sion qual­i­ty, if it is in those de­ci­sions, may be far more im­por­tant than its quan­tity in­di­cates. (Why do hu­mans have such large brains, when an­i­mals like chim­panzees do so many or­di­nary ac­tiv­i­ties seem­ingly as well with a frac­tion of the ex­pense? Why is lan­guage worth­while? Per­haps be­cause of con­sid­er­a­tions like these. We may be at our most hu­man while fill­ing out the pa­per­work for life in­sur­ance.)

Rea­sons for doubt. The pre­train­ing the­sis, while log­i­cally im­pec­ca­ble—how is a model sup­posed to solve all pos­si­ble trick ques­tions with­out un­der­stand­ing, just guess­ing?—n­ever struck me as con­vinc­ing, an ar­gu­ment ad­mit­ting nei­ther confu­ta­tion nor con­vic­tion. It feels too much like a magic trick: “here’s some in­for­ma­tion the­o­ry, here’s a hu­man bench­mark, here’s how we can en­code all tasks as a se­quence pre­dic­tion prob­lem, hey presto, in­tel­li­gence!” There are lots of al­go­rithms which are Tur­ing-com­plete or ‘uni­ver­sal’ in some sense; there are lots of al­go­rithms like AIXI which solve AI in some the­o­ret­i­cal sense (Schmid­hu­ber & com­pany have many of these cute al­go­rithms such as ‘the fastest pos­si­ble al­go­rithm for all prob­lems’, with the mi­nor catch of some con­stant fac­tors which re­quire com­put­ers big­ger than the uni­verse).

Why think pre­train­ing or se­quence mod­el­ing is not an­other one of them? Sure, if the model got a low enough loss, it’d have to be in­tel­li­gent, but how could you prove that would hap­pen in prac­tice? (Train­ing char-RNNs was fun, but they had­n’t ex­actly rev­o­lu­tion­ized deep learn­ing.) It might re­quire more text than ex­ists, count­less petabytes of data for all of those sub­tle fac­tors like log­i­cal rea­son­ing to rep­re­sent enough train­ing sig­nal, amidst all the noise and dis­trac­tors, to train a mod­el. Or maybe your mod­els are too small to do more than ab­sorb the sim­ple sur­face-level sig­nals, and you would have to scale them 100 or­ders of mag­ni­tude for it to work, be­cause the scal­ing curves did­n’t co­op­er­ate. Or maybe your mod­els are fun­da­men­tally bro­ken, and stuff like ab­strac­tion re­quire an en­tirely differ­ent ar­chi­tec­ture to work at all, and what­ever you do, your cur­rent mod­els will sat­u­rate at poor per­for­mance. Or it’ll train, but it’ll spend all its time try­ing to im­prove the sur­face-level mod­el­ing, ab­sorb­ing more and more lit­eral data and facts with­out ever as­cend­ing to the higher planes of cog­ni­tion as planned. Or…

But ap­par­ent­ly, it would’ve worked fine. Even RNNs prob­a­bly would’ve worked—­Trans­form­ers are nice, but they seem mostly be about effi­cien­cy.¹⁷ (Train­ing large RNNs is much more ex­pen­sive, and do­ing BPTT over mul­ti­ple nodes is much harder en­gi­neer­ing-wise.) It just re­quired more com­pute & data than any­one was will­ing to risk on it un­til a few true-be­liev­ers were able to get their hands on a few mil­lion dol­lars of com­pute.

1. Q: Did any­one pre­dict, quan­ti­ta­tive­ly, that this would hap­pen where it did?

   A: Not that I know of.

2. Q: What would fu­ture scaled-up mod­els learn?

   GPT-2-1.5b had a cross-en­tropy Web­Text val­i­da­tion loss of ~3.3 (based on the per­plex­ity of ~10 in Fig­ure 4, and log₂(10) = 3.32). GPT-3 halved that loss to ~1.73 judg­ing from Brown et al 2020 and us­ing the scal­ing for­mula (2.57 × (3.64 × 10³)^-0.048). For a hy­po­thet­i­cal GPT-4, if the scal­ing curve con­tin­ues for an­other 3 or­ders or so of com­pute (100–1000×) be­fore cross­ing over and hit­ting harder di­min­ish­ing re­turns, the cross-en­tropy loss will drop to ~1.24 (2.57 × (3.64 × (10³ × 10³))^−0.048).

   If GPT-3 gained so much meta-learn­ing and world knowl­edge by drop­ping its ab­solute loss ~50% when start­ing from GPT-2’s lev­el, what ca­pa­bil­i­ties would an­other ~30% im­prove­ment over GPT-3 gain? (Cut­ting the loss that much would still not reach hu­man-level, as far as I can tell.¹⁸) What would a drop to ≤1, per­haps us­ing wider con­text win­dows or re­cur­ren­cy, gain?

   A: I don’t know.

3. Q: Does any­one?

   A: Not that I know of.¹⁹

Prospects

What can we ex­pect from fu­ture DL work? Will GPT-3 kick­start an arms race where soon we will be dis­cussing, blase, what would seem now like lu­di­crously far­fetched schemes like bidi­rec­tional mul­ti­modal Trans­former 100× the size trained on 100× the data (video/text/PDFs-as-images/photo/robotics) with sup­ple­men­tary su­per­vised learn­ing as the back­bone of a MuZe­ro-like learn­ing+­plan­ning DRL agent run­ning on thou­sands of tasks (such as cod­ing) si­mul­ta­ne­ous­ly?

The ex­is­tence of the hard­ware over­hang im­plies that the lim­it­ing fac­tor here is less hard­ware than hu­man: will any or­ga­ni­za­tion treat GPT-3 as a Sput­nik mo­ment and in­vest ag­gres­sively in scal­ing pro­grams? Is there a GPT-4-equivalent brew­ing away in­side Deep­Mind or Google Brain’s TPU pods now? They aren’t stu­pid, they have the hard­ware, they have the bud­gets, they have the peo­ple.

But I think they lack a vi­sion. As far as I can tell: they do not have any such thing, be­cause Google Brain & Deep­Mind do not be­lieve in the scal­ing hy­poth­e­sis the way that Sutskev­er, Amodei and oth­ers at OA do. Just read through ma­chine learn­ing Twit­ter to see the dis­dain for the scal­ing hy­poth­e­sis. (A quar­ter year on from GPT-3 and count­ing, can you name a sin­gle dense model as large as the 17b Turing-NLG—never mind larger than GPT-3?)

Google Brain is en­tirely too prac­ti­cal and short­-term fo­cused to dab­ble in such es­o­teric & ex­pen­sive spec­u­la­tion, al­though Quoc V. Le’s group oc­ca­sion­ally sur­prises you. They’ll dab­ble in some­thing like GShard, but mostly be­cause they ex­pect to be likely to be able to de­ploy it or some­thing like it to pro­duc­tion in Google Trans­late.

Deep­Mind²⁰ holds what we might call the “weak scal­ing hy­poth­e­sis”: they be­lieve that AGI will re­quire us to “find the right al­go­rithms” effec­tively repli­cat­ing a mam­malian brain mod­ule by mod­ule, and that while these mod­ules will be ex­tremely large & ex­pen­sive by con­tem­po­rary stan­dards (which is why com­pute is im­por­tant, to give us “a more pow­er­ful tool with which to hunt for the right al­go­rithms”), they still need to be in­vented & fine­tuned piece by piece, with lit­tle risk or sur­prise un­til the fi­nal as­sem­bly. Each piece, how­ev­er, it­self can scale: there’s no mag­i­cal in­tel­li­gence gland or quan­tum woo which cre­ates a bright line be­tween hu­mans and, say, chim­panzees or ro­dents. (As much as we hu­mans ex­trav­a­gantly ad­mire our own ca­pa­bil­i­ties like lan­guage or log­ic, those are rel­a­tively mi­nor flour­ishes on the ba­sic brain—each or­gan­ism solves the same ba­sic prob­lems, like ex­plo­ration, long-term mem­o­ry, learn­ing world-mod­els, as­so­ci­at­ing re­wards with spe­cific ac­tions, meta-learn­ing, etc.) As such, once you have a rat-level AGI, a hu­man-level AGI is just more so. (And rats are a lot eas­ier to ex­per­i­ment on.) That is how you get DM con­trap­tions like Agen­t57 which throw the kitchen sink at the wall to see what sticks, and why they place such em­pha­sis on neu­ro­science as in­spi­ra­tion and cross-fer­til­iza­tion for re­verse-engi­neer­ing the brain. (See also Sam Alt­man’s pod­cast in­ter­view com­ments on OA’s ad­van­tage vs un­named ri­vals with more com­pute is be­cause the lack of com­pute makes them stay “small and fo­cused”—“for sure” like a startup ap­proach.) When some­one seems to have come up with a scal­able ar­chi­tec­ture for crack­ing a hard prob­lem, like Al­p­haZero or Al­phaS­tar, they are will­ing to pour on the gas to make it scale, but oth­er­wise, in­cre­men­tal re­fine­ment on ALE and then DMLab-30 is the game plan. They have been bit­ing off and chew­ing pieces of the brain for a decade, and it’ll prob­a­bly take an­other decade or two of steady chew­ing if all goes well. Be­cause they have locked up so much tal­ent and have so much pro­pri­etary code and be­lieve all of that is a ma­jor moat to any com­peti­tor try­ing to repli­cate the com­pli­cated brain, they are fairly easy­go­ing. You will not see DM ‘bet the com­pany’ on any moon­shot; Google’s cash­flow is­n’t go­ing any­where (and DM’s bud­get), and slow and steady wins the race.

Go­ing be­yond that, most other re­search labs like Tesla or FAIR are ir­rel­e­vant and un­in­ter­est­ed. Chi­nese AI com­pa­nies are a ques­tion mark: past the lan­guage bar­ri­er, I seem to dis­cern in­ter­est in AGI & lit­tle of the re­flex­ive West­ern op­po­si­tion, and com­pa­nies like Baidu oc­ca­sion­ally re­lease im­por­tant re­search (such as the early scal­ing pa­per Hes­t­ness et al 2017), but over­all, Chi­nese AI may be over­es­ti­mat­ed, and they seem to suffer from a kind of Dutch dis­ease—­fund­ing for sur­veil­lance tech­nol­o­gy, and for nar­row e-com­merce nich­es, is so plen­ti­ful that other ar­eas are ne­glect­ed.

OA, lack­ing any­thing like DM’s long-term fund­ing from Google or its enor­mous head­count, is mak­ing a star­tup-like bet that they know an im­por­tant truth which is a se­cret: “the scal­ing hy­poth­e­sis is true!” So, sim­ple DRL al­go­rithms like PPO on top of large sim­ple ar­chi­tec­tures like RNNs or Trans­form­ers can emerge, ex­ploit­ing the bless­ings of scale, and meta-learn their way to pow­er­ful ca­pa­bil­i­ties, en­abling fur­ther fund­ing for still more com­pute & scal­ing, in a vir­tu­ous cy­cle. This is why OA had to re­vise its cor­po­rate form: lack­ing any enor­mous en­dow­ment or ex­tremely deep­-pock­eted pa­tron like Google, where does it get the money to scale (or hire ma­chine learn­ing engineer/researchers who can com­mand salaries in the mil­lion­s)? OA has to earn the nec­es­sary mon­ey, so in a move like Mozilla Foun­da­tion own­ing Mozilla Cor­po­ra­tion (to sell Fire­fox search en­gine place­men­t), or the Her­shey or­phan­age own­ing Her­shey Choco­late or the Girl Scouts li­cens­ing their cook­ies, Ope­nAI switched from a pure non­profit funded by do­na­tions to a non­profit which owns a for-profit subsidiary/startup, “Ope­nAI LP”, which can take in­vest­ments and en­gage in for-profit ac­tiv­i­ties. OA LP, while con­trolled by OA, can then shoot for the moon. And if OA is wrong to trust in the God of Straight Lines On Graphs, well, they never could com­pete with DM di­rectly us­ing DM’s fa­vored ap­proach, and were al­ways go­ing to be an al­so-ran foot­note, so they have no re­gret.

While all of this hy­po­thet­i­cally can be repli­cated rel­a­tively eas­ily (n­ever un­der­es­ti­mate the amount of tweak­ing and spe­cial sauce it takes) by com­peti­tors if they wished (the nec­es­sary amounts of com­pute bud­gets are still triv­ial in terms of Big Sci­ence or other in­vest­ments like Al­phaGo or Al­phaS­tar or Way­mo, after al­l), said com­peti­tors lack the very most im­por­tant thing, which no amount of money or GPUs can ever cure: the courage of their con­vic­tions. They are too hide­bound and deeply philo­soph­i­cally wrong to ever ad­mit fault and try to over­take OA un­til it’s too late. How can we talk se­ri­ously about any kind of mil­i­tary Man­hat­tan Project when the US mil­i­tary does­n’t even let its de­vel­op­ers use Ten­sor­flow or Py­Torch, or about gov­ern­ment projects in the shadow of coro­n­avirus? This might seem ab­surd (surely the Bit­ter Lesson/scaling hy­poth­e­sis have now earned enough prior prob­a­bil­ity to be taken se­ri­ously and re­ceive ma­jor re­search in­vest­ments to test how far they can go, es­pe­cially given how im­por­tant the im­pli­ca­tions are), but look at the re­peated crit­i­cism of OA every time they re­lease a new ex­am­ple of the scal­ing hy­poth­e­sis, from GPT-1 to Dactyl to OA5 to GPT-2 to iGPT to GPT-3… To para­phrase St Au­gustine, most peo­ples’ re­ac­tion to the Bit­ter Les­son or scal­ing hy­poth­e­sis is “grant me scale & com­pute—but not yet”.²¹

A crit­i­cal in­di­ca­tor will be whether or­ga­ni­za­tions be­yond ‘the usual sus­pects’ (Mi­crosoft Ze­RO-2 team has reached 1t-s­cale train­ing, but there is also Nvidia, Sales­force, Al­len, Google DM/GB, Connor/EleutherAI, Face­book FAIR) start par­tic­i­pat­ing or if they con­tinue to dis­miss scal­ing. At least as of 2020-10-26, 152 days lat­er, no model has come near GPT-3, and in­deed, no model has even ex­ceeded Turing-NLG’s 17b.²²

Critiquing The Critics

Keep­ing track. GPT-3 in 2020 makes as good a point as any to take a look back on the past decade. It’s re­mark­able to re­flect that some­one who started a PhD be­cause they were ex­cited by these new “ResNets” would still not have fin­ished it by now—that is how re­cent even resnets are, never mind Trans­form­ers, and how rapid the pace of progress is. In 2010, one could eas­ily fit every­one in the world who gen­uinely be­lieved in deep learn­ing into a mod­er­ate-sized con­fer­ence room (as­sisted slightly by the fact that 3 of them were busy found­ing Deep­Mind). Some­one in­ter­ested in ma­chine learn­ing in 2010 might have read about some in­ter­est­ing stuff from weirdo diehard con­nec­tion­ists in rec­og­niz­ing hand-writ­ten dig­its us­ing all of 1–2 mil­lion pa­ra­me­ters, or some mod­est neural tweaks to stan­dard voice-recog­ni­tion hid­den Markov mod­els. In 2010, who would have pre­dicted that over the next 10 years, deep learn­ing would un­dergo a Cam­brian ex­plo­sion caus­ing a mass ex­tinc­tion of al­ter­na­tive ap­proaches through­out ma­chine learn­ing, that mod­els would scale up to 175,000 mil­lion pa­ra­me­ters, and that these enor­mous mod­els would just spon­ta­neously de­velop all these ca­pa­bil­i­ties?

No one. That is, no one aside from a few diehard con­nec­tion­ists writ­ten off as will­ful­ly-de­luded old-school fa­nat­ics by the rest of the AI com­mu­nity (n­ever mind the world), such as Moravec, Schmid­hu­ber, Sutskever, Legg, & Amod­ei? One of the more shock­ing things about look­ing back is re­al­iz­ing how un­sur­pris­ing and eas­ily pre­dicted all of this was if you lis­tened to the right peo­ple. In 1998, 22 years ago, Moravec noted that AI re­search could be de­cep­tive, and hard­ware lim­its meant that “in­tel­li­gent ma­chine re­search did not make steady progress in its first 50 years, it marked time for 30 of them!”, pre­dict­ing that as Moore’s law con­tin­ued, “things will go much faster in the next 50 years than they have in the last 50.” Moravec fur­ther ob­served that part of the rea­son for rapid progress was the hard­ware over­hang: while su­per­com­put­ers of the nec­es­sary power would ex­ist long be­fore the con­nec­tion­ist rev­o­lu­tion be­gan, no one would be al­lowed to use them, as they would be de­voted to ‘more im­por­tant’ (pres­ti­gious) hard STEM work, like “physics sim­u­la­tions” (ie cli­mate sim­u­la­tions & nu­clear bombs)²³, and “AI re­search must wait for the power to be­come more afford­able.” Afford­able mean­ing a work­sta­tion roughly ~$1,854^$1000₁₉₉₈; suffi­ciently cheap com­pute to ri­val a hu­man would ar­rive some­time in the 2020s, with the 2010s see­ing afford­able sys­tems in the lizard–­mouse range. As it hap­pens, the start of the DL rev­o­lu­tion is typ­i­cally dated to AlexNet in 2012, by a grad stu­dent us­ing 2 GTX 580 3GB GPUs (launch list price of… $657^$500₂₀₁₀, for a sys­tem build cost of per­haps $1,901^$1500₂₀₁₂). 2020 saw GPT-3 ar­rive, and as dis­cussed be­fore, there are many rea­sons to ex­pect the cost to fall, in ad­di­tion to the large hard­ware com­pute gains that are be­ing fore­cast for the 2020s.

The ac­cel­er­at­ing pace of the last 10 years should wake any­one from their dog­matic slum­ber and make them sit up­right. And there are 28 years left in Moravec’s fore­cast…

The temp­ta­tion, that many do not re­sist so much as revel in, is to give in to a dé­for­ma­tion pro­fes­sion­nelle and dis­miss any model as “just” this or that(“just bil­lions of IF state­ments” or “just a bunch of mul­ti­pli­ca­tions” or “just mil­lions of mem­o­rized web pages”), miss­ing the for­est for the trees, as Moravec com­mented of chess en­gi­nes:

The event was no­table for many rea­sons, but one es­pe­cially is of in­ter­est here. Sev­eral times dur­ing both match­es, Kas­parov re­ported signs of mind in the ma­chine. At times in the sec­ond tour­na­ment, he wor­ried there might be hu­mans be­hind the sce­nes, feed­ing Deep Blue strate­gic in­sight­s!…In all other chess com­put­ers, he re­ports a me­chan­i­cal pre­dictabil­ity stem­ming from their undis­crim­i­nat­ing but lim­ited looka­head, and ab­sence of long-term strat­e­gy. In Deep Blue, to his con­ster­na­tion, he saw in­stead an “alien in­tel­li­gence.”

…Deep Blue’s cre­ators know its quan­ti­ta­tive su­pe­ri­or­ity over other chess ma­chines in­ti­mate­ly, but lack the chess un­der­stand­ing to share Kas­parov’s deep ap­pre­ci­a­tion of the differ­ence in the qual­ity of its play. I think this di­chotomy will show up in­creas­ingly in com­ing years. En­gi­neers who know the mech­a­nism of ad­vanced ro­bots most in­ti­mately will be the last to ad­mit they have real minds. From the in­side, ro­bots will in­dis­putably be ma­chi­nes, act­ing ac­cord­ing to me­chan­i­cal prin­ci­ples, how­ever elab­o­rately lay­ered. Only on the out­side, where they can be ap­pre­ci­ated as a whole, will the im­pres­sion of in­tel­li­gence emerge. A hu­man brain, too, does not ex­hibit the in­tel­li­gence un­der a neu­ro­bi­ol­o­gist’s mi­cro­scope that it does par­tic­i­pat­ing in a lively con­ver­sa­tion.

But of course, if we ever suc­ceed in AI, or in re­duc­tion­ism in gen­er­al, it must be by re­duc­ing Y to ‘just X’. Show­ing that some task re­quir­ing in­tel­li­gence can be solved by a well-de­fined al­go­rithm with no ‘in­tel­li­gence’ is pre­cisely what suc­cess must look like! (Other­wise, the ques­tion has been thor­oughly begged & the prob­lem has only been pushed else­where; com­puter chips are made of tran­sis­tors, not es­pe­cially lit­tle ho­mun­culi.)

Hind­sight is 20⁄20. Even in 2015, the scal­ing hy­poth­e­sis seemed highly du­bi­ous: you needed some­thing to scale, after all, and it was all too easy to look at flaws in ex­ist­ing sys­tems and imag­ine that they would never go away and progress would sig­moid any month now, soon. Like the ge­nomics rev­o­lu­tion where a few far-sighted seers ex­trap­o­lated that the nec­es­sary n for GWASes would in­crease ex­po­nen­tially & de­liver pow­er­ful PGSes soon, while sober ex­perts wrung their hands over “miss­ing her­i­tabil­ity” & the mirac­u­lous com­plex­ity of bi­ol­ogy & scoff about how such n re­quire­ments proved GWAS was a failed par­a­digm, the fu­ture ar­rived at first slowly and then quick­ly. Yet, here we are: all honor to the fa­nat­ics, shame and hu­mil­i­a­tion to the crit­ics!²⁴ If only one could go back 10 years, or even 5, to watch every AI re­searchers’ head ex­plode read­ing this pa­per… Un­for­tu­nate­ly, few heads ap­pear to be ex­plod­ing now, be­cause hu­man ca­pac­ity for hind­sight & ex­cuses is bound­less (“I can get that much with fine­tun­ing, any­way I pre­dicted it all along, how bor­ing”) and, un­for­tu­nate­ly, “there is no fire alarm” for AGI. (If you are still cer­tain that there is near-zero prob­a­bil­ity of AGI in the next few decades, why? Did you pre­dic­t—in writ­ing—­ca­pa­bil­i­ties like GPT-3? Is this how you ex­pect AI fail­ure to look in the decades be­fore­hand? What spe­cific task, what spe­cific num­ber, would con­vince you oth­er­wise? How would the world look differ­ent than it does now if these crude pro­to­type in­sec­t-brain-sized DL sys­tems were not on a path to suc­cess?)

Au­thor­ity with­out ac­count­abil­i­ty. What should we think about the ex­perts? Pro­jec­tions of fail­ure were made by em­i­nent, re­spectable, se­ri­ous peo­ple. They spoke in con­sid­ered tones of why AI hype was ex­ces­sive and might trig­ger an “AI win­ter”, and the fun­da­men­tal flaws of fash­ion­able ap­proaches and why brute force could not work. These state­ments were made rou­tinely in 2014, 2015, 2016… And they were wrong. I am aware of few is­su­ing a mea culpa or re­flect­ing on it.²⁵ It is a puz­zling fail­ure, and I’ve re­flected on it be­fore.

Phat­ic, not pre­dic­tive. There is, how­ev­er, a cer­tain tone of voice the bien pen­sant all speak in, whose sound is the same whether right or wrong; a tone shared with many state­ments in Jan­u­ary to March of this year; a tone we can also find in a 1940 Sci­en­tific Amer­i­can ar­ti­cle au­thor­i­ta­tively ti­tled, “Don’t Wor­ry—It Can’t Hap­pen”, which ad­vised the reader to not be con­cerned about it any longer “and get sleep”. (‘It’ was the atomic bomb, about which cer­tain sci­en­tists had stopped talk­ing, rais­ing pub­lic con­cerns; not only could it hap­pen, the British bomb project had al­ready be­gun, and 5 years later it did hap­pen.)

The iron law of bu­reau­cra­cy: Cathe­dral goth­ic. This tone of voice is the voice of au­thor­ity.
The voice of au­thor­ity in­sists on calm, and peo­ple not “pan­ick­ing” (the chief of sin­s).
The voice of au­thor­ity as­sures you that it won’t hap­pen (be­cause it can’t hap­pen).
The voice ut­ters sim­ple ar­gu­ments about why the sta­tus quo will pre­vail, and con­sid­ers only how the wild new idea could fail (and not all the pos­si­ble op­tion­s).
The voice is not, and does not deal in, un­cer­tain­ty; things will ei­ther hap­pen or they will not, and since it will not hap­pen, there is no need to take any pre­cau­tions (and you should not worry be­cause it can’t hap­pen).
The voice does not be­lieve in draw­ing lines on graphs (it is rank nu­merol­o­gy).
The voice does not is­sue any nu­mer­i­cal pre­dic­tions (which could be fal­si­fied).
The voice will not share its source code (for com­pli­cated rea­sons which can­not be ex­plained to the laity).
The voice is op­posed to un­eth­i­cal things like ran­dom­ized ex­per­i­ments on vol­un­teers (but will over­look the in­sult).
The voice does not have a model of the fu­ture (be­cause a model im­plies it does not al­ready know the fu­ture).
The voice is con­cerned about its pub­lic im­age (and un­kind gos­sip about it by other speak­ers of the voice).
The voice is al­ways sober, re­spectable, and cre­den­tialed (the voice would be pleased to write an op-ed for your na­tional mag­a­zine and/or news­pa­per).
The voice speaks, and is not spo­ken to (you can­not ask the voice what ob­jec­tive fact would change its mind).
The voice never changes its mind (un­til it does).
The voice is never sur­prised by events in the world (only dis­ap­point­ed).
The voice ad­vises you to go back to sleep (right now).

When some­one speaks about fu­ture pos­si­bil­i­ties, what is the tone of their voice?

Links

AI:

• Mat­ters Of Scale:

  • GPT-3: see above
  • “Mea­sur­ing the Al­go­rith­mic Effi­ciency of Neural Net­works”, Her­nan­dez & Brown 2020 (blog/in­ter­view; the first pro­to­type is never the best one, but given enough com­pute & time, you can re­fine it and fig­ure out how it should have been done all along, and this pa­per quan­ti­fies the neural net hard­ware over­hang just since 2012: “it now takes 44× less com­pute to train…to the level of AlexNet”. Un­sur­pris­ing—eg the ex­pe­ri­ence curve in lin­ear pro­gram­ming: Bixby 2002; see Grace 2013/Yud­kowsky 2013. We don’t know how to train the right kind of neural nets and make huge mis­takes with the sim­plest things, as ca­pa­bil­ity jumps like resnets or Effi­cient­Net or R2D2 oc­ca­sion­ally re­mind us.)
  • “In­tel­li­Code Com­pose: Code Gen­er­a­tion Us­ing [GPT-2] Trans­former”, Svy­atkovskiy et al 2020 (un­clear if ap­pli­ca­tion of Ze­RO-2; see also the GPT-3 few-shot code com­ple­tion abil­i­ties)
  • “GrokNet: Uni­fied Com­puter Vi­sion Model Trunk and Em­bed­dings For Com­merce” (blog; one mod­el, 7 datasets, 89m im­ages, 83 losses/tasks, and +8% search qual­ity boost world­wide)

• “Deep neu­roethol­ogy of a vir­tual ro­dent”, Merel et al 2019 (me­dia)

• “Go-Ex­plore 2: First re­turn then ex­plore”, Ecoffet et al 2020

• “Learn­ing to Sim­u­late Dy­namic En­vi­ron­ments with GameGAN”, Kim et al 2020 (project page; an un­ex­pected ap­pear­ance of a Neural Tur­ing Ma­chine)

• “Ex­plor­ing Bayesian Op­ti­miza­tion: Break­ing Bayesian Op­ti­miza­tion into small, size­able chunks”, Ag­ni­hotri & Ba­tra 2020

• “This Word Does Not Ex­ist” (GPT-2); “This Fur­sona Does Not Ex­ist (TFDNE)” ed­i­tor (a sim­ple but high­-qual­ity StyleGAN 2 face model of fur­ries, also avail­able on Art­breeder; in­ter­est­ing for how the fur flew due to le­gal fuzzi­ness & some artists act­ing like an­i­mals, howl­ing about ‘theft’ & free fur­sonas be­ing a wolf in sheep’s cloth­ing up­set­ting their peck­ing or­der²⁶—though the cre­ator has out­foxed the pa­per tiger threats, these kit­tle­some ques­tions will dog ML as DL mod­els mul­ti­ply like rab­bits)

Ge­net­ics:

• Every­thing Is Her­i­ta­ble:

  • “Lo­cal ge­netic cor­re­la­tion analy­sis re­veals het­ero­ge­neous eti­o­logic shar­ing of com­plex traits”, Zhang et al 2020 (“autism/IQ r_g…could be ex­plained by 2 eti­o­log­i­cal­ly-dis­tinct ge­netic sig­na­tures w/bidirectional lo­cal ge­netic cor­re­la­tions”)
  • “Iden­ti­fi­ca­tion of 370 loci for age at on­set of sex­ual and re­pro­duc­tive be­hav­iour, high­light­ing com­mon ae­ti­ol­ogy with re­pro­duc­tive bi­ol­o­gy, ex­ter­nal­iz­ing be­hav­iour and longevity”, Mills et al 2020
  • “Genome-wide as­so­ci­a­tion study of school grades iden­ti­fies a ge­netic over­lap be­tween lan­guage abil­i­ty, psy­chopathol­ogy and cre­ativ­ity”, Ra­jagopal et al 2020 (“math per­for­mance was se­verely affected whereas lan­guage per­for­mance (Dan­ish and Eng­lish) was rel­a­tively un­affected or en­hanced in those with psy­chi­atric dis­or­ders”)
  • “GWAS of De­pres­sion Phe­no­types in the Mil­lion Vet­eran Pro­gram and Meta-analy­sis in More than 1.2 Mil­lion Par­tic­i­pants Yields 178 In­de­pen­dent Risk Loci”, Levey et al 2020
  • “A Sin­gle Gene Causes The­ly­tok­ous Partheno­gen­e­sis, the Defin­ing Fea­ture of the Cape Hon­ey­bee Apis mel­lif­era capen­sis”, Yagound et al 2020
  • “In­sights into the ge­netic ar­chi­tec­ture of the hu­man face”, White et al 2020

• Re­cent Evo­lu­tion:

  • “Sex-bi­ased re­duc­tion in re­pro­duc­tive suc­cess dri­ves se­lec­tive con­straint on hu­man genes”, Gard­ner et al 2020; “Genome-wide analy­sis iden­ti­fies ge­netic effects on re­pro­duc­tive suc­cess and on­go­ing nat­ural se­lec­tion at the FADS lo­cus”, Math­ieson et al 2020 (pre­vi­ous­ly: Bar­ban et al 2016/Tropf et al 2016/Ver­weij et al 2017)
  • “Dis­en­tan­gling se­lec­tion on ge­net­i­cally cor­re­lated poly­genic traits us­ing whole-genome ge­nealo­gies”, Stern et al 2020

• En­gi­neer­ing:

  • “Re­cent ad­vances in bovine in vitro em­bryo pro­duc­tion: re­pro­duc­tive biotech­nol­ogy his­tory and meth­ods”, Ferré et al 2020
  • Atomic gar­den mu­ta­tion breed­ing

Statistics/meta-science/mathematics:

• “Vari­abil­ity in the analy­sis of a sin­gle neu­roimag­ing dataset by many teams”, Botvinik-Nezer et al 2020
• “Re­mem­ber­ing John Con­way’s FRACTRAN, a ridicu­lous, yet sur­pris­ingly deep lan­guage”, Regi­nald Braith­waite (how does the re­cent­ly-de­ceased John Con­way’s 1980 es­olang lead to the Col­latz con­jec­ture?)
• “Tum­bling toast, Mur­phy’s Law and the fun­da­men­tal con­stants”, Matthews 1995 (overview; an­throp­ics size ar­gu­ment from Press 1980; see also Ba­con et al 2001/Borghi 2012)

Politics/religion:

• Re­view of The Cul­tural Rev­o­lu­tion
• “The Vol­un­tari­ness of Vol­un­tary Con­sent: Con­sent Searches and the Psy­chol­ogy of Com­pli­ance”, Som­mers & Bohns 2019 (peo­ple are bad at pre­dict­ing re­sis­tance to po­lice re­quests; see also Christin et al 2012)
• Op­er­a­tion INFEKTION (see also Gor­don 1997)
• “Progress Stud­ies for As­pir­ing Young Schol­ars” (ex­per­i­men­tal on­line sum­mer class for high school stu­dents by Ja­son Craw­ford on de­vel­op­ment)

Psychology/biology:

• “Un­der­stand­ing im­mu­nity through the lens of dis­ease ecol­ogy”, Hedrick 2017²⁷ (“…for the past few thou­sand years, we hu­man be­ings have been the most dis­eased species on earth”; fol­lowup to Hedrick 2004)
• “How san­i­ta­tion con­quered dis­ease long be­fore vac­cines or an­tibi­otics”, Ja­son Craw­ford
• “Every­day Life as an In­tel­li­gence Test: Effects of In­tel­li­gence and In­tel­li­gence Con­text”, Gor­don 1997
• “Ob­jec­tive and sub­jec­tive ex­pe­ri­ences of child mal­treat­ment and their re­la­tion­ships with psy­chopathol­ogy”, Danese & Widom 2020 (noth­ing in psy­chol­ogy makes sense ex­cept in the light of in­di­vid­u­al-d­iffer­ences)
• “Brain­less but Mul­ti­-Head­ed: De­ci­sion Mak­ing by the Acel­lu­lar Slime Mould Physarum poly­cephalum”, Beek­man & Latty 2015
• “I’m paid bi­week­ly, just not by lep­rechauns: Eval­u­at­ing valid-but-in­cor­rect re­sponse rates to at­ten­tion check items”, Cur­ran & Hauser 2019 (how do “lizard­man con­stant” re­spon­ders jus­tify it? Or, ‘free re­sponse is the devil’)

Tech­nol­o­gy:

• “Re­flec­tions on How De­sign­ers De­sign with Data”, Bigelow et al 2014 (why are data vi­su­al­iza­tions so bad—­su­per­fi­cially pretty but mis­lead­ing or use­less? Be­cause many de­sign­ers don’t look at the data, avoid au­toma­tion & cre­ate man­u­ally so they can fo­cus on pretty shapes/colors & en­joy­ing fid­dling with it, and ig­nore read­ers)
• “Do Ads Harm News Con­sump­tion?”, Yan et al 2020 (“Users who adopt ad block­ers sub­se­quently con­sume 20% more news ar­ti­cles cor­re­spond­ing to 10% more cat­e­gories. The effect per­sists over time…”; see my ad page)
• “The 1-Bit In­stru­ment: The Fun­da­men­tals of 1-Bit Syn­the­sis, Their Im­ple­men­ta­tional Im­pli­ca­tions, and In­stru­men­tal Pos­si­bil­i­ties”, Troise 2020

Eco­nom­ics:

• “In Ohio, the Amish Take On the Coro­n­avirus” (sup­ply and de­mand: masks can be eas­ily made any­where if prices are al­lowed to rise & they are not il­le­gal to sell)
• “The Story of Amer­i­ca’s Most Pro­lific Coun­ter­feiter” (how Frank Bourassa tricked a Swiss mill into sell­ing him the unique U.S. dol­lar linen-pa­per to cre­ate $317^$250₂₀₁₂m in per­fect coun­ter­feit money & mostly got away with it)

Fic­tion:

• “My Im­mor­tal As Al­chem­i­cal Al­le­gory”, SSC

Misc:

• “Dead Reck­on­ing: The 18^th cen­tury mis­ad­ven­tures of HMS Wa­ger and her re­luc­tant crew” (the Wa­ger Mutiny)

Books

Fic­tion:

• The Bat­tle Be­tween the Frogs and the Mice: A Tiny Home­ric Epic, trans­lated Stallings 2009 (re­view)

Music

Touhou:

• “Sept Jours sans Elle (Vo­cal)” (Raven’s Jig; Une Se­maine chez les Écar­lates {2018}) [clas­si­cal]
• “Un Jour Joueur” (Raven’s Jig; Une Se­maine chez les Écar­lates {2018}) [clas­si­cal]
• “Bons et mau­vais Jours” (Raven’s Jig; Une Se­maine chez les Écar­lates {2018}) [clas­si­cal]

MLP:

• “Morn­ing in Bal­ti­mare” (Mane in Green; II. The Jour­ney [The Quest of the Lost Sap­phire—Ep. 2] {2017}) [in­stru­men­tal rock]
• “Love and Re­flec­tion” (Dionte George; Ig­nite {2020}) [jazz]
• “Sec­ond Prances (Vo­cal VIP)” (Etherium Apex ft. Nicole Carino {2020}) [elec­tron­ic]
• “Spun” (The Waste­land Wail­ers feat. Brit­tany Church & Hay­mak­er; Ig­nite {2020}) [coun­try]
• “Eq­ui­ter­ian Em­pire” (Car­bon Mae­stro; Ce­les­tial Di­vide OST) [orches­tral]
• “The Storm Is Com­ing VIP [S­in­gle Pur­pose Remix]” (Un­dreamed­Panic feat. Meta­jok­er; Ig­nite {2020}) [rock]
• “Mare Cog­ni­tum” (Idyl­lia feat. Vel­vet R. Wings; Ig­nite {2020}) [orches­tral rock]
• “Fire City (Day & Night)” (Wan­der­ing Artist; Ig­nite {2020}) [orches­tral]
• “What Re­mains” (To­talspark; Ig­nite {2020}) [Liq­uid Drum & Brass]

Dou­jin:

• “Come, Sweet Death [Komm, süsser Tod]” (Platina Jazz feat. Niklas Gabrielsson; Anime Stan­dards Vol. 6 {2019}) [jazz]
• “Hope” (Simp­sonill {2017}) [elec­tron­ic]