The Hyperbolic Time Chamber & Brain Emulation

A time dilation chamber as thought experiment on the power of pure thought, with comparison to computer AGI advantages/disadvantages.
transhumanism, anime, AI, SF
2012-08-292018-09-02 finished certainty: likely importance: 9


A time di­la­tion tool from an anime is dis­cussed for its prac­ti­cal use on Earth; there seem sur­pris­ingly few uses and none that will change the world, due to the se­vere penal­ties hu­mans would in­cur while us­ing it, and ba­sic con­straints like Am­dahl’s law limit the sci­en­tific us­es. A com­par­i­son with the po­si­tion of an Ar­ti­fi­cial In­tel­li­gence such as an em­u­lated hu­man brain seems fair, ex­cept most of the time di­la­tion dis­ad­van­tages do not ap­ply or can be ame­lio­rated and hence any speedups could be quite effec­tively ex­ploit­ed. I sug­gest that skep­tics of the idea that speedups give ad­van­tages are im­plic­itly work­ing off the crip­pled time di­la­tion tool and not mak­ing al­lowance for the disanalo­gies.

, a pop­u­lar (but medioc­re) shonen fight­ing ani­me, in­cludes a cute bit of SF in it—the Hy­per­bolic Time Cham­ber (HTC), which can be thought of a re­verse in which time speeds up for 2 peo­ple: the HTC opens and closes once a day re­al­time, but in­side its lit­tle pocket uni­verse, a full year passes for 2 peo­ple, giv­ing a 365x speedup. There’s no such thing as a HTC-within-a-HTC, so uses are lim­ited to just that one 365x speedup—no 3652 speedups.

Ig­nor­ing the DBZ-spe­cific as­pects of the HTC like the per­son limit or in­creased grav­ity or air and tem­per­a­ture changes, one won­ders: what one would do with a HTC in the real world?

Uses for time acceleration

In DBZ, the only use seems to be for train­ing mon­tages to let char­ac­ters power up to fight a new alien or mar­tial artist, but world-de­stroy­ing mar­tial artists seem to be rare in the real world, so we could­n’t use it for that.

Could we use it for reg­u­lar mar­tial arts train­ing? The DBZ ar­ti­cle for the HTC men­tions no non-e­mer­gency use, and a lit­tle thought leads us to con­clude that, prob­a­bly not: in­side the HTC, time passes as nor­mal, which means that you don’t save any time. All the HTC is do­ing is re­ar­rang­ing rel­a­tive time be­tween groups. If you step in, you still age a full year be­fore step­ping out, and you will now die a year early by the re­al­time cal­en­dar. So what’s the point? We can think of a few us­es—imag­ine some­one who gets in­jured just be­fore the Olympic­s—but let’s face it, that may be con­ve­nient for a few peo­ple, but it’s hardly chang­ing the world. A whole time-ac­cel­er­ated pocket uni­verse… Surely we can think of some­thing less point­less than tweak­ing ath­let­ics?

Downsides

Ac­tu­al­ly, it’s worse than point­less—the HTC is a su­per-: you can­not leave at any point be­fore the year is up, you can­not com­mu­ni­cate in any way, noth­ing goes in or out, and you have only what you brought with you (and al­ways what you brought with you).

Autarchy

Un­der such con­di­tions, a year in the HTC could well be con­sid­ered “cruel and un­usual pun­ish­ment”; no doc­tor would vol­un­teer for it, so any pris­on­ers in the HTC face a se­ri­ous risk of death from any cause un­less a fel­low pris­oner that day/year had med­ical train­ing. The food will also suck, as any food will have to be stor­able for a year (it would­n’t do to starve to death a day be­fore the door opens back to the real world); you can’t grow your own be­cause there is no ap­par­ent sun, soil, or sea­sons in­side the HTC, and while you could prob­a­bly bring in a green­house & soil and fer­til­ize with re­cy­cled food, how are you go­ing to power the lights in your green­house? Drag in a minia­ture nu­clear power plant or a ? And we haven’t even con­sid­ered how much time one would spend (waste) on this smal­l­-s­cale agri­cul­ture; main­tain­ing was a ful­l-time job for many high­ly-skilled peo­ple, and if that was true for the HTC, there would hardly be any point in it.

If one punted on the prob­lems of main­tain­ing a high qual­ity of life and posited a ded­i­cated re­searcher, well, the HTC is still not use­ful. They will find it hard to take with them an en­tire li­brary or lab­o­ra­to­ry, many in­gre­di­ents are too ex­pen­sive or per­ish­able to buy in ad­vance just be­cause the re­searcher might need them, but if they don’t have ac­cess to pretty much every­thing, they’ll quickly hit some sort of bar­rier where one email or or­der would let them fin­ish a project but that email can’t be sent for up to a year. (Imag­ine a re­searcher who en­ters the HTC—and his lap­top’s hard-drive dies. Oops. Hope he had back­ups or spares, of his data and his lap­top and every­thing else for that mat­ter.) Omit­ting these con­cerns, re­search is a so­cial process in the sense that one is often dis­cussing or ex­plain­ing or de­fend­ing the re­search, and with­out these in­ter­ac­tions it is easy to go down blind al­leys, make mi­nor-seem­ing but fa­tal mis­takes1, wind up rein­vent­ing some­thing stan­dard in an­other field, etc. One can eas­ily waste a month this way, and so a year. A group would help, but groups are sus­cep­ti­ble to group­think and will still go down blind al­leys or sim­ply lack rel­e­vant ex­per­tise. (In this re­spect, the Mil­len­nial Maths in are highly un­re­al­is­tic; any group of aca­d­e­mics which clos­eted them­selves for a mil­len­nium would over­whelm­ingly likely be a sheer waste of hu­man cap­i­tal.)

One might won­der about other kinds of ed­u­ca­tion in the HTC like math­e­mat­ics, but all the above points ap­ply to any rea­son to live in the HTC: why would you ac­cept all those bur­dens to spend a year learn­ing some­thing… when you could just live that same year in the real world at much less cost and a far higher stan­dard of liv­ing? It cer­tainly would be nice to go into the HTC for a few weeks and come back with a dozen PhD­s—but not if you emerge aged 40, hav­ing lived the best years of your life in a prison cell and prob­a­bly deep in debt too!

Aging

Speak­ing of a few years in the HTC, what about bi­o­log­i­cal ag­ing? Peo­ple don’t or­di­nar­ily spend half their lives ac­quir­ing mul­ti­ple de­grees out­side the prison of an HTC, why would they vol­un­tar­ily do so in­side it? It’s the same trade, after all: half your life for mul­ti­ple de­grees. This point has been made in fic­tional treat­ments of time-ac­cel­er­a­tion clas­sic short story “The Six Fin­gers of Time” where the pro­tag­o­nist is given the abil­ity to slow down time by a mys­te­ri­ous an­cient con­spir­a­cy, and while he tries to un­cover their se­crets, he dies of old age—they had let him slow down time, but not his in­her­ent nat­ural ag­ing. Or YA novel , where the orig­i­nal dis­cov­erer of the HTC dies be­fore his fam­ily ex­pects it, look­ing sus­pi­ciously like an old man; the im­pli­ca­tion is that he spent so much time in the HTC in­ves­ti­gat­ing it that the nor­mal ag­ing while in­side it used up a good chunk of his lifes­pan. Iron­i­cal­ly, if he had been able to sur­vive an­other mon­th, he would have seen the res­o­lu­tion of the mys­tery. The pro­tag­o­nist ben­e­fits some­what from his own year in the HTC, but for idio­syn­cratic rea­sons. Sim­i­larly in the pro­tag­o­nist dis­cov­ers a method of time ac­cel­er­a­tion which she re­solves to use to save her world from cer­tain doom by launch­ing a gen­er­a­tion ship to be ac­cel­er­ated and dis­cover some sal­va­tion; but she and the first gen­er­a­tion (a good chunk of their world’s sci­en­tific com­mu­ni­ty) fully ex­pect to per­ish of age long be­fore the gen­er­a­tion ship re­turns just years later in re­al­time. In a more con­ven­tional ex­am­ple, we may ad­mire how prison & re­venge give the Count in a great deal of fo­cus dur­ing his ed­u­ca­tional prison stay but how many of us would agree to be im­pris­oned the same way if there were no hid­den for­tune wait­ing for us at the end?

This is a fun­da­men­tal is­sue and prob­a­bly why time-ac­cel­er­a­tion is a un­der­used trope in sci­ence fic­tion com­pared to time di­la­tion or time trav­el: the down­side is sim­ply too ap­par­ent.

Some uses

Zero-sum competition

The point about the HTC ‘re­ar­rang­ing rel­a­tive time’ for ath­letes and the orig­i­nal use in DBZ—train­ing to save the world when every minute counts—­sug­gests one class of prob­lems: things which are ex­tremely time-sen­si­tive with mul­ti­ple com­pet­ing groups and ze­ro-sum or win­ner-take-all dy­nam­ics.

With large sums of money at stake, we can hand-wave the su­per-su­per­max prison points: oil com­pa­nies only have to pay oil rig work­ers a few score or hun­dreds of thou­sands of dol­lars for work­ing sev­eral weeks at a time on oil rigs, and sci­en­tists and as­tro­nauts com­pete for po­si­tion in iso­lated fa­cil­i­ties as Antarc­tic bases or the In­ter­na­tional Space Sta­tion. All of these are far less iso­lated or bur­den­some than the HTC, but by more than a few fac­tors? Seems un­like­ly. So a few mil­lion dol­lars may suffice to cover the costs of a small group, es­pe­cially if they can reuse in­fra­struc­ture from pre­vi­ous days/years.

Are there busi­ness prob­lems where a year’s head­start is worth at least a few mil­lion dol­lars? Sure! Many pro­gram­ming tasks come to mind: would Google pay a few mil­lion to lock up the core An­droid coders to take care of a years’ worth of out­stand­ing bugs and to-do items? Would Ap­ple do some­thing sim­i­lar? What about any hedge fund? It seems plau­si­ble that every day of a HTC could be booked or even auc­tioned off.

The neg­a­tives here in­clude the lack of com­mu­ni­ca­tion and it­er­a­tion: when the group heads through the door, that’s the last they’ll hear from the world for a year. They can’t re­lease a pro­to­type at the 6 month mark and see how it does after a month. If they fall to group­think and take the wrong ap­proach, there are no out­siders who will say that their ap­proach is crazy and elab­o­rate and why don’t they just do the stan­dard thing? Worse, if they dis­cover they for­got a key piece of doc­u­men­ta­tion or hard­ware or they run out of chips or they need a par­tic­u­lar ex­pert or some­thing, the next time they can get it is… a year lat­er. Oops. Hope that was­n’t a fa­tal er­ror. (Even if they had com­mu­ni­ca­tion, it’d only bound the loss­es: if it takes a sec­ond to load a web­page in re­al­time, then it will take them >365 sec­onds or >6 min­utes.) This lack of it­er­a­tion runs counter to many busi­ness styles and is en­tirely an­ti­thet­i­cal to mod­ern tech busi­nesses which prize con­stant feed­back and abil­ity to change ideas & ap­proaches on a dime.

Still, with mul­ti­-ter­abyte hard dri­ves, one could just take a copy of all doc­u­men­ta­tion and source code (or per­haps bring along a few In­ter­net Archive-style “petaboxes” and store a copy of a small frac­tion of the In­ter­net), and for some tasks like stock­-mar­ket re­search, it’s plau­si­ble one could bring every­thing one needs. Hedge funds would prob­a­bly ben­e­fit from be­ing able to send in their quants for a year of con­cen­trated re­search and scoop the com­pe­ti­tion.

Non-zero-sum uses

The more con­crete a field, the less the ben­e­fit. Most com­mer­cial ser­vices would be im­pos­si­ble: you can’t cut some­one’s hair in the real world from the HTC, al­though with loads of equip­ment you could work on a ro­bot which cuts hair. You can’t run clin­i­cal drug ex­per­i­ments on a group of pa­tients from in­side a HTC ei­ther; for that mat­ter, you’ll have a hard time bring­ing along rats or mon­keys. But you could read a lot of pa­pers on rats. (But not nec­es­sar­ily do much; for ex­am­ple, meta-analy­ses will be hard be­cause fre­quently au­thors do not in­clude the ex­act num­bers one needs, and so one has to con­tact them—ex­actly what can’t be done in the HTC.) Pure math­e­mati­cians might ben­e­fit, but by and large, math­e­mat­ics is not so com­pet­i­tive & time-sen­si­tive that stick­ing some math­e­mati­cians into the HTC would be worth the pre­mi­um.

Which is not to say there are no con­crete us­es. One cute ex­am­ple would be stor­age of goods: in­stead of an art & wine , just stick your wine & cheese & other goods in the HTC and let them age a year every day un­til ripened to per­fec­tion. More valu­ably, one could by­pass tests and just age a prod­uct di­rect­ly; want to know if the will work or the be read­able for 10,000 years un­der ideal con­di­tions? That’s just 10,000 days or 28 years away. (We could also ex­pect an efflo­res­cence of coun­ter­feit art, doc­u­ments, and goods for the same rea­son­s.) Bet­ter yet, want to run fast pri­mate ag­ing ex­per­i­ments? If you can front the money and ei­ther au­to­mate the care & feed­ing of the sub­jects or find lab tech­ni­cians will­ing to spend their lives in pris­on, you can run as many as you please.

These would­n’t be rev­o­lu­tion­ary im­prove­ments, though (with the ex­cep­tion of ag­ing re­search which might rev­o­lu­tion­ize hu­man so­ci­ety if the re­sults were use­ful).

Self-contained vs not

More gen­er­al­ly, we could say that ap­plies to use of HTC: any task has se­r­ial and par­al­lel el­e­ments, but if some el­e­ments are made cheaper or even free, the time to ac­com­plish the task still de­pends on the other bot­tle­neck el­e­ments. El­e­ments which can be done in com­plete iso­la­tion and which ben­e­fit from rel­a­tive speedups cor­re­spond to par­al­lel el­e­ments, and el­e­ments which must be done in the real world cor­re­spond to the se­r­ial el­e­ments. With a HTC, the HTC-elements will quickly speed up, but tasks will now bot­tle­neck on re­al-world tasks. (Imag­ine Google sends its An­droid pro­gram­mers into the HTC and they re­turn a day later bear­ing a repos­i­tory groan­ing with new patch­es; the fea­tures still have to be tested in a real world con­text, re­viewed, in­fra­struc­ture up­dat­ed, and fi­nally ac­tu­ally trans­mit­ted to the cus­tomers who may be­gin us­ing them.)

One op­por­tu­nity is to look at Am­dahl’s law as a pos­i­tive, and look at the com­puter ver­sion of an iso­lated team in the HTC beaver­ing away on a pro­ject: a bunch of servers work­ing on an ex­tremely hard se­r­ial prob­lem. For ex­am­ple, sim­u­lat­ing a long evo­lu­tion of . Many prob­lems in sci­en­tific com­pu­ta­tion or where there is more than a day’s mar­gin might also ben­e­fit from what is effec­tively a su­per-fast proces­sor with 1-day la­ten­cy. Fur­ther, such a su­per­com­put­ing fa­cil­ity in an HTC faces prob­lems with re­place­ment parts and get­ting the elec­tric­ity such com­pu­ta­tions will con­sume (and how much would you have to pay the sysad­mins & tech­ni­cians to be im­pris­oned for a year?), and so its ca­pac­ity will come at a pre­mium com­pared to the equiv­a­lent re­al-world prob­lem; a prob­lem like hash crack­ing which is triv­ially par­al­leliz­able would not ben­e­fit from such a fa­cil­i­ty. Elec­tric­ity is the dom­i­nant cost of com­put­ing power these days, so a HTC must save on elec­tric­ity or jus­tify its cost pre­mi­um. In­stead of throw­ing one re­ally ex­pen­sive HTC server at the prob­lem for a year, throw 365 cheap pow­er-effi­cient servers at it for a day as many tech com­pa­nies are able to do, or just run it on a cloud com­put­ing plat­form.

These points do not ap­ply to any com­pu­ta­tion which is in­her­ently se­r­ial and can­not be run on more than a few com­put­ers. One such cat­e­gory of non-par­al­leliz­able prob­lems (as­sum­ing NC ≠ P) is the com­plex­ity class , which in­cludes such eco­nom­i­cally im­por­tant tasks as op­ti­miza­tion. How­ev­er, the im­por­tant un­par­al­leliz­able prob­lems (at least lin­ear pro­gram­ming) typ­i­cally have very fast ap­prox­i­mate or heuris­tic solvers, and op­ti­miza­tion prob­lems tend to as­ymp­tote and ex­pe­ri­ence se­verely . Is there a prob­lem where the time-limit is so tight and the ad­di­tional op­ti­miza­tion so valu­able that it would pay for a year of pre­mium power con­sump­tion & com­pu­ta­tion? I don’t know. Maybe there is.

So, many busi­ness ap­pli­ca­tions would not ben­e­fit, many re­search tasks would not ben­e­fit, and I haven’t thought of any im­por­tant ar­eas of life which would ben­e­fit from a HTC. Some peo­ple would find it con­ve­nient to re-arrange their lives even at some cost, ar­rang­ing big blocks of time for some self­-con­tained things (for ex­am­ple, work­ing on one’s own pro­ject­s), but the ben­e­fit would be lim­it­ed; I would analo­gize to , which can be em­ployed to free up a block of 8 hours (skip­ping a night of sleep) but at a cost (money). If one be­lieves that modafinil use comes with no health penal­ties or re­cov­ery sleep, it ar­guably is bet­ter than a HTC be­cause it can be used in more con­ve­nient chunks and you re­main in the real world while us­ing it, run­ning at re­al­time. Yet, while modafinil is pop­u­lar among a few groups, it has not rev­o­lu­tion­ized the world.

In gen­er­al, a world with one or many HTCs would look a great deal like our own, al­though in some ar­eas, there will be sud­den bursts of progress as HTC groups re­turn from their ex­pe­di­tions with their prizes and likely a one-time eco­nomic boost as HTC-specific ap­pli­ca­tions are dis­cov­ered.

Real HTCs

Well, cute and in­ter­est­ing, but why do we care about this SF trope from DBZ?

Be­cause the HTC can be analo­gized to an em­u­lated brain or an “up­load”! The 365x speedup of peo­ple in the HTC could be the speedup of a brain on a su­per­com­puter after con­sid­er­able op­ti­miza­tion2. (One could ar­gue that early up­loads will run at far less than re­al-time as they will be cre­ated as soon as hard­ware is just pow­er­ful enough to run them at all, and be com­pletely un­com­pet­i­tive & re­search pro­jects; but then again, their cre­ation could come long after the hard­ware ex­ists, wait­ing on bot­tle­necks like scan­ning of brain­s—the “hard­ware over­hang” ques­tion.) A com­put­er, like a HTC, can­not be nested to give a speedup; a vir­tual com­puter will usu­ally rule slower rel­a­tive to re­al­time. A brain on a com­puter with­out any pe­riph­er­als like a ro­bot will be iso­lated from the real world, just like the peo­ple in the cham­ber, and so on.

We found the HTC not use­ful in prac­tice; does this con­clu­sion also fol­low for up­loads? Should we ex­pect up­loads to strug­gle in the mar­ket­place, find­ing val­ued niches but not caus­ing in­creases in world GDP growth rates or any sort of Sin­gu­lar­i­ty?

Emulations are not HTCs

While the sim­i­lar­i­ties are strik­ing, so are the dis­sim­i­lar­i­ties:

  1. a com­puter can have com­mu­ni­ca­tions with the In­ter­net & world; a 365x slow­down may be painful, but it is bet­ter than a fixed de­lay of 0–365 days.

    Even when the slow­down hits, there is the op­tion—­much re­duced in the HTC—of switch­ing to an en­tirely differ­ent task. (Sim­i­lar to the com­put­ing world’s re­ac­tion to clock speed stag­na­tion and the rise of mul­ti­-cores, with the at­ten­dant pres­sure on Am­dahl’s law3: process-level par­al­lelism. You can’t do just one thing faster, so you might as well do many things slow­er.) This elim­i­nates many of the ob­jec­tions. If it re­ally can’t find any­thing to do with its time, an em­u­la­tion can al­ways slow it­self down to re­al-time.

  2. the over­head of liv­ing in the Hy­per­bolic Time Cham­ber is re­duced; a com­puter in the real world ben­e­fits from all the real world in­fra­struc­ture like power plants or semi­con­duc­tor chip fabs. There are some power sav­ings from , but there’s not much rea­son to oth­er­wise run as fast as pos­si­ble. (This per­mits many more minds to run sped-up as com­pared to hu­mans liv­ing in the HTC, re­duc­ing fur­ther the dis­ad­van­tage of #1 and also in­creas­ing the value of be­ing sped-up.)

  3. A per­son in the HTC is a rel­a­tively fixed quan­ti­ty, es­pe­cially since many re­sources will be un­avail­able; an em­u­lated brain has ac­cess to those re­sources per #1, but also has many op­tions differ­ent from a reg­u­lar hu­man. (A much-dis­cussed top­ic; see eg. .)

  4. An em­u­lated brain is free of a ma­jor time limit for reg­u­lar hu­mans: ag­ing. While a hu­man could not afford to get 12 PhDs even if a HTC ex­ist­ed—be­cause that would con­sume the most pro­duc­tive decades of his life—an em­u­lated brain could. This breaks the sym­me­try fur­ther.

Be­tween these 4 dis­ana­logic points, an up­load avoids some of the dis­ad­van­tages that ren­ders the HTC non­com­pet­i­tive and gains some ad­van­tages which may make it more com­pet­i­tive, and make the sud­den im­prove­ments a much more gen­eral phe­nom­e­non.

Ex­pect­ing any dra­matic changes from up­loads or AGIs in gen­eral has been mocked by crit­ics as an over-valu­ing of “brains in a box” or, pace Gene Wolfe, mag­i­cal think­ing (“The would-be sor­cerer alone has faith in the effi­cacy of pure knowl­edge”). If we look at such crit­i­cism, do the ar­gu­ments seem to as­sume a model of think­ing in which the upload/AI is trapped in the HTC, or does it re­sem­ble an upload/AI out­side the HTC?

See Also


  1. The in­abil­ity to cri­tique your own re­sults or ideas as ca­pa­bly as some­one else can seems to have deep roots in psy­chol­ogy and sup­port evo­lu­tion­ary ac­counts of rea­son­ing as evolved pri­mar­ily for ar­gu­ing and con­vinc­ing other peo­ple, not truth-seek­ing. See also “rub­ber-duck­ing”.↩︎

  2. Al­though it’s un­likely that the ex­act speedup would be near 365x, as power & heat­ing con­straints dom­i­nate the prob­lem; Ger­ald­Mon­roe points out that a straight­for­ward com­par­i­son of tran­sis­tor vs neu­ron switch­ing speed leads to fac­tors like 25 mil­lion, and mod­ern CPUs are lim­ited in speed mostly by heat dis­si­pa­tion is­sues—the stan­dard 2/4GHz CPU could run at 5GHz+ if one had pow­er­ful cool­ing. Heat con­cerns led to ar­gue that dat­a­cen­ters of up­loaded brains would even­tu­ally re­lo­cate to the deep sea for max­i­mal cool­ing (and hence, speed).↩︎

  3. Am­dahl’s law is also rel­e­vant to the eco­nom­ics of up­loaded brains: sup­pose one be­lieved that spe­cial­ized or “tool” AIs will al­ways out­per­form any up­loaded brain or AGI at a spe­cific task, and every im­prove­ment that speeds up the uploads/AGIs im­proves the tool AIs just as much, such that the uploads/AGIs never sur­pass the tool AI; does this im­ply that there will be no uploads/AGIs out­side niches like re­search, as hu­mans us­ing tool AIs are more profitable? Holden Karnof­sky seems to think some­thing sim­i­lar when he does­n’t think that com­pet­i­tive pres­sure will force peo­ple run­ning tool AIs to even­tu­ally switch to run­ning AGIs; Nick Sz­abo ex­plic­itly be­lieves uploads/AGIs can never be profitable given tool AI com­pe­ti­tion:

    Even if there was such a thing as a “gen­eral in­tel­li­gence” the spe­cial­ized ma­chines would soundly beat it in the mar­ket­place. It would be far from a close con­test.

    I dis­agree. The mar­ket is not purely tool AI vs AGI. Hu­mans do not in­crease their speed even if tool AIs are in­creas­ing their speed ar­bi­trar­i­ly. There­fore, a hu­man+­tool-AI sys­tem’s per­for­mance as­ymp­tot­i­cally ap­proaches the limit where the tool-AI part takes zero time and the hu­man part takes 100% of the time. Time pres­sures may force a shift to ever more tool AI sys­tems and even­tu­ally tool-AI+AGI sys­tems when that be­comes pos­si­ble. (“Greater use of highly adapt­able and flex­i­bly au­tonomous sys­tems and processes can pro­vide [sub­stan­tial] time-do­main op­er­a­tional ad­van­tages over ad­ver­saries who are lim­ited to hu­man plan­ning and de­ci­sion speeds…”) The mo­ment that al­go­rith­mic progress or Moore’s law means that an AGI even slightly out­per­forms a hu­man at us­ing the tool-AI, the same eco­nomic rea­sons you were count­ing on as your sal­va­tion sud­denly turn on you and drive the re­place­ment of any hu­mans in the loop. Since hu­mans are a known fixed quan­ti­ty, if an AGI can be im­proved—even if at all times it is strictly in­fe­rior to a tool AI at the lat­ter’s spe­cial­iza­tion—then even­tu­ally an AGI+tool-AI sys­tem will out­per­form a hu­man+­tool-AI sys­tem (bar­ring ex­otic un­proven as­sump­tions about as­ymp­totic lim­it­s).

    At­tempts to evade this by split­ting up or com­bin­ing tool AIs ei­ther don’t avoid this logic or wind up ac­cept­ing the con­clu­sion: if every hu­man skill has been trans­ferred to tool-AIs, then a com­plex of tool-AIs now forms an AGI which out­per­forms all hu­mans by de­fi­n­i­tion; if not every hu­man skill has been trans­ferred, such as “em­ploy­ing tool-AIs as most ap­pro­pri­ate for the mo­ment”, then there is the large eco­nomic niche for AGIs which I have iden­ti­fied with my Am­dahl’s law ar­gu­ment. So ei­ther there ex­ist AGI which out­per­form all hu­mans, or there ex­ists eco­nomic pres­sure to use AGI. For ex­am­ple, if one ar­gued that a com­plex of tool-AIs would not share world­views or data ap­pro­pri­ately and need a hu­man to co­or­di­nate them, well, why can’t an AGI do this and be su­pe­rior to the hu­mans per Am­dahl’s law?

    What hu­man is in the loop on high fre­quency trad­ing? Who was in the loop when Knight Cap­i­tal’s mar­ket maker was los­ing hun­dreds of mil­lions of dol­lars? The an­swer is that no one was in the loop be­cause hu­mans in the loop would not have been eco­nom­i­cally com­pet­i­tive. That’s fine when it’s “just” bil­lions of dol­lars at stake and com­pa­nies can de­cide to take the risk for them­selves or not—but the stakes can change, ex­ter­nal­i­ties can in­crease.

    Here’s an­other near-fu­ture test/example: how do we hu­mans deal with ? Drones are ex­plod­ing in pop­u­lar­i­ty, are in­creas­ing their ca­pa­bil­i­ties con­stant­ly, and are cov­eted by count­less se­cu­rity agen­cies and pri­vate groups for their tremen­dous use in all sorts of roles both be­nign and dis­turb­ing. Just like AIs would be. The tool vs gen­eral AI dis­tinc­tion maps nicely onto drones as well: a tool AI cor­re­sponds to a drone be­ing man­u­ally flown by a hu­man pi­lot some­where, while a gen­eral AI would cor­re­spond to an au­tonomous drone which is car­ry­ing out some mis­sion (blast in­sur­gents?). So, here is a near-fu­ture test of the ques­tion ‘are peo­ple likely to let tool AIs ’drive them­selves’ for greater effi­cien­cy?’—sim­ply ask whether in, say, a decade there are au­tonomous drones car­ry­ing tasks that now would only be car­ried out by pi­loted drones. If in a decade we learn that au­tonomous drones are killing peo­ple, then we have an an­swer to our tool AI ques­tion: it does­n’t mat­ter be­cause given a tool AI, peo­ple will just turn it into a gen­eral AI.↩︎