The Hyperbolic Time Chamber & Brain Emulation

A time dilation chamber as thought experiment on the power of pure thought, with comparison to computer AGI advantages/disadvantages.
transhumanism, anime, AI, SF
2012-08-292018-09-02 finished certainty: likely importance: 9


A time dila­tion tool from an anime is dis­cussed for its prac­ti­cal use on Earth; there seem sur­pris­ingly few uses and none that will change the world, due to the severe penal­ties humans would incur while using it, and basic con­straints like Amdahl’s law limit the sci­en­tific uses. A com­par­i­son with the posi­tion of an Arti­fi­cial Intel­li­gence such as an emu­lated human brain seems fair, except most of the time dila­tion dis­ad­van­tages do not apply or can be ame­lio­rated and hence any speedups could be quite effec­tively exploit­ed. I sug­gest that skep­tics of the idea that speedups give advan­tages are implic­itly work­ing off the crip­pled time dila­tion tool and not mak­ing allowance for the disanalo­gies.

, a pop­u­lar (but medioc­re) shonen fight­ing ani­me, includes a cute bit of SF in it—the Hyper­bolic Time Cham­ber (HTC), which can be thought of a reverse in which time speeds up for 2 peo­ple: the HTC opens and closes once a day real­time, but inside its lit­tle pocket uni­verse, a full year passes for 2 peo­ple, giv­ing a 365x speedup. There’s no such thing as a HTC-within-a-HTC, so uses are lim­ited to just that one 365x speedup—no 3652 speedups.

Ignor­ing the DBZ-spe­cific aspects of the HTC like the per­son limit or increased grav­ity or air and tem­per­a­ture changes, one won­ders: what one would do with a HTC in the real world?

Uses for time acceleration

In DBZ, the only use seems to be for train­ing mon­tages to let char­ac­ters power up to fight a new alien or mar­tial artist, but world-de­stroy­ing mar­tial artists seem to be rare in the real world, so we could­n’t use it for that.

Could we use it for reg­u­lar mar­tial arts train­ing? The DBZ arti­cle for the HTC men­tions no non-e­mer­gency use, and a lit­tle thought leads us to con­clude that, prob­a­bly not: inside the HTC, time passes as nor­mal, which means that you don’t save any time. All the HTC is doing is rear­rang­ing rel­a­tive time between groups. If you step in, you still age a full year before step­ping out, and you will now die a year early by the real­time cal­en­dar. So what’s the point? We can think of a few uses—imag­ine some­one who gets injured just before the Olympic­s—but let’s face it, that may be con­ve­nient for a few peo­ple, but it’s hardly chang­ing the world. A whole time-ac­cel­er­ated pocket uni­verse… Surely we can think of some­thing less point­less than tweak­ing ath­let­ics?

Downsides

Actu­al­ly, it’s worse than point­less—the HTC is a super-: you can­not leave at any point before the year is up, you can­not com­mu­ni­cate in any way, noth­ing goes in or out, and you have only what you brought with you (and always what you brought with you).

Autarchy

Under such con­di­tions, a year in the HTC could well be con­sid­ered “cruel and unusual pun­ish­ment”; no doc­tor would vol­un­teer for it, so any pris­on­ers in the HTC face a seri­ous risk of death from any cause unless a fel­low pris­oner that day/year had med­ical train­ing. The food will also suck, as any food will have to be stor­able for a year (it would­n’t do to starve to death a day before the door opens back to the real world); you can’t grow your own because there is no appar­ent sun, soil, or sea­sons inside the HTC, and while you could prob­a­bly bring in a green­house & soil and fer­til­ize with recy­cled food, how are you going to power the lights in your green­house? Drag in a minia­ture nuclear power plant or a ? And we haven’t even con­sid­ered how much time one would spend (waste) on this smal­l­-s­cale agri­cul­ture; main­tain­ing was a ful­l-­time job for many high­ly-skilled peo­ple, and if that was true for the HTC, there would hardly be any point in it.

If one punted on the prob­lems of main­tain­ing a high qual­ity of life and posited a ded­i­cated researcher, well, the HTC is still not use­ful. They will find it hard to take with them an entire library or lab­o­ra­to­ry, many ingre­di­ents are too expen­sive or per­ish­able to buy in advance just because the researcher might need them, but if they don’t have access to pretty much every­thing, they’ll quickly hit some sort of bar­rier where one email or order would let them fin­ish a project but that email can’t be sent for up to a year. (Imag­ine a researcher who enters the HTC—and his lap­top’s hard-­drive dies. Oops. Hope he had back­ups or spares, of his data and his lap­top and every­thing else for that mat­ter.) Omit­ting these con­cerns, research is a social process in the sense that one is often dis­cussing or explain­ing or defend­ing the research, and with­out these inter­ac­tions it is easy to go down blind alleys, make minor-seem­ing but fatal mis­takes1, wind up rein­vent­ing some­thing stan­dard in another field, etc. One can eas­ily waste a month this way, and so a year. A group would help, but groups are sus­cep­ti­ble to group­think and will still go down blind alleys or sim­ply lack rel­e­vant exper­tise. (In this respect, the Mil­len­nial Maths in are highly unre­al­is­tic; any group of aca­d­e­mics which clos­eted them­selves for a mil­len­nium would over­whelm­ingly likely be a sheer waste of human cap­i­tal.)

One might won­der about other kinds of edu­ca­tion in the HTC like math­e­mat­ics, but all the above points apply to any rea­son to live in the HTC: why would you accept all those bur­dens to spend a year learn­ing some­thing… when you could just live that same year in the real world at much less cost and a far higher stan­dard of liv­ing? It cer­tainly would be nice to go into the HTC for a few weeks and come back with a dozen PhD­s—but not if you emerge aged 40, hav­ing lived the best years of your life in a prison cell and prob­a­bly deep in debt too!

Aging

Speak­ing of a few years in the HTC, what about bio­log­i­cal aging? Peo­ple don’t ordi­nar­ily spend half their lives acquir­ing mul­ti­ple degrees out­side the prison of an HTC, why would they vol­un­tar­ily do so inside it? It’s the same trade, after all: half your life for mul­ti­ple degrees. This point has been made in fic­tional treat­ments of time-ac­cel­er­a­tion clas­sic short story “The Six Fin­gers of Time” where the pro­tag­o­nist is given the abil­ity to slow down time by a mys­te­ri­ous ancient con­spir­a­cy, and while he tries to uncover their secrets, he dies of old age—they had let him slow down time, but not his inher­ent nat­ural aging. Or YA novel , where the orig­i­nal dis­cov­erer of the HTC dies before his fam­ily expects it, look­ing sus­pi­ciously like an old man; the impli­ca­tion is that he spent so much time in the HTC inves­ti­gat­ing it that the nor­mal aging while inside it used up a good chunk of his lifes­pan. Iron­i­cal­ly, if he had been able to sur­vive another mon­th, he would have seen the res­o­lu­tion of the mys­tery. The pro­tag­o­nist ben­e­fits some­what from his own year in the HTC, but for idio­syn­cratic rea­sons. Sim­i­larly in the pro­tag­o­nist dis­cov­ers a method of time accel­er­a­tion which she resolves to use to save her world from cer­tain doom by launch­ing a gen­er­a­tion ship to be accel­er­ated and dis­cover some sal­va­tion; but she and the first gen­er­a­tion (a good chunk of their world’s sci­en­tific com­mu­ni­ty) fully expect to per­ish of age long before the gen­er­a­tion ship returns just years later in real­time. In a more con­ven­tional exam­ple, we may admire how prison & revenge give the Count in a great deal of focus dur­ing his edu­ca­tional prison stay but how many of us would agree to be impris­oned the same way if there were no hid­den for­tune wait­ing for us at the end?

This is a fun­da­men­tal issue and prob­a­bly why time-ac­cel­er­a­tion is a under­used trope in sci­ence fic­tion com­pared to time dila­tion or time trav­el: the down­side is sim­ply too appar­ent.

Some uses

Zero-sum competition

The point about the HTC ‘rear­rang­ing rel­a­tive time’ for ath­letes and the orig­i­nal use in DBZ—train­ing to save the world when every minute counts—­sug­gests one class of prob­lems: things which are extremely time-sen­si­tive with mul­ti­ple com­pet­ing groups and zero-­sum or win­ner-­take-all dynam­ics.

With large sums of money at stake, we can hand-wave the super-­su­per­max prison points: oil com­pa­nies only have to pay oil rig work­ers a few score or hun­dreds of thou­sands of dol­lars for work­ing sev­eral weeks at a time on oil rigs, and sci­en­tists and astro­nauts com­pete for posi­tion in iso­lated facil­i­ties as Antarc­tic bases or the Inter­na­tional Space Sta­tion. All of these are far less iso­lated or bur­den­some than the HTC, but by more than a few fac­tors? Seems unlike­ly. So a few mil­lion dol­lars may suf­fice to cover the costs of a small group, espe­cially if they can reuse infra­struc­ture from pre­vi­ous days/years.

Are there busi­ness prob­lems where a year’s head­start is worth at least a few mil­lion dol­lars? Sure! Many pro­gram­ming tasks come to mind: would Google pay a few mil­lion to lock up the core Android coders to take care of a years’ worth of out­stand­ing bugs and to-do items? Would Apple do some­thing sim­i­lar? What about any hedge fund? It seems plau­si­ble that every day of a HTC could be booked or even auc­tioned off.

The neg­a­tives here include the lack of com­mu­ni­ca­tion and iter­a­tion: when the group heads through the door, that’s the last they’ll hear from the world for a year. They can’t release a pro­to­type at the 6 month mark and see how it does after a month. If they fall to group­think and take the wrong approach, there are no out­siders who will say that their approach is crazy and elab­o­rate and why don’t they just do the stan­dard thing? Worse, if they dis­cover they for­got a key piece of doc­u­men­ta­tion or hard­ware or they run out of chips or they need a par­tic­u­lar expert or some­thing, the next time they can get it is… a year lat­er. Oops. Hope that was­n’t a fatal error. (Even if they had com­mu­ni­ca­tion, it’d only bound the loss­es: if it takes a sec­ond to load a web­page in real­time, then it will take them >365 sec­onds or >6 min­utes.) This lack of iter­a­tion runs counter to many busi­ness styles and is entirely anti­thet­i­cal to mod­ern tech busi­nesses which prize con­stant feed­back and abil­ity to change ideas & approaches on a dime.

Still, with mul­ti­-ter­abyte hard dri­ves, one could just take a copy of all doc­u­men­ta­tion and source code (or per­haps bring along a few Inter­net Archive-style “petaboxes” and store a copy of a small frac­tion of the Inter­net), and for some tasks like stock­-­mar­ket research, it’s plau­si­ble one could bring every­thing one needs. Hedge funds would prob­a­bly ben­e­fit from being able to send in their quants for a year of con­cen­trated research and scoop the com­pe­ti­tion.

Non-zero-sum uses

The more con­crete a field, the less the ben­e­fit. Most com­mer­cial ser­vices would be impos­si­ble: you can’t cut some­one’s hair in the real world from the HTC, although with loads of equip­ment you could work on a robot which cuts hair. You can’t run clin­i­cal drug exper­i­ments on a group of patients from inside a HTC either; for that mat­ter, you’ll have a hard time bring­ing along rats or mon­keys. But you could read a lot of papers on rats. (But not nec­es­sar­ily do much; for exam­ple, meta-­analy­ses will be hard because fre­quently authors do not include the exact num­bers one needs, and so one has to con­tact them—ex­actly what can’t be done in the HTC.) Pure math­e­mati­cians might ben­e­fit, but by and large, math­e­mat­ics is not so com­pet­i­tive & time-sen­si­tive that stick­ing some math­e­mati­cians into the HTC would be worth the pre­mi­um.

Which is not to say there are no con­crete uses. One cute exam­ple would be stor­age of goods: instead of an art & wine , just stick your wine & cheese & other goods in the HTC and let them age a year every day until ripened to per­fec­tion. More valu­ably, one could bypass tests and just age a prod­uct direct­ly; want to know if the will work or the be read­able for 10,000 years under ideal con­di­tions? That’s just 10,000 days or 28 years away. (We could also expect an efflo­res­cence of coun­ter­feit art, doc­u­ments, and goods for the same rea­son­s.) Bet­ter yet, want to run fast pri­mate aging exper­i­ments? If you can front the money and either auto­mate the care & feed­ing of the sub­jects or find lab tech­ni­cians will­ing to spend their lives in pris­on, you can run as many as you please.

These would­n’t be rev­o­lu­tion­ary improve­ments, though (with the excep­tion of aging research which might rev­o­lu­tion­ize human soci­ety if the results were use­ful).

Self-contained vs not

More gen­er­al­ly, we could say that applies to use of HTC: any task has ser­ial and par­al­lel ele­ments, but if some ele­ments are made cheaper or even free, the time to accom­plish the task still depends on the other bot­tle­neck ele­ments. Ele­ments which can be done in com­plete iso­la­tion and which ben­e­fit from rel­a­tive speedups cor­re­spond to par­al­lel ele­ments, and ele­ments which must be done in the real world cor­re­spond to the ser­ial ele­ments. With a HTC, the HTC-elements will quickly speed up, but tasks will now bot­tle­neck on real-­world tasks. (Imag­ine Google sends its Android pro­gram­mers into the HTC and they return a day later bear­ing a repos­i­tory groan­ing with new patch­es; the fea­tures still have to be tested in a real world con­text, reviewed, infra­struc­ture updat­ed, and finally actu­ally trans­mit­ted to the cus­tomers who may begin using them.)

One oppor­tu­nity is to look at Amdahl’s law as a pos­i­tive, and look at the com­puter ver­sion of an iso­lated team in the HTC beaver­ing away on a pro­ject: a bunch of servers work­ing on an extremely hard ser­ial prob­lem. For exam­ple, sim­u­lat­ing a long evo­lu­tion of . Many prob­lems in sci­en­tific com­pu­ta­tion or where there is more than a day’s mar­gin might also ben­e­fit from what is effec­tively a super-­fast proces­sor with 1-day laten­cy. Fur­ther, such a super­com­put­ing facil­ity in an HTC faces prob­lems with replace­ment parts and get­ting the elec­tric­ity such com­pu­ta­tions will con­sume (and how much would you have to pay the sysad­mins & tech­ni­cians to be impris­oned for a year?), and so its capac­ity will come at a pre­mium com­pared to the equiv­a­lent real-­world prob­lem; a prob­lem like hash crack­ing which is triv­ially par­al­leliz­able would not ben­e­fit from such a facil­i­ty. Elec­tric­ity is the dom­i­nant cost of com­put­ing power these days, so a HTC must save on elec­tric­ity or jus­tify its cost pre­mi­um. Instead of throw­ing one really expen­sive HTC server at the prob­lem for a year, throw 365 cheap pow­er-­ef­fi­cient servers at it for a day as many tech com­pa­nies are able to do, or just run it on a cloud com­put­ing plat­form.

These points do not apply to any com­pu­ta­tion which is inher­ently ser­ial and can­not be run on more than a few com­put­ers. One such cat­e­gory of non-­par­al­leliz­able prob­lems (as­sum­ing NC ≠ P) is the com­plex­ity class , which includes such eco­nom­i­cally impor­tant tasks as opti­miza­tion. How­ev­er, the impor­tant unpar­al­leliz­able prob­lems (at least lin­ear pro­gram­ming) typ­i­cally have very fast approx­i­mate or heuris­tic solvers, and opti­miza­tion prob­lems tend to asymp­tote and expe­ri­ence severely . Is there a prob­lem where the time-limit is so tight and the addi­tional opti­miza­tion so valu­able that it would pay for a year of pre­mium power con­sump­tion & com­pu­ta­tion? I don’t know. Maybe there is.

So, many busi­ness appli­ca­tions would not ben­e­fit, many research tasks would not ben­e­fit, and I haven’t thought of any impor­tant areas of life which would ben­e­fit from a HTC. Some peo­ple would find it con­ve­nient to re-arrange their lives even at some cost, arrang­ing big blocks of time for some self­-­con­tained things (for exam­ple, work­ing on one’s own pro­ject­s), but the ben­e­fit would be lim­it­ed; I would analo­gize to , which can be employed to free up a block of 8 hours (skip­ping a night of sleep) but at a cost (money). If one believes that modafinil use comes with no health penal­ties or recov­ery sleep, it arguably is bet­ter than a HTC because it can be used in more con­ve­nient chunks and you remain in the real world while using it, run­ning at real­time. Yet, while modafinil is pop­u­lar among a few groups, it has not rev­o­lu­tion­ized the world.

In gen­er­al, a world with one or many HTCs would look a great deal like our own, although in some areas, there will be sud­den bursts of progress as HTC groups return from their expe­di­tions with their prizes and likely a one-­time eco­nomic boost as HTC-specific appli­ca­tions are dis­cov­ered.

Real HTCs

Well, cute and inter­est­ing, but why do we care about this SF trope from DBZ?

Because the HTC can be analo­gized to an emu­lated brain or an “upload”! The 365x speedup of peo­ple in the HTC could be the speedup of a brain on a super­com­puter after con­sid­er­able opti­miza­tion2. (One could argue that early uploads will run at far less than real-­time as they will be cre­ated as soon as hard­ware is just pow­er­ful enough to run them at all, and be com­pletely uncom­pet­i­tive & research pro­jects; but then again, their cre­ation could come long after the hard­ware exists, wait­ing on bot­tle­necks like scan­ning of brain­s—the “hard­ware over­hang” ques­tion.) A com­put­er, like a HTC, can­not be nested to give a speedup; a vir­tual com­puter will usu­ally rule slower rel­a­tive to real­time. A brain on a com­puter with­out any periph­er­als like a robot will be iso­lated from the real world, just like the peo­ple in the cham­ber, and so on.

We found the HTC not use­ful in prac­tice; does this con­clu­sion also fol­low for uploads? Should we expect uploads to strug­gle in the mar­ket­place, find­ing val­ued niches but not caus­ing increases in world GDP growth rates or any sort of Sin­gu­lar­i­ty?

Emulations are not HTCs

While the sim­i­lar­i­ties are strik­ing, so are the dis­sim­i­lar­i­ties:

  1. a com­puter can have com­mu­ni­ca­tions with the Inter­net & world; a 365x slow­down may be painful, but it is bet­ter than a fixed delay of 0–365 days.

    Even when the slow­down hits, there is the option—­much reduced in the HTC—of switch­ing to an entirely dif­fer­ent task. (Sim­i­lar to the com­put­ing world’s reac­tion to clock speed stag­na­tion and the rise of mul­ti­-­cores, with the atten­dant pres­sure on Amdahl’s law3: process-level par­al­lelism. You can’t do just one thing faster, so you might as well do many things slow­er.) This elim­i­nates many of the objec­tions. If it really can’t find any­thing to do with its time, an emu­la­tion can always slow itself down to real-­time.

  2. the over­head of liv­ing in the Hyper­bolic Time Cham­ber is reduced; a com­puter in the real world ben­e­fits from all the real world infra­struc­ture like power plants or semi­con­duc­tor chip fabs. There are some power sav­ings from , but there’s not much rea­son to oth­er­wise run as fast as pos­si­ble. (This per­mits many more minds to run sped-up as com­pared to humans liv­ing in the HTC, reduc­ing fur­ther the dis­ad­van­tage of #1 and also increas­ing the value of being sped-up.)

  3. A per­son in the HTC is a rel­a­tively fixed quan­ti­ty, espe­cially since many resources will be unavail­able; an emu­lated brain has access to those resources per #1, but also has many options dif­fer­ent from a reg­u­lar human. (A much-dis­cussed top­ic; see eg. .)

  4. An emu­lated brain is free of a major time limit for reg­u­lar humans: aging. While a human could not afford to get 12 PhDs even if a HTC exist­ed—be­cause that would con­sume the most pro­duc­tive decades of his life—an emu­lated brain could. This breaks the sym­me­try fur­ther.

Between these 4 dis­ana­logic points, an upload avoids some of the dis­ad­van­tages that ren­ders the HTC non­com­pet­i­tive and gains some advan­tages which may make it more com­pet­i­tive, and make the sud­den improve­ments a much more gen­eral phe­nom­e­non.

Expect­ing any dra­matic changes from uploads or AGIs in gen­eral has been mocked by crit­ics as an over-­valu­ing of “brains in a box” or, pace Gene Wolfe, mag­i­cal think­ing (“The would-be sor­cerer alone has faith in the effi­cacy of pure knowl­edge”). If we look at such crit­i­cism, do the argu­ments seem to assume a model of think­ing in which the upload/AI is trapped in the HTC, or does it resem­ble an upload/AI out­side the HTC?

See Also


  1. The inabil­ity to cri­tique your own results or ideas as capa­bly as some­one else can seems to have deep roots in psy­chol­ogy and sup­port evo­lu­tion­ary accounts of rea­son­ing as evolved pri­mar­ily for argu­ing and con­vinc­ing other peo­ple, not truth-seek­ing. See also “rub­ber-­duck­ing”.↩︎

  2. Although it’s unlikely that the exact speedup would be near 365x, as power & heat­ing con­straints dom­i­nate the prob­lem; Ger­ald­Mon­roe points out that a straight­for­ward com­par­i­son of tran­sis­tor vs neu­ron switch­ing speed leads to fac­tors like 25 mil­lion, and mod­ern CPUs are lim­ited in speed mostly by heat dis­si­pa­tion issues—the stan­dard 2/4GHz CPU could run at 5GHz+ if one had pow­er­ful cool­ing. Heat con­cerns led to argue that dat­a­cen­ters of uploaded brains would even­tu­ally relo­cate to the deep sea for max­i­mal cool­ing (and hence, speed).↩︎

  3. Amdahl’s law is also rel­e­vant to the eco­nom­ics of uploaded brains: sup­pose one believed that spe­cial­ized or “tool” AIs will always out­per­form any uploaded brain or AGI at a spe­cific task, and every improve­ment that speeds up the uploads/AGIs improves the tool AIs just as much, such that the uploads/AGIs never sur­pass the tool AI; does this imply that there will be no uploads/AGIs out­side niches like research, as humans using tool AIs are more prof­itable? Holden Karnof­sky seems to think some­thing sim­i­lar when he does­n’t think that com­pet­i­tive pres­sure will force peo­ple run­ning tool AIs to even­tu­ally switch to run­ning AGIs; Nick Szabo explic­itly believes uploads/AGIs can never be prof­itable given tool AI com­pe­ti­tion:

    Even if there was such a thing as a “gen­eral intel­li­gence” the spe­cial­ized machines would soundly beat it in the mar­ket­place. It would be far from a close con­test.

    I dis­agree. The mar­ket is not purely tool AI vs AGI. Humans do not increase their speed even if tool AIs are increas­ing their speed arbi­trar­i­ly. There­fore, a human+­tool-AI sys­tem’s per­for­mance asymp­tot­i­cally approaches the limit where the tool-AI part takes zero time and the human part takes 100% of the time. Time pres­sures may force a shift to ever more tool AI sys­tems and even­tu­ally tool-AI+AGI sys­tems when that becomes pos­si­ble. (“Greater use of highly adapt­able and flex­i­bly autonomous sys­tems and processes can pro­vide [sub­stan­tial] time-­do­main oper­a­tional advan­tages over adver­saries who are lim­ited to human plan­ning and deci­sion speeds…”) The moment that algo­rith­mic progress or Moore’s law means that an AGI even slightly out­per­forms a human at using the tool-AI, the same eco­nomic rea­sons you were count­ing on as your sal­va­tion sud­denly turn on you and drive the replace­ment of any humans in the loop. Since humans are a known fixed quan­ti­ty, if an AGI can be improved—even if at all times it is strictly infe­rior to a tool AI at the lat­ter’s spe­cial­iza­tion—then even­tu­ally an AGI+tool-AI sys­tem will out­per­form a human+­tool-AI sys­tem (bar­ring exotic unproven assump­tions about asymp­totic lim­it­s).

    Attempts to evade this by split­ting up or com­bin­ing tool AIs either don’t avoid this logic or wind up accept­ing the con­clu­sion: if every human skill has been trans­ferred to tool-AIs, then a com­plex of tool-AIs now forms an AGI which out­per­forms all humans by def­i­n­i­tion; if not every human skill has been trans­ferred, such as “employ­ing tool-AIs as most appro­pri­ate for the moment”, then there is the large eco­nomic niche for AGIs which I have iden­ti­fied with my Amdahl’s law argu­ment. So either there exist AGI which out­per­form all humans, or there exists eco­nomic pres­sure to use AGI. For exam­ple, if one argued that a com­plex of tool-AIs would not share world­views or data appro­pri­ately and need a human to coor­di­nate them, well, why can’t an AGI do this and be supe­rior to the humans per Amdahl’s law?

    What human is in the loop on high fre­quency trad­ing? Who was in the loop when Knight Cap­i­tal’s mar­ket maker was los­ing hun­dreds of mil­lions of dol­lars? The answer is that no one was in the loop because humans in the loop would not have been eco­nom­i­cally com­pet­i­tive. That’s fine when it’s “just” bil­lions of dol­lars at stake and com­pa­nies can decide to take the risk for them­selves or not—but the stakes can change, exter­nal­i­ties can increase.

    Here’s another near-­fu­ture test/example: how do we humans deal with ? Drones are explod­ing in pop­u­lar­i­ty, are increas­ing their capa­bil­i­ties con­stant­ly, and are cov­eted by count­less secu­rity agen­cies and pri­vate groups for their tremen­dous use in all sorts of roles both benign and dis­turb­ing. Just like AIs would be. The tool vs gen­eral AI dis­tinc­tion maps nicely onto drones as well: a tool AI cor­re­sponds to a drone being man­u­ally flown by a human pilot some­where, while a gen­eral AI would cor­re­spond to an autonomous drone which is car­ry­ing out some mis­sion (blast insur­gents?). So, here is a near-­fu­ture test of the ques­tion ‘are peo­ple likely to let tool AIs ’drive them­selves’ for greater effi­cien­cy?’—sim­ply ask whether in, say, a decade there are autonomous drones car­ry­ing tasks that now would only be car­ried out by piloted drones. If in a decade we learn that autonomous drones are killing peo­ple, then we have an answer to our tool AI ques­tion: it does­n’t mat­ter because given a tool AI, peo­ple will just turn it into a gen­eral AI.↩︎