September 2021 Gwern.net newsletter with links on TODO
September 2021’s Gwern.net newsletter is now out; previous, August 2021 (archives). This is a collation of links and summary of major changes, overlapping with my Changelog; brought to you by my donors on Patreon.
As Norbert Wiener remarked of the A-bomb, the only nuclear secret worth keeping from Stalin was the secret that it was feasible: knowledge of the implementation details isn’t that important, as much work as they may take, compared to the general idea & the knowledge that it works. I am reminded of the joke about the repairman, but to rewrite it for DL: “Here’s a 10-line diff fixing your AGI; my compute-bill is 10 million petaflop-days.”; “What‽ But it only takes 0.1 million to train it!” “Yes, 0.1m to train it with the diff, and 9.90m to know which 10 lines.”↩︎