“Deep Reinforcement Learning” (Comprehensive overview of recent work on deep methods for reinforcement learning, culminating in the Atari player, with additional comments on extending to continuous games, Go, and why ‘deep models’, the apparent third member of the reinforcement learning trinity of value/policy/model learning, has failed & may be redundant. Gradient descent is magic.)
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/changes/additions.
Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular /r/Gwern subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).
A subreddit for posting links of interest and also for announcing updates to gwern.net (which can be used as a RSS feed). Submissions are categorized similar to the monthly newsletter and typically will be collated there.
A lawn is an area of soil-covered land planted with grasses and other durable plants such as clover which are maintained at a short height with a lawnmower and used for aesthetic and recreational purposes. Lawns are usually composed only of grass species, subject to weed and pest control, maintained in a green color, and are regularly mowed to ensure an acceptable length. Lawns are used around houses, apartments, commercial buildings and offices. Many city parks also have large lawn areas. In recreational contexts, the specialised names turf, pitch, field or green may be used, depending on the sport and the continent.
We combine constraints on galaxy formation histories with planet formation models, yielding the Earth-like and giant planet formation histories of the Milky Way and the Universe as a whole. In the Hubble Volume (10^13 Mpc^3), we expect there to be 10^20 Earth-like and 10^20 giant planets; our own galaxy is expected to host 10^9 and 10^10 Earth-like and giant planets, respectively. Proposed metallicity thresholds for planet formation do not significantly affect these numbers. However, the metallicity dependence for giant planets results in later typical formation times and larger host galaxies than for Earth-like planets. The Solar System formed at the median age for existing giant planets in the Milky Way, and consistent with past estimates, formed after 80 virialised dark matter haloes continues to collapse and form stars and planets, the Universe will form over 10 times more planets than currently exist. We show that this would imply at least a 92 civilisation the Universe will ever have, independent of arguments involving the Drake Equation.
Isaac Newton published few of his works, and only those he considered perfect after long delays. This leaves his system the world, as described in the Principia and elsewhere, incomplete, and many questions simply unaddressed, like the fate of the Sun or role of comets. But in 2 conversations with an admirer and his nephew, the elderly Newton sketched out the rest of his cosmogony.
According to Newton, the solar system is not stable and must be adjusted by angels; the Sun does not burn perpetually, but comets regularly fuel the Sun; and the final result is that humanity will be extinguished by a particularly large comet causing the sun to flare up, and requiring intelligent alien beings to arise on other planets or their moons. He further gives an anthropic argument: one reason we know that intelligent races regularly go extinct is that humanity itself arose only recently, as demonstrated by the recent innovations in every field, inconsistent with any belief that human beings have existed for hundreds of thousands or millions of years.
This is all interestingly wrong, particularly the anthropic argument. That Newton found it so absurd to imagine humanity existing for millions of years but only recently undergoing exponential improvements in technology demonstrates how counterintuitive and extraordinary the Industrial & Scientific Revolutions were.
Background: Results from previous studies show that the cognitive ability of offspring might be irreversibly damaged as a result of their mother’s mild iodine deficiency during pregnancy. A reduced intelligence quotient (IQ) score has broad economic and societal cost implications because intelligence affects wellbeing, income, and education outcomes. Although pregnancy and lactation lead to increased iodine needs, no UK recommendations for iodine supplementation have been issued to pregnant women. We aimed to investigate the cost-effectiveness of iodine supplementation versus no supplementation for pregnant women in a mildly to moderately iodine-deficient population for which a population-based iodine supplementation programme—for example, universal salt iodisation—did not exist.
Methods: We systematically searched MEDLINE, Embase, EconLit, and NHS EED for economic studies that linked IQ and income published in all languages until Aug 21, 2014. We took clinical data relating to iodine deficiency in pregnant women and the effect on IQ in their children aged 8–9 years from primary research. A decision tree was developed to compare the treatment strategies of iodine supplementation in tablet form with no iodine supplementation for pregnant women in the UK. Analyses were done from a health service perspective (analysis 1; taking direct health service costs into account) and societal perspective (analysis 2; taking education costs and the value of an IQ point itself into account), and presented in terms of cost (in sterling, relevant to 2013) per IQ point gained in the offspring. We made data-supported assumptions to complete these analyses, but used a conservative approach that limited the benefits of iodine supplementation and overestimated its potential harms.
Findings: Our systematic search identified 1361 published articles, of which eight were assessed to calculate the monetary value of an IQ point. A discounted lifetime value of an additional IQ point based on earnings was estimated to be £3297 (study estimates range from £1319 to £11 967) for the offspring cohort. Iodine supplementation was cost saving from both a health service perspective (saving £199 per pregnant woman [sensitivity analysis range –£42 to £229]) and societal perspective (saving £4476 per pregnant woman [sensitivity analysis range £540 to £4495]), with a net gain of 1·22 IQ points in each analysis. Base case results were robust to sensitivity analyses.
Interpretation: Iodine supplementation for pregnant women in the UK is potentially cost saving. This finding also has implications for the 1·88 billion people in the 32 countries with iodine deficiency worldwide. Valuation of IQ points should consider non-earnings benefits—eg, health benefits associated with a higher IQ not germane to earnings.
Artificial reinforcement learning (RL) is a widely used technique in artificial intelligence that provides a general method for training agents to perform a wide variety of behaviours. RL as used in computer science has striking parallels to reward and punishment learning in animal and human brains. I argue that present-day artificial RL agents have a very small but nonzero degree of ethical importance. This is particularly plausible for views according to which sentience comes in degrees based on the abilities and complexities of minds, but even binary views on consciousness should assign nonzero probability to RL programs having morally relevant experiences. While RL programs are not a top ethical priority today, they may become more significant in the coming decades as RL is increasingly applied to industry, robotics, video games, and other areas. I encourage scientists, philosophers, and citizens to begin a conversation about our ethical duties to reduce the harm that we inflict on powerless, voiceless RL agents.
Red is a 2010 American action comedy film loosely inspired by the limited comic-book series of the same name created by Warren Ellis and Cully Hamner and published by the DC Comics imprint Homage. The film stars Bruce Willis, Morgan Freeman, John Malkovich, Helen Mirren, Karl Urban, Mary-Louise Parker, Brian Cox, Julian McMahon, Richard Dreyfuss, Ernest Borgnine, Rebecca Pidgeon, and James Remar, with German film director Robert Schwentke directing a screenplay by Jon Hoeber and Erich Hoeber. The title of the film is derived from the designation of former Central Intelligence Agency (CIA) agent Frank Moses, meaning "Retired, Extremely Dangerous".
Lady Jane is a 1986 British costume-drama romance film, directed by Trevor Nunn, written by David Edgar, and starring Helena Bonham Carter as the title character. It tells the story of Lady Jane Grey, her marriage to Lord Guildford Dudley, and her reign as the "Nine Days' Queen" following the death of Edward VI of England.
Subscription page for the monthly gwern.net newsletter. There are monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews. You can also browse the archives since December 2013.
Newsletter tag: archive of all issues back to 2013 for the gwern.net newsletter (monthly updates, which will include summaries of projects I’ve worked on that month (the same as the changelog), collations of links or discussions from my subreddit, and book/movie reviews.)