--- title: "May 2020 News" description: "May 2020 Gwern.net newsletter: GPT-3 scaling, implications, deep theory; anime GAN updates, and 1 book review." thumbnail: /doc/ai/nn/transformer/gpt/2020-brown-gpt3-figure13-meanperformancescalingcurve.png thumbnail-text: "Figure 1.3 from Brown et al 2020 (OpenAI, GPT-3), showing roughly log-scaling of GPT-3 parameter/compute size vs benchmark performance on all text/natural language benchmarks test." created: 2019-12-26 status: finished confidence: log css-extension: dropcaps-de-zs backlink: False ... May 2020's [Gwern.net](/newsletter/2020/05 "'May 2020 News', Gwern 2019") [newsletter](https://gwern.substack.com/ "'Gwern.net newsletter (Substack subscription page)', Gwern 2013") is now out; previous, [April 2020](/newsletter/2020/04 "'April 2020 News', Gwern 2019") ([archives](/doc/newsletter/index)). This is a collation of links and summary of major changes, overlapping with my [Changelog](/changelog); brought to you by my donors on [Patreon](https://www.patreon.com/gwern). # Writings - Ganbooru prototype: released: [256px BigGAN trained on Danbooru2019](/biggan#danbooru2019e621-256px-biggan "Release: 256px BigGAN trained on Danbooru2019+e621 for 607k iterations; promising prototype for GAN anime generation"){.include} [Danbooru2019 Figures dataset](/crop#danbooru2019-figures "Danbooru2019 Figures dataset is a large-scale Character anime illustration dataset of _n_ = 855,880 images (248GB) cropped from Danbooru2019 using the AniSeg anime character detection model. The images are cropped to focus on a single character's entire visible body, extending 'portrait' crops to 'figure' crops. This is useful for tasks focusing on individual characters, such as character classification or for generative tasks."){.include} - **Gwern.net**: - experimental `` [mobile image optimization](https://groups.google.com/g/hakyll/c/aFH9LHKyDZ8/m/-zY0SHdUBAAJ) ([since removed](/design-graveyard#srcset-mobile-optimization "‘Design Graveyard § srcset Mobile Optimization’, Gwern 2010")) - [`popups.js`](/static/js/popups.js "'popups.js', Achmiz 2019"): +support for reverse-footnote popups
Mailing List Switch
The newsletter moved this month to Substack due to reaching the TinyLetter 5000-subscriber limit. Please let me know of any issues beyond the known issue of length truncation. (Note that reading the website version on desktop is the recommended way for annotations etc.)
# On GPT-3: Meta-Learning, Scaling, Implications, And Deep Theory {#gpt-3} On ["GPT-3: Language Models are Few-Shot Learners", Brown et al 2020](https://arxiv.org/abs/2005.14165#openai "'GPT-3: Language Models are Few-Shot Learners', Brown et al 2020") ([poems](https://arxiv.org/pdf/2005.14165.pdf&org=openai#page=48 "Figure F.1: Four uncurated completions from a context suggesting the model compose a poem in the style of Wallace Stevens with the title 'Shadows on the Way'") & my followup [GPT-3 Creative Writing](/gpt-3 "Creative writing by OpenAI's GPT-3 model, demonstrating poetry, dialogue, puns, literary parodies, and storytelling."), compare [my old finetuned GPT-2 poetry](/gpt-2 "'GPT-2 Neural Network Poetry', Branwen & Presser 2019"); [random samples](https://justpaste.it/7eovk "GPT-3 Github JSON dump reformatted to readable HTML"); ["OpenAI API"](https://openai.com/blog/openai-api/) with real-world demos) [**Moved to "The Scaling Hypothesis".**](/scaling-hypothesis "‘The Scaling Hypothesis’, Gwern 2020"){.include-annotation .redirect-from-id} # Media ## Links **AI**: - [Matters Of Scale]{.smallcaps}: - GPT-3: see above; for GPT-3 compared to humans on the absolute scale of character prediction, see [Scaling Hypothesis, footnote 18](/scaling-hypothesis#human-perplexity) - ["Measuring the Algorithmic Efficiency of Neural Networks"](https://arxiv.org/abs/2005.04305#openai), Hernandez & Brown 2020 ([blog](https://openai.com/research/ai-and-efficiency "'AI and Efficiency: We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months', Hernandez & Brown 2020")/[interview](https://80000hours.org/podcast/episodes/danny-hernandez-forecasting-ai-progress/ "How to measure and forecast the most important drivers of AI progress"); the first prototype is never the best one, but given enough compute & time, you can refine it and figure out how it should have been done all along, and this paper quantifies the neural net hardware overhang just since 2012: "it now takes 44× less compute to train...to the level of AlexNet". Unsurprising---eg the [experience curve](https://en.wikipedia.org/wiki/Experience_curve_effects) in [linear programming](https://en.wikipedia.org/wiki/Linear_programming): [Bixby 2002](/doc/cs/algorithm/2002-bixby.pdf "Solving Real-World Linear Programs: A Decade and More of Progress"); see [Grace 2013](/doc/ai/scaling/2013-grace.pdf#miri "Algorithmic Progress in Six Domains")/[Yudkowsky 2013](/doc/ai/scaling/2013-yudkowsky.pdf#miri "Intelligence Explosion Microeconomics"). We don't know how to train the right kind of neural nets and make huge mistakes with the simplest things, as capability jumps like resnets or EfficientNet or [R2D2](https://openreview.net/forum?id=r1lyTjAqYX#deepmind "'R2D2: Recurrent Experience Replay in Distributed Reinforcement Learning', Kapturowski et al 2018") occasionally remind us.) - ["IntelliCode Compose: Code Generation Using [GPT-2] Transformer"](https://arxiv.org/abs/2005.08025#microsoft-openai "'IntelliCode Compose: Code Generation Using Transformer', Svyatkovskiy et al 2020"), Svyatkovskiy et al 2020 (unclear if application of [ZeRO-2](https://www.microsoft.com/en-us/research/blog/zero-2-deepspeed-shattering-barriers-of-deep-learning-speed-scale/ "'ZeRO-2 & DeepSpeed: Shattering barriers of deep learning speed & scale', Team 2020"); see also the GPT-3 few-shot code completion abilities) - ["GrokNet: Unified Computer Vision Model Trunk and Embeddings For Commerce"](/doc/ai/scaling/2020-bell.pdf#facebook) ([blog](https://ai.meta.com/blog/powered-by-ai-advancing-product-understanding-and-building-new-shopping-experiences/ "GrokNet: Powered by AI: Advancing product understanding and building new shopping experiences"); one model, 7 datasets, 89m images, 83 losses/tasks, and +8% search quality boost worldwide) - ["Deep neuroethology of a virtual rodent"](https://openreview.net/forum?id=SyxrxR4KPS#deepmind), Merel et al 2019 ([media](https://spectrum.ieee.org/ai-powered-rat-valuable-new-tool-neuroscience "AI-Powered Rat Could Be a Valuable New Tool for Neuroscience: Researchers from DeepMind and Harvard are using a virtual rat to see what neural networks can teach us about biology")) - ["Go-Explore 2: First return then explore"](/doc/reinforcement-learning/exploration/2021-ecoffet.pdf#uber "'First return, then explore', Ecoffet et al 2021"), Ecoffet et al 2021 - ["Learning to Simulate Dynamic Environments with GameGAN"](https://arxiv.org/abs/2005.12126#nvidia), Kim et al 2020 ([project page](https://nv-tlabs.github.io/gameGAN/#nvidia "'Learning to Simulate Dynamic Environments with GameGAN [homepage]', Kim et al 2020"), [code](https://github.com/nv-tlabs/GameGAN_code); an unexpected appearance of a [Neural Turing Machine](https://arxiv.org/abs/1410.5401#deepmind "Graves et al 2014")) - ["Exploring Bayesian Optimization: Breaking Bayesian Optimization into small, sizeable chunks"](https://distill.pub/2020/bayesian-optimization/), Agnihotri & Batra 2020 - ["This Word Does Not Exist"](https://www.thisworddoesnotexist.com/) (GPT-2); ["This Fursona Does Not Exist (TFDNE)"](https://www.thisfursonadoesnotexist.com/ "'This Fursona Does Not Exist', Arfafax 2020") [editor](https://colab.research.google.com/drive/1O5XbpMRU9i83mLAiTrMufCqmImgTRI7A "'This Fursona Does Not Exist—Fursona Editor (Tensorflow Version)', Arfafax 2020") (a simple but high-quality [StyleGAN 2 face model](/face "'Making Anime Faces With StyleGAN', Gwern 2019") of [furries](https://en.wikipedia.org/wiki/Furry_fandom), also available on [Artbreeder](https://www.artbreeder.com/); interesting for how [the fur flew](https://gizmodo.com.au/2020/05/the-internet-furry-drama-raising-big-questions-about-artificial-intelligence/ "The Internet Furry Drama Raising Big Questions About Artificial Intelligence") due to legal fuzziness & some artists acting like animals, howling about 'theft' & free fursonas being a wolf in sheep's clothing upsetting their pecking order^[Don't worry: we already have short-shorts & ear-[TIPS](https://en.wikipedia.org/wiki/United_States_Treasury_security#TIPS) to hedge against fursona inflation. That said, we advise taking a large position in equineties image macro funds to benefit from a flight to quality and herding: it'll be a bear market for kinky bonds---and that's no bull.]---though the creator has outfoxed the paper tiger threats, these kittlesome questions will dog ML as DL models multiply like rabbits) **Genetics**: - [Everything Is Heritable]{.smallcaps}: - ["Local genetic correlation analysis reveals heterogeneous etiologic sharing of complex traits"](https://www.biorxiv.org/content/10.1101/2020.05.08.084475.full), Zhang et al 2020 ("autism/IQ _r~g~_...could be explained by 2 etiologically-distinct genetic signatures w/bidirectional local genetic correlations") - ["Identification of 370 loci for age at onset of sexual and reproductive behavior, highlighting common aetiology with reproductive biology, externalizing behavior and longevity"](https://www.biorxiv.org/content/10.1101/2020.05.06.081273.full "‘Identification of 370 genetic loci for age at first sex and birth linked to externalizing behavior’, Mills et al 2021"), Mills et al 2020 - ["Genome-wide association study of school grades identifies a genetic overlap between language ability, psychopathology and creativity"](https://www.biorxiv.org/content/10.1101/2020.05.09.075226.full), Rajagopal et al 2020 ("math performance was severely affected whereas language performance (Danish and English) was relatively unaffected or enhanced in those with psychiatric disorders") - ["GWAS of Depression Phenotypes in the Million Veteran Program and Meta-analysis in More than 1.2 Million Participants Yields 178 Independent Risk Loci"](https://www.medrxiv.org/content/10.1101/2020.05.18.20100685.full), Levey et al 2020 - ["A Single Gene Causes Thelytokous Parthenogenesis, the Defining Feature of the Cape Honeybee _Apis mellifera capensis_"](/doc/genetics/cloning/2020-yagound.pdf "'A Single Gene Causes Thelytokous Parthenogenesis, the Defining Feature of the Cape Honeybee Apis mellifera capensis', Yagound et al 2020"), Yagound et al 2020 - ["Insights into the genetic architecture of the human face"](https://www.biorxiv.org/content/10.1101/2020.05.12.090555.full), White et al 2020 - [Recent Evolution]{.smallcaps}: - ["Sex-biased reduction in reproductive success drives selective constraint on human genes"](https://www.biorxiv.org/content/10.1101/2020.05.26.116111.full), Gardner et al 2020; ["Genome-wide analysis identifies genetic effects on reproductive success and ongoing natural selection at the _FADS_ locus"](https://www.biorxiv.org/content/10.1101/2020.05.19.104455.full), Mathieson et al 2020 (previously: [Barban et al 2016](/doc/genetics/heritable/correlation/2016-barban.pdf "Genome-wide analysis identifies 12 loci influencing human reproductive behavior")/[Tropf et al 2016](https://www.biorxiv.org/content/10.1101/049163.full "Mega-analysis of 31,396 individuals from 6 countries uncovers strong gene-environment interaction for human fertility")/[Verweij et al 2017](https://www.nature.com/articles/ejhg2017105 "Sexual dimorphism in the genetic influence on human childlessness")) - ["Disentangling selection on genetically correlated polygenic traits using whole-genome genealogies"](https://www.biorxiv.org/content/10.1101/2020.05.07.083402.full), Stern et al 2020 - [Engineering]{.smallcaps}: - ["Recent advances in bovine _in vitro_ embryo production: reproductive biotechnology history and methods"](https://www.cambridge.org/core/journals/animal/article/review-recent-advances-in-bovine-in-vitro-embryo-production-reproductive-biotechnology-history-and-methods/4C4A7C008A6014ADBFDECCFED12FAE13 "'Recent advances in bovine in vitro embryo production: reproductive biotechnology history and methods', Ferré et al 2020"), Ferré et al 2020 - [Atomic garden](https://en.wikipedia.org/wiki/Atomic_gardening) [mutation breeding](https://en.wikipedia.org/wiki/Mutation_breeding) **Statistics/Meta-Science**: - ["Variability in the analysis of a single neuroimaging dataset by many teams"](/doc/statistics/bias/2020-botviniknezer.pdf), Botvinik-Nezer et al 2020 - ["Remembering John Conway’s FRACTRAN, a ridiculous, yet surprisingly deep language"](https://raganwald.com/2020/05/03/fractran.html), Reginald Braithwaite (how does the recently-deceased [John Conway's](https://en.wikipedia.org/wiki/John_Horton_Conway) [1980](/doc/cs/computable/1987-conway.pdf "'FRACTRAN: A Simple Universal Programming Language for Arithmetic', Conway 1987") [esolang](https://en.wikipedia.org/wiki/FRACTRAN) lead to the [Collatz conjecture](https://en.wikipedia.org/wiki/Collatz_conjecture)?) - ["Tumbling toast, Murphy’s Law and the fundamental constants"](/doc/science/physics/1995-matthews.pdf), Matthews 1995 ([overview](/doc/statistics/bias/1997-matthews.pdf "'The Science of Murphy's Law: Life's little annoyances are not as random as they seem: the awful truth is that the universe is against you', Matthews 1997"); [anthropics](https://en.wikipedia.org/wiki/Anthropic_principle) size argument from [Press 1980](/doc/science/physics/astronomy/1980-press.pdf "Man's size in terms of fundamental constants"); see also [Bacon et al 2001](/doc/science/physics/2001-bacon.pdf "A closer look at tumbling toast")/[Borghi 2012](/doc/science/physics/2012-borghi.pdf "On the tumbling toast problem")) **Politics/religion**: - ["The Voluntariness of Voluntary Consent: Consent Searches and the Psychology of Compliance"](/doc/sociology/2019-sommers.pdf), Sommers & Bohns 2019 (people are bad at predicting resistance to police requests; see also [Christin et al 2012](/doc/technology/2012-christin.pdf "It's All About The Benjamins: An empirical study on incentivizing users to ignore security advice")) - [Operation INFEKTION](https://en.wikipedia.org/wiki/Operation_INFEKTION) (see also [Gordon 1997](/doc/iq/1997-gordon.pdf#page=47 "Everyday Life as an Intelligence Test: Effects of Intelligence and Intelligence Context: Conspiracy Rumors in Everyday Life")) - ["Progress Studies for Aspiring Young Scholars"](https://progressstudies.school/ "'Progress Studies for Aspiring Young Scholars: An online summer program in the history of technology for high school students', Scholars 2020") (experimental online summer class for high school students by [Jason Crawford](https://jasoncrawford.org/) on development) **Psychology/biology**: - ["Understanding immunity through the lens of disease ecology"](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5711578/ "'Understanding Immunity through the Lens of Disease Ecology', Hedrick 2017"), Hedrick 2017[^Hedrick] ("...for the past few thousand years, we human beings have been the most diseased species on earth"; followup to [Hedrick 2004](https://www.cell.com/immunity/fulltext/S1074-7613\(04\)00307-3 "The Acquired Immune System: A Vantage from Beneath"); [commentary](https://www.lesswrong.com/posts/Ce5cdvLkKyu6nTbh9/the-greatest-host-2)) - ["How sanitation conquered disease long before vaccines or antibiotics"](https://blog.rootsofprogress.org/draining-the-swamp "'Draining the swamp: How sanitation conquered disease long before vaccines or antibiotics', Crawford 2020"), Jason Crawford - ["Everyday Life as an Intelligence Test: Effects of Intelligence and Intelligence Context"](/doc/iq/1997-gordon.pdf), Gordon 1997 - ["Objective and subjective experiences of child maltreatment and their relationships with psychopathology"](/doc/psychology/personality/2020-danese.pdf), Danese & Widom 2020 (nothing in psychology makes sense except in the light of individual-differences) - ["Brainless but Multi-Headed: Decision Making by the Acellular Slime Mould _Physarum polycephalum_"](/doc/biology/2015-beekman.pdf "'Brainless but Multi-Headed: Decision Making by the Acellular Slime Mould Physarum polycephalum', Beekman & Latty 2015"), Beekman & Latty 2015 - ["I’m paid biweekly, just not by leprechauns: Evaluating valid-but-incorrect response rates to attention check items"](/doc/psychology/personality/2019-curran.pdf), Curran & Hauser 2019 (how do ["lizardman constant"](/doc/sociology/survey/lizardman/index "'Lizardman Constant in Surveys', Gwern 2013") responders justify it? Or, 'free response is the devil') [^Hedrick]: Some interesting references on **viral evolution**: - [Coevolution Of Virulence]{.smallcaps}: - [_Experimental Epidemiology_](/doc/genetics/selection/artificial/1936-greenwood-experimentalepidemiology.pdf "'Experimental Epidemiology', Greenwood et al 1936"), Greenwood et al 1936 ([editorial](/doc/genetics/selection/artificial/1936-jama.pdf "'Experimental Epidemiology', JAMA editors")) - ["Population biology of infectious diseases: Part I"](/doc/genetics/selection/natural/1979-anderson.pdf)/["Part II"](/doc/genetics/selection/natural/1979-may-2.pdf "'Population biology of infectious diseases: Part II', May & Anderson 1979"), Anderson & May 1979 - ["Coevolution of hosts and parasites"](/doc/genetics/selection/natural/1982-anderson.pdf), Anderson & May 1982 - [[Passaging](https://en.wikipedia.org/wiki/Serial_passage)]{.smallcaps}: - ["Experimental Evolution of Parasites"](/doc/genetics/selection/artificial/1998-ebert.pdf), Ebert 1998 - ["History of Sabin attenuated poliovirus oral live vaccine strains"](/doc/genetics/selection/artificial/1973-sabin.pdf), Sabin & Boulger 1973 (making [Sabin's](https://en.wikipedia.org/wiki/Albert_Sabin) [polio vaccine](https://en.wikipedia.org/wiki/Polio_vaccine#Attenuated_2) by dozens of passages through monkeys & monkey tissues) **Technology**: - ["Reflections on How Designers Design with Data"](/doc/design/visualization/2014-bigelow.pdf), Bigelow et al 2014 (why are data visualizations so bad---superficially pretty but misleading or useless? Because many designers don't look at the data, avoid automation & create manually so they can focus on pretty shapes/colors & enjoying fiddling with it, and ignore readers) - ["Do Ads Harm News Consumption?"](https://arxiv.org/abs/2005.06840 "‘How Does the Adoption of Ad Blockers Affect News Consumption?’, Yan et al 2020"), Yan et al 2020 ("Users who adopt ad blockers subsequently consume 20% more news articles corresponding to 10% more categories. The effect persists over time..."; see [my ad page](/banner "'Banner Ads Considered Harmful', Gwern 2017")) - ["The 1-Bit Instrument: The Fundamentals of 1-Bit Synthesis, Their Implementational Implications, and Instrumental Possibilities"](/doc/cs/hardware/2020-troise.pdf), Troise 2020 **Economics**: - ["In Ohio, the Amish Take On the Coronavirus"](https://www.nytimes.com/2020/04/09/us/politics/amish-coronavirus-ohio.html "'In Ohio, the Amish Take On the Coronavirus: A famously traditional community has mobilized to help hospitals with medical supplies, even as it struggles with reconciling its communal way of life with the dictates of social distancing.', Williamson 2020") (supply and demand: masks can be easily made anywhere *if* prices are allowed to rise & they are not illegal to sell) - ["The Story of America's Most Prolific Counterfeiter"](https://www.gq.com/story/the-great-paper-caper "'The Great Paper Caper: Years of running drugs and boosting cars left Frank Bourassa thinking: There’s got to be an easier way to earn a dishonest living. That’s when he nerved up the idea to make his fortune. (Literally.) Which is how Frank became the most prolific counterfeiter in American history—a guy with more than $200 million in nearly flawless fake twenties stuffed in a garage. How he got away with it all, well, that’s even crazier.', Tower 2014") (how Frank Bourassa tricked a Swiss mill into selling him the unique [U.S. dollar](https://en.wikipedia.org/wiki/Federal_Reserve_Note) linen-paper to create [$250]($2012)m in perfect [counterfeit money](https://en.wikipedia.org/wiki/Counterfeit_money) & mostly got away with it) **Fiction**: - ["_My Immortal_ As Alchemical Allegory"](https://slatestarcodex.com/2020/05/26/my-immortal-as-alchemical-allegory/ "'My Immortal As Alchemical Allegory', Alexander 2020"), SSC **Misc**: - ["Dead Reckoning: The 18^th^ century misadventures of HMS _Wager_ and her reluctant crew"](https://www.damninteresting.com/dead-reckoning/ "'Dead Reckoning: The 18th century misadventures of HMS Wager and her reluctant crew', Bellows 2019") (the [_Wager_](https://en.wikipedia.org/wiki/HMS_Wager_\(1739\)) [Mutiny](https://en.wikipedia.org/wiki/Wager_Mutiny)) ## Books **Fiction**: - [_The Battle Between the Frogs and the Mice: A Tiny Homeric Epic_](https://www.amazon.com/gp/product/1589881427/), translated Stallings 2009 ([review](/review/book#the-battle-between-the-frogs-and-the-mice-stallings-2019 "'Book Reviews § The Battle Between the Frogs and the Mice, Stallings 2019', Gwern 2013"){.include}) **Nonfiction**: - [Review of _The Cultural Revolution_](/review/cultural-revolution "'Review Of The Cultural Revolution, Dikötter 2016', Gwern 2019"){.include} ## Music [**Touhou**](https://www.reddit.com/r/TOUHOUMUSIC/search/?q=author%3Agwern&sort=new&restrict_sr=on&t=all): - ["Sept Jours sans Elle (Vocal)"](https://www.youtube.com/watch?v=5OmnD4zRrfw) (Raven's Jig; [_Une Semaine chez les Écarlates_](https://www.youtube.com/watch?v=H6iMYgpnqBU) {2018}) [classical] - ["Un Jour Joueur"](https://www.youtube.com/watch?v=bSjqmUMqH58) (Raven's Jig; _Une Semaine chez les Écarlates_ {2018}) [classical] - ["Bons et mauvais Jours"](https://www.youtube.com/watch?v=xiY5HFHD4Js) (Raven's Jig; _Une Semaine chez les Écarlates_ {2018}) [classical] **MLP**: - ["Morning in Baltimare"](https://www.youtube.com/watch?v=gaHzrCMWlwY) (Mane in Green; _II. The Journey [The Quest of the Lost Sapphire---Ep. 2]_ {2017}) [instrumental rock] - ["Love and Reflection"](https://www.youtube.com/watch?v=M_sViQ2rHnQ) (Dionte George; [_Ignite_](https://poniesatdawn.bandcamp.com/album/ignite) {2020}) [jazz] - ["Second Prances (Vocal VIP)"](https://www.youtube.com/watch?v=gJ-bLIjz1jY) (Etherium Apex ft. Nicole Carino {2020}) [electronic] - ["Spun"](https://www.youtube.com/watch?v=tEsDbLYym-w) (The Wasteland Wailers feat. Brittany Church & Haymaker; _Ignite_ {2020}) [country] - ["Equiterian Empire"](https://www.youtube.com/watch?v=rOsHR7x9eTA) (Carbon Maestro; _Celestial Divide OST_) [orchestral] - ["The Storm Is Coming VIP \[Single Purpose Remix\]"](https://www.youtube.com/watch?v=Uj852XOZyEA) (UndreamedPanic feat. Metajoker; _Ignite_ {2020}) [rock] - ["Mare Cognitum"](https://www.youtube.com/watch?v=CFOwIOJt4KI) (Idyllia feat. Velvet R. Wings; _Ignite_ {2020}) [orchestral rock] - ["Fire City (Day & Night)"](https://poniesatdawn.bandcamp.com/album/dreamwalkerstrack/fire-city-day-night) (Wandering Artist; _Ignite_ {2020}) [orchestral] - ["What Remains"](https://www.youtube.com/watch?v=5-r-dY0mzdQ) (Totalspark; _Ignite_ {2020}) [Liquid Drum & Brass] **Doujin**: - ["Come, Sweet Death [_Komm, süsser Tod_]"](https://www.youtube.com/watch?v=wRMkOY9OrbU) (Platina Jazz feat. Niklas Gabrielsson; [_Anime Standards Vol. 6_](https://www.amazon.com/Anime-Standards-Vol-Platina-Jazz/dp/B07YG536TT) {2019}) [jazz] - ["Hope"](https://www.youtube.com/watch?v=Bp12S4uM5Fs) (Simpsonill {2017}) [electronic]