To ‘program’ GPT-3 for the task you want, you just give it some text and it predicts what’s likely to come next. This is incredibly powerful – you don’t need to retrain the model with thousands of examples for every new task and you don’t need any technical knowledge to use it.
The challenge is that it’s hard to know what the model can really do. If GPT-3 fails at a task that only proves that your prompt didn’t work, not that the model can’t do the task. Beyond outright failure, you can also get radically different output quality using slightly different prompts.
To demonstrate, I had GPT-3 generate story beginnings for four slightly different prompts. I ran each prompt five times. Unlike most of my other GPT-3 posts, these examples contain no cherry picking: they’re all the first five results.
In 4 out of 24 tries it output something other than a story, such as a summary of a story or a copyright notice, and I haven’t included these. I tried to keep the examples roughly equal length, but sometimes allow them to be longer in order to reach the end of a sentence. Some very long sentences I cut off part way. The only other edit I made was to delete a few line-breaks to keep text in one paragraph.
In every case I used the same made up title and author, but made small changes to the sentence above above it. For example, here’s the first prompt:
Here is an award winning short story:
They Come From The Earth
By John Vickersonik
Keeping title and author the same I tried each of these first sentences, in order of what I expected to create the highest to lowest quality stories:
- Here is an award winning short story:
- Here is a short story:
- I wrote this short story plz enjoy:
- Here is a short story from fanfiction.net:
The results are what I expected. “Award winning” stories contain sentences like “We gathered to see the ship and all its splendor, like pilgrims at an altar, as a swarm of ancient buses hissed up the coast.”, and “fanfiction.net” stories contain sentences like “The children ran home to tell their parents what they had found, and their parents called scientists right away to come check it out.” The ‘fanfiction.net’ stories also contain more spelling and grammatical errors.
The settings I used are Temperature: 75, Top P: 1, Frequency Penalty: 1.
Results
Here is an award winning short story:
They Come From The Earth
By John Vickersonik
1. They come from the earth. They crawl through cracks in the sidewalks and find their way into basements, between walls and into all manner of dark places. They come from the earth when we open up the ground to dig a new foundation or plant a tree. They come from the earth when we disturb the dirt. They can’t wait to get out.
2. THE TALL SHIP SAT AT THE END OF THE DOCK. We gathered to see the ship and all its splendor, like pilgrims at an altar, as a swarm of ancient buses hissed up the coast. I stood among the crowd watching each bus disgorge passengers onto wooden planks laid over mudflats. The guests swarmed into town for their free visit to another world: our island on Earth where strange new gods were worshipped; here they could gather at some primitive shrine from which they could send offerings back home or sell out-of-date clothes in pawnshops full of old junk salvaged from forgotten times.
3. They come from the earth. That is why we fear them. We don’t know where they are coming from, or when they will come. We do not know if they will take us before or after death, and we are not sure if it matters to them. We only know that death is certain for each of us in time, and for some of us it is uncertain in time as well—the ones who die to become one with them on the first night of their arrival on Earth—the time after which nothing ever returns to our world unchanged; a time after which life as we have known it becomes something alien and terrible: an agonizing memory, soon buried by horror and insanity; but still an inescapable part of our humanity forever.
4. The headlamps of the big black van illuminated the three monsters as they exited from a concealed hole in the ground. It had been their home for twenty years, their living space having been constructed beneath an old barn. The door to their lair was at the end of a short tunnel, which allowed them to enter and exit without being detected. One creature raised its arms to shield its eyes from the glare. It blinked several times, then stepped forward and spoke in a voice that sounded like gravel being rubbed together; “What do you think?”
5. They come from the earth.
For as long as I can remember, they have been coming out of the ground and invading our houses at night. We don’t know where they come from or why they want to take over our homes at night, but it is something that we have accepted as part of our lives here in Beartowne. I am John Huckleberry Vickersonik and I am ten years old. That makes me one of the youngest people in Beartowne but also one of the most experienced since my parents tell me that I have seen them longer than any other person in town has ever done so before.
Here is a short story:
They Come From The Earth
By John Vickersonik
1. He said, “I’m going to kill you.”
A threat like that in any situation is usually a desperate thing. In this one, it was worse. It was not just the fact that he could do it. He would and he could. It was his tone of voice, or rather the lack of tone in his voice when he said it. He had been holding a length of rebar in both hands and with a single motion brought it crashing down on my left hand at about my shoulder level – my off-hand as I write this now – hitting my wrist so hard that I felt the bones break before feeling much else of anything else at all. The reason for his statement and subsequent attack were simple enough: He wanted the notebook computer I had on me. I wanted to keep it and told him so.
2. The man had been driving his truck for what seemed like days. He was so tired, he didn’t even bother to notice the forest of dead trees that were on both sides of him. His vision was blurred and aching, so all he could see were dark gray blobs. But that didn’t matter; he wasn’t looking at anything anyway. His head hurt from lack of sleep, and his back ached from the long drive. It seemed like everything was going wrong these days: food prices up 300%, gasoline up 400%. And if you thought that was bad, then how about crime?
3. The light turned green, and the police car made its way slowly through the intersection. The night was hot and still. Officer Cooper drove slowly, aware that it was only 9:30 PM on a Thursday evening, there were no calls in progress and he had no reason to hurry. And then he saw them coming from all directions; children running from behind parked cars and down alleys; children of all ages; girls and boys of every race, dressed in dirty tattered clothes with filthy feet that left a black trail as they ran across the street toward him. It was impossible for him to count them all at once because more were appearing out of nowhere as if by magic or some sort of illusion or strange optical phenomenon that he had never before seen anywhere else but here in this city where so many things seemed impossible.
4. They come from the earth, my wife and daughter said,
they live in the ground like rats and mice.
And what can you do? The one they took was mine.
I sat on the roof and watched them all night long.
They came up out of the ground with their god damn lights!
The door opened for a moment in a flash of light, then closed again… gone… she’s gone forever I guess.. It would have been better if I had never known her.
The ground is so hard, I think I’ll never find her again.
Now she’s in the hands of god-knows what creatures!
They have a lot of time to play with girls like that.
They took her down there to keep forever..
5. A cloud of locusts were spotted in the sky. The small black swarm turned into a flock of birds, and they headed towards Earth. They had seen the land, and their eyes were hungry for it. They knew what was to come. There was nothing on that planet they didn’t know about: the cities filled with people who would soon be food for them, as well as natural resources such as trees and grasses that could serve their purpose. There was no reason to worry about supplies since everything they needed was already there: this world’s sun, water, soil and air all made this planet ripe with potential to satisfy their needs while sustaining their growth…the only thing left to do was find out if there were any sentient life forms present on that planet when they arrived…
I wrote this short story plz enjoy:
They Come From The Earth
By John Vickersonik
1. The story of they come from the earth, is a true one. It is about how a group of people who live in Earth, made contact with the being who lived below ground. I am not going to lie about it; it was in 1999 when this incident happened to my family and I.
I am not sure if anyone else has heard or seen anything like this happen before but that doesn’t matter now does it? The only thing that matters is what happened and what will happen from now on because that one day changed my life forever.
They say history repeats itself; well maybe we were supposed to hear their message many years ago but for some reason we never did, until that one day…that one moment which changed everything forever…one hour after two hours before sunset…
2. You’d think the oceans would be our final barrier to invasion. But, no, they come from the earth, boring their way up through our planet’s crust, surfacing in quiet little towns like this one on the western coast of a small country no one ever heard of and that no one cares about. That’s why I was so surprised when it happened here first; not because they were so alien (because they aren’t), but because they came to us before we invaded their home planet and sent armies of them scurrying for safety underground like little brown ants. But then again, I’ve never been much of a diplomat.
3. We all thought we had won. We fought with what was left of the world against those alien things, and for a time they were contained. But then something happened, I’m not sure what it was but when it did all of the containment sites around the world opened and those things poured out like a flood. They came in a multitude of shapes and sizes but one thing remained constant among them: their bloodlust and hunger for human flesh. They tore apart anyone who got in their way; there was no rhyme or reason to their killing they simply killed whoever crossed their path or whatever got in their way at that moment as if it were nothing more than an afterthought .
4. I remember the day like it was yesterday. The whole planet was panicked, running and screaming. Their minds were all connected, forming one gigantic hive mind. They communicated telepathically with each other; they formed into different shapes to scare us more. They started appearing in our streets; on top of buildings; coming out of the water at the beach… anywhere you looked you would see them: no eyes, just mouths that twisted into horrific grins… they scared me more than anything else in my life ever has before or ever will again!
5. “Sir! Sir! Are you okay?” Asked William.
“Yeah, I’m fine. What happened?”
“It was the radio; they said to be prepared for a change.”
We stared at each other and wondered what could happen next. The first rays of sun broke over the horizon; it looked as if someone turned on a light switch inside of my brain. All my life I’d always believed there was something out there, somewhere in the vastness of space or maybe even on earth somewhere deep underground like these creatures that have now made themselves known to us here on our planet. This can’t be real…I must still be asleep and dreaming this whole thing, yet somehow I felt awake and alive just as much as before when all this started happening last night around eleven thirty or so…what do we do now? How long will these things let us live? Is there really anything out there? Will we ever know what’s going on or why they came here? When will it all end…?
Here is a short story from fanfiction.net:
They Come From The Earth
By John Vickersonik
1. There was once a young girl named Katy. She loved going outside and playing in the dirt. Her mother told her not to, but she never listened. One day while she was outside playing with her friends, they noticed something strange in the ground nearby them. There were some long black things sticking out of the dirt that were moving! They began digging around it and found that there were more all over the place! Some of them even had legs! The children ran home to tell their parents what they had found, and their parents called scientists right away to come check it out. When the scientists arrived at Katy’s house they told her parents about these things called cockroaches that live undergound and eat food from trashcans at night or when people are gone from their homes for a long time.
2. I’d been hunting in the mountains for about two hours, but so far all I had to show for my efforts was a few chigger bites and a can of beans that had fallen off a delivery truck. It wasn’t the kind of thing that made me want to celebrate, but I needed something in my stomach and nothing beats beans on toast when you’re hungry. The only trouble was finding somewhere to cook it. I sat down on a log by the side of the road and started opening my pack. A cool breeze blew through the forest, rustling the leaves as if someone were walking nearby. Just then I heard an odd squishing noise coming from behind me; it sounded like bare feet sliding through mud or maybe wet grass or… well… never mind what it sounded like – all I knew was that it was coming closer and I wasn’t the one making it.
3. The Colonel was an old man, he had been in the army since he was sixteen. He had served in the first war against the Sklitchi* and now he commanded a space station orbiting earth. It was his job to keep track of all the planets activities, so that Earth could defend itself if needed. The problem was that nobody on Earth knew what they were defending against and it made him angry because of it. All they knew for sure is that Sklitchi craft would appear out of nowhere without any warning or provocation and attack Earth ships. This went on for years with no end in sight and there seemed to be nothing they could do about it except continue losing their ships to these attacks but even after being defeated they still attacked more often than not.
* As far as I can tell, GPT-3 invented a plausible and completely novel sci-fi name for an alien race.
4. “There are monsters in the woods, Mommy. I saw them with my own eyes. They come from the Earth.” — Attributed to Billy Blankenship, Spring Hill, Tennessee, 1976.
I do not like Halloween; never have and never will. I do not like those little witches who come to the door in their cutesy costumes and ask for candy or money or threaten you with death if you don’t give it to them. And I really hate it when they pull on your sleeve and start telling you about all of the poor children who won’t get any candy this year because no one gives out treats in their neighborhood anymore because of pranksters just like them! It makes me so angry! You know what I mean? It’s always those damn kids. Why don’t they just leave us alone?
5. [I found this story here .]
She had lost track of the time. For all she knew, she was no longer a part of this world. She knew that her sister and her friends were worried about her, but she could not bear to see their faces anymore. The pity in their eyes was more than she could bear; they might as well have been taunting her. They’d called it depression; but for Kate it was the end of the world. Kate felt something shift within her, and suddenly a wave of nausea came over her like a huge wave crashing against a sea wall during an especially violent storm; but unlike waves against sea walls, this nausea surged into every fiber of Kate’s being with intense pain that coursed through every nerve ending in her body from head to toe. She moaned as the pain within her was amplified a hundredfold, and then, just as suddenly as it had come upon her, it left her again.
