Since GPT-3 came out, the advice on how to write a prompt keeps growing. Be polite. Tell the model it is an expert. Write IMPORTANT in capitals and ask it to think carefully. Put a clear instruction at the top. Most of this advice comes from someone who tried it once on the API and it worked. There is no second run behind it.
I read these tips and sorted them in my head without measuring anything. Politeness and “you are an expert” I took for superstition. A clear instruction I took for craft, the one part of the prompt that at least cannot hurt. I can’t run GPT-3 at home. But the same few shot trick works on GPT-2, so I put both groups on the same task and counted.
The setup
The task is SST-2 from GLUE: 872 short movie reviews from the dev set, 444 of them positive. A model that says “positive” to everything gets 50.9%.
The prompt starts with an instruction: “Classify the sentiment of each movie review as positive or negative.” Then come 4 examples from the training set, 2 positive and 2 negative, as Review: ... and Sentiment: positive. Then the review to classify. The model does not generate anything. I compare the probability of ” positive” and ” negative” as the next word, so the same prompt always gives the same number.
The models are GPT-2 medium (355M parameters) and GPT-2 large (774M) in float32, on 4 cores of an AMD EPYC 7713 in a virtual machine. The scoring gives the same predictions on torch 1.11 with transformers 4.17. For medium I ran 217 prompts, for large 99 of them.
I chose 10 sets of 4 examples with a fixed seed. Every variant runs on the same 10 sets, so the change in the prompt is the only difference.
The tips
On medium the plain instruction gives 59.3% on average over 10 sets. “Please … Thank you!” gives 62.8%, “You are an expert film critic.” 60.7%, IMPORTANT with “Think carefully and be accurate.” 60.3%. Polite is better than plain on 5 sets and worse on 4, which is a coin flip. Capitals are better on 7 and worse on 2, but 10 sets are too few to be sure of that.
But look at the rows of the figure instead of the averages. With the plain instruction the set of examples alone moves the result from 48.9% to 82.1%. That is 33 points from the choice of 4 reviews. The words I was ready to argue about moved the mean by 1 to 4.
Where I was wrong
The instruction was the part I believed in, so I also ran the prompt without it. Without the instruction medium gets 68.8%, 9.5 points more, better on 8 sets of 10 and worse on 1. Then I tried 2 wrong instructions. “Translate each sentence into French.” gives 65.5% and “Classify each movie review by whether it is about a comedy or a drama.” gives 64.5%, also more than the right one.
Without examples it is the same. The bare Review: ... Sentiment: gives 73.2%, with the plain instruction on top 60.8%, with the polite one 53.9%. With “please” the model answers positive to 97% of reviews.
I spent some time looking for a bug. I checked that the instruction really ends up in the prompt and that both label words are single tokens. The cached prefix also gave the same tokens as the full text. Everything was fine. I don’t know why this instruction hurts medium.
The order of the same 4 examples
For 3 of the sets I ran all 24 orders of the same 4 examples, not a word changed. On set 0 the accuracy goes from 50.9% to 78.3%, on set 1 from 50.2% to 84.5%, on set 2 from 50.1% to 82.1%. Zhao and colleagues showed this on GPT-3 last year. Lu and colleagues did it on GPT-2 with exactly this design, SST-2 and all 24 orders of 4 examples, so it is not new. But I did not expect it to be bigger than everything else in my grid.
The bad orders are not random. There are 12 orders with the labels + - + - over the 3 sets. In all of them the model answers “positive” to 99.2% or more of the reviews and the accuracy is 50.9% to 51.5%. It looks like the model continues the pattern of the labels instead of reading the review. The mirror order - + - + is not so clean, there the share of positive answers goes from 1.0% to 51.5%.
GPT-2 large
On large my story about the instruction fell apart. With 4 examples the wording hardly matters. Polite is 1.3 points above plain and no instruction 0.6 below. The 2 wrong instructions are now a bit worse than plain, French by 2.4 points and lower on 8 sets of 10. So the previous section belongs to medium only. The set of examples still moves the plain prompt from 53.6% to 87.6%.
I had time only for the 24 orders of set 0 on large. They go from 51.6% to 88.3%, but a different order breaks it. + - + - is fine here, 75.0% to 88.3%. Large breaks when the last 2 examples have the same label. After + + - - it answers positive to only 2.5% to 5.2% of reviews, after - - + + to 87.4% to 96.7%. Zhao and colleagues call this recency bias, the model leans to the label it saw last. It fits large only: on medium + - + - ends with a negative example and the model still says positive.
Without examples the wording matters on large a lot. The bare prompt gets 86.4%, polite 84.1%, capitals 78.7%, plain 71.9%, expert 68.6%. On medium polite was 6.9 points worse than plain. On large it is 12.2 points better. But every zero shot number is 1 prompt with no sets behind it. I can’t say how much of these 18 points is the wording and how much is this exact sentence. That bare prompt at 86.4% is also above every variant with 4 examples on large, where the best mean is 65.8%.
What else moved the number
The label words matter more than the polite words. With “good” and “bad” medium gets 68.4%, with “A” and “B” 49.0%, below answering positive to everything.
Calibration helps medium a little and large more. Zhao and colleagues ask the model about an empty input like “N/A” and divide the answer probabilities by what it says there. With the plain instruction this moves medium from 59.3% to 62.0% and large from 64.5% to 72.0%, better on 7 sets of 10 for both. On the zero shot prompt of medium it goes the other way, from 60.8% to 55.4%. After the correction the model says positive only to 7.7% of reviews.
Random labels in the examples cost medium 4.2 points, close to what Min and colleagues reported last month for bigger models. Flipped labels cost 6.7, but medium does not learn to flip from them. It just says positive more often, 89% of the time.
What I did not check
- GPT-3 itself. Bigger models may well read the instruction better.
- Other tasks. Movie review sentiment is the easiest case. On a task where the model needs the instruction to know what to do, removing it would probably hurt.
- More sets. 10 sets show a spread of 33 points, but a difference of 3 between wordings stays inside the noise. Polite could be 2 points better in reality and I would not see it.
For small models I will now try a few orders of the examples before I touch any wording. And I don’t take the instruction for the safe part anymore, even if on large it was only neutral.