Dmitriy Semenkevich.

A year ago I stripped every please and thank you out of my prompt templates because of a joke Sam Altman made about electricity. I never checked what I saved. 20171 calls later the polite wrapper costs 24 dollars per million requests on sonnet and moves the answer no more than a control wrapper with no social word in it. The wrapper that did move things was the one telling the model how much to write: Just the answer takes haiku on GSM8K from 88.5 percent down to 56.0.

A version bump inside one model family looked like a patch release, so I measured what it does to retrieval. 2.19 of the top 10 slots change while nDCG@10 moves in the third decimal, a fixed 0.8 cosine cutoff goes from passing 97.5% of results to passing 1.8%. The index that is halfway through the migration returns 0 relevant documents for half the corpus.

Code review here runs in 3 stages and I had never picked the size of any of them on evidence. After 191 model calls the stage I would have defended hardest had not once disagreed with itself. The thing that kept turning out to be wrong was my own apparatus: 2 comment lines in the control file took 62 findings down to 0.