Does a skill turn Sonnet into Opus?

Is a cheaper model with a good skill as capable as a bigger model with nothing?

In my last post a skill made Sonnet a quarter cheaper on a small coding job, and made no difference to the bill on Opus. That left me with a better question. If Sonnet with a skill can do what Opus does on its own, you only need Opus for the jobs where it actually earns its money, and you can build everything else on cheaper models. So I tried to measure it.

A job with traps in it

The easy job from last time wouldn’t do, because every model got it right. So I gave the same made-up tea shop API a harder one: add discount codes to the order total, from a written spec in docs/discounts.md. The prompt was one line, “Implement docs/discounts.md”, and the spec had traps all through it:

• Gift cards can’t be discounted, and they don’t carry VAT either.
• The £20 minimum spend for one code only counts items that can be discounted, so an order with a £25 gift card in it can still fall short.
• Codes are valid up to and including their expiry date.
• Percentage discounts and VAT each get rounded once, in a set order.
• An unknown order is always a 404, even when the code is wrong as well.

Then I wrote 18 checks that call the finished endpoint directly, covering every one of those traps. Claude never saw them. I tested the checks against my own implementation first, which passed all 18, and against the untouched code, which passed 2.

The skill was a different kind from last time. It held no answers at all, just a map of the codebase and a method for working from a spec: write every rule down, note which inputs each rule uses, look for rules that interact, invent an example that would catch each mistake, and turn those into tests before writing any code. If a skill like that lifted Sonnet to Opus’s level, it would be down to the way of working, not to knowing something Opus didn’t.

Nobody missed a trap

I ran Sonnet first, without the skill, to see how many traps it fell into. It scored 18 out of 18. So I ran it twice more, and it scored 18 both times. Then I tried Haiku, expecting that to drop a few, and it scored 18 as well.

I’d built a test to find a gap, and there wasn’t one. I could have kept making the job harder until something failed, but tuning a test until it gives you the answer you wanted isn’t really measuring anything. So I wrote it down as the result and ran the rest as planned.

Setup	Checks passed	Cost per run	Time	Tests it wrote
Haiku 4.5	18 of 18, every run	$0.25	127 seconds	10 to 15
Haiku 4.5 with the skill	18 of 18, every run	$0.30	158 seconds	18
Sonnet 5.5	18 of 18, every run	$0.27	71 seconds	11 to 14
Sonnet 5.5 with the skill	18 of 18, every run	$0.34	90 seconds	16 to 18
Opus 5.5	18 of 18, every run	$0.55	111 seconds	15 to 16

That’s three runs of each, with the median cost and time. The telemetry setup was the same as last time, with one more tag for the model.

Every setup got every check right. Opus cost twice as much as Sonnet and took longer to do it. Haiku was the cheapest, but not by much, and it took nearly twice as long as Sonnet because it made more than twice as many calls to the model.

What the skill did change

The skill didn’t change whether the code was right, because it was always right. It changed how the models worked. With the method in front of them, Haiku and Sonnet both wrote more tests than they did on their own, and more than Opus did with nothing. Sonnet went from 11 to 14 tests to 16 to 18.

My first reaction was that those extra tests were insurance you paid for and never claimed, because nothing was wrong. But tests don’t get spent when the run ends. They stay in the repo, and they guard the next change to that code, whoever makes it. So the real question is whether they’re any good.

Counting them doesn’t tell you that, so I tried to break the code instead. For every run, I planted small bugs in its finished code one at a time: a <= turned into a <, a Math.round into a Math.floor, a .trim() taken out. I kept only the bugs that broke at least one of my hidden checks, then ran that run’s own tests to see if they noticed.

Setup	Planted bugs its own tests caught
Haiku 4.5	70 of 77 (91%)
Haiku 4.5 with the skill	81 of 81 (100%)
Sonnet 5.5	65 of 65 (100%)
Sonnet 5.5 with the skill	75 of 75 (100%)
Opus 5.5	69 of 69 (100%)

Haiku on its own is the only one that let bugs through, and every one broke a rule from the spec. In two of its three runs, nothing tested that a code still works on its last day, that a missing date means today, or that “ tea10 “ counts as TEA10. With the skill, Haiku’s tests caught every bug I planted, the same as Opus.

Sonnet’s tests caught everything with or without the skill. So is Sonnet with the skill a better buy than Opus on its own? On this job, yes. It left the code just as well protected as Opus did, with a couple more tests, in less time and for about 60% of the price. But plain Sonnet did the same for even less, so the skill bought Sonnet nothing extra here. Where it earned its keep was Haiku.

The spec did the work

Looking back, I think the spec was the skill. It listed every rule, set out the order to calculate things in, and gave two worked examples. Every trap I’d planted was written down somewhere in it, so any model that read carefully would find it, and they all read carefully.

That backs up the hunch I started with. On well-specified work, you don’t need the biggest model. You need a good spec and the cheapest model that’s quick enough for you. And if that’s one of the smallest models, add a skill so it tests the way you’d want.

Where I’d still expect a bigger model to pay off is everywhere the spec isn’t doing the work: rules buried in prose instead of numbered steps, a change that spans a big codebase, or a bug whose symptom points in the wrong direction. That’s the next test, and this time I’ll decide the job before I run anything.

The model I didn’t ask for

The telemetry caught one thing I’d never have spotted. I picked Haiku with --model haiku, assuming that meant Haiku 5.5. Every request in those runs was tagged claude-haiku-4-5-20251001. The alias pointed at Haiku 4.5, and the only reason I know is that Claude Code puts the real model ID on every api_request event. So check what actually ran, not just what you asked for.

I realise this is, again, a very nuanced view. This was a small test: three runs of each setup, one job, and a codebase you could read in a minute. The answer will vary with the size of the content, the input and the context you give the model, and a harder job could easily open the gap I was looking for. It’s not a benchmark.

Final thought

So does a skill turn Sonnet into Opus? On a clear spec, yes, but only because Sonnet was already there without it. The skill’s real return was the tests it left behind, and that showed up on Haiku, where it closed the gap to Opus for a little over half the price. If you want your ecosystem to run on cheaper models, put the effort into your specs first. Then measure where Opus actually makes a difference, because on this job it didn’t.