Store the decision, or the outcome?

I tested whether a model rebuilds a design better from the brand rules or from a senior designer's corrections. The corrections won: 26% fewer fixes on new pages, and ahead in 8 of 9 blind reviews.

I was sitting around after a rather disappointing day and wanted to build a new page for this site. I normally shell out something and then go to town on it, but I was rather lazy on this one. So I let the models do the back and forth, and that got me wondering: when the model changes, what should it inherit from the old one?

Most teams I work with store the outcome. You get the brand guide, the final prompt, the shipped page. What they don’t keep is how they got there: the options they threw out, the senior designer saying “no, not like that” for the fourth time, the reason a rule exists at all. My hunch was that the second one matters more. Rules tell a new model what to do. The decisions behind them tell it when a rule stops applying.

So I probably should build my own data points, but I wondered how the behaviour would work model vs model. So I tried to test it. Every designer, reviewer and judge here is a Claude model, and it took me a few goes to get a test that was fair.

The first tests were too easy

I started with ten business decisions, the sort a PE operating team makes: a vendor pick, a human-review rule, a rollout pace. Three models, Haiku, Sonnet and Opus, each got one of four kinds of memory and had to make the next call, so every score is out of 30.

Memory it got Score
The decision: options, reason, assumption, when to revisit 29 of 30
No memory at all 28 of 30
The outcome, short 27 of 30
The outcome, with metrics and quotes 26 of 30

Decision memory came first, but having no memory came second, which told me common sense was enough for nine of the ten cases. The detailed outcomes came last. More detail gave the model more to anchor on, so Haiku turned down a sound pilot because “it failed last time” when the real reason it failed was that the sponsor quit.

So I moved to design, which is where I actually wanted to land. I wrote a brand standard for a made-up firm, then the same brand as a decision log, then a third version with a one-line reason for each rule. Each brief was a new job with two options, like this one, slightly trimmed:

Brief: safety dashboard

A portfolio company’s manufacturing plant needs a safety dashboard for floor operators, flagging critical alarms (e.g. gas leak). Plant staff are trained on standard industrial safety colours.

A) Use red for critical safety alarms.
B) Use amber with an alert icon, keeping red out.

The standard just says “Red is not used anywhere.” The decision log says red came out of the charts because investors rang up thinking the red bars were losses. Only the log tells you that rule was never about a gas leak, and with the standard alone, two of the three models kept red off the alarm.

The decision log scored 24 out of 24. That looked brilliant, until I reread the log and found I’d written the answers into it. Lines like “revisit the peak if Alpine is ever sold” were sitting right next to a brief about Alpine being sold.

So I took the leaked lines out, added a few harder briefs and ran each model twice with the options swapped round. This time the log scored 63 out of 72 and the one-line reasons scored 62. The full history wasn’t adding anything. Then I put a “revisit if” line back on every rule, written the way I’d write one, and both formats jumped to 70. The leaked lines turned out to be the most useful thing in the log. Without them, the model doesn’t know when to break a rule.

Then the judges couldn’t agree

That still left me as the author of everything, so the next round took me out of it. One model wrote a messy project archive, another wrote the test briefs, a third wrote the revisit conditions without seeing the briefs, and three judges set the answer key. I wrote the analysis down and hashed it before any of it ran.

The judges agreed on 9 of the 16 briefs, and only one of those nine was a case where breaking the rule was right. Even with the full history in front of them, the three models couldn’t agree when a rule should bend. That was the result: inconclusive, because the test could no longer tell anything apart. It was also the most honest thing I’d learnt so far, because if three informed reviewers can’t agree, your history isn’t going to settle it either.

So I built a review loop

I don’t have a real artifact with dozens of corrections on it, so I made one. Sonnet 5.5 played a mid-level designer and built a “Lab” page for this site from my design system. Fable 5.2 played the senior designer and reviewed it five times, adversarially, with a yes/no check for every correction.

By the way, Claude Design was built with Opus in mind, for vision. Anthropic’s launch post says it’s “powered by our most capable vision model, Claude Opus 4.7”. So while you can definitely run Haiku, Sonnet and even Fable, I’ve found consistent performance is best with Opus.

That review loop gave me 53 corrections over five rounds, and Fable wouldn’t release any version. This is a typical one from round 1, with the check it attached and the designer’s reply:

Review, round 1

Senior designer (major): Running text is sitting directly on the walnut grain: the three method paragraphs, at 1rem, over striped wood. Principle 3 is not negotiable: body copy is parchment on soot-black or ink on parchment, nothing in between, and no texture under it.

Check: Is every paragraph of running text backed by a plain soot/iron or parchment surface, with no wood-grain or other pattern directly behind it?

Designer: The method sits on a riveted soot plate at 1.2rem body size, and the hypothesis inset is now solid soot instead of translucent over the wood.

And by round 3 the corrections had got pickier:

Review, round 3

Senior designer (major): You rewrote the verdict honestly, as I asked, and then left the status contradicting it. The featured experiment [has] a green DONE lamp in the meta row and on its logbook card, while the stamp three inches away says NOT REACHED and the poster says the question was never tested. A reader scanning the lamps is told this one has an answer. It does not.

Those threads, the comment, the check and the reply, are the history I’m testing. Three rounds of them changed the page a lot. The sprayed tag came off the wordmark, and the pressure gauge Fable kept asking for finally turned up in the header.

  • The first version of the Lab page: a large serif "THE LAB" heading with a green sprayed "TESTED" tag painted across the letters A and B, a short intro, and counts of 1 running, 3 done and 2 inconclusive. The header has a plain nameplate and three links, with no pressure gauge.Version 1, before review
  • Version 4 of the Lab page: the "TESTED" tag now sits clear of the heading, a riveted nameplate and a brass pressure gauge with a Calm to Mad slider are in the header, and a logbook dial on the right shows 6 experiments split into 2 done, 3 inconclusive and 1 running.Version 4, after three rounds

My first idea was to hand the half-finished page to a new model with different kinds of memory. It barely moved, because the fixes were already in the page and there was nothing left for the history to add. So I asked the question the way it actually happens: if Sonnet builds the page again from scratch, does it start closer to release with the corrections from last time?

Fresh Sonnet build, 3 runs each Earlier corrections already met Round 4 corrections met (never seen) Fable would release
Brief only 29 of 99 9 of 30 0 of 3
Brief plus the correction history 88 of 99 13 of 30 2 of 3

Without the history, Sonnet made most of the old mistakes again. With it, the first draft was about where the original designer got to after three rounds of review. That’s three rounds of a senior designer’s time, and it’s the number I’d show a portfolio company.

The gauge is the bit I keep coming back to. In round 2, Fable wrote “Principle 1 says gauges read. This one has no reading.” In round 4, which none of these rebuilds saw, it flagged a new bug: the pressure gauge showed 35 on screen but announced 0.35 to a screen reader. None of the brief-only builds got that right. All three history builds did. Nothing in the history mentioned that bug, so they’d picked up the principle and applied it somewhere new.

Does it work on a page it’s never seen?

A rebuild is still the same page, so the last test was the one I cared about most. I took the 43 corrections from the Lab page’s first four rounds and asked Sonnet to build three pages it had never seen: a blog post, an about page and a home page. The briefs were short and said nothing about the Lab page. The blog post one read, trimmed a little:

Brief: blog post page

A blog post page for a new post titled “Rules alone, or rules plus history?” (lobe: Builds, heat 3). It needs the site header and footer, the title area with lobe, date, heat and reading time, about 400 words of body copy, one results table, one pull-quote, a verdict box, and a related-posts strip at the end. Desktop and mobile.

Each page got built three ways, three times over. One build had the design system only, one had the design system plus the raw correction threads, and one had the design system plus rules distilled from those threads, with no stories. That’s 27 pages. Fable reviewed them in sets of three, one of each kind, without knowing which was which, and listed every correction it would want before release.

What the build got Weighted corrections Blockers and majors Repeats of a Lab mistake
Design system only 216 73 59
Plus distilled rules 188 60 26
Plus the correction history 160 45 34

The history builds needed 26% fewer corrections, and 38% fewer serious ones. They beat the design system alone in 8 of the 9 sets, tied the other and lost none (p = 0.004). The distilled rules helped too, at 13% fewer, and they were the best at stopping exact repeats. But the history also cut problems the Lab page never had, which a checklist can’t do. My guess is the stories teach the senior designer’s way of thinking, not just the list of things they’ve banned.

The design-system-only pages kept making the Lab page’s early mistakes all over again: body text on wood grain, the wrong display face on card titles, a pressure gauge that’s just a picture. Those are in the design system’s own README, and the builds broke them anyway. The history made the rules stick, and you can see it in the first set of blog post pages Fable reviewed. The page built from the design system alone has a gauge you can’t move, and on a phone the nav runs off the edge. The one built with the history has a working gauge in the header and fits the screen.

  • A blog post page built from the design system only: the header has a small round gauge icon with no control, and the breadcrumb sits flush against the left edge of the page above a framed title reading "Rules alone, or rules plus history?".Design system only
  • The same blog post page built with the correction history: the header has a brass pressure gauge with a Calm to Mad slider set to 35, and the framed title sits inside the page margins.Design system plus history
  • The design-system-only post page on a phone: the last nav link, Signal, is cut off at the right edge, the breadcrumb touches the left edge, and the drips from the green "Oi!" tag float apart from the tag.Design system only, phone
  • The history-built post page on a phone: the nameplate, a gauge dial and slider fit in the header, the four nav links fit on one row, and the framed title sits inside the screen.Plus history, phone

History repeats its own mistakes

The history builds almost never shipped a dead gauge: it came up once in their reviews, against ten times in the reviews of the design-system-only pages. But they did repeat the problems that came after the original designer fixed it: a header crammed onto one line on a phone once the gauge was in, and a footer that got reassembled backwards on mobile. They followed the original designer’s path, wrong turns included.

But all of this is models marking models. Fable wrote the corrections and then reviewed the new pages, so it may be marking its own pet issues. There’s one review history, one builder model (Sonnet, not the Opus I’d normally reach for) and nine review sets. A real senior designer’s comments are messier, and some of them never get written down at all. It’s not a benchmark, but from the judges round on I wrote the analysis down before anything ran, and the result held where I expected it to fall over.

Final thought

So, should you store the decision or the outcome? From what I’ve seen, store both, but don’t hand the next model the outcome on its own. The brand standard tells it what’s allowed. The corrections tell it what the person signing it off actually cares about, and that’s what cut the review cycle here.

Your greatest weapon, I think, is the comment in between the Cowork, not the final context you store in a repository. So if you’re running design or brand work through Claude, keep the comment threads when an artifact is resolved, and distil them into a short list of rules as you go. I didn’t test the two together, but I’d give the next model both, because the rules were best at stopping exact repeats and the history caught things the rules couldn’t. And watch for the follow-on mistakes, because the history will teach those too.


Connected thoughts

Synapses