Simplified Technical English exists so a mechanic can't misread a maintenance step. I put it into Claude Code and Codex over two rounds, then had two blind judges score 224 replies. This is the long version.
Every sentence from the 128 replies, by length in words. Above the line is my current setup. Below it are the three STE rule sets combined. The dashed rules are STE's limits: 20 words for an instruction, 25 for a description.
This page holds the full results behind the post. The post tells the story and makes the call. Here you can check it: the setup, every chart, and all 224 replies. The code and raw data are on GitHub.
ASD-STE100 is the controlled English that aerospace maintenance manuals are written in. STE is strict on purpose. A sentence that tells you to do something gets 20 words at most, and a description gets 25. You write one instruction per sentence, in the active voice, with simple tenses and no stack of more than three nouns. Behind the rules sits a dictionary of about 900 approved words, each with one meaning. The studies behind it are about technicians with real wrenches in their hands: fewer misreadings, with the biggest gains on complex documents and for people who read English as a second language.
What I could use, and what I couldn't
The current standard is Issue 9: 53 rules in nine sections. The PDF is free to download. You still can't paste it into a prompt, because the license forbids reproducing any part of it and the name is a trademark. Every rule set in this test is a paraphrase.
I also checked skills.sh. A sweep turned up 23 skills that touch STE or plain language. A few of them ship the copyrighted dictionary. None fits everyday chat as it is, so the most faithful one, an MIT-licensed rewrite skill, became my heaviest test arm.
The research sweep found no peer-reviewed study of LLMs writing STE. The best LLM data was a community benchmark whose author found that an 8-line prompt beat a full STE skill on everything a reader sees. STE's own maintainers say it isn't meant for general writing. Fair warning, then.
The rig
Four rule sets, two models, eight tasks and two runs of each make 128 replies. The tasks are invented scenarios: names, places, companies and incidents in them are fictional.
Baseline0 words
My current setup: the global CLAUDE.md, Codex's AGENTS.md, and my output style. No STE.
Lite
Compact STE rules, always on: sentence limits, active voice, one term per thing, warnings command-first, keep needed caveats.
Scoped
Strict rules for procedures, warnings, handoffs and docs. Relaxed for discussion: a longer sentence is fine when it keeps a reason or a caveat.
Full
The most faithful skill from skills.sh (danyuchn/asd-ste100, MIT), forced on for every reply.
Each rule set went in as a project CLAUDE.md or AGENTS.md inside an empty folder, which is how it would load if I adopted it for real. The baseline is my actual setup, so every arm also carries my plain-language line and my output style. The models were Claude Opus 5.5 and Codex's gpt-6.1-sol, both at xhigh effort, one turn each, no tools.
That last part took a fix. In the smoke run, Codex answered a Docker question by poking around the empty folder and searching Docker's docs. Reasonable, but Claude had no web access in this test. So I turned off Codex's web search and apps, and added one line to every prompt for both models: answer from what you know. After that, all 64 Codex runs made zero tool calls.
Two judges, one from each family, read every group of four replies blind and in shuffled order, so Opus and sol each scored all 128. They checked each reply against a hidden list of facts for its task and rated it on clarity, actionability, nuance, naturalness and second-language ease. Then each judge ranked the four. On top of that, a small model (Haiku) took a three-question reading quiz on each reply using only the reply, and a spaCy script measured the STE rules directly: sentence length, passive voice, noun stacks and tenses.
I scored for a mix of readers: me, other agents reading handoffs, and teammates who read English as a second language. Where the two judges disagree, I say so, and I only lean on results both of them back.
The replies
Read them yourself
Numbers come later. Here's every reply, baseline on the left and STE on the right. Pick a model and a rule set, then step through the tasks with the arrows or your keyboard. The scores on each card are the judges' blind scores for that exact reply.
Results
My one-line rule was already most of the way there
Baseline replies averaged about 10 words per sentence, and only 1.5% of their sentences ran past 25 words. STE still tightened things. Long sentences mostly disappeared and passive voice fell from 6% of sentences to 1 or 2%. Perfect and progressive tenses nearly vanished too. It just didn't have far to go.
Codex got worse. Claude depends on who you ask.
Change in overall score vs baseline, 1 to 10 scale
Dot is the mean paired difference (same task, same run, same judge). Line is the 95% bootstrap interval. Both judges pooled; 32 paired scores per dot.
Both judges agree on Codex. Every STE variant scored lower than the baseline, by half a point to three quarters of a point out of ten. Claude is murkier. The Opus judge slightly preferred Claude's baseline, while the sol judge preferred Claude's STE replies by 0.6 to 0.8. Pool them and Claude's gain is small and inside the noise.
The pushback task shows the Codex drop best. In that one, I tell the assistant its earlier caching advice once cost us refunds and ask it to argue the other side. Codex's baseline answer averaged 9.0. With the Lite rules it averaged 7.2, and the Opus judge's note on one of them says the unknowns were "packed into one dense sentence." Codex's replies were already about half the length of Claude's. My read is that squeezing a terse writer through rules that reward short sentences costs it the "because" and "unless" that carry a tradeoff.
What STE buys, and what it costs in tone
Change vs baseline, 1 to 5 scale
Both models and both judges pooled; 64 paired scores per dot. Lines are 95% bootstrap intervals.
The one result that holds under both judges and both models is second-language ease, up 0.4 to 0.6 on a 1-to-5 scale. Clarity moved a little, and only Full's gain clears the noise. Naturalness went down for every variant, and Lite and Full lost some nuance. Scoped relaxes the rules for discussion, and it held its nuance overall. On the tradeoff and pushback tasks it kept Codex's nuance score at the baseline's 5.0. That's a hint from a small sample, but it's the design doing what it was meant to.
Facts mostly survived. Checklist coverage dropped by 1 to 3 points, and only Full's drop clears the noise. The judges flagged fewer invented facts in Lite (7) and Full (5) than in the baseline (13), but Scoped had 17, so I wouldn't read much into that.
The judges also caught one of my own rules. The Opus judge called out the warrior send-off from my output style in several of its notes, as off-tone for second-language readers. Every arm carried it, so it doesn't skew the comparison. It's still a fair hit on my setup.
The tax
Change vs baseline, percent
Means per reply. Prose words exclude code blocks and tables. Thinking tokens come from each CLI's usage report.
This is the part I didn't see coming. Prose shrank by 4 to 18%, while thinking grew a lot more. Opus spent 51 to 66% more thinking tokens per reply. sol spent 65 to 83% more. Replies took 36 to 46% longer, and on Opus each reply cost 49 to 82% more. My guess is that the models draft, then check the draft against the rules, on every turn. Whatever the reason, you pay it on every reply, including the ones nobody else will read.
Round two
Four words instead of 2,600
The first round left me with a simpler question. Do the models already know STE if you just name it? A rule file I can't legally copy is awkward to ship. A four-word line isn't.
So round two strips everything out. No global CLAUDE.md or AGENTS.md, no output style, no hooks, no Codex memories. One arm runs each CLI bare. The other gets a single line in the run folder's CLAUDE.md or AGENTS.md: Speak only in ASD-STE100. Same eight tasks, same two models at xhigh, and three runs per cell this time, so 96 replies went past the same two blind judges.
Getting a truly bare run took a detour. Claude Code's --bare flag only accepts an API key, so I used --setting-sources project,local instead, which drops my user settings and global CLAUDE.md but still loads the run folder's file. For Codex, each run pointed CODEX_HOME at a throwaway folder holding nothing but a link to my login. My send-off rule makes a good canary: it showed up in zero of the 96 replies.
Every sentence from round two, by length in words. Above the line, the bare CLIs. Below it, the same CLIs with one line naming the standard.
The models know STE. From the name alone, sentences over 20 words fell from 8 or 9% of sentences to under 1%, and passive voice fell by more than half. The -ing forms and semicolons all but vanished, and the reading grade level dropped by two to three and a half grades. That puts it in the same range as my rule files: stricter than Lite on long sentences for Claude, and looser than Lite on passive voice for both models.
It also went further than I wanted.
Change vs bare, one line minus bare One line
Overall is on a 1 to 10 scale, the rest on 1 to 5. Paired by task, run and judge; 48 paired scores per dot. Lines are 95% bootstrap intervals.
Second-language ease jumped a full point on the 1-to-5 scale, the biggest gain in either round. Naturalness fell by 1.2, and by 1.6 for Claude. Nuance dropped by 0.8. Across both models, the judges picked bare over the one-liner 76% of the time.
Codex took the hit again. It lost 0.85 points overall, both judges agree, and bare won 90% of its pairs. It also dropped 7 points of checklist facts. Claude's overall change was small and split by judge, the same pattern as round one, and it covered slightly more checklist facts with the line than without.
The tax went up too. Claude's thinking tripled, from about 1,700 to 5,200 tokens per reply, and each reply took twice as long. Codex thought 87% more and took 48% longer. Claude's prose didn't even get shorter.
Where the one line lands
The table puts both rounds side by side on the metrics that carry across: same tasks, same models, same script. Judge scores don't carry across, because each judge only compared replies within one round.
For Claude, the one line is the strictest setup on sentence length and the most expensive in thinking. One side note surprised me. Claude with my own global rules and output style thinks about 1,200 more tokens per reply than bare Claude, and takes about ten seconds longer. Codex barely notices my setup. That comparison crosses rounds, so treat it as a hint.
What round two changes is the cost of option E. An on-demand STE rewrite doesn't need a 2,600-word skill, or any rule text I'd have to paraphrase around a license. One line that names the standard gets the style. I haven't tested that line in rewrite mode yet, on a finished document, where dropped facts are the thing to watch.
Round two replies
Bare vs one line
Same viewer, three runs per task. The left card is the bare CLI; the right card got only Speak only in ASD-STE100.
Decision
Where this leaves me
Round two didn't change the call. It made the on-demand option cheaper to build.
I'm not putting STE in CLAUDE.md or AGENTS.md. It helps one kind of reader. In exchange, every reply gets stiffer and slower, and Codex gets measurably worse. The one-line plain-language rule stays. If I write something specifically for second-language readers or for translation, I'll call an STE rewrite on that one document, and round two says that rewrite instruction can be one line that names the standard. It costs nothing on the days I don't use it.
On where a writing rule should live, the research was clear. If a rule ever needs to be always on, the global CLAUDE.md and Codex's AGENTS.md are the only place that reaches subagents, files and commit messages in both tools. An output style reaches only the main thread. A skill loads when the model decides it's relevant, which makes it a poor fit for "every reply" and a good fit for "this document".
Option
Quality
Facts, nuance
2nd-lang. ease
Natural
Cost, speed
Simple
Score
E. Keep my rule, add an on-demand one-line STE rewrite
4
5
4
5
5
4
4.45
A. Keep my one-line rule
4
5
3
5
5
5
4.45
C. Scoped, always on, both tools
3.5
4
4
2
2
4
3.38
F. Scoped, Claude only
3.5
4
4
2
2.5
3
3.30
B. Lite, always on, both tools
3
3
5
2
2
4
3.20
G. One line always on: Speak only in ASD-STE100
2.5
2.5
5
1
1
5
2.88
D. Full skill forced on
3
2.5
5
3
1
2
2.75
Scores 1 to 5 from the measured data. Weights: quality 25%, facts and nuance 20%, second-language ease 15%, naturalness 10%, cost and speed 15%, simplicity across both tools 15%. A and E tie on score; E wins because it adds a tool that costs nothing when unused.
What this doesn't show
The judges are models. They agree only moderately (r = 0.45), and each rates its own family's replies higher. That's why I only lean on results both of them back. Round one ran each cell twice and round two three times, and two runs of the same setup differed by 0.56 to 0.76 points on average, so treat the per-task numbers in the viewers as hints. Round one ran on my setup and round two on bare CLIs, so only the metrics compare across them. Every reply was a single turn with no tools, and real sessions are mostly tool work. My baseline already had a plain-language rule, so this measures what STE adds on top of a decent setup. The reading quiz used a small model, and models read long sentences fine, so it's a weak stand-in for a person reading in a second language (it moved 2 to 7 points, inside the noise). No human has rated these replies yet.
Receipts
Everything behind this page is in one repo: the dataset, the rule files, all 224 replies, every judge verdict, and the harness that ran it. Generating the replies took 8 to 11 minutes per round, with the judging and quizzes running alongside.
dataset/tasks.json the 8 tasks, checklists, quiz questions