Working with AI, part 1: why AI assistants start using your project's made-up names and codes, how to keep replies in plain English, and a Claude Code script that checks each reply

Work with an AI assistant on one project for long enough and it starts replying with names and codes you made up weeks ago. The replies make sense to the AI and to nobody else, and sometimes not even to you. Asking it to write plainly works for a few replies, but then the made-up names come back. What lasts is an instruction saved in the assistant's settings, plus the habit of asking "what does that mean?" every single time. In Claude Code, a short script can also check each reply for you.

This is part 1 of Working with AI, a short series on what goes wrong when you use Claude, ChatGPT or similar tools for real work, and what to change.

An everyday example

Say you're planning a kitchen renovation with ChatGPT. Early on you call the cheaper layout "Plan B" and the supplier with the long waiting list "the slow guys". Two weeks later the assistant writes: "Plan B still works if the slow guys confirm by Friday."

You know what that means. Your partner won't, and neither will the builder if you forward it. A month from now you may not be sure yourself.

What happened to me

I use Claude on a long robotics project: software that lets several robots work together, with machine learning helping them decide how to move. One main assistant coordinates several helper agents, which are separate copies of the AI that it hands smaller jobs to. A few weeks in, most replies were full of our own made-up names. Tests had code numbers instead of names, and ideas had two-word names we'd invented along the way. A typical reply looked like this (the names here are invented):

Q7 passed, so the green path holds.

Written for a stranger, it says:

The seventh test passed, so the approach we chose on Monday still works.

I had to stop and work out the first version, and it meant nothing to anyone I forwarded it to. Asking for plain English worked for a few replies, then the codes came back.

Made-up names can also hide mistakes. Repeat a name often enough and it sounds like an agreed fact, even when nobody remembers what it stands for. The next post in this series is about one such mistake.

Why it happens

The assistant doesn't keep track of which words are yours and which are everyone's. Your made-up names are short, they point at exactly the right thing, and they're all over the conversation, so to the AI they look like good words to use.

A one-off request to "write plainly" is one message among hundreds, and it gets a smaller share of the AI's attention as the chat grows. Many tools also summarise or drop older parts of a long chat, and your request can go with them. Helper agents add to it: the made-up names travel with each handover and look a bit more official each time.

What to do

1. Put the request in your saved instructions. Use ChatGPT's custom instructions, a Claude Project's instructions, or a CLAUDE.md file in Claude Code (Anthropic's coding tool). These are included in every conversation, so they don't get lost like a chat message.
2. Ask "what does that mean?" every time. Don't work it out yourself and move on. Asking makes the assistant write the meaning out and reminds it that you want plain words.
3. Name things so a stranger could guess what they are. "The cheaper layout" is better than "Plan B".
4. Imagine forwarding it. If someone who wasn't there couldn't follow a summary, ask for a rewrite before you act on it.
5. Start a new conversation when made-up names pile up. Open it with a short summary in plain words, and the old names stay in the old chat.

Paste this

Add this to your custom instructions or project instructions, and change it to suit you:

## How to write to me
- Write for someone who hasn't seen this conversation.
- Don't use code numbers or names we made up. Describe
  the thing in plain words.
- If a code or name has to appear (a file name or product
  code I'll search for), put its meaning right next to it.
- Before sending a summary, check that it would make sense
  if I forwarded it to a colleague.
- If I ask "what does that mean?", rewrite the whole reply
  in plain words.

It won't stop every made-up name, but they come back much less often, and you have a written rule to point at.

If you use Claude Code

This section has code. If you don't use Claude Code, skip to the next section: the habits and the instruction above do most of the work.

Claude Code can run small scripts of your own at set moments, called hooks. A Stop hook runs each time Claude finishes a reply. It can let the reply through, or send it back with a reason, and Claude writes it again. Save this example as ~/.claude/hooks/plain-words.py and put your own names in the list:

#!/usr/bin/env python3
"""Stop hook: send a reply back if it uses our made-up project names."""
import json
import re
import sys

# Your made-up names. Matching ignores upper and lower case.
MADE_UP = ["Q7", "B12", "green path", "the slow guys"]


def main():
    data = json.load(sys.stdin)
    if data.get("stop_hook_active"):
        return  # this is already the rewrite: let it through
    reply = data.get("last_assistant_message") or ""
    reply = re.sub(r"`[^`]*`", " ", reply)  # skip text in backticks
    found = [name for name in MADE_UP
             if re.search(r"\b" + re.escape(name) + r"\b", reply, re.I)]
    if found:
        print(json.dumps({
            "decision": "block",
            "reason": "Your reply uses made-up project names: "
                      + ", ".join(found)
                      + ". Rewrite it in plain words for someone new."
        }))


try:
    main()
except Exception:
    pass  # if the check itself breaks, let the reply through

Claude Code hands the script the finished reply as last_assistant_message. Three details matter:

• Text in backticks is skipped. Our own check once sent back a reply that was explaining which names were banned, and so had to list them. Backticks let Claude mention a name on purpose, as in "open Q7.csv (the seventh test)".
• When a reply is already the rewrite, Claude Code sets stop_hook_active to true and the script lets it through. That way it can't loop.
• If the script itself breaks, the reply goes through.

Then switch it on in ~/.claude/settings.json:

{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          { "type": "command", "command": "python3 ~/.claude/hooks/plain-words.py" }
        ]
      }
    ]
  }
}

To test it, go to ~/.claude/hooks and feed it a plain reply, a reply with made-up names, and the same reply as a rewrite (Python 3.12.3):

echo '{"stop_hook_active": false, "last_assistant_message": "The seventh test passed."}' | python3 -I plain-words.py; echo "exit=$?"
echo '{"stop_hook_active": false, "last_assistant_message": "Q7 passed, so the green path holds."}' | python3 -I plain-words.py; echo "exit=$?"
echo '{"stop_hook_active": true, "last_assistant_message": "Q7 passed, so the green path holds."}' | python3 -I plain-words.py; echo "exit=$?"

exit=0
{"decision": "block", "reason": "Your reply uses made-up project names: Q7, green path. Rewrite it in plain words for someone new."}
exit=0
exit=0

No output means the reply goes through. Only the second is sent back.

A tool that checks your writing

avoid-ai-writing is a free, open-source tool that removes common AI writing habits, such as stock phrases and padding. It works as a Claude Code skill (a set of instructions Claude loads for that job), and includes a detector you can run on a file. To install it:

git clone https://github.com/conorbronsdon/avoid-ai-writing ~/.claude/skills/avoid-ai-writing

Then ask Claude to "remove AI-isms from post.md", or run the detector, which needs Node.js:

node ~/.claude/skills/avoid-ai-writing/bin/avoid-ai-writing.js --source-mode rendered-markdown post.md

It prints a score out of 100 (lower is better) and what it found.

It looks for AI writing habits, not your made-up names, but you can add those. The skill accepts a house style file through a --style option, and that file's register part is a list of plain-word instructions. Save something like this as house.json:

{
  "name": "My project",
  "genre": "replies and notes about my project",
  "register": [
    "Never use these made-up project names: Q7, B12, green path, the slow guys. Describe the thing in plain words instead.",
    "Write for someone who hasn't seen the project."
  ],
  "mechanics": {
    "quotes": "straight"
  }
}

Then ask Claude: "Remove AI-isms from post.md with --style ./house.json." The mechanics part is required; it sets rules like the kind of quotation marks, and the repo's scripts/check-style.js checks those. Nothing checks the register list. Claude follows it while rewriting, and a file that still says "Q7" passes the script. Treat it as an instruction, like the one in "Paste this", not a guarantee.

The detector can't tell when a normal word is used with the wrong meaning, either. An earlier draft of this post scored 1 out of 100, and a person reading it still found a word used in a sense it doesn't have. Before you send anything important, have someone from outside the project read it.