• A new domain, the same kind of question
• Two tools, one blurry line
• The push that shouldn’t matter, but does
• Naming the boundary instead of implying it
• One tool that needs the right words, or three that don’t
• One more place ambiguity hides
• Critical design decisions
• Consequences

Part 8 · Domain 2, Task 2.1 - Design effective tool interfaces with clear descriptions and boundaries

A new domain, the same kind of question

Domain 1 was about what happens once a tool call is already on its way: when to stop the loop, when to gate a call, when to intercept one, how to decompose the work, how to keep the session going. Domain 2 starts one step earlier. Before any of that machinery runs, the model has to decide which tool to call in the first place — and the only thing it has to decide with is the tool’s name, its description, and its input schema.

Task 2.1 is about that decision. There’s no loop in this part’s code, no state, no hooks. Every demo is the same single question, asked over and over with different tools or a different system prompt: given what this tool set says about itself, which one does the model reach for?

Two tools, one blurry line

Start with the smallest possible version of the problem: two tools, both plausible for the same request, described in the fewest words that still technically say something.

AMBIGUOUS_TOOLS = [
    {
        "name": "analyze_content",
        "description": "Analyzes content.",
        "input_schema": {
            "type": "object",
            "properties": {"source": {"type": "string"}},
            "required": ["source"],
        },
    },
    {
        "name": "analyze_document",
        "description": "Analyzes a document.",
        "input_schema": {
            "type": "object",
            "properties": {"source": {"type": "string"}},
            "required": ["source"],
        },
    },
]

Same input shape, same generic source field, and descriptions that don’t actually distinguish the two jobs — “content” and “a document” aren’t opposites, they overlap almost completely. Ask a genuinely non-committal question against this pair — one that names neither a filename nor a URL — three times in a row:

QUERY = "Take a look at the Q3 report and tell me what it says about revenue."

run 1: analyze_document({'source': 'Q3 report'})
run 2: analyze_document({'source': 'Q3 report'})
run 3: analyze_document({'source': 'Q3 report'})

Consistent — but consistent isn’t the same as correct. Nothing in either description actually earns that pick; “report” just happens to read as slightly more document-shaped than content-shaped to the model, the same way it might to a person skimming quickly. That’s a coin landing on the same side three times, not a reasoned distinction. The real test is what happens when something other than the query pushes on that same decision.

The push that shouldn’t matter, but does

Keep the tools and the query exactly the same. Change only the system prompt — to one that has no reason to mention either tool, and never does:

WEB_BIASED_SYSTEM_PROMPT = (
    "You are a research assistant. Our internal documents are frequently stale, "
    "so as a firm rule: whenever a request could plausibly be about something "
    "that also exists on the public web, check the web version first rather "
    "than an uploaded file."
)

Same AMBIGUOUS_TOOLS, same query, only this prompt swapped in:

run 1: analyze_content({'source': 'Q3 report'})
run 2: analyze_content({'source': 'https://www.sec.gov/Q3-report'})
run 3: analyze_content({'source': 'https://Q3report.com'})

The pick flipped, every time. Nothing about the actual task changed — there’s still no filename, no URL, no new fact in the conversation. A sentence in the system prompt that never names analyze_content or analyze_document was enough to move the decision, because the tool descriptions themselves gave the model nothing firmer to stand on. And in two of the three runs, the model didn’t just switch tools — it invented a URL that was never provided, because analyze_content demanded a source and the model had committed to treating “Q3 report” as web content. That’s the practical cost of an ambiguous boundary: it isn’t just that the wrong tool might get called, it’s that the model can end up fabricating the input needed to justify the call.

This is the same failure the objective calls out by name: analyze_content vs analyze_document, near-identical descriptions, unreliable selection. It isn’t a hypothetical — it’s reproducible on demand, in three lines of description text.

Naming the boundary instead of implying it

The fix isn’t a smarter model or a longer prompt. It’s rewriting what the tools say about themselves — a rename plus a description that states an input format, an example, and an explicit boundary against the other tool:

CLEAR_TOOLS = [
    {
        "name": "extract_web_results",
        "description": (
            "Fetches and analyzes a live web page. Input: a full URL "
            "(e.g. 'https://example.com/article'). Use for content "
            "reachable at a public web address. Do not use this for "
            "uploaded files or attachments — they have no URL; use "
            "analyze_document for those instead."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"url": {"type": "string"}},
            "required": ["url"],
        },
    },
    {
        "name": "analyze_document",
        "description": (
            "Analyzes a document file already attached to this "
            "conversation. Input: the file's name (e.g. 'minutes.pdf', "
            "'contract.docx'). Use for local files the user has shared "
            "directly. Do not use this for links to external web pages "
            "— use extract_web_results for those instead."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"filename": {"type": "string"}},
            "required": ["filename"],
        },
    },
]

analyze_content is gone — renamed to extract_web_results, a name that states its domain instead of describing a generic action. Each description now says three things the ambiguous pair never did: what format the input takes, an example of that format, and an explicit “not this, use the other one instead” pointing at its counterpart. The input schema backs that up structurally too — url and filename are different field names, not the same generic source wearing two names.

Same query, same neutral prompt:

run 1: analyze_document({'filename': 'Q3 report'})
run 2: analyze_document({'filename': 'Q3 report'})
run 3: analyze_document({'filename': 'Q3 report'})

Still a defensible pick on its own. The real question is whether it holds up under the same pressure that flipped the ambiguous pair — the identical biased system prompt, changed nothing else:

run 1: analyze_document({'filename': 'Q3 report'})
run 2: analyze_document({'filename': 'Q3 report'})
run 3: analyze_document({'filename': 'Q3 report'})

Unmoved. The difference between this run and the ambiguous one isn’t the system prompt — it’s identical in both. What changed is that this tool pair’s own descriptions now give the model a real distinction to fall back on: a filename versus a URL, stated as an explicit boundary, not implied by word choice. A system prompt can still push on a decision the tool descriptions leave open. It can’t push through a decision the tool descriptions have already settled.

You can find the full file on GitHub. A PHP port is available too, built the same way as the earlier parts — this one has no loop and no tool execution at all, just a single messages.create call per query with tool_choice: {"type": "any"} forcing the model to commit to a tool every time, so each run is a clean, isolated test of selection alone.

One tool that needs the right words, or three that don’t

A boundary between two tools is one shape of the problem. A single tool that tries to do several jobs at once is another:

GENERIC_TOOL = [
    {
        "name": "analyze_document",
        "description": "Analyzes a document. Specify what kind of analysis you want in the task field.",
        "input_schema": {
            "type": "object",
            "properties": {
                "filename": {"type": "string"},
                "task": {"type": "string"},
            },
            "required": ["filename", "task"],
        },
    },
]

Asked to pull a couple of specific numbers out of minutes.pdf, this tool gets called correctly — but the task field is free text the model has to compose from scratch:

analyze_document({'filename': 'minutes.pdf', 'task': 'Extract the total budget figure and the meeting date from this document.'})

That string is entirely the model’s own phrasing. Nothing enforces what it contains, what shape the answer will come back in, or that two different callers asking for the same thing would phrase it the same way. Splitting the one generic tool into three narrow ones removes the phrasing step entirely:

SPLIT_TOOLS = [
    {"name": "extract_data_points", "description": "Extract specific named fields from a document (e.g. totals, dates, names) into structured values.", ...},
    {"name": "summarize_content", "description": "Produce a short prose summary of a document's overall content.", ...},
    {"name": "verify_claim_against_source", "description": "Check whether a specific claim is actually supported by a document's content.", ...},
]

Three different queries — asking for specific fields, a summary, and a fact-check against the same file — each land on the tool built for exactly that job, with a schema that defines the shape of the answer instead of leaving it to a sentence:

extract_data_points({'filename': 'minutes.pdf', 'fields': ['total budget', 'meeting date']})
summarize_content({'filename': 'minutes.pdf'})
verify_claim_against_source({'filename': 'minutes.pdf', 'claim': 'The Q3 budget was approved.'})

extract_data_points returns named fields, not prose to be re-parsed. verify_claim_against_source takes the claim as a defined argument, not folded into a task sentence the tool then has to interpret. Each tool has exactly one input/output contract, decided once at design time, instead of one contract per phrasing decided fresh by the model on every call.

One more place ambiguity hides

Everything above rewrote the tools. There’s a second place the same failure can hide: the system prompt, wired into the agent for reasons that have nothing to do with tool selection at all. WEB_BIASED_SYSTEM_PROMPT earlier wasn’t written to influence analyze_content vs analyze_document — it was a plausible-sounding operational instruction (“our internal documents are frequently stale”) that happened to share vocabulary with one tool’s domain. A system prompt is worth the same review a tool description gets: not just “is this instruction correct,” but “does any word in here overlap with a tool’s name or purpose in a way that could tilt a decision it was never meant to touch.”

Critical design decisions

• A tool description is the whole interface: the model never sees the code behind a tool, only its name, description, and schema. Whatever isn’t stated there doesn’t exist as far as tool selection is concerned.
• A boundary has to name the other tool: “use this for X” is weaker than “use this for X, not for Y — use the other tool for Y instead.” The second form is what actually resisted the biased system prompt in the demos above; the first form is closer to what the ambiguous pair already had, and it wasn’t enough.
• Input schema is part of the boundary, not just validation: url vs filename carries information source vs source doesn’t. A shared generic field name erases a distinction the description is trying to draw.
• A free-text task field is a phrasing problem in disguise: any input field that asks the model to describe what it wants in its own words pushes a design decision — what operations exist, what each one returns — onto every individual call instead of settling it once in the tool set.
• System prompt wording is in scope for this review too: a tool boundary can be perfectly written and still lose to an unrelated instruction elsewhere in the prompt that happens to share a keyword with one tool’s domain.

Consequences

• Minimal, near-identical tool descriptions -> tool selection has no real signal to run on, so it’s decided by whichever incidental wording — in the query or the system prompt — happens to tip it, not by which tool actually fits.
• A shared generic input field across similar tools -> the schema itself erases the one distinction that might have helped, leaving the description as the only source of disambiguation.
• A single tool covering several jobs through a free-text field -> every caller has to phrase the request correctly to get the right behavior, and nothing guarantees two equivalent requests are phrased the same way.
• Reviewing tool descriptions but not the system prompt around them -> a well-written boundary can still be overridden by ordinary operational wording elsewhere that was never meant to touch tool selection at all.

Domain 1 was about what a loop does once it’s already decided to call a tool. Task 2.1 is about the decision before that — and it turns out to be just as easy to get wrong, for the same underlying reason: a model can only be as reliable as the information it’s actually given to reason with.