Question 140
A tool's description states it edits code, but the backend actually runs tests and never writes files. What is the primary problem?
Explanation
Correct Answer
Tool descriptions are a planning interface: the model uses them to decide which callable can perform a task. If the description promises edits but the backend only runs tests, the model can dispatch the wrong operation, infer nonexistent changes and confuse result handling. Align the advertised capability, parameter schema, permission class and response semantics with the real implementation. A test tool should return actual test outcomes; an edit tool should report the files it actually changed. Validate the contract with a representative call rather than trusting its English label.
Incorrect Answers
Markdown quantity is not the problem when a tool named or described as an editor actually runs tests without writing source. Even a beautifully formatted result cannot correct the model's mistaken expectation about side effects. The operator needs the tool contract to state what the callable really does, the arguments it takes and the evidence it returns. Otherwise an agent may report that a patch was applied when no file changed.
Optional parameters affect how a tool accepts requests, but adding more fields would not correct a fundamentally inaccurate behavior description. The model is choosing an editing capability based on documentation that conceals the backend's read-only test execution. Fix the registered tool name, description and output contract to reflect actual behavior, then validate routing and observed side effects. More argument flexibility cannot turn a test runner into an editor.
A long context window might affect model attention, but it cannot explain why the registered tool says it edits code while its implementation never writes. Even a model with perfect recall would select a misleading capability if the catalog describes it inaccurately. Correct the contract and test the resulting tool behavior. Do not attribute a concrete registry mismatch to general prompting or model memory limitations.



Claude Certified Architect - Foundations (CCA-F) Practice Tests