Guide Box and Notion agents to stop retrying unavailable tools

[william-openai]

Summary

• add a Box skill guardrail that treats Tool <name> not found as terminal for that tool during the current task
• add the same Notion guardrail to all four Notion entry skills
• direct Notion agents to use search and fetch when those available fallbacks are sufficient
• normalize Notion skill, example, reference, and eval-rubric mentions of the special-cased live tools from notion-search / notion-fetch to search / fetch

Why

Box and Notion expose some MCP tools conditionally by account or workspace, while the app schema snapshot can still advertise those tools broadly. When the provider returns Tool <name> not found, retrying that same tool with different arguments or later in the task cannot recover.

Notion's ChatGPT/Codex tool surface special-cases the base tools as search and fetch. Other actions retain their notion-* names, such as notion-create-pages.

In the trailing seven-day analysis (2026-05-26 00:00 UTC through 2026-06-02 00:00 UTC):

• Box had 2,181 dynamic-tool not found failures out of 5,123 total errors. At least 447 were redundant same-turn retries.
• Notion had 547,562 dynamic-tool not found failures out of 909,812 total errors. At least 133,861 were redundant same-turn retries.

This is a guidance-level mitigation. It should reduce avoidable repeat calls after the first terminal error, but it does not prevent the initial call caused by the schema/runtime mismatch.

Validation

• git diff --check -- plugins/box plugins/notion
• rg -n "Notion:notion-(search|fetch)|\bnotion-(search|fetch)\b" plugins/notion (no matches)
• JSON validation with jq empty for all changed Notion eval files
• node plugins/plugin-eval/scripts/plugin-eval.js analyze plugins/box --format markdown (no failures; existing Box budget/structure warnings remain)
• node plugins/plugin-eval/scripts/plugin-eval.js analyze plugins/notion --format markdown (currently crashes on the unchanged existing interface.defaultPrompt string shape: ((intermediate value) || []).join is not a function)