Labs / AI Engineering
AI EngineeringWorking~5h
Build a function-calling router that rejects bad arguments before any tool runs
A router that takes a natural-language request, selects the correct tool from a registry of 5+ tools with distinct typed schemas (including tools with overlapping or confusable purposes), and validates the model's proposed arguments against the tool's schema before execution — checking types, required fields, enum constraints, and value ranges. Invalid or underspecified calls are rejected with a structured error fed back to the model for a repair attempt, and only schema-valid calls ever reach the actual tool execution layer.
Что это доказывает
Строка, которую вы сможете защитить на собеседовании.
Строка для резюме
Built a function-calling router with JSON-schema argument validation across 5+ tools, blocking 100% of malformed tool calls pre-execution and driving a self-correction retry loop, verified against a 25-case test suite of ambiguous and adversarial routing requests.
- Can design a tool registry with clean, unambiguous schemas and correctly route between tools that plausibly overlap
- Treats schema validation as a hard gate before execution, not a soft prompt-level suggestion
- Builds a repair loop that feeds validation errors back to the model instead of failing silently or crashing
- Tests routing correctness against deliberately ambiguous and malformed inputs, not just the happy path
Краткое описание
Что вы создаёте, шаг за шагом.
- 01Define a registry of 5+ tools with typed JSON schemas, including at least 2 pairs of tools whose names or purposes are plausibly confusable (e.g. get_order_status vs get_order_history)
- 02Implement routing: the model selects a tool and proposes arguments from a natural-language request, and the router captures the raw proposed call before any execution
- 03Implement a hard validation gate that checks the proposed arguments against the selected tool's schema (types, required fields, enums, ranges) and blocks execution on any violation
- 04On validation failure, return a structured error describing exactly what's wrong and feed it back to the model for one repair attempt, re-validating the repaired call before allowing execution
- 05Build a 25+ case test suite covering correct routing, tool-confusion cases, missing required fields, wrong types, out-of-range enum/values, and at least 3 adversarial cases where the request tries to talk the model into skipping a required field or using an invalid value
- 06Log every routing decision, validation result, and repair attempt so a run can be replayed and audited
Доказательство
Готово, когда эти проверки пройдены.
The router selects the correct tool for all unambiguous routing cases in the test suite
automated test
Every malformed or schema-invalid tool call in the test suite is blocked before reaching tool execution, with zero invalid calls executed
automated test
At least 2 of the tool-confusion test cases route to the correct tool despite plausible overlap with a similarly-named tool
automated test
A repair attempt on a validation failure produces a re-validated, schema-valid call in at least 3 of the deliberately malformed test cases
automated test
Стек
Pythonan LLM API with function/tool callinga JSON Schema validation library
Метод Sage
frame → route → decide → prove
Вы сохраняете
A schema-validated function-calling router + repair loop + 25-case routing/validation test suite with audit log