The Labs / AI Engineering
AI EngineeringWorking~5h
Build a structured-output extractor with schema-conformance guarantees
An extraction pipeline that takes messy real-world text (e.g. job postings, invoices, support tickets, resumes — pick one real domain) and reliably returns data conforming to a strict JSON schema, even when the source text is incomplete, ambiguous, or adversarially malformed. You implement schema-constrained generation, validation against the schema, and an automatic repair loop for validation failures, then prove conformance across a stress-test suite including edge cases designed to break naive parsing.
它证明了什么
你可以在面试中捍卫的一句话。
简历亮点
Built a structured-output extraction pipeline (schema-constrained generation + JSON Schema validation + automatic repair loop) achieving 100% schema conformance across a 30+ case stress-test suite including malformed and adversarial inputs.
- Can go beyond 'ask the model for JSON and hope' to actually guaranteeing conformance
- Understands JSON Schema validation and how to use it as a hard gate, not a suggestion
- Can design a repair/retry loop that fixes failures instead of just erroring out
- Tests against edge cases and adversarial inputs, not just the happy path
简介
你逐步构建的内容。
- 01Pick one real extraction domain (e.g. job posting -> {title, company, salary_range, remote, required_skills[]}) and write a strict schema (required fields, types, enums, nested arrays) using JSON Schema or a Pydantic/Zod model
- 02Implement the extractor using the model's native structured-output or tool-calling mode (not just a 'return JSON' instruction in the prompt)
- 03Validate every output against the schema programmatically; on validation failure, feed the specific validation error back to the model and retry with a bounded attempt limit (e.g. max 3 repair attempts) rather than retrying blind
- 04Build a stress-test suite of >=30 inputs: clean cases, missing-field cases, ambiguous cases (data present but unclear), and adversarial cases (text that tries to inject fake JSON or break the parser)
- 05Run the full suite and report the conformance rate, the repair-loop success rate (how many failures were fixed by retry vs. exhausted attempts), and which input patterns caused the most failures
- 06Handle the unrecoverable case explicitly: after exhausting repair attempts, return a typed error/failure result instead of silently returning invalid or null data
证明
当这些通过时，即完成。
Every output that is returned as 'success' validates against the JSON Schema with zero exceptions across all 30+ stress-test cases
automated test
At least one deliberately malformed/ambiguous input triggers the repair loop and is successfully corrected within the attempt limit
automated test
An input engineered to be unrecoverable (e.g. required data genuinely absent from the text) returns an explicit typed failure, not a silently invalid or fabricated JSON object
automated test
A conformance report exists showing pass rate, repair success rate, and a breakdown of failure patterns by input category
artifact
技术栈
Pythonan LLM API with structured-output/tool-calling supporta JSON Schema validator (e.g. jsonschema/Pydantic/Zod)
Sage Method
frame → decide → prove
你将保留
A structured-output extractor + JSON Schema + stress-test suite + conformance report