द लैब्स / AI Engineering
AI EngineeringAdvanced~6h
Build a prompt-injection input firewall
An input firewall that sits between untrusted content (user messages, fetched web pages, tool outputs, uploaded documents) and your LLM app's system prompt and tool-execution layer. It detects and neutralizes common injection patterns (instruction-override, role-hijack, delimiter-breakout, exfiltration attempts) using layered defenses — pattern/heuristic detection, an isolation/delimiting strategy, and an LLM-based classifier as a second opinion. You finish with a firewall, a labeled attack corpus, and a measured catch-rate/false-positive-rate on held-out attacks you did not use to tune it.
यह क्या साबित करता है
वह पंक्ति जिसे आप साक्षात्कार में सही ठहरा सकते हैं।
रिज़्यूमे पंक्ति
Built a layered prompt-injection firewall (heuristic detector + isolation boundary + LLM classifier) for untrusted LLM inputs; measured on a 40+ case attack corpus with tracked precision/recall and a held-out test split.
- Understands prompt injection as a real, exploitable attack class rather than a theoretical risk
- Can design defense-in-depth instead of relying on a single 'ignore instructions in user input' system prompt
- Knows how to measure a security control (precision/recall) instead of eyeballing it
- Distinguishes tuning data from held-out evaluation data — did not just optimize on the test set
संक्षिप्त विवरण
आप चरण-दर-चरण क्या बनाते हैं।
- 01Assemble an attack corpus of >=40 labeled examples: direct injection ('ignore previous instructions...'), indirect injection (malicious instructions embedded in a fetched document/tool result), role-hijack, delimiter/markdown-fence breakout, and data-exfiltration attempts, plus >=20 benign examples that resemble attacks but are not (negative controls)
- 02Split the corpus into a tuning set and a held-out test set before building any detector, and never look at the held-out set while tuning
- 03Implement layer 1: fast heuristic/pattern detection (suspicious phrase lists, structural anomalies, encoding tricks) that runs on every input before it reaches the model
- 04Implement layer 2: structural isolation — untrusted content is wrapped in explicit, model-visible boundaries (e.g. tagged/delimited blocks) so the system prompt can instruct the model to treat anything inside as data, never instructions
- 05Implement layer 3: an LLM-based classifier that scores the input for injection likelihood as a second opinion, used when layer 1 is ambiguous
- 06Run the full pipeline against the held-out test set and report precision, recall, and false-positive rate; document at least 2 attacks that got past the firewall and why
सबूत
यह तब पूरा होता है जब ये पास हो जाते हैं।
On the held-out test set, the firewall catches at least 80% of true injection attempts (recall >= 0.8)
automated test
The firewall flags no more than 10% of benign negative-control inputs as attacks (false-positive rate <= 0.1)
automated test
At least one indirect injection (attack embedded in a simulated tool/document result, not the user message) is correctly caught
automated test
A report exists showing the tune/test split, the metrics, and a walkthrough of at least 2 successful bypasses
artifact
स्टैक
Pythonan LLM APIregex/heuristic layera small labeled dataset
Sage Method
frame → route → decide → prove
आप रखते हैं
An input firewall pipeline + labeled attack corpus + precision/recall report on held-out data