Sentinel Test Generation Agent
A multi-step agent that writes test cases from tickets, grounded in a code graph — and refuses to let untested work proceed downstream.
- hourly per-host write limits on ticketing and test management
- 10 / 100hourly per-host write limits on ticketing and test management
- ticket-to-case mapping that makes coverage checkable rather than assumed
- 1 → manyticket-to-case mapping that makes coverage checkable rather than assumed
- grounding sources: code graph, historical analyses, existing case corpus
- 3grounding sources: code graph, historical analyses, existing case corpus
Test coverage lagged because writing cases by hand competed with shipping. Sentinel generates them from the ticket, grounds itself in a graph of the actual code, and hands reviewed cases to the test management system. The part I care about most is the contract: automation downstream fails loudly when the cases it should be tagging do not exist.
The problem
Test-case generation and test automation ran as two pipelines against the same ticket, with nothing enforcing that one had happened before the other. Cases were generated but not always linked; automation ran without the coverage tags it needed. Reconciling the two was manual, so it was skipped, and coverage gaps only surfaced later as missing traceability on priority work.
Constraints
- Context sources — historical root-cause analyses, the existing case corpus, the code itself — are only reachable through tool calls, not direct database access.
- Generated cases land in a system of record that real QA processes depend on, so a malformed upload is worse than no upload.
- Priority coverage requires traceable identifiers linking automated scenarios back to the cases they satisfy.
- Write access to ticketing and test-management systems needs rate limiting, because an agent loop can generate load a human never would.
Decisions
What I chose, why, and what I turned down to get there.
Fail the downstream pipeline instead of proceeding without coverage
The ticket key became the integration contract: generation persists a one-to-many mapping from ticket to case identifiers, and automation reads that mapping before it starts. If no cases are linked, it stops with a structured validation error rather than running and reporting success on untested work. A pipeline that silently proceeds without coverage is worse than one that stops, because it produces false confidence.
Considered and rejected
- Merging both pipelines into one system — larger blast radius and a rewrite, when a shared key and a hard check achieve the actual goal
A code graph rather than text search for grounding
An evaluation of a lighter option came first and was rejected: it could not express the relationships that mattered. The adopted approach indexes files, classes, and functions as nodes with calls, imports, and inheritance as edges, so the agent can answer which code paths a change actually reaches. Test generation needs to know what a change touches, and that is a graph traversal, not a similarity match.
Considered and rejected
- A simpler code-indexing tool — evaluated first, rejected for insufficient relationship depth
Parallel tool calls within a single agent turn
Gathering context from several sources sequentially made each turn slow enough that people noticed. Allowing concurrent calls in one turn cut that latency directly. Agent responsiveness is mostly a function of how tool calls are scheduled, not how fast the model is.
Fix the parser rather than the symptom
Steps were splitting incorrectly on a delimiter. The cause was HTML-entity-encoded angle brackets being broken up by a naive parser — so the fix was decoding plus a lookahead pattern, verified against known cases with before-and-after step counts. Separately, a priority field was being set from a hardcoded value that did not match the target system's project-specific identifiers. Both were data-contract bugs at a boundary, which is where this class of defect concentrates.
What is shipped and what is not
This is the longest-running agent in the portfolio and it is still moving, so the honest description separates delivered work from planned work. Delivered: generation from tickets, upload to the test management system with correct folder handling and labelling, code-graph grounding, parallel tool calls, and an end-to-end run validated with a QA engineer against real tickets rather than only internal testing.
- In progress: extending generation to the core product security domain, with a reviewer sub-agent that critiques generated cases before a human sees them.
- Designed and not yet built: the enforced traceability contract described above, and automated remediation of failed test analysis.
- Presenting a roadmap as shipped would be the same failure this portfolio argues against everywhere else. The delivered core is real; the rest is stated as intent.
Rate limits as a design input
Write paths carry per-hostname limits — ten requests an hour on ticket writes, a hundred on test-management writes. An agent that retries is an agent that can hammer a system of record, and the limit is what makes giving it write access defensible in the first place.
Stack
- n8n
- Model Context Protocol
- Neo4j
- CLI agents
- Python
- Jira
- AWS Bedrock
What was mine
The agent, its tools, the grounding design, and the integration contract are mine. Test cases are reviewed and owned by the QA engineers who use them, and the validation run was done jointly with a QA engineer.