Documentation Automation Platform
A multi-agent pipeline that writes product documentation from source code and tickets — and stops to ask questions instead of guessing.
- sequential context steps required before any drafting begins
- 4sequential context steps required before any drafting begins
- codebase searches mandated per article to confirm exact product terminology
- 2+codebase searches mandated per article to confirm exact product terminology
- repositories scanned for changes feeding the refresh queue
- 6repositories scanned for changes feeding the refresh queue
- queue item processed at a time, trading throughput for correctness
- 1queue item processed at a time, trading throughput for correctness
Documentation went stale because writing it required engineers who had other work. I built the agent pipeline that drafts it from the code and the tickets, then reworked that pipeline after production use exposed the real failure: an agent given thin context writes confident, wrong prose. The fix was to make gathering context a mandatory phase with a gate where the agent must ask before it drafts.
The problem
The first version drafted from a ticket description alone — typically a one-line summary written for a project manager, not an engineer. It did not reliably read the linked engineering tickets, never searched the codebase to confirm what things were actually called, and never surfaced an ambiguity. The output was plausible and imprecise: approximate UI labels, missed edge cases, and multiple revision rounds per article. The agent was not short of capability; it was short of context, and it had no way to say so.
Constraints
- The ticket description cannot be trusted as the specification — it is written for a different audience than the documentation.
- Published articles are customer-facing, so an approximate field name is a defect, not a rough edge.
- Related engineering tickets are often not formally linked, so discovery cannot rely on link graphs alone.
- Generated documentation must stay reviewable by non-engineers rather than being published straight to customers.
Architecture
Decisions
What I chose, why, and what I turned down to get there.
A clarification gate that blocks drafting
The agent compiles a numbered list of what it cannot resolve from the available evidence and waits for answers before writing a word. This is the single most valuable behaviour in the pipeline: the failure mode of a documentation agent is not refusing to write, it is writing confidently from a gap. Making the gap explicit converts a silent assumption into a question someone can answer in seconds.
Considered and rejected
- Letting the agent proceed on best-effort assumptions — faster per run, and the source of every revision round it was meant to eliminate
Mandate codebase search before drafting, not as a fallback
At least two searches per article, one broad on the feature and one narrow on a specific component or configuration key. Documentation has to use the product's exact terminology, and the only authoritative source for what a field is called is the code that names it. Tickets describe intent; the codebase describes what shipped.
Search for related tickets rather than trusting links
A bounded keyword search over recently-tested tickets discovers work that is relevant but was never formally linked, deduplicated against what the link graph already returned. Bounding it by recency and status keeps the context pool relevant instead of merely large — an unbounded search would trade one context problem for another.
Distil the writing rules from published work
Rather than hand-authoring a style guide, the rules were derived from the last several published release notes — how entries open, the tone and sentence length, the recurring constructions. Critically, patterns that violated the existing rules were noted and deliberately not encoded, so the result matches the house voice without inheriting its bad habits. Style guidance written from scratch drifts from the real corpus; this cannot.
Considered and rejected
- Hand-written tone guidance — already existed, and was what the output was drifting from
Validate formatting in the workflow, not only in the prompt
A malformed comment format persisted after the prompt was corrected, so a validation and correction layer went into the workflow node itself. When a model is the only thing enforcing a structural contract, that contract is advisory. Defense in depth is cheaper than a class of formatting bugs that reappear whenever the prompt changes.
Keeping the refresh bounded
A collector scans merged pull requests across six repositories, extracts which product areas changed, and maintains a queue with per-item state — first seen, last seen, last run, retry count. A scheduled runner then processes that queue rather than regenerating everything. Documentation refresh is incremental work, and treating it as a full rebuild would waste most of every run.
- The queue is processed one item at a time, sequentially, on purpose: parallel runs collided over shared template files. Throughput was the correct thing to trade for correctness here, because the deadline is weekly.
- Item names are canonicalized and deduplicated, so repeated pull requests touching the same area update one queue row instead of creating several.
- Retries and failure handling live in the queue rather than in the agent, so a transient failure does not lose the work.
Why the review surface matters
Drafted documentation is proposed as a pull request and reviewed before publication, never published directly. The whole pipeline is built on the assumption that an agent's output needs a human decision at the end — which is what the review application exists to make practical rather than painful.
Stack
- n8n
- Model Context Protocol
- Python
- CLI agents
- Confluence
- Zendesk
- GitHub
- AWS Bedrock
What was mine
The pipeline, the context-enrichment design, the queue model, and the agent prompts are mine, built out from an epic opened by engineering leadership. The documentation team owns the published articles and the review decision.