Documentation QA
This section covers the quality assurance process for the NimbusWiz technical documentation. The process verifies documentation claims against the prototype source code, produces QA reports, and drives a targeted correction pass.
The QA pipeline
I drafted nimbuswiz-doc-audit, a structured AI prompt ("skill") built for Claude that executes the following tasks:
- Reads the NimbusWiz component tree directly
- Walks every documented UI claim against the corresponding source file
- Distinguishes unimplemented prototype controls from genuine documentation errors
- Writes two structured report files
The audit cross-references every documented claim against what the prototype actually renders. Because the skill is repeatable and deterministic in its output format, each run produces a comparable dated snapshot rather than a one-off narrative review.
The audit checks the following parameters:
- Page titles and
<h1>and<h2>headings - Button and CTA labels
- Sidebar link names and navigation destinations
- Form field labels, step labels, and step order
- Filter dropdown option text
- UI patterns (modal vs. banner vs. side panel, wizard vs. flat form)
- Features described in docs but absent from the prototype
- Features in the prototype not covered in the docs
Each claim gets one of four status markers:
| Symbol | Meaning |
|---|---|
| ✅ | The documented claim matches the prototype exactly. |
| ❌ | The documented claim differs from what the prototype renders or does. |
| 🔍 | Not found in source. The described element was searched for and is absent from the prototype. |
| ➕ | In the prototype source but missing from the docs. |
The audit separates prototype scope gaps (controls with no onClick handler) from genuine doc-vs-prototype mismatches. Scope gaps go into a dedicated table in the flow analysis report, not into the mismatch count. This keeps the accuracy numbers meaningful: a button that isn't yet wired is a scope decision, not a documentation error.
After the audit, a targeted correction pass uses the two reports as inputs. The correction agent applies specific fixes for each finding: wrong labels, wrong page names, incorrect navigation sequences, and missing planned-feature notes. It doesn't rewrite pages from scratch.
The audit is designed to run repeatedly. Each run produces new dated files alongside the previous ones, so you can compare snapshots and track whether issues have been fixed or regressed.
The two report outputs
Accuracy report
A per-page inventory of every verified claim, organized by guide and page area.
The first audit found issues in button and field labels, page names, and feature flows. Several documented features (the First Run Setup Wizard, a Confirm Rollback dialog, time-range filters) had no corresponding implementation in the prototype. Several prototype surfaces (the Developer Hub tabs, Settings security tab, Terraform integration) had no documentation at all.
Following corrections based on the audit, I ran a second audit. Claim accuracy improved from roughly 12% to roughly 79%, and no regressions were found.
Flow analysis report
A task-flow comparison between what the docs describe and what the prototype wires up.
For each documented workflow, the report shows what the docs instruct the user to do, what the user actually encounters in the prototype, and a navigability verdict. It also includes a prototype scope gaps table and a confirmed prototype bugs list.
View the complete PDF.
Human in the loop
The final sign-off on the docs QA process is always the human, not AI. A human check of accuracy and usefulness of every new piece of content to be published is non-negotiable.