A well-organized answer can be convincing while resting on a wrong assumption. In AI-assisted research, ask which claims the decision depends on and what supports them. This is a proposed method, not a client case or a guarantee that a model will not fail.
The workflow
- 01Decision question
- 02Claims
- 03Sources and excerpts
- 04Checks and conflicts
- 05Memo and review
Start with a specific decision
“Research the market” allows almost any report to look useful. State the decision: test an offer, choose a supplier or defer a build. Define options, constraints and timing. Identify information that would change the decision.
In a fictional manual-service-versus-application choice, the question is whether the defined audience has a recurring problem and a manual trial can test it affordably. If the report will change no action, rewrite the question.
Split the question into claims
Ask what source fits, whether current data or interviews are required, and whether a claim is stated by the source or inferred. A link at the end is insufficient if you cannot tell which sentence it supports.
Use a ledger: claim ID, wording, type, source, date, excerpt and limits. Types are fact, inference, assumption and recommendation. When a material fact changes, revisit connected inferences and recommendations rather than editing one sentence.
Match the source to the claim
A current price needs the provider or a valid quote. User opinion needs sample context. Rules need the relevant official source. Marketing can explain a product but is not independent evidence of impact. Inspect originals instead of repeating summaries.
Preserve disagreements. Dates, countries or measures may differ. Expose unresolved conflict rather than choosing a convenient number. Name unverified points and who could supply the missing evidence.
Review claims before prose
NIST discusses confabulation: confident erroneous content, including misleading reasoning or citations. Separate verification from writing. A reviewer returns to the source and inspects the claim rather than only its style.
Try deliberate failures in a test copy: “Every customer prefers option one,” a link that does not support its sentence, an old fact or contrary evidence. These are workflow tests, not AI certification. If review misses them, change the rules and test again.
Make the memo inspectable
State question, options, evidence with limits, recommendation and conditions for changing it. Missing evidence calls for the smallest collection step. Do not hide a large gap beneath a strong recommendation. Temporary decisions need owners and review dates.
Use: known; unknown; options; why this recommendation; what changes our view; next step. Page count matters less than making assumptions challengeable. Link material claims to ledger rows.
Match review to consequence
Private research, publishing a claim, contacting a client, purchasing and changing a live system are different actions needing appropriate clear permission. A “final” filename is not approval. Bind review to the version and revisit material changes.
One person or agent filling all roles is sequential review, not independent review. Files help reveal errors but do not replace judgment. Complete artifacts cannot prove a decision correct.
A small exercise
Take an old AI report and choose five decision-critical sentences. Fact or inference? Which source? Does it support the claim? Would a wrong sentence change the recommendation? Start there.
Sources and limits
New article for preview review. Examples are fictional. Risk discussion draws on NIST; the proposed workflow is not NIST endorsement.