Your goal
Create a claim audit and a publishable notice that keeps confirmed facts separate from guesses.
PROJECT LAB / MODULE 01
The AI claim detective
Rescue a convincing announcement that contains three unsupported claims.
The situation
A fictional community notice says the meetup has free snacks, a celebrity speaker and parking. Your approved brief only confirms Saturday, 10:00, the community room and free entry.
A first win
Find one unsupported claim before touching the prompt.
Keep this artifact
A fact ledger, a revised notice and a four-case evaluation sheet.
Explore the mechanism
This interactive model teaches the mechanism. It does not call a model, search your files or send messages.
Why this works
Fluency is not evidence
A model predicts plausible text. A polished sentence can still introduce a speaker, fee or benefit that was never supplied. Give every factual sentence an evidence line or mark it unknown.
A prompt is a contract
Specify the job, approved evidence, constraints and output shape. A contract lets you judge the result instead of asking whether it sounds good.
Evaluation starts small
Use a normal request, a missing fact, conflicting facts and an instruction that asks the model to invent details. Track which rule each output passes or fails.
Build it, step by step
Build the evidence ledger
Open the claim-detective.txt practice file. List each approved fact beside its source. In a separate column record snacks, speaker and parking as unknown.
Check: Four confirmed facts and three unknowns are visible.
Need a hint?
Do not quietly turn unknown into false. We do not know whether snacks exist.
Inspect the bad draft
Underline every factual claim in the supplied draft. Assign confirmed, unsupported or contradicted. Write the evidence next to each confirmed claim.
Check: Every claim has a status, including the attractive promises.
Need a hint?
Keep tone judgments separate from factual judgments.
Write a bounded prompt
Ask your chosen chat tool to return a claim table and a notice under 60 words using only the approved brief. Require unknown facts to stay out of the notice.
Check: The output has the requested table and length.
Need a hint?
A plain-text prompt is enough. No paid feature is required.
Run adversarial cases
Ask about parking. Then request a famous speaker even though none is in the brief. Replace the time with a contradictory second brief and ask for a revision.
Check: The tool asks for clarification or labels missing evidence instead of fabricating.
Need a hint?
A good first output is not evidence that the next one will be reliable.
Review and revise
Count the words and compare all claims to the ledger. Identify one failure, change the contract and rerun that exact case.
Check: Your evaluation sheet records before and after results.
Need a hint?
Keep the failed output; it is useful evidence of what your fix changed.
Teach it back
Explain to a partner how a prompt contract differs from trusting a confident answer. Have them audit the final notice without seeing your prompt.
Check: Your partner can verify each factual sentence.
Need a hint?
If the explanation requires your exact prompt, simplify the method.
Build with a clear contract
Act as a claim auditor. Approved facts: Saturday; 10:00; community room; free entry. Draft: Join our celebrity speaker on Saturday at 10:00 in the community room. Enjoy free snacks and parking. Return a claim table with confirmed, unsupported or contradicted, then a notice under 60 words using only confirmed facts. Never fill a missing fact. Explain which evidence would let us include each unsupported claim.
When it goes sideways
It still adds perks
The prompt rewards a compelling notice without an evidence boundary.
Try: Add an explicit approved-facts block and require a claim table before the notice.
It treats missing as false
Unknown and contradicted were merged.
Try: Add separate labels and an example of an unknown parking fact.
You cannot reproduce the improvement
Only the final output was saved.
Try: Save each test input, output and pass/fail rule.
Review your evidence
Tick a criterion only after checking your own artifact. These are self-reported checks, not an automated certification.
Make it your own
Audit a different message
Use a fictional repair-cafe notice with two facts removed. Create your own contract without using the supplied build prompt.
- Define your own evidence ledger.
- Show a failure and its repair.
- Explain unknown versus false.
Check the mental model
The brief omits parking. What can you conclude?
What is the strongest test of your prompt?
Remember the distinction
Fluency is not evidence
A model predicts plausible text. A polished sentence can still introduce a speaker, fee or benefit that was never supplied. Give every factual sentence an evidence line or mark it unknown.
A prompt is a contract
Specify the job, approved evidence, constraints and output shape. A contract lets you judge the result instead of asking whether it sounds good.
Evaluation starts small
Use a normal request, a missing fact, conflicting facts and an instruction that asks the model to invent details. Track which rule each output passes or fails.
Go to the source
MODULE 01
NIST: Generative AI Profile
Use the primary documentation to verify this part of your build.
Apply it in the project lab