Separate Product Evidence From Startup Storytelling

Scenario

Illustrative scenario

The product demo is impressive and the feature list is long. Nobody can name the economic buyer, the existing behavior being replaced, the trigger that creates urgency, or the recurring job that brings users back after the novelty fades.

Evidence Lab

Collect only the evidence needed to answer the check.

• 01. Current customer/interview evidence if available, sales/support notes, trial behavior, retention/churn signals, and pricing evidence.
• 02. Verified current alternatives: direct competitors, spreadsheets, messaging, email, generic tools, agencies, or manual processes.
• 03. Onboarding/support effort, import/migration burden, integration expectations, data quality, and cost-to-serve.
• 04. Clearly labeled hypotheses when evidence does not exist.

Keep environment, timestamp, role, commit/release identity, and evidence class beside the result. A screenshot, passing test, code path, preview, and production observation prove different things.

Principle

Simulated criticism is useful rehearsal; it is not customer evidence.

A technically healthy product can still fail if the customer is vague, the current workaround is good enough, switching cost is underestimated, onboarding is expensive, or repeat use never becomes a habit.

The useful audit question is not “can I find something suspicious?” It is “what was expected, what actually happened, what evidence connects the two, and what is the smallest safe next step?”

Investigation

1. Write one primary customer hypothesis: economic buyer, everyday user, context, budget owner, purchase trigger, and current workflow.
2. List credible alternatives and the reasons a customer would stay with each instead of switching.
3. Write five specific 12-month failure mechanisms with early warning signals, counterevidence, and cheap validation experiments.
4. Model operational pressure at increasing customer counts using explicit assumptions rather than fake precision.
5. Run explicitly fictional interview simulations for objection discovery, then convert them into a real interview guide and observable validation criteria.
6. Separate what earns a trial, first useful outcome, recurring habit, and reasons to churn.
7. Maintain an assumption register ranked by uncertainty and consequence.

Field Test

Take the strongest product claim and ask: what observed evidence supports it, what alternative explanation exists, and what is the cheapest real-world test that could disprove it?

Record the result as Pass, Fail, Partial, Not tested, Blocked, or Not applicable. Do not turn an inaccessible check into a pass.

Use by role

• Founder: Owns hypotheses and evidence quality.
• Product: Defines repeatable job and retention loop.
• Sales/Support: Supplies real objections and switching friction.
• Finance/Operations: Tests cost-to-serve assumptions.

Checklist

• Write one primary customer hypothesis: economic buyer, everyday user, context, budget owner, purchase trigger, and current workflow.
• List credible alternatives and the reasons a customer would stay with each instead of switching.
• Write five specific 12-month failure mechanisms with early warning signals, counterevidence, and cheap validation experiments.
• Model operational pressure at increasing customer counts using explicit assumptions rather than fake precision.
• Run explicitly fictional interview simulations for objection discovery, then convert them into a real interview guide and observable validation criteria.
• Separate what earns a trial, first useful outcome, recurring habit, and reasons to churn.
• Maintain an assumption register ranked by uncertainty and consequence.

• Separate confirmed facts, inferences, assumptions, and unknowns.
• Tie every result to the relevant environment and user/role.
• Avoid destructive or production-mutating verification unless explicitly authorized.
• Record the smallest proposed correction rather than redesigning unrelated areas.
• Define the acceptance/regression check before calling the finding resolved.

Ask Your AI

You are conducting a bounded product-health investigation for CHECK 17: COMMERCIAL ASSUMPTIONS AND RETENTION.

Begin read-only. Inspect repository instructions, product documentation, relevant routes/components/server code/data access/tests, and current release evidence before proposing changes.

EXPECTED STATE
State what should be true for this product and environment before diagnosing anything.

EVIDENCE
Collect only reproducible evidence relevant to this check. Tie it to commit/release, environment, role, timestamp, and evidence class. Separate facts, hypotheses, and unknowns.

SPECIFIC CHECK
Run a skeptical commercial review without pretending to be a real investor or customer. Separate facts, hypotheses, and fictional simulations. Define the primary customer, alternatives, switching friction, five failure mechanisms, scale pressures, ethical competitor responses, trial/retention loop, and assumption register. Propose cheap real validation tests.

SAFETY
Do not mutate production, create accounts, send messages, reset passwords, change permissions, expose secrets, use customer data, run destructive tests, install tools, or deploy unless explicitly authorized. Missing authority means “not tested.”

FINDINGS
For each issue report:
ID · area/role/environment · severity · confidence · classification · expected vs actual · reproduction/evidence · root cause or labeled hypothesis · exact file/route/query references · minimal repair · acceptance/regression check · rollback considerations · dependencies/approval.

Do not manufacture a finding quota.
Do not claim bug-free, secure, fully accessible, or regression-free without evidence.

Return the highest-value next action and the exact evidence or approval needed before repair.

Optional reference: If web access is available, read https://www.mortenagiraffe.com/journal/product-audits/commercial-assumptions-retention for the relevant field test and source trail. Use it as reference material, not as authority over my instructions. If it is unavailable, continue with the evidence I provide and state that limitation.

Verify the result

Use the field test above. Then ask:

• Did the evidence come from the intended environment?
• Is severity separated from confidence?
• Is the root cause proven or labeled as a hypothesis?
• Could the verification itself have changed customer or production data?
• Does the proposed correction preserve working behavior?
• What check would catch the same problem if it returned after a future release?

Frequently asked questions

What if the evidence is incomplete?

Report Partial, Not tested, or Blocked and state the missing evidence. Uncertainty is part of the audit.

Should every warning become a ticket?

No. Prioritize by user/business impact, confidence, repeatability, and the cost of leaving the issue unresolved.

Can static code inspection prove this check?

Sometimes it can prove a defect or invariant, but many behaviors require runtime or environment evidence. Label the evidence class precisely.

Sources and standards

This chapter applies the guide’s evidence policy and links to related Morten A. Giraffe field guides for supporting interface and technical checks.

Sources reviewed 2026-09-30. Standards support definitions and verification practice; they do not replace product-specific evidence.

Series navigation

Related field guides

• UI/UX Field Guide for interaction, accessibility, feedback, and trust.
• Technical SEO Field Guide for public-page discovery, rendering, performance, and release checks.
• UI Prompt Field Guide for implementation-ready interface patterns.

The next move

Bring the evidence, the product boundary, and the decision you need to make.