Validation · Product Hypotheses · A/B Testing · MVP · PTOS

Hypothesis, Thresholds, and Cheap Tests: A Validation Protocol for Product Managers

A step-by-step guide to Validation: from formulating a 'killable' hypothesis and setting thresholds, to choosing the cheapest verification methods and making a decision (Go/No-go/Reframe).

Hypothesis, Thresholds, and Cheap Tests: A Validation Protocol for Product Managers

Validation is a critical stage in product development that allows separating "great ideas" from truly needed solutions. Its goal is not to confirm your guesses, but to create conditions where hypotheses can fail, and to do so as cheaply and early as possible.

In PTOS, Validation is a filter between an idea and expensive development. It protects team time, budget, and focus, moving the discussion from opinions to facts.

Fundamental Principle

Interest is not proof. Proof is a change in behavior or willingness to pay.

Validation Protocol: 5 Battle Steps

This protocol is a simple but strict algorithm that will help you conduct validation effectively and honestly.

Step 1. Write down a hypothesis so that it can be killed

Your hypothesis must be falsifiable. That is, there must be clear conditions under which it will be considered incorrect.

Format (DoD Validate): "If we do A, segment S will do B, metric M will change by Δ."
- A: Your proposed change (e.g., "reduce the number of fields in the registration form").
- S: Target user segment ("new mobile app users").
- B: Change in behavior ("will complete registration more often").
- M(Δ): Measurable result ("conversion to registration will increase by 5%").
Why: Clear formulation protects against vague conclusions and "fitting" results.

Step 2. Set thresholds before the test (pre-commitment)

This is the main safeguard against self-deception. Before starting the test, clearly define:

Success threshold: For example, "Conversion will increase by 5% or more."
Failure threshold: For example, "Conversion will not change or will decrease."
Decision based on result: What you will do in each case (scale / improve / roll back / delete).
Why: At the moment of receiving results, emotions can take over. Pre-defined criteria help make objective decisions.

Step 3. Choose the cheapest way to get an honest signal

You don't have to immediately launch an A/B test for millions of users. There are many tools that provide valuable signals with minimal cost.

Examples of verification methods:
- Fake-door: To check real interest in a feature.
- Prototype test (clickable prototype): To understand if users can complete the path.
- Concierge MVP: Manual service execution for a small group to check value.
- Pilot: Launch for a limited segment with clear goals.
- A/B test: To check causality when there is something to compare.
Why: Resource savings. The cheaper the test, the more hypotheses you can check.

Step 4. Collect at least 2 sources of reality

Relying on one source is almost guaranteed to lead to errors. Use data triangulation for a more reliable picture.

Examples:
- Interviews (qualitative data) + product data (quantitative).
- Observation of behavior + willingness to pay (e.g., subscribing to a waitlist with prepayment).
Why: Mixing qualitative and quantitative data provides a more complete and reliable picture.

Step 5. Make a decision: Go / No-go / Reframe

Based on the validation results, you must arrive at one of three clear decisions:

Go: Hypothesis confirmed, thresholds met, scaling risks manageable. Proceed to Build or Scale.
No-go: Hypothesis not confirmed, thresholds failed. Abandon the idea or return to Discovery.
Reframe: Results are ambiguous or new insights have emerged. Reformulate the hypothesis or return to Discovery for a deeper understanding of the problem.
Why: Validation should end with action, not stagnation in uncertainty.

This Validation protocol transforms idea testing from an intuitive process into a scientific method that allows your team to learn quickly and create products demanded by the market.