Evaluate · Decision Making · Product Strategy · Pivot · PTOS

From Scale to Kill: 4 Permissible Outcomes of Evaluate and How to Make a Decision

A detailed breakdown of the four permissible outcomes of the Evaluate phase (Scale, Iterate, Rollback, Kill), with clear criteria for each and an understanding of why Evaluate must conclude with a decision, not just an observation.

From Scale to Kill: 4 Permissible Outcomes of Evaluate and How to Make a Decision

The Evaluate phase is not just about "looking at metrics." It's the culmination of the learning cycle that must conclude with one of four specific decisions. If you merely "noted" the results and moved on, you didn't conduct an Evaluate; you just wasted time.

In PTOS, the fundamental principle of Evaluate is:

Evaluate only exists when, based on its results, a decision has been made in advance and thresholds defined: success / failure / unclear.

Let's break down what these decisions can be and what to do in each case.

1. Scale

This is the most desired outcome. It means your change works and is ready for expansion.

• When to make this decision:
  • Success thresholds, defined before launch, are met (e.g., target metric increased by N%).
  • Guardrail metrics (side effects) are normal (e.g., support load didn't increase, retention didn't drop).
  • The effect is stable within the defined measurement window.
• What to do next:
  • Expand the rollout to the next segment or to 100% of the audience.
  • Formalize operational processes (update documentation, train the team).
  • Continue to monitor guardrails, because new, unexpected effects may appear at a larger scale.

2. Iterate

This is the most common outcome. A signal exists, but it's insufficient for a full-scale launch.

• When to make this decision:
  • The result falls into the "grey area" (e.g., metric increased, but less than expected).
  • A signal exists, but the Activation Path (path to value) breaks somewhere.
  • There are minor negative side effects that can be fixed.
• What to do next:
  • Do not scale! This is a key rule. Premature scaling of a "grey area" is scaling a problem.
  • Formulate a new, more precise hypothesis aimed at resolving the identified problem.
  • Make one next verifiable step (next bet), not a "package of improvements."
  • Create a stop-doing list: what we stop doing to maintain focus.

3. Rollback

This is a sign of a mature, not weak, team. Rollback is not failure, but a managed decision that protects users and the business.

• When to make this decision:
  • Guardrail metrics are violated (e.g., errors increased, retention dropped in a key segment).
  • The harm from the change outweighs the benefit.
  • User trust is at risk.
• What to do next:
  • Roll back the change (partially or completely).
  • Conduct a root cause analysis: what exactly went wrong?
  • Adjust the hypothesis, thresholds, or test design for the next attempt.
  • Stop all communications and promises related to this change.

4. Kill

The most difficult decision from an emotional standpoint, but often the most correct one.

• When to make this decision:
  • Failure thresholds are reached.
  • User behavior has not changed, repeatability is absent.
  • Several iterations have not brought significant effect.
• What to do next:
  • Turn off or completely remove the feature from the product.
  • Document what was learned. This is the most important step. What assumptions were wrong? What did we learn about our users?
  • Return to the Discover phase with a new, deeper understanding of the problem.

Strict adherence to this decision matrix transforms Evaluate from a ritualistic dance around dashboards into a powerful engine for product growth, based on honesty, discipline, and continuous learning.