Deploy ≠ Release: Top Companies' Practices for Safe Feature Rollouts (Feature Flags, Canary Releases, Progressive Delivery)
An overview of advanced practices from Martin Fowler, Microsoft, Netflix, Google, and Etsy, which allow separating code deployment from feature release, ensuring controlled and safe rollout of changes.
Deploy ≠ Release: Top Companies' Practices for Safe Feature Rollouts
In modern product development, speed and safety are not enemies, but allies. The ability to quickly and reliably deliver value to users is a key competitive advantage. But how to do this without risking product stability and user trust? The answer lies in a simple but powerful principle: deploy is not release.
Leading technology companies such as Meta, Microsoft, Netflix, and Google have long implemented practices that allow them to roll out changes dozens of times a day, minimizing risks. Let's see how they do it and what can be applied in your work.
1. Separating Deploy and Release with Feature Flags
This is a fundamental practice. Deploy is a technical operation of deploying new code to production servers. Release is a product decision about when and to whom this new code will be available.
- How it works: New functionality is "wrapped" in a
feature flag(orfeature toggle). The code goes into production but remains inactive. It can be enabled at any time via a special admin panel, without the need for a new deploy. - Why it's cool:
- Risk reduction: If something goes wrong, the feature can be instantly "killed" (kill switch).
- Control: You decide who sees the new feature—only the internal team, 1% of users, or everyone at once.
- Acceleration: Development is not blocked by waiting for a "release window." Teams can deploy code as soon as it's ready.
2. Progressive Delivery: Rolling Out in "Rings" and "Bake Time"
Instead of rolling out a feature to 100% of users immediately (and hoping for the best), staged rollout is used.
- How it works: Microsoft calls this "rings." First, the feature is enabled for the closest group (e.g., the development team itself), then for company employees, then for a small percentage of external users, and so on. There's a pause between each stage—
bake time. - Why it's cool:
- Early problem detection:
Bake timeallows identifying problems that don't manifest immediately (memory leaks, performance issues under load). - Limiting "blast radius": If a problem does occur, it will affect only a small group of users, not the entire audience.
- Early problem detection:
3. Canary Release and Automatic Degradation Assessment
A "canary in a coal mine" is a version of your application with a new feature that is rolled out to a very small but real audience, while the majority continues to use the stable version.
- How it works: The system automatically compares key metrics (errors, latencies, conversions) between the "canary" and the main version. Netflix uses its own Kayenta system for this.
- Why it's cool:
- Data-driven decisions: Decisions about further rollout or rollback are made not based on feelings ("seems okay"), but on statistically significant data about degradation or improvement of metrics.
- Automation: Reduces human factor and accelerates decision-making.
4. Regulating Release Speed via SLO and Error Budget
Speed should not come at the expense of quality. Google SRE popularized the concept of Error Budget.
- How it works: An
Service Level Objective(SLO) is defined for each service, for example, 99.9% availability. This means you have an "error budget" of 0.1%. If over a certain period (e.g., a month) the service exceeds this budget (i.e., performs worse than promised), all new releases, except critical fixes, are halted. The team switches to improving stability. - Why it's cool:
- Balance of speed and reliability: Creates a natural mechanism that forces the team not to sacrifice quality for speed.
- Objective criteria: The decision to "release or fix" is made based on pre-agreed rules, not arguments.
Conclusion
"Frequent releases" are not heroism or chaos. They are the result of a well-thought-out process architecture where small, isolated changes are delivered to production and enabled controllably. By implementing feature flags, progressive delivery, and error budgets, you stop fearing releases and turn them from a risky event into a routine, safe operation of value delivery.