Canary Release Benefits

The term “canary deployment” comes from an old coal mining technique. These mines often contained carbon monoxide and other dangerous gases that could kill the miners. Canaries are more sensitive to airborne toxins than humans, so miners would use them as early detectors. The birds would often fall victim to the gas before it reached the miners. This approach helped ensure the miners’ safety—one bird dying or falling ill could save multiple human lives. In canary development, the small part of user base which receives a new version acts as the canary. It detects potential bugs and disruption without affecting every other system running. Canary releases are typically short-lived and used to validate whether a release meets the requirements it set out to.

Wait, but what is a canary deployment or a canary release?

Canary releases (also “canary deployments”) and feature flag rollouts are two common feature release strategies for testing in the production environment, increasing the safety of continuous delivery, deploying faster and more often.

Canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure and making it available to everybody.

The initial steps for implementing canary deployment are: create two clones of the production environment, have a load balancer that initially sends all traffic to one version, and create new functionality in the other version. When you deploy the new software version, you shift some percentage – say, 10% – of your user base to the new version while maintaining 90% of users on the old version. If that 10% reports no errors, you can roll it out to gradually more users, until the new version is being used by everyone. If the 10% has problems, you can roll it right back, and 90% of your users will have never even seen the problem.

Also, Canary releases can be used as a way to implement A/B testing due to similarities in the technical implementation. However, it is preferable to avoid conflating these two concerns: while canary releases are a good way to detect problems and regressions, A/B testing is a way to test a hypothesis using variant implementations. If you monitor business metrics to detect regressions with a canary, also using it for A/B testing could interfere with the results. On a more practical note, it can take days to gather enough data to demonstrate statistical significance from an A/B test, while you would want a canary rollout to complete in minutes or hours.

Canary release benefits

• Limits possible errors from any operational problems when there is a gradual deployment of the new functionality.
• Reduces risk of negative outcomes impacting a large percentage of user base when releasing a new functionality to users.
• Automates the entire release life cycle.
• Helps increase value if the system, application, and user behaviour is observable and well-instrumented.
• Manages incompatibilities between API versions and database schema changes, supporting a good testing strategy.