Feature toggling provided by Unleash feature toggle service is a fantastic technique. It decouples the process of putting new code into production from the process of enabling new functionality to your users. Turning on features in production, at run time, keeps you in control. This way of working reduces risk because you are now able to turn off feature toggles in seconds if problems are detected. It is usually much faster to disable a feature toggle than rolling back the application to the previous version or rolling out a new version to fix the problem.
Even though you are able to disable a misbehaving feature toggle within seconds, you still have the human factor. A person needs to detect that the feature is misbehaving. The problem then have to be correlated to the problem with the feature toggling being updated. What if she forgets to check the logs or the metrics?
Our friends over at Keptn have integrated their framework for continuous delivery and automated operations with Unleash. This allowing to automatically react on production issues with state changes of feature toggles in Unleash feature toggle service. Read on to learn more about Keptn, and the benefits of integrating it with a feature toggle service such as Unleash.
What you are going to learn
Imagine you have an application deployed and released in your production environment, but some features are still hidden behind a feature flag. You now want to enable a new feature via Unleash. However, the new feature is now impacting the user experience in a bad way. Thanks to the integration of Unleash feature toggle service and Keptn this issue will be mitigated automatically by Keptn. Kepth will reach out to Unleash to turn off the problematic feature toggle.
What else can Keptn do for you?
Keptn is an open-source platform which solves continuous delivery for the enterprises. Keptn is defined as «an event-based control plane for continuous delivery and automated operations for cloud-native applications».
Keptn uses a declarative approach, avoiding tons of scripting to define your pipeline. It also makes it very easy to roll out your application to multiple environments, also known as stages. A workflow in Keptn is triggered by sending cloud-events to Keptn. Keptn will then start the deployment process automatically, following the strategy defined in its specification, the shipyard file. After the deployment, Keptn starts the tests, evaluates the quality gates. In case of a problem, Keptn rolls back the last changes. In this sense, Ketpn acts as an orchestrator of your deployment and testing tools.
Keptn not only makes it super simple to consistently and automatically roll-out your application. It also integrates with monitoring solutions such as Prometheus and Dynatrace. It uses monitoring data to automatically evaluate the quality gates to decide if a specific version of an application should be promoted to production. In addition, it will automatically solve problems in a production stage based on monitoring data. One example is instance by scaling up the number of pods if the application suddenly starts to increase CPU usage.
Automatically turn of flaky features
Let’s imagine that you turn on a new feature by flipping the switch on the feature toggle protecting it from your users in production. Some of your key metrics start falling. It could be that your application starts responding slow, due to a flaky API integration. Or worse, that some of your key business metrics are dropping significantly, and affecting your revenue. What if you forget to check the logs? Or that you are busy doing other things? Why should the computers not be able to detect the problem and revert the feature toggle change automatically?
Luckily, this is where the automation in Keptn saves us. Keptn is integrated with monitoring solutions to receive alerts or problems tickets and to automatically start counter actions based on the detected issues. With its built-in integration to Unleash feature toggle service, Keptn can control the state of feature toggles and change their state. Keptn will do so if one of those toggles is responsible for a bad user experience for the end user. Keptn will contact the Unleash API to disable the problematic feature toggle and fix the problem automatically for us. Having Unleash + Keptn in place gives you risk-free toggling of feature flags thanks to its self-healing capabilities. This reduces mean-time-to-remediation (MTTR) to just a couple of minutes by automatically turning off the problematic feature flag. Moreover, having this automation also reduces context switches for developers since they don’t need to remediate issues. Instead they can be informed at a later point in time and can focus on their tasks at hand since the issue has already been mitigated.
Self-healing in action
Have a look at this short video to see how Keptn and Unleash feature toggle service work together to auto-remediate feature flags that cause issues in your production environment: