Cloud Custodian helps cloud and security engineering teams enforce real-time governance policies that enable organizations to be well-managed in the cloud (also known as cloud governance). Cloud Custodian started as an open source project six years back. Being open source, it harnessed agility and speed required to keep up with the pace of public clouds and the breadth of use cases. Fast forward to today, thousands of companies are using Cloud Custodian to power their cloud governance initiatives.
This article was originally published by Stacklet.
Back in 2014, when Capital One decided to go Cloud-first, many people thought they were crazy and that no bank could ever migrate to the public cloud. At the time, large regulated enterprises would usually cautiously — almost reluctantly — move to the cloud, where the pace of development would vastly outpace their ability to onboard new services. Finding success wouldn’t have been possible without Cloud Custodian, an open source tool created by Kapil Thangavelu, co-founder of Stacklet, while at Capital One.
Operationalizing and Managing Cloud Custodian at Scale is Challenging
Over the past six months, I had the opportunity to talk with several Cloud Custodian users about using Cloud Custodian and the challenges they face at scale. Users often asked: What’s the best way to enforce policies across a thousand cloud accounts spanning multiple regions? Or how can policy lifecycles be managed without breaking resources? And how can we steer precious engineering resources away from the tedium of managing Cloud Custodian itself and instead focus their attention on policies?
Another element that came up regularly was the need for visualizing and analyzing the output of Cloud Custodian policies. Effectively, Cloud Custodian produces large volumes of complex compliance logs, eventually requiring teams to develop separate analytics pipelines to make sense of the data and share the results with other stakeholders, including business teams responsible for cloud compliance.
Confidence in the intelligence gathered is directly linked to the quality of the data collected. Cloud Custodian policies are applications, and as such, they sometimes error out, time out, or lack permissions to execute. Monitoring, alerting, and reacting to these events is critical to administrators to maintain the trust relationship with the rest of the organization, but this is something Cloud Custodian does not provide out-of-the-box.
Every piece of feedback we got underlined that what users ultimately care about is the confidence that the governance policies they write will be enforced. How that happens is almost irrelevant, provided the results can be trusted.
Scaling Cloud Custodian With Stacklet Platform
We recently introduced Stacklet Platform. Stacklet Platform provides advanced features to help users operate Cloud Custodian at any scale. Stacklet Platform commoditizes the deployment, execution, and reporting of policies so that development teams can focus on the policy code rather than burdensome orchestration.
Stacklet Platform provides a new management console that lets users configure, deploy, and orchestrate governance policies across multiple cloud platforms and accounts. Fully compliant with Cloud Custodian’s existing configuration mechanisms, Stacklet Platform allows administrators to group cloud accounts according to their operational and business needs and manage collections of policies bound to specific regulatory requirements.
Stacklet Platform monitors policy executions and reports on policy health thus reducing the overhead of managing hundreds of controls in thousands of cloud accounts. Data gathered from policy execution is then converted into actionable intelligence, helping cloud and security teams detect compliance issues, respond via immediate action, or collaborate with the responsible development teams. Business teams can use the Stacklet Platform management console to view account posture and compliance reports in a self-service manner.
As part of the platform, we released two additional products below. (Stay tuned for detailed blogs on these in the coming weeks!)
- Stacklet AssetDB: A holistic, real-time streaming database of all cloud resources and configurations that can be queried via SQL. Stacklet AssetDB can be used to accelerate Cloud Custodian execution and reduce pressure on cloud APIs.
- Stacklet Policy Packs: These are bundles of policies that provide out-of-the-box access to regulatory benchmarks such as CIS and PCI DSS across all clouds. Some packs target specific business cases such as cost optimization or publicly reachable assets. These packs are created and maintained by the Stacklet engineering team.
It’s only the beginning for Stacklet Platform. We have loads of new features in the backlog addressing notification optimization, advanced remediation workflows, and advanced policy lifecycle management. But at the end of the day, it’s about you as a user. If you have a need, a want, an idea, or a question that you’d like to share with us, we want to hear your voice. Feel free to send me an email Sam at stacklet.io or connect with me on LinkedIn. If you are interested in seeing the product in action, please sign up here.
