SITUATION
A media conglomerate’s digital division realized it needed to implement AWS best practices in order to more effectively architect and maintain the applications that serve media content to the more than 50 million U.S. customers. Its mission-critical applications include a content management system that continuously distributes content from 4,000 publishers and 120 news stations; as well as technology that supports various TV networks. This division strives to deliver applications and services in a DevOps model, with an eye toward Continuous Integration/Continuous Delivery (CI/CD). However, the model grew lopsided with only a handful of DevOps engineers supporting several dozen application developers.
And although the standard development, integration, QA and production environments were built solidly on the Amazon Web Services (AWS) platform, they had been constructed manually through the AWS console with little documentation.
The situation significantly impacted the DevOps team:
- Dependency on tribal knowledge meant more time-consuming infrastructure updates.
- Engineers spent a disproportionate amount of time reacting to outages
- Little effort could be devoted to future projects.
To address the issues, leadership embarked on a journey to fully document and automate more of its AWS infrastructure to enable more efficient updating, reduced outages, and more repeatable processes for setting up and maintaining environments. However, the organization lacked the expertise to pursue the issue expediently.
Based on a long-term relationship with AIM Consulting, the organization chose to partner with AIM’s Cloud & Operations practice on the project, and the solution was soon at hand.
SOLUTION
An AIM senior cloud consultant reviewed the client’s major AWS infrastructure components, recommended the use of AWS CloudformationInfrastructure as Code (IaC) templates for the environment, and trained and worked alongside DevOps resources to be self-sufficient in Cloudformation use. IaC templates enable an entire infrastructure to be modeled in a text file, resulting in a single source of truth, faster troubleshooting and elimination of manual actions and custom scripts.
Following the infrastructure review, AIM demonstrated how to use AWS Cloudformation to deploy their Virtual Private Cloud (VPC) network, carving the various network tiers into a collection of Cloudformation templates to deploy an isolated network environment. After the successful demonstration, AIM recommended a security group audit, which included tightening existing security groups and removing others that had become obsolete. AIM also recommended and deployed a three-tier subnet architecture and measures to strengthen identity and access management.
AIM then showed how to layer Cloudformation templates to deploy entire environments, beginning with a simple dev environment. Working interactively with existing and newly onboarded DevOps resources, AIM proactively guided the team in constructing various environments, until eventually the team could deploy an entire environment on its own in just four hours. With no prior Cloudformation knowledge, the team was self-sufficient in building environments in this fashion within a few short months.
As a trusted advisor, AIM recommended and employed best practices throughout. Although Cloudformation is in itself a form of declarative documentation, AIM also provided further instruction on the client’s internal Confluence page describing how the infrastructure was built.
RESULTS
AIM’s guidance helped to stabilize the client’s DevOps infrastructure by building in repeatable processes with Cloudformation templates. The engagement paid down massive technical debt and produced immediate results in terms of dramatically reduced outages and increased release velocity.
The DevOps team has increased fourfold, a fluid team no longer reliant on tribal knowledge or manual processes. Additionally, the massive reduction in time to deploy new environments — from 80 hours to 4 — has allowed the team much more freedom to move forward with future projects such as building out the CI/CD pipeline, transitioning to Amazon’s Elastic Kubernetes Service (EKS), and enhancing disaster recovery capabilities. IaC represents a foundational component for all these efforts.
The engagement with AIM continues, with a new AIM cloud consultant assisting the DevOps team on its CI/CD venture and the original consultant advising and training the team on a regular basis to help fine-tune the new infrastructure.
Get In Touch
Whether you need help with technology strategy and implementation or have an in-flight project in need of additional resources, AIM is here to help.
Fill out the form below and one of our experts will be in touch.
