The Growth of CI/CD in Network Automation
Contributed By: Morgan Stern, VP Automation Strategy, Itential
Among the more enjoyable tasks that my job affords me is the opportunity to discuss both goals and challenges CSP network and orchestration teams face as they relate to network automation.
While these conversations typically span several different topics, trends do emerge; and while some of the ideas we discuss never materialize, others do gain traction and become the standard approach for those teams.
Last year, one trend that saw huge momentum was the intersection of network automation and CI/CD. Several CSPs I met with were developing pipelines to manage their network lifecycle activities. This made sense as the practices that changed the way software was developed, tested, and deployed could yield the same types of benefits to the processes for integrating and deploying network automations and orchestration.
While it felt like everyone wanted to discuss CI/CD, the focus of their pipelines varied significantly across different CSPs. For example, some wanted to create pipelines for the specific configs in an “Infrastructure as Code” (IaC) model. Others, wanted to create pipelines for their automation artifacts – the scripts, workflows, templates, etc. – seeking a more structured process for development, testing and deployment now that automation activities were evolving from task-centric to end-to-end process focused. . And in other cases, some CSPs wanted to develop pipelines for managing the automation platforms themselves, to provide the capability to automate the testing, integration, and deployment of the automation/orchestration engines through the version upgrade and patch processes.
After much investigation, which included talking with CSPs around the world, as well as with engineers and SMEs across a few different disciplines, three common themes emerged:
- The desire for implementing orchestration/automation CI/CD pipelines was in response to the growing dependency and complexity of automation activities within the CSPs.
Automation has graduated from small scripts used by individual engineers to solve specific problems, to automation architectures that solve for end-to-end activities. During this evolution, the limitations of some approaches became obvious. Certain practices did not scale well and created an enormous amount of technical debt in the attempt to take tools that were designed to solve one problem (automate a single task) and apply them to a much larger and more complex problem (automate a business/technical process).
- The problem set that any given CSP was trying to solve with CI/CD was a reasonably accurate indicator of where that CSP was in their automation and orchestration journey.
Infrastructure as Code (IaC) was the starting point for most organizations, but as the infrastructure required to execute automations became more complex, teams realized that they needed to address the complexity of developing, testing, and deploying the automation assets themselves to ensure interoperability, versioning, and security of assets like new scripts, workflows, and templates. The next challenge was how to create pipelines to ensure that, without disruption to the production environment, the automation tools and platforms were integrated, tested, and upgraded securely..
- Pipeline development created additional opportunities for automation, particularly for testing activities.
Multiple CSPs wanted the ability to dynamically instantiate working environments to improve the quality of their testing efforts. This created the need for capability to replicate portions of the production network and network services within the lab on demand, and then to have mechanisms for storing and managing network snapshots that could be accessed, versioned, and automatically updated to reflect architectural or vendor changes in production.
I’m excited to observe the best practices that emerge from these activities, and the new tools CSPs employ to manage their automation pipelines. As a network automation vendor, we are always looking for features to simplify pipeline development, and we expect other vendors and the Open-Source community to do the same. With these new capabilities, automation activities should accelerate dramatically, driving significant incremental business benefits for productivity and agility and opening the door for new service offerings made possible through these innovations.