The 5 steps to intelligent networks
In a previous post, I described the changing needs driving computer network automation. I also drew a distinction between the various levels of maturity involved in network automation: scripting, template-driven, orchestration and intent-driven:
1. Scripting. The operator uses a scripting language such as Python to write programs that send commands over the network to the individual devices. The commands can be sent using the configuration language of the device (command line, REST API, SNMP, etc.). This is usually adequate for small-scale networks.. However, the code needs to be maintained as devices change and new devices are introduced. . The operator is usually not a professional programmer and doesn’t operate under software development lifecycle principles. Bugs are common, especially in failure scenarios.
2. Template-driven automation. Automation tools such as Ansible were designed for software deployment, but are now extended to allow management of devices. Ansible allows device configuration steps to be specified in a re-usable YAML template called playbooks. Here the common tasks such as connecting to the device, handling failures and proper sequencing are hidden by the tool. Device vendors contribute plug-ins that abstract these tasks for their devices. Repetitive tasks (e.g., connect an application to the network) are abstracted into fill-in-the-blanks templates. Configuration tasks are executed by filling in the blanks (e.g., the address of the target device, the address of the computer hosting the application). Tasks can be composed into multi-step processes. This is a huge improvement over scripting since there is very little code to write. Multiple device types can be handled in one process. Network configuration can be done lock-step with other changes such as software deployment. (Tools like Ansible use imperative automation: the playbook author specifies the sequence of steps). However, there may still be tasks that are difficult to automate using templates (e.g. complex multi-step processes across multiple devices with a lot of conditional logic). Depending on the plugin, the tool may be sensitive to the current configuration of the network device — making it hard to predict the impact of running the tool.
3. Orchestration. Orchestration tools such as Terraform are similar to template-driven automation in the sense that they also have templates whose blanks can be filled with inputs, and hide the logic of connecting/executing commands. In addition, they are automatically able to figure out dependencies between steps and execute steps in parallel (if there are no dependencies between steps) or in sequence (hence: orchestration). Terraform can also eliminate steps if it determines that the device is already in the state defined in the template. Terraform and similar tools use so-called declarative automation: the desired configuration state is declared in the template file and, regardless of the current configuration state of the device, the tool will determine the steps necessary to converge the current configuration to the desired state. At the highest levels of sophistication, operators can delete the entire network configuration state and re-create it from scratch in an instant. These tools can also check for configuration drift — the difference between the desired state and the actual configuration state. By executing the tool in a closed loop of configure/detect drift/configure, operators can ensure that the network state is always as close to desired state as possible.
4. Intent-based networking. While orchestrators are highly capable, the language used to specify network configuration state is usually in the language of the network device (e.g., routes, load balancing policies, access control lists). What if one could generate the configuration state directly from the business requirements (intent)? If a new version of an application is to be deployed, the business requirement could be expressed as “Deploy version 8.2 of application X without interrupting the current users of the current version. If the new version does not perform as well as the old version, then roll back the deployment to version 8.1”. This could be translated into a strategy such as blue/green deployment or canary deployment, leading to the configuration of traffic management devices such as load balancers. There isn’t any open source tool that can be described to be “intent-driven”, but niche examples are: the horizontal pod autoscaler in Kubernetes, security groups in Apache CloudStack and OpenStack and the intent framework in OpenDaylight.
Once the network has been configured, it tends to drift from the desired state — hardware failures, device upgrades, new devices and computers, operator errors and so on. Ideally, the configuration automation runs in a tight loop, always converging the network state to the desired state. Data from the network — traffic metrics, hardware and software state need to be analyzed to determine the optimal set of changes required to bring the network back to desired state. Since the pace of change is rapidly increasing, and the volume of data is large, this analysis also needs to be automated. Machine learning and big data techniques are needed to help out with these tasks.
The evolution of network automation to closed-loop systems powered by machine learning and big data is not unlike the evolution of autonomous vehicles (“self-driving”). Self-driving vehicles take instructions such as “drive from point A to point B safely”. They process high-volume sensor data using machine learning algorithms to continuously correct the vehicle’s course and to react to external and internal events. Similarly, self-driving networks will take tons of telemetry from various parts of the network and analyze the data to implement high-level intent such as “ensure that at least 99% of the users experience a latency of less than 1 second while using the least amount of resources”. Self-driving networks are not imminent, but the drive towards increasing automation makes them inevitable.
