By Richard Lander, CTO | April 12, 2024
Application Orchestration is a new approach to software delivery in which software controllers orchestrate the delivery of applications and their dependencies into a runtime environment.
The context for this blog post is cloud-native software, i.e. containerized applications running on Kubernetes. App orchestration isn't constrained to these technologies, however it is most beneficial there so we'll be talking about the topic in that context.
Before we get into the details, it's helpful to start with common practice as it exists today.
Pipeline-based delivery systems (Continuous Delivery and GitOps) are commonplace today. They generally manage a chain of CLI programs that humans use to configure software systems and deliver them into their runtime environment.
This approach is useful in that practitioners start in testing and development with CLI tools like Helm and Kustomize. They iron out their Helm charts and Kustomize configs in early stages and these CLI tools also lend themselves well to shell scripting to improve velocity and repeatability while iterating. Then these charts and configs can be migrated over to GitOps systems like Argo CD for automated releases.
Taking Argo CD as a good example, DevOps engineers can use Argo's custom Kubernetes resources to define how the charts and configs get tied together. Everything can be stored in source control and allow git triggers to fire everything off when updates are made to those assets.
This all sounds great in theory and works fine in simple environments. However, the more complex software delivery gets, the more brittle and toil-laden this approach becomes.
In simple use cases, templating and overlay tools like Helm and Kustomize are wonderful. They deliver the needed configurability in short order. This is a large reason why they've become so prevalent in software delivery to Kubernetes environments. Platform teams get a killer return on investment early in development.
However, as time passes and requirements stack up, the templates in Helm charts accumulate many variables as well as the need for conditional and looping logic. Templating languages are pretty awful for this. There is a reason why web application developers keep as much logic as possible out of the templates that render HTML. General purpose programming languages are far superior for providing variable substitution and for encoding complex logic. The same applies to overlays. You can only overlay so many values in different text files before it becomes a maze that is difficult to navigate if you haven't walked it recently.
The relationship between complexity of your software delivery concerns and maintainability of your pipeline-based delivery systems look something like this:
Many applications use managed services as a part of their application stack. Let's say your production workload uses an RDS database in AWS. Spinning up an instance of your application, including its database, is challenging using pipeline-based delivery when that delivery system is designed to primarily handle Kubernetes resources. If you're using Terraform to deploy your RDS instances, you need to capture outputs, such as the database network endpoint, and feed that into the Kubernetes configuration for your application - Helm or Kustomize. These overlapping concerns are troublesome using different tools that are not natively interoperable. You can accumulate a lot of glue to keep the system together. And if any small piece of glue comes unstuck, the whole pipeline breaks down.
The best case end result is:
The worst case end result is pipelines that break down at critical times and require troubleshooting complex templates and configs.
Kubernetes alone is not an application platform. It provides an excellent foundation for one but it needs many support services installed on it to provide network ingress, TLS termination, DNS management, observability, secrets management and many more. The Cloud Native Computing Foundation landscape has a dizzying number of projects to provide solutions, so your options are not limited. However, tying your applications into these support service dependencies is non-trivial, even once you've found and installed the right support service solutions. Pipeline-based delivery can install those support services, but that must happen as a prerequisite to workload delivery. That is to say, a platform team must spin up each Kubernetes cluster and install all the support services that might be needed by tenant workloads. As the capabilities of your platform grow, this leads to many support services deployed when they may not be needed. All of this is because a pipeline-based delivery is not well suited to installing support services in response to the requirement of a workload that is being delivered. They just aren't cut out for the logical processing required for this kind of system.
As a consequence, you often have to deploy a lot of support services that consume considerable compute resources but are not always used. A testing or dev cluster will take longer to spin up and have much more in it that is needed or wanted. The alternative is to provide a complex set of different environments that support different workloads, obligating the platform user (developer) to have granular knowledge of the different environment profiles.
Kubernetes clusters themselves are a challenge to manage in pipeline-based systems. Every organization needs multiple clusters for different tiers of operation (testing, dev, staging, production), different locations, and possibly different cloud providers. Kubernetes clusters are, themselves, dependencies for a cloud native application. Providing those on-demand when application delivery is triggered is pretty challenging. Projects like cluster-api offer a way to do this with Kubernetes custom resources, but in a pipeline-based system, the business logic you can apply to managing these clusters is limited, and you're always managing yaml manifests that require careful configuration for everything to go right.
Usually, dedicated platform teams are required to pre-provision Kubernetes clusters for use, increasing toil and decreasing the agility and productivity of engineering teams.
The fact of the matter is that cloud native applications have a lot of complex dependencies. And pipeline-based delivery is not well suited to managing complex dependencies.
Application Orchestration uses a control plane with purpose built controllers for different parts of the system rather than CLI tools that are chained together in a pipeline.
In this simplified example there are just two controllers:
Elegant management of overlapping configuration concerns between infrastructure and workload management is illustrated by these examples:
Application orchestration is distinctly different from pipeline systems in the following ways:
True orchestration of software delivery becomes possible with application orchestration. Overlapping concerns for configuring all of a workload's dependencies can be elegantly handled in this manner. Some of the advantages that come about are:
Application orchestration is, in part, a framework for platform engineers to provide the most efficient software delivery operations to their developers and operations teams.
As mentioned earlier, the CLI tools used in pipeline-based systems make it very easy to execute early stage POCs and MVPs. This is very helpful in getting software delivery systems up and running quickly.
Application orchestration offers a comparable time-to-launch for simple use cases that can leverage the existing primitive objects in an app orchestration system. However, the custom abstractions for delivering the software you built will often require building controllers instead of wrangling configs and templates. This requires a bit more up-front engineering. Good software development kits and frameworks can mitigate this, however you're always dealing with compiled, containerized, released software controllers rather than a simple git push for a config to add functionality.
The trade-off boils down to time-to-launch vs returns over time:
The pipeline-based approach has a wealth of options that are relatively straight forward to install and use. Again, this helps in getting up and running quickly.
App orchestration requires a control plane that is, in and of itself, a complex distributed software system. Good installation and bootstrapping mechanisms mitigate this. So can managed service providers that offer hosted app orchestration control planes. But it is, nonetheless, a challenge to navigate.
The holy grail of a self service experience for developers becomes available to all teams, large and small, with app orchestration. Simple use cases can be managed with primitive objects and general-purpose controllers. Complex systems will require extension and customization, but the end result will pay dividends and provide increasing returns over time.
App orchestration offers platform engineers the opportunity to provide significantly outsized value from what they've been able to deliver in the past. It dovetails exceptionally well with the Kubernetes operator pattern and allows community collaboration around commonly encountered use cases and challenges. It means they don't have to re-invent the wheel each time they develop a new cloud-native application platform or extend an existing one.
App orchestration offers an opportunity for an industry standard that makes platform engineering more effective by orders of magnitude.
In a word: Efficiency. As modern cloud native software becomes ever more feature rich, complex and distributed, efficiency in software delivery will become more critical. Greater efficiency leads to lower costs and better software as the net results.
Threeport is an open source application orchestrator. It was developed by platform engineers with a wealth of experience and a recognition that a better system for software delivery is possible. The Threeport CLI is available to download on GitHub. You can install a Threeport control plane locally or in AWS and try it out for yourself.
Qleet provides fully managed Threeport control planes. The engineers at Qleet developed Threeport and are available to support you every step of the way in leveraging application orchestration for your company's software delivery.