The Growth of Platform Engineering
By Richard Lander, CTO | May 10, 2024
Platform engineering is gaining considerable adoption. If this is not evident from your own personal experience, check out the CloudBees research report. That report aligns with what I've seen and it stands to reason. Software is getting more complex and sophisticated in response to user demand. This leads to delivery requirements becoming more demanding. Consequently, the systems used to deliver software are becoming more involved. Ergo, organizations are finding they need to invest in their application platforms to get the job done. Or build very large DevOps teams to wrangle configurations and fill the gap.
Opportunity Costs
Developer productivity is critical. The more features devs can bring to a organization's users, the more value they build for that org. As the software grows in complexity and features, and as the delivery requirements expand, if the platform doesn't meet those needs then developer productivity suffers.
Being able to smoothly, reliably deliver software to its runtime environment means less toil and less expense. And there is less opportunity cost incurred. This factor is greatly under-appreciated. The more time practitioners spend fiddling with deployment configs and troubleshooting production issues, the less time they spend improving their software and its support systems.
Those organizations that get ahead of the competition in platform engineering stand to pull well ahead of their competition.
DevOps vs Platform Engineering
I think of DevOps as a subset of platform engineering. I realize many think of this in reverse, but that's just a product of DevOps practitioners being more numerous - and that platform engineering, as a discipline, is younger. Broadly speaking, platform engineering deals with the application platforms used to deliver an organization's software. DevOps has come to signify a particular approach to this problem that focuses on config management tools and continuous delivery pipelines. The world is beginning to realize that the DevOps approach that has worked well in the past is approaching its expiry date. The methods of the past are becoming less and less effective.
For a comparison of these two topics you can check out my blog posts:
What it boils down to is this: DevOps has been a practice of delivering software with tools in a pipeline-based system. Platform engineering includes building software systems that manage software delivery. The software engineering approach has a higher up-front cost, but with improving returns over time. You can read about the diminishing returns with pipeline-based delivery in my blog post: Introducing Application Orchestration.
Here's the crux: building software to manage delivery is becoming a must as software architectures become more complex. The config-wrangling DevOps approach just doesn't scale with the complexity of modern cloud native software.
DevOps: An Endangered Species
The software engineering resources that are emerging for platform engineering will become very plain for all to see over the next few years. The open source software, libraries, and SDKs that are already available provide platform engineering teams with a wealth of raw material with which to build application platforms. Expect this trend to continue.
The Kubernetes operator pattern is a great example of this. It is an under-used approach today, but this will inevitably change out of necessity. For more information on what a Kubernetes operator is and how operators can be used to build application platforms, see my blog post: What is a Kubernetes Operator?.
This is not exactly at odds with a DevOps approach to software delivery in the short term. In fact, in its early stages, platform engineering can greatly benefit DevOps practices by reducing the complexity of the configurations that are wrangled with DevOps tools. However, over the long haul, it will become plain to see that the DevOps config management toolchains can be switched out for software systems that are more intelligent and reliable.
If your job title includes "DevOps," I would strongly suggest the following:
- Invest less in your proficiency with various DevOps tools like Helm and Terraform.
- Invest more in your software development skills, particularly with Go.
This will leave you better positioned to evolve with the times.
Nothing New Under the Sun
Not too long ago, we were configuring servers to run our software with Ansible, Chef and Puppet. (Some are still doing this.) Server config automation was great for certain use cases. However, as the amount of software under management grew, managing server inventory, scheduling workloads to servers, bin packing workloads onto servers and service connection concerns became quite a pain point. Then along came Kubernetes that provided an API and control plane for managing those concerns. That software layer replaced the config management tools for servers.
Now we have a config management layer atop Kubernetes that has become quite unwieldy and toilsome to manage. The sheer number of runtime variables and resources needed to run workloads, multiplied by the number of workloads under management, has become untenable without a large, expensive DevOps team.
Platform engineering will deliver the software layer to deliver software into cloud native environments. We have a great blog post that explores this topic: Nothing New Under the Sun.
Threeport
Threeport is an application orchestrator that provides a foundation for platform engineering. It is that software layer for delivering software into cloud native environments. It is open source and free to try out.
Qleet
Qleet is a managed Threeport provider. Threeport is trivial to try out for testing and development. If you try it and like what you see, checkout Qleet to provide you with a Threeport control plane without any of the management overhead.
