Between Kubernetes, AWS, and observability tools, deploying new software has become nightmarishly complex. Learn how we’re trying to streamline the process with Diego.

Releasing software was supposed to be easier by now. We’ve got all these tools, we’ve got resources like Stack Overflow, and we’ve got the Cloud. But somehow, most engineering leaders we talk to feel it’s only gotten more complicated

In this article, we’ll explain: 

• Why releasing software is more complex than it’s ever been – especially for smaller teams
• Challenges in the current development landscape
• How we’re trying to address this complexity through our new platform Diego (don’t worry, we’ll keep the sales pitch to a minimum).

If you’re a bootstrapped or early-stage startup, we’d love to know if this reflects your own experience – tell us in the comments below.

Software might be eating the world, but DevOps complexity is grinding it to a halt

If you’ve been following Tech Amigos, you’ll know that we’re a group of DevOps engineers specialising in cloud setup for SMBs. We’ve been at it for several years now, so we’ve talked to a lot of CTOs and heads of engineering at smaller companies (20-200 employees).

Here’s what we’re seeing: The pressures to ship high quality software faster are immense. For a bootstrapped company, adding new features might be the one thing keeping the lights on; for pre- or post-seed startups, it’s often all investors are interested in (especially with today’s heightened focus on revenue metrics).

However, and somewhat counter-intuitively, shipping software has gotten more difficult and error prone. Specifically when it comes to SMBs, we’re finding that it’s not writing new features that’s causing the bottleneck – but rather the complexity of deploying code to the cloud!

Yes, you might be able to release an MVP quickly, but creating repeatable and scalable processes is an entire different story. Talking with CTOs and heads of engineering, we kept running into the same issues.

The challenges slowing down software teams:

• Skilled DevOps engineers are hard to come by. A majority of IT leaders have reported that hiring skilled DevOps people is a challenge, according to a 2021 survey (PDF). For smaller companies that can’t afford to keep up with the salary arms race, this is an even bigger problem.

• Dealing with tooling sprawl. Everything you can do on the cloud nowadays comes in dozens of different flavours – from your choice of cloud provider to your container orchestration platform. Tooling choices are endless and confusing; evaluating software becomes a costly distraction.

• Configuring the cloud stack correctly. Every platform under the sun promises a complete self-service experience, but the reality is often less straightforward. Something as simple as setting up an S3 bucket to store some documents comes with a host of configuration choices, and the same applies to every microservice, database, and component you use. Many of the people we spoke to had nagging suspicions that things were set up incorrectly and that this would come back to haunt them (they were usually right).

The sophistication and variety of today’s tooling has become its Achilles’ heel. There’s an entire ecosystem built for DevOps experts to squeeze out that extra bit of efficiency from every last part of the development pipeline. But for a small company that just needs to ship an app today, getting started can be extremely difficult. 

DevOps issues such as managing dependencies, Kubernetes configuration, and CI/CD pipelines end up eating 10-30% of development resources – slowing down the company’s core revenue-generating activities (as was covered in this report from 2021). 

Searching for repeatable patterns to build the 80/20 stack 

The Pareto principle tells us that roughly 80% of consequences come from 20% of actions. Does this apply to software development? Yes – the first 20% of correct design choices will give you most of the benefits when it comes to performance, security, and stability; the rest you can optimise later.

While no two projects are identical, we’ve found that many early stage companies or smaller software teams are implementing the same patterns:

• Standard open-source toolchain: For most companies and most projects, there is a set of open source tools that strikes the right balance between control and usability. GitHub, Kubernetes, ArgoCD, Prometheus and Grafana give you what you need to write well-structured code and push it to the cloud. They are all open-source so you’re not locked in to a vendor before you know exactly what you need. 

• Good-enough configuration rather than perfectly-tuned everything: Options are endless and fiddling with them might eventually produce a cleaner pipeline or reduce milliseconds of latency, but that’s rarely what makes or breaks a startup. Instead, we would start with an MVP-type setup that ticks just-enough boxes to enable short cycle times without jeopardising security, stability or resilience.

• Implementing best practices for development workflows: There are right and wrong ways to handle namespace management, resource provisioning and dependencies. We would review the way things that were done in the past, fix them when necessary, and ensure that best practices are adhered to moving forward.

After dozens of projects with different companies, we found that these basic patterns are repeated time and again – right down to the choice of naming conventions and tooling. And like any good developer, we found ourselves asking: why repeat a process manually, when you can automate it?

How Diego productises deployment standards and best practices

Diego is our answer to the DevOps challenge that’s paralysing smaller software teams. Diego is a platform that provides an end-to-end development workflow using a set of open-source tools, mainly Kubernetes and ArgoCD. With Diego in place, developers can build, deploy, and monitor containerised applications in the cloud – on a solid foundation of enterprise-grade DevOps infrastructure.

To put it in another way, we’ve packaged all our years of DevOps experience, and all the repeating patterns we had been implementing for our customers, into a software platform that any developer can manage. Diego provides ready-to-use environments, allowing you to quickly deploy secure, scalable, and resilient containerised applications.

What does this mean in practice? It means that instead of us coming in and solving problems for software teams, we give them the tools they need to solve the same problems instantly: cloud and Kubernetes setup, security, infrastructure management, application management, deployment, and observability. You can find the details here.

Our vision is to give smaller teams the ability to ship high-quality software quickly, even if they don’t have large DevOps teams to optimise their cloud deployments. No more trial and error or whim-based configuration decisions that end up creating a mess of repos and dependencies. Instead, Diego provisions the resources and environments needed for each feature, app, or microservice – and lets developers work their magic from there.

We’d love your feedback.

Diego is a new product. We’ve just launched with a small group of design partners and would love to hear your feedback – are we in the right direction? What else would you like to see? How can we further simplify the process of building, deploying, and monitoring apps for organisations like yours? Tell us in the comments.

If you’re ready to give Diego a spin, get in touch here and we’ll set you up with a free trial.