#53- Open Source is harder than you think

Hey there! This is Platform Weekly, your platform engineering chef delivering your weekly taste of cloud-native goodness straight to your table 🥐Let’s get bakin’

Open source is harder than you think

A month ago HashiCorp shocked the cloud native world by changing their OSS licensing around Terraform, which sparked a huge debate and response from the community (e.g. Open Tofu). It’s debatable whether the whole thing should have been much of a surprise or whether people could have expected it, looking at the growth trajectory of the company and comparing it to similar past cases.

The tension between open source software or OSS and proprietary offerings existed ever since the beginning of software development itself really. And I’m sure I don’t have to highlight just how important OSS has been in the history of our industry, from Linux and C++ all the way to more modern tools in the platform engineering stack (K8s, Docker, terraform, etc.).

I myself spent the last year and a half working to get our own OSS contribution off the ground and learned that it’s really not that easy. And while I’m certainly not justifying HashiCorp or saying that I agree with what they’ve done and how, I think it’s important to understand the nuances of both open-sourcing and maintaining a piece of software.

Take Score for example, a platform-agnostic workload specification that lets developers describe what their workload needs to run in an environment-agnostic way (e.g. I need a Postgres, without having to specify which version or any other implementation details).

We launched Score in Q4 2022 and it was immediately off to the races. We focused a lot on distribution and making sure we’d get lots of eyes on it. And it worked. We quickly skyrocketed past 1k stars on GitHub, then 2k, then 5k. Score then trended twice on GitHub globally, at a time when LLMs were getting their first hype cycle and every other repo was AI.

We initially thought that if we got enough excitement and press around the project, contributors would come. Well, we were wrong. Once we got to 6-7k stars, momentum slowed down a lot. And while we had lots of people reporting bugs and helping us drive development, we didn’t manage to get to a critical mass of core contributors that would actually build out CLI implementations (which is really what makes the concept of the workload spec a powerful tool that engineers can use in their day to day).

We made the mistake of assuming that with all the good press, the community development would really start. But it never did. And we just didn’t have the resources internally to drive the entire thing on our own, at the expense of developing the core offering (the Platform Orchestrator).

Does that mean Score was a failure? Not at all. Score is still used every day by a growing number of teams, especially by large enterprises in combination with our Platform Orchestrator. But it’s still far from becoming what we originally dreamt of i.e. a truly platform-agnostic spec. The architecture is there, but we’re missing the implementations to fully prove the concept.

What this shows though is that even for a team with a decent level of resources (we are no Google of course) and for a project that got lots of initial traction, it can be tricky to keep the momentum going. OSS is essential for software engineering and, for the world really. But it can be damn hard. You live, you learn, you stay crunchy.