Team Topologies and DevOps Topologies in practice

[Original article at medium.com]

In this article, I’ll share my experience with adopting DevOps Topologies and Team Topologies in a company that I worked for few years ago: OneFootball .

Reaching over 100 million monthly active users worldwide in 2024, OneFootball is the most popular football media platform for the new generation of football fanatics. Founded in 2008, the company has a global reach, available in 12 languages and is currently one of the best-rated sports app in the world, based on millions of App Store and Google Play reviews.

Prior to 2017, Onefootball had less than 10 million monthly active users and it operated with a small, old-fashion startup engineering org, comprising a mobile team, a backend team, a frontend team, a few DevOps Engineers and a product team. This setup revealed its shortcomings around 2017 when a decision was made to transition product development into cross-functional teams.

Although the shift into cross-functional teams marked a positive change for software development, there were no significant plans for evolving the DevOps model, resulting in the formation of a pseudo “Platform” team — which had little to do with what we know today as Platform Engineering. Added to scalability issues, this team inherited several responsibilities that did not align with those of other product teams, indicating that the department had a match with the DevOps Anti-Type E.

During the implementation of Kubernetes and other CI/CD improvements together with my friend Rodrigo Vieira Del Monte , it became necessary to support not only the new cloud infrastructure but also the emerging demand for tools and automation. Consequently, it was decided to hire and engage DevOps engineers for each product team.

Initially, the Platform team focused on AdTech tools, web raffles and advent calendars (why not?) while refactoring the user API and transforming the entire cloud infrastructure. Later on, the DevOps Engineers assisted the product teams to take responsibility for managing and monitoring application and infrastructure related to their scope. Over time, all DevOps engineers collaborated to ensure uniformity and adherence to standards across the cloud infrastructure. Nevertheless, with DevOps engineers spread among each product team, it became apparent that the department functioned within a different Anti-Type, a blend of Anti-Types C and D, although all engineers (not only DevOps) took responsibility for Ops and on-call.

Despite the suboptimal team topologies for collaboration, it is important to acknowledge the collective efforts made by all engineers to keep the entire ecosystem working, even if it required trade offs between teams. This statement extends to all engineers and managers who demonstrated maturity by trusting the process and supporting the change at the time — which eventually paid off.

All efforts we made culminated in a successful product launch during one of the company’s busiest periods: The FIFA World Cup 2018. It became a major success with an all-time high number of users accessing the platform. The process of transforming an unstable product into a global success is covered in a interview I did for the FutureStack18 conference in Berlin and in a later participation as a speaker in the AWS Summit Berlin 2019:

However, it became clear that collaboration between teams had to be improved. The emergence of Site Reliability Engineering (SRE) as a concept shed some light on how to fix some of the collaboration issues we had, prompting the creation of a cross-functional SRE team, albeit initially as Anti-Type H. The main reason was that we ended up spending few months working on products and tools while the engagement with other engineers decreased a bit.

The view ahead was exciting, though. The new SRE team, comprising previous DevOps, Backend, and Frontend Engineers inherited systems and tools that had been overlooked previously. Eventually, the team elevated the engineering department’s maturity by introducing higher reliability standards and practices such as Production Readiness Reviews, Production Meetings, Blameless Post-mortems, among others.

At that point, it’s safe to say that the SRE team became a Type 7 SRE team:

Additionally, the SRE team developed a tool for generating Service Level Objectives (SLOs), filling a gap not addressed by major SaaS players at that time. The tool ideation is well described by my dear team mates at that time, Jim van der Waal and Dominik Garcia Bertapelle , in this article:

With the possible introduction of gRPC alongside authentication/authorization challenges, product teams required a close attention for supporting both their projects and operations. The SRE team decided to adopt a tiered engagement approach, temporarily migrating into Type 5.

Reflecting on the engineering department’s transformation, it is evident that the success of each DevOps topology was a result of a high engineering maturity, trust among teams, clear ownership and a commitment to innovation. The introduction of the SRE team significantly improved microservices’ reliability and fostered a productive interaction between development and operations, particularly through the implementation of SLOs. But we still had room for improvement.

At some point in 2019 I became aware of Team Topologies through a presentation in the DevOps Enterprise Summit:

At that time, I wasn’t aware of anybody in the industry speaking about the impact of team topologies. When I learned about the possibilities, I couldn’t help myself but to get in touch with Manuel Pais. Few weeks later, we both arranged a Team Topologies Workshop at Onefootball, an unique opportunity I had to meet with Manuel Pais and few colleagues from other companies to understand how to organise teams for success and fast flow.

With this workshop, I was able to deep dive in an issue that no one was talking about: cognitive load and how it affects software development. After the workshop, one thing became clear: there’s no right approach when setting teams up, but several ways of doing it wrong. And that obviously encouraged me to look into my domain of practice.

At one point, the SRE team served as an enabling team, bridging the reliability gap within the engineering department and providing guidance akin to a "technical consulting team", with a strong sense of collaboration and engagement.

On another point, it functioned as a specialized stream-aligned team, developing DevEx tools and produtcs like the SLO tool while managing low-level APIs such as for authentication/authorisation — activities and tasks that made more sense as we've scaled.

In conclusion, it was really nice to see this whole transformation that the engineering department at Onefootball went throughout the years. The success of each one of the DevOps topologies were a result of:

• High-engineering maturity and synergy
• Trust that all teams and the majority of the engineers had on each other
• End-to-end responsibility, a high sense of accountability within the product teams
• A relentless pursue for staying ahead in the game with knowledge and innovation

All these things really helped to smooth out the transformation process. However, we couldn't have done it without having the guidance from both DevOps Topologies and later on with Team Topologies.

What did I learn from these transformations? Exactly what is in the page 77 of the Team Topologies book:

“Adopt and Evolve Team Topologies that Match Your Current Context”.

This part of the book specifically mentions that we should be careful not to be reactive, but to look at the long term. It’s important to highlight that those transformations happened in the course of 4 years and they have yielded long-term value for the business — the success of the FIFA World Cup 2018 stands as a prime example that speaks for itself. I have an immense gratitude to have been part of this journey at Onefootball.