Skip to content

OrganizationPractitioner9 min read

Team models for observability

Platform team, center of excellence or federated model: decide who builds observability, who owns what goes into it and who pays, with a split of responsibilities and the signs that say when to change models.

For: architects · team managers · executives and CIOsPrerequisites: No technical background needed: a rough idea of how IT teams are organized is enough.

Reading mode

Monday morning, the online ordering site slows down. The sales director asks who monitors that journey. The infrastructure team says it provides the monitoring tool, not the dashboards. The team that builds the site says it never had time to connect it to that tool. The same day, finance asks why the tool’s bill has doubled in a year. Nobody can answer, because nobody knows who uses it.

This scenario is made up. It still shows something I see often: an observability platform, meaning the set of tools that collect, store and display an application’s metrics, logs and traces, rarely fails for purely technical reasons. It lasts when three questions have a written answer: who builds the platform, who owns what goes into it and who pays. The team model is how an organization answers all three at once.

A condominium is a good picture of the problem. The building manager looks after the common areas: the elevator, the boiler, the roof. Each owner is responsible for their own apartment. Shared charges are split according to a rule everyone knows. When one of those three rules is missing, the arguments start.

This page compares three team models, proposes an ownership boundary and a table of responsibilities, then covers funding and the signs that say it is time to change models.

The book Team Topologies by Matthew Skelton and Manuel Pais (IT Revolution, 2019) provides a useful vocabulary for this topic. It describes four team types and three interaction modes.

For observability, the stream-aligned team is a product team that delivers a service and runs it. The platform team provides collection, storage, standard dashboards and alerting as self-service. The enabling team temporarily helps product teams build skills, for example instrumenting their code with OpenTelemetry. Instrumenting means adding what the code needs to emit metrics, logs and traces; OpenTelemetry is the most widely used open standard for doing so. The fourth type, the complicated-subsystem team, is rarely needed for observability, except for a very specialized component.

The three interaction modes are collaboration, X-as-a-Service (one team consumes what another provides without having to ask it each time) and facilitating. This mapping to observability is my reading of the book, not a definition by its authors. I recommend it because it forces you to say how teams work together, not only who does what.

In the first model, a dedicated platform team builds and runs the platform as a product: the collection pipelines that carry the data, storage, visualization, alerting and ready-to-use instrumentation templates. Product teams use it as self-service. The CNCF white paper on platforms (CNCF Platforms White Paper) stresses this product approach: internal users, gathered requirements, a carefully designed experience.

The second model is the center of excellence. A small cross-functional group defines the standards, trains and advises. Those standards cover naming conventions, semantic conventions (the common names OpenTelemetry proposes to describe a request, a service or a database), alerting rules and post-mortem templates. The center of excellence does not necessarily run the platform, which may be a purchased service or operated by the operations team. Its authority rests on expertise and leadership backing.

The third is the federated model. A central core runs the shared foundation and each product team names an observability champion. That champion spends an explicit share of their time on their team’s instrumentation, quality objectives and dashboards. Champions form a community led by the core team.

Platform teamCenter of excellenceFederated
What is centralizedthe tool and its operationstandards and expertisethe foundation and the community
Main strengthconsistency, economies of scale, self-servicelight start, spreads good practicesclose to the business, peer-driven adoption
Main riskbecoming a bottleneck or a ticket deskproducing standards nobody appliesheterogeneity, champions with no real time
When to choose it, in my viewdozens of teams, need for cost controlearly stage, purchased tool, few teamsalready decentralized organization, autonomous product teams
Success conditiona platform product owner and usage indicatorsa leadership mandate and standards checked automaticallychampion time written into objectives

These models are not mutually exclusive. Many organizations start with a center of excellence, set up a platform team when volume justifies it, then federate through champions. I recommend naming the current model and the target model, even if they are hybrids. An organization that does not know which model it is in does not know who decides.

Whatever the model, I recommend writing down a simple boundary between what the platform owns and what product teams own. It is the condominium rule of common areas and private units. In a 2006 ACM Queue interview, Werner Vogels stated the “you build it, you run it” principle. It applies to observability too: the team that delivers a service owns its signals, meaning the metrics, logs and traces that service emits.

The table below uses a few terms worth setting out. An SLI (service level indicator) measures what the user experiences, for example the share of successful payments; an SLO (service level objective) is the target set on that indicator. Retention is how long data is kept. A data contract describes what a service is allowed to send. Cardinality is the number of distinct combinations a metric’s labels take: the higher it is, the more storage costs (the page Cardinality explained tells the full story).

Owned by the platformOwned by the product team
Collectors, pipelines, storage, default retentioninstrumentation of its code and dependencies
Availability and performance of the platform itselfSLIs and SLOs of its services, with the business
Instrumentation templates, libraries, standard dashboardsdashboards specific to its service
Generic alerting rules (platform, shared infrastructure)alerts on its symptoms and the related runbooks
Data contracts and default cardinality budgetscompliance with its data contract and requests for exceptions
Measuring and showing costs per teamcost trade-offs on its own signals

The platform has SLOs of its own: availability of ingestion (receiving the data), the delay before a signal shows up, query speed. An observability platform that does not observe itself loses the teams’ trust at the first gap in the data.

A RACI matrix is a table that says, for each activity, who does the work (R, responsible), who answers for it and decides (A, accountable), who is consulted (C) and who is informed (I). Lesson 3 of the CIO path proposes one at leadership level. This one goes one level down, to day-to-day activities, for a platform team model with champions. It is an illustrative example, to be adapted in a workshop.

ActivityPlatform teamProduct teamObservability championSecurityBusiness
Run the platform and its SLOsA, RIICI
Instrument a new serviceCARII
Define the SLO of a customer journeyCRCIA
Create a paging alertCARII
Add a high-cardinality attributeARCII
Collect potentially personal dataCRCAI
Decide on an observability budget overrunRCCIA
Evolve instrumentation standardsA, RCCCI

A “paging alert” is one that rings the on-call person’s phone. Two rules matter for this table: a single A per row and a review at least once a year. The interactive RACI matrix checks the first one and shows each role’s load.

Three funding modes coexist. They follow the FinOps progression (FinOps is the practice of making teams accountable for their infrastructure spending) described in lesson 4 of the CIO path.

With a central budget, IT carries all the spending. Nothing more than a budget is needed, but no team has an incentive to cut what it sends. Showback shows each team what it consumes without making it pay. It requires a monthly review and a team label on every piece of incoming data, enforced at ingestion. Chargeback makes each team pay for its consumption. It needs a reliable measurement that everyone accepts, as well as an internal budget mechanism.

I recommend starting with showback as soon as the platform is shared, even if chargeback never comes. Seeing your own consumption is often enough to change behavior, much like a detailed statement of service charges in a condominium. I also recommend funding the foundation (operations, standards, training) centrally: charging it back pushes teams to work around it.

When the time to get a first dashboard on a new service keeps growing, the platform has become a ticket desk. Invest in self-service.

When several teams have built their own monitoring tools, their needs are not covered or not heard. Gather them the way you would for a product.

When standards exist but code reviews do not check them, the center of excellence has no leverage. Automate the check in continuous integration, the pipeline that tests every code change before it goes to production.

When champions no longer have time for the role, the federated model is not funded. Their time has to be written into their objectives.

Finally, if nobody can answer “who owns this signal?”, the ownership boundary is not written down.

The indicators for running a platform as a product (time to first dashboard, developer satisfaction, self-service rate, cost per instrumented service) are detailed in lesson 5 of the CIO path.

  • The current model and the target model are named
  • The ownership boundary between the platform and product teams is written down
  • The platform has its own SLOs and publishes them
  • The operational RACI was built in a workshop and has a single A per row
  • Each team sees what it consumes
  • The shared foundation is funded centrally
  • Roles (platform engineer, product owner, champion) have a job description
  • The signs for changing models are reviewed at least once a year