PeoplePractitioner10 min read
5. People and culture
Support the roles that are changing, recognize four typical profiles, establish the blameless post-mortem, run a sustainable on-call rotation, treat the platform as a product, recruit and retain.
For: team managers · executives and CIOsPrerequisites: None.
The platform rolls out in a few months; the culture that brings it to life takes years. That gap explains many failures. The dashboards are ready, the SLOs documented, yet in the middle of a crisis the teams fall back on their old reflexes. This lesson sticks to what is specific to observability. It starts from two convictions: the transformation moves forward in small successive groups and no tool replaces the direct conversation between a manager and their team.
From gatekeeper to doctor
Section titled “From gatekeeper to doctor”For a long time, operations worked like a gatekeeper’s job. You watched health indicators, reacted to alerts, escalated. Talent came from intimate knowledge of the infrastructure: a good caretaker knows every noise in the building. That model runs into three changes. First the cloud, where you no longer see inside the machine, only a programming interface (API). Then distributed architectures, where nobody knows hundreds of components intimately. Finally more and more frequent releases, which change the system every week.
The new job looks more like a doctor’s. Equipped with traces, metrics and logs linked together, they reason through hypotheses and elimination, the way a diagnosis is made from test results. They know how to read the system rather than know it by heart and they document so the team learns. The skills that become central are reading a dashboard, querying structured logs, understanding dependencies and reasoning in percentiles and SLOs. A percentile answers a simple question: “below what time do 99% of requests fall?” It tells you what the worst-served users experience, which an average hides.
Resistance to this change is not a lack of will. It says that a professional identity, built on knowledge nobody ever wrote down, is being called into question. Some phrases close the conversation: “you need to adapt,” “it’s not that different,” “this is the future.” Others open it: “I understand this transition is uncomfortable,” “your knowledge is still valuable, here is how we will recognize it,” “what concretely worries you?”
Four profiles to support
Section titled “Four profiles to support”The table below describes four typical profiles found in most operations teams. A real person never fits a box entirely, but the grid helps you prepare one-on-one conversations.
| Archetype | What motivates them | Possible path | Main risk | Levers |
|---|---|---|---|---|
| Senior system administrator | recognition of their expertise | senior SRE, platform engineer | leaving and taking unwritten knowledge with them | pairing with a younger SRE, public recognition, OpenTelemetry training, a role in the SLOs of legacy systems |
| Monitoring lead | operational usefulness, less noise | production SRE, incident commander | burnout before the transformation | healthy on-call, training in reliability engineering, recognition of noise-reduction work |
| Infrastructure project manager | structure and control | platform product manager, observability project manager | ending up without a defined role | an explicit transformation mission, objectives, product management training |
| System architect | consistency and durability of choices | platform architect, staff engineer | slowing things down out of skepticism | involve them early in decisions, make them the technical arbiter, give them a public voice |
Sylvain, in the journal, matches the first profile. The lever that worked for him is in the last column: public recognition and a role on the systems he knows best.
The blameless post-mortem
Section titled “The blameless post-mortem”In a healthy organization, an incident is an expensive gift: it reveals a blind spot in the system. In an unhealthy one, it is a fault to pin on someone and teams end up hiding incidents. The post-mortem, the review meeting held after an incident, is the ritual that moves you from one to the other, provided it is blameless.
It rests on four rules. You look for what made the error possible, not for a culprit: a person who made a mistake in production mostly showed that the mistake was possible. The account is factual, with a timestamped timeline, facts kept separate from interpretations and no “they should have.” Actions are few and assigned: a handful of actions with an owner, a date and a success criterion are worth more than fifty with no owner. Finally, the document circulates internally so other teams benefit from it.
The CIO’s role is decisive. They must state publicly that they will not sanction a good-faith error made during an incident. Then they must prove it at the first serious incident, including by defending the operator in front of the executive committee. Without that proof, the promise is worthless.
The document usually follows the same outline: summary, timeline, root cause analysis, aggravating factors, corrective actions, lessons learned and what went well.
Sustainable on-call
Section titled “Sustainable on-call”On-call works like the duty pharmacy: someone has to be able to step in at night and at weekends. An exhausted on-call team makes worse decisions in a crisis, restores service more slowly and eventually leaves. This wear can be measured just as well as technical debt.
Benchmarks
Section titled “Benchmarks”Google’s book Site Reliability Engineering, in its chapter on on-call, gives three orders of magnitude that are often cited. An engineer should spend no more than 25% of their time on call. They should handle no more than two incidents per twelve-hour shift. The rotation should have at least eight people for a single-site team (six per site across two sites). These are benchmarks, not standards. What matters is choosing your own and measuring them.
The first is to compensate. In France, the Labour Code requires on-call periods to be compensated, either financially or with time off (Article L3121-9). Beyond the law, on-call that goes unrecognized drives the best people away.
The second is to limit the noise. An alert at night should stay exceptional and require immediate human action. The alert fatigue simulator estimates the real load of an on-call week.
You also need to respect the right to disconnect: someone who is not on call does not get called; exceptions are written down and compensated. I recommend tracking perceived load with a monthly indicator, subjective and shared within the team, which spots drift before burnout sets in. Finally, after a major incident, the team involved takes time to recover without having to justify it.
The CIO is permanently on crisis call. They need a trusted deputy, real time offline and written escalation procedures.
The platform as a product
Section titled “The platform as a product”Platform engineering treats the observability infrastructure as a product whose customers are the developers. The central team stops answering tickets one by one; it builds self-service tools, a bit like a bank moving from the counter to the mobile app. An internal developer portal brings together service documentation, instrumentation templates, standard dashboards and procedures. Backstage, created by Spotify and now a CNCF project, is one example; commercial solutions also exist.
Four indicators help steer the platform. Set the targets for your own context.
| Indicator | What it measures |
|---|---|
| Time to first dashboard | time between creating a service and its first usable dashboard |
| Developer satisfaction | willingness to recommend the platform, measured each quarter |
| Self-service rate | share of requests resolved without the platform team stepping in |
| Cost per instrumented service | platform spend divided by the number of services covered |
Recruit, train, retain
Section titled “Recruit, train, retain”Recruit
Section titled “Recruit”The market for SRE (site reliability engineering) and observability profiles remains tight. For salaries, rely on a recent compensation survey for your local job market rather than on a figure read in a training course: they change too fast.
Three channels work. Employee referrals work well with a clear job description and a short process. Internal reskilling of administrators trained in observability costs less than an external hire and preserves knowledge of the information system. Work-study programs work provided there is real supervision, like the mentoring Sylvain provides in the journal.
The people dimension series details the approach: profiles, the job description, the interview and the first 90 days, with a job description generator.
Retain
Section titled “Retain”Five factors carry weight. First, the quality of the tooling, a reliability culture where incidents do not end in sanctions and a balanced on-call load. Next, written career paths (SRE, staff, principal, management), with their criteria and pay. Last, recognition, including public recognition: letting people publish, speak at conferences, write.
Exercises
Section titled “Exercises”The first is profile mapping, in two hours. List the ten to twenty key people in your operations and assign each one to a typical profile. Note the risks and levers, then schedule a conversation with the two whose departure would cost the most.
The second is an on-call audit, over half a day. For each rotation, record the size, how often people are on duty, night alerts per week, the compensation and whether there is recovery time. Compare with the benchmarks above.
The third is a post-mortem replay, two hours as a team. Take a past incident, redo the post-mortem with the four rules and compare with the original document. The differences are a good measure of your cultural maturity.
To make all this last, here is a two-year HR roadmap to adapt.
| Period | Actions |
|---|---|
| Quarter 1 | archetype mapping, one-on-one interviews, updated job descriptions, first on-call audit |
| Quarter 2 | individual training plans for senior profiles, blameless post-mortem in two pilot teams |
| Quarter 3 | first targeted hire, first internal reskilling cycle, mentoring in both directions |
| Quarter 4 | review of departures, publication of career paths, first assessment of the blameless post-mortem |
| Year 2 | rollout to all teams, developer portal, expanded work-study program, consolidated pay scale |
And a checklist for the CIO:
- I know by name the ten most critical people in my operations
- The archetypes and their paths are mapped
- The blameless post-mortem is in place in at least one team
- Every on-call rotation is sized, compensated and measured
- SRE and platform career paths are written, pay included
- Every senior profile has a training budget and plan
- Departures of SRE and platform profiles are tracked
- I spend time every month listening directly to the operations teams
Going further
Section titled “Going further”- The people dimension and the alert fatigue simulator.
- The error budget: multi-window alerts that wake people up for nothing less often.