Traits 1 to 5: hiring
(1) They name the failure mode they fear most about the role and the team, unprompted, in the first 20 minutes of an interview. (2) They have killed something recently and can name what they learned, in specific terms with a timeline. (3) They can read a billing dashboard (AWS, GCP, Azure, OpenAI, Anthropic, the model provider of their choice) and tell you what is driving cost this month. (4) They can name the model and provider they used last week, the prompt version, and why they picked it over the alternatives, not just 'GPT' or 'Claude'. (5) They write a one-page strategy memo on a current question in under 30 minutes, with a clear position and three supporting reasons, in plain English. None of these are technical traits in the narrow sense, they are operating traits. The r/cscareerquestions and r/ExperiencedDevs subreddits both have well-developed threads on hiring senior individual contributors and tech leads; the high-signal questions cluster around traits 1 to 5 every time, regardless of whether the role is engineering, growth, or product.
Traits 6 to 10: operating model
(6) They run two-week sprints with written success criteria for each sprint, signed by the team lead and reviewed at sprint close. (7) They have an eval harness that ran in CI yesterday, with a green or red signal a stakeholder outside the team can read. (8) They keep a kill list alongside the roadmap (work shipped and then shut down, with a documented reason) and publish it internally on at least a quarterly cadence. (9) They publish weekly demos of working software, working journeys, working agents, not weekly status decks; the demos are a calendar event with a clear audience and a recording. (10) They review against marginal contribution, not against effort spent or features delivered; a sprint that shipped three features and moved no needles is a sprint to learn from, not celebrate. If fewer than five of these are true in your team, you do not yet have a performance team; you have an effort team in performance clothing. Will Larson's writing at lethain.com on staff-engineer operating models is the closest reference text in a software context; the same patterns generalise cleanly to digital and growth teams.
How to hire for these traits
Stop hiring on resume keyword match. The keywords are easily faked; the traits are not. Use a working-session interview where the candidate walks through a real problem from your actual roadmap and you watch how they reason: which questions they ask, where they push back on the brief, what they assume, what they verify. Ask them to draft a one-page strategy memo on something they shipped or killed in their last role, given 30 minutes alone in a room. Ask them what they killed last quarter, by name, with the postmortem. Three working sessions tell you more about a senior hire than ten phone screens. The Triplebyte and Karat writing on technical interview design has the methodological backbone; the Lara Hogan resilient management writing is the leadership analogue. Both predate the AI agent shift but transfer cleanly.
How to build the operating model
Start with the eval harness. Pick one campaign, one feature, or one agent. Write a test set with ground-truth labels (50 cases is enough for the first iteration, 200 within a quarter). Make the eval visible in CI with a green or red signal in the team Slack. The rest of the operating model (two-week sprints, kill list, weekly demos, marginal-contribution review) falls out of having a real measurement substrate, because the substrate forces every other practice to be honest. Without the eval, every other operating practice drifts into performance theatre within two quarters. The r/dataengineering and MLOps communities have spent five years arguing about how to do this right and the conclusion is consistent: start with the eval, layer the operating practices on top, and the team self-organises around the substrate. The Google MLOps maturity model and the Microsoft Responsible AI Standard both document the same sequence from different angles.
Further reading
Real, named sources the editor can swap in for specific URLs. We do not auto-link these because the right link changes over time. If you find a great primary source, write us and we will update the note.
- r/cscareerquestions, r/ExperiencedDevs. High-signal threads on hiring senior IC and tech leads. The traits cluster the same way.
- Will Larson, lethain.com. The reference text for staff-engineer-level operating models. Generalizes to digital teams.
- r/dataengineering, MLOps community. Where the eval-first operating model has been argued out over five years.
- Quora: Hiring senior engineers / Heads of Growth. Honest first-hand accounts of what works and what is theatre.
- Yao et al., "ReAct" (2022). The paper that reframed agent work as operating loops. Worth a read for any digital team thinking about AI.
Comments