The five jobs of a digital performance agency
A real digital performance agency owns five things at once. The measurement model (what counts as success, how it is attributed, what the holdout looks like, how often we refresh the control cohort). The activation surfaces (where the message lands: paid search, paid social, programmatic, owned email, push, SMS, in-app, web). The creative system (what gets shown, the rubric for variant generation, the brand guardrails the agent works inside). The optimisation loop (what we change next week, why, what we shut down). The executive reporting layer (what the CFO sees, on what cadence, with what supporting evidence). Most agencies own one or two of these and bolt the rest on through subcontractors or partner pitches. The result is a deck that says 'performance' on every page and a P&L that does not move. The five-jobs frame originated in operating-model writing in the early 2020s, including HBR pieces by Marc Pritchard at P&G on the rebundling of marketing operations, and now shows up in most agency RFP templates the buyer side writes.
Measurement first, creative second
The order matters. A campaign whose success metric is engagement rate will reliably hit engagement rate and reliably fail to grow revenue. Performance agencies should agree the success metric before the first creative is briefed, and the metric should be one a CFO would defend in a board meeting: marginal contribution against a holdout, payback period, contribution margin per acquired customer, retention-weighted LTV. If your agency has not asked you what your gross margin or contribution margin is in the first three meetings, they are not a performance agency. They are an engagement agency with a performance label. The r/PerformanceMarketing subreddit has a recurring thread (search 'attribution honesty') that breaks down which metrics survive boardroom scrutiny and which do not. Mark Ritson's Marketing Week columns make the same point about brand-versus-activation balance from a different angle, and Byron Sharp's How Brands Grow is the textbook reference for why the right metric is rarely the most-tracked metric.
How AI changed the job in the last 18 months
Specialised AI agents now do the work that used to take a team of three: variant generation across channels, send-time decisioning per profile, post-campaign analysis with hypothesis generation, audience design from warehouse data, anomaly detection in performance dashboards. The agency's job is no longer to do the work, it is to design the agent, set the success criteria, write the eval harness, and stay accountable for the outcome. We ship these in production in under two weeks for clients on the agent side, then operate them on a 90-day on-call before handing over. The shift from execution to agent design is best captured in Anthropic's Building with Claude documentation (the eval-first guidance, refreshed quarterly), the ReAct paper (Yao et al., 2022) on reasoning-and-acting loops, and Schick et al.'s Toolformer paper (2023) on tool-augmented language models. All three make the same point in different words: the moat is the eval, not the model.
Red flags when picking one
Three patterns to walk away from. (1) An agency that cannot show a written eval harness for any campaign they have run in the last 12 months. The absence of an eval is the absence of measurement discipline; you can also ask to see their kill list for the same period and learn the same thing. (2) An agency that bills by the hour instead of by the agreed outcome. The hourly-billing pattern shows up on r/marketing and r/agency threads weekly; it correlates so strongly with weak measurement discipline that we treat it as the single highest-signal red flag. (3) An agency that lists "AI" as a service line without naming a model (Claude, GPT-4, Gemini, a specific open model), a deployment region (EU, US, in-region for regulated industries), or a regulatory framework (SOC 2, GDPR, India DPDP, sector-specific). Performance is a discipline; "AI performance" without specifics is a marketing claim. Walk fast.
How to evaluate a digital performance agency in 30 minutes
Four questions, in order. Question 1: 'Show me the last campaign you killed because the numbers did not land. Name it, walk me through what you tried, what failed, what you learned, and where the postmortem lives.' If they cannot name one, they have not learned the discipline of killing things. Question 2: 'How do you write success criteria, and where is the example I can read in the next ten minutes?' A real performance shop has at least one redacted SOW or campaign brief they can share. Question 3: 'How do you handle row-level security on the activation layer between the warehouse, the CDP, and the channel?' The good ones have a written pattern, usually attribute-based access control mapped to sandboxes. Question 4: 'What is your kill clause? When can either side walk away with no penalty, and have you ever invoked it?' These four questions surface around 80 percent of what you need to know about whether the agency is operationally serious or PowerPoint-serious, inside 30 minutes.
Further reading
Real, named sources the editor can swap in for specific URLs. We do not auto-link these because the right link changes over time. If you find a great primary source, write us and we will update the note.
- r/PerformanceMarketing. Working practitioners debating attribution models, vendor evaluation, and agency contracts.
- r/marketing. Recurring threads on agency vs in-house, hourly billing red flags, and KPI hygiene.
- Stack Overflow tag [marketing-attribution]. Technical Q&A on attribution modeling, control groups, and uplift measurement.
- Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (2022). The paper that reframed AI 'execution' as 'designing the agent loop'. Still the cleanest read on what changed.
- Sequoia Capital, "Generative AI's Act Two" (2023). The piece most agency strategists are quietly reading when they say "AI is moving up the stack".
- Anthropic's Building with Claude documentation. The eval-first guidance that has changed how serious performance shops scope agent work.
Comments