Insight · Funders
Why funders keep learning about their impact from the people least able to tell them the truth, and what a structural fix would have to look like.
There is a structural oddity at the heart of philanthropy and public funding that almost no one names, because naming it is awkward for everyone involved. The funder wants to know whether its money worked. The only party positioned to tell it is the grantee. And the grantee's continued existence depends on the answer being yes.
This is the reporting loop: a closed circuit in which the organisation being evaluated is also the organisation supplying the evaluation, to a funder who will decide, on the strength of that evaluation, whether to fund it again. Everyone in the loop is acting in good faith. The loop still produces a systematically flattering picture, because it is built to.
This piece is about why that loop persists, why the usual fixes do not fix it, and what a genuine alternative would have to look like. It is also, in the interest of being straight with the reader, the thinking behind a service the Lab has built for funders. We would rather explain the reasoning than pretend the reasoning is disinterested.
No one designs the reporting loop. It assembles itself out of reasonable decisions.
A funder cannot personally observe every project it backs, so it asks grantees to report. Reporting is expensive for a grantee, so the funder standardises it into indicators. The indicators become the terms on which the next grant is judged, so the grantee, rationally, manages toward the indicators. And because the grantee both defines much of what is measured and writes the account of it, the report drifts, gently, honestly, toward the version of events that supports renewal.
Add the emotional reality and the drift gets stronger. The people writing these reports believe in their work, often rightly. They have watched it help someone. When they round a participation figure up, or describe a mixed result in its best light, they are not lying; they are advocating for something they have seen do good. Advocacy and assessment are different activities, and the reporting loop asks one person to do both at once, for an audience holding their funding.
The result is not fraud. It is something harder to correct than fraud: a whole sector's evidence base tilted a few degrees toward optimism, everywhere, all the time, by people doing their sincere best.
Once you see the loop as a system, its blind spots are predictable. Three kinds of knowledge are structurally hard to get from inside it.
The most basic one. When the reporter and the reported-on are the same party, no amount of good faith closes the gap between a self-assessment and an assessment. The funder is reading a case, not a finding, and cannot fully tell which parts are which.
Because each grantee reports in its own terms, a portfolio of grants yields a stack of incompatible stories. The funder who wants to ask the obvious portfolio question, which of these delivered the most change per pound, finds there is no common denominator to divide by. Every grant counted differently, so nothing can be ranked.
The deepest blind spot. Real impact is the difference between what happened and what would have happened anyway. That second quantity is invisible from inside a project; the grantee sees the people it served, not the people it did not, and not the change that would have occurred without it. Self-report almost always credits the intervention with the whole of the observed change, because the part that would have happened anyway is, by its nature, not in the room.
These are not failures of effort. A grantee working flat out cannot report its way out of a problem that is built into its position in the loop.
The sector is aware of the loop, if not by that name, and has tried to fix it. The fixes mostly strengthen the loop while appearing to challenge it.
More indicators. The instinct, when reporting feels thin, is to ask for more. But more self-reported indicators are more of the same signal, not a different one. Twenty optimistic numbers are not more independent than five; they are just heavier. The loop is untouched, and the grantee is more burdened.
Better software. The largest wave of investment has gone into platforms: dashboards that collect grantee data and render it in real time, benchmarked, board-ready. This is genuinely useful for workflow, and genuinely beside the point for truth. A dashboard is a lens, and a lens faithfully magnifies whatever you point it at. Point it at self-report and you get faster, prettier, more confident self-report. The software industry has, largely, automated the loop rather than broken it.
Logic models and theories of change. These are real improvements in rigour, and the Lab uses them. But a theory of change written by the grantee, and reported against by the grantee, is still inside the loop. It makes the self-assessment more structured. It does not make it independent.
The common thread is that each fix operates on the reporting, and the problem is not the reporting. The problem is who is doing it, and their position relative to the thing being judged.
There is only one move that changes the structure rather than decorating it: put an independent party, with no stake in the renewal decision, into the field, to gather primary evidence from the people the grant was meant to serve.
That sentence contains the whole difference, and each part of it is load-bearing.
Independent means the evidence-gatherer is not the grantee and does not benefit from the grant continuing. The incentive that tilts the loop is simply absent.
In the field means the evidence is primary, observed and asked directly, not aggregated from forms. It is the difference between reading that a service reached people and sitting with the people to learn what it changed. See the Lab's in-depth interview guide for the technique behind that.
From the people the grant was meant to serve means the counterfactual becomes reachable. Talk to the people who were not served, or who were served and gained nothing, and the change that would have happened anyway starts to become visible, because you are no longer only in the room with the successes.
None of this replaces the grantee's report. It re-positions it: from the sole source of truth to one input among several, checked against evidence gathered by someone with no reason to flatter it. The grantee is freed to do what it is actually good at, the work, while the assessment is done by a party built to assess.
This is not a novel insight about evaluation. It is the ordinary standard in any field where the stakes are high enough: audited accounts are not prepared by the company alone, clinical results are not certified by the drug's maker. Social funding has tolerated self-certification longer than most sectors would, largely because independent field verification is genuinely hard and genuinely expensive. That is a reason it is rare. It is not a reason it is unnecessary.
The clearest place the loop distorts is the social return figure. Social return on investment, done properly, is one of the most useful numbers a funder can hold: every pound in, so many pounds of social value out. Done through the loop, it is close to meaningless, because the same optimism that tilts a narrative compounds savagely in a ratio.
An honest SROI subtracts four things the loop tends to ignore: the change that would have happened anyway, the change caused by others, the change that merely moved a problem elsewhere, and the way benefits fade over time. Each subtraction lowers the ratio. Each is exactly the kind of unflattering adjustment a party inside the loop has no incentive to make. Which is why a verified SROI and a self-reported one can differ not by a little but by a multiple, and why the verified one is the only one worth putting in front of a board.
The point of verification is not suspicion of grantees. It is that a number a funder will make decisions on should be able to survive someone with no stake asking, of every figure, where did this come from. A ratio that resolves to its evidence is defensible. A ratio that resolves to a grantee's projection is a hope with a decimal point.
Breaking the loop costs something, and it is worth being honest about what.
It costs money, because field verification is real work and cannot be automated into a form. It costs a certain comfort, because independent evidence sometimes says a beloved grant is not working, and that is harder to receive than a glowing self-report. And it costs a change of posture: from reading grantees' accounts of themselves to commissioning an account from outside.
What it buys is the thing the loop cannot provide at any volume of reporting: a portfolio picture a funder can actually trust, compare, and defend. The ability to say, with evidence, that this grant delivered and that one did not, and to move the next pound accordingly. Over a portfolio and over years, that is not a marginal gain. It is the difference between funding in the dark with a good dashboard and funding with the lights on.
The Lab exists to study how technologies and interventions actually meet real people, independently, in the field, from the human side first. A funder's portfolio is that same question asked at scale: across many grants, what is actually reaching the people it was meant to reach, and what is it changing.
So the service we have built for funders is not a new capability bolted on. It is our core work, independent field research and honest impact measurement, pointed at a portfolio and fed into a platform a funder can run a board meeting from. The dashboard is the part everyone can see. The part that matters is underneath it: every figure checked, in the field, by a team with no reason to tell a funder what it wants to hear.
The reporting loop has held for decades because breaking it is hard. It is not impossible. It just requires someone outside the loop to go and look.
This is an independent insight piece by Transitions Lab. For the accompanying service, see For Funders. To discuss a portfolio, see Contact.