Method
What numbers can measure, and what only people can tell you. The Lab's plain-language guide to the two kinds of research evidence, and how they work together.
Every serious study of impact eventually meets the same fork. Do you count, or do you ask? Do you measure how many and how much across a large group, or do you sit with a smaller number of people and understand how and why? The honest answer, most of the time, is both. But the two produce different kinds of evidence, and knowing which one answers which question is the difference between a study that informs a decision and one that merely decorates it.
This page sets out the distinction plainly, and then makes the case that most organisations under-invest in: what qualitative evidence can tell you that numbers, on their own, never will.
Quantitative research measures. It turns experience into numbers that can be counted, compared, and generalised across a population. Its strength is scale and comparability: it can tell you that adoption rose by 34 per cent, that 62 per cent of users are women, that the effect was larger in one region than another. Its discipline is sampling and measurement, and its characteristic output is a figure with a confidence interval around it.
Qualitative research explains. It works in words rather than numbers, and stays close to the reasoning, the meaning, and the texture of what happened. Its strength is depth: it can tell you why adoption rose, what the change meant to the people living it, and what they would fix first. Its discipline is interviewing and interpretation, and its characteristic output is a theme, grounded in evidence, that explains a pattern the numbers only describe.
Neither is softer or harder than the other. A biased survey produces confident, precise, wrong answers just as easily as a leading interview produces shallow ones. Both are rigorous when done well, and both are worthless when done badly.
| Quantitative | Qualitative | |
|---|---|---|
| Answers | How many, how much, how often | How, why, and what it means |
| Works in | Numbers, at scale | Words, in depth |
| Strength | Comparability & generalisation | Reasoning, meaning, the unexpected |
| Discipline | Sampling & measurement | Interviewing & interpretation |
| Output | A figure, with a margin of error | A theme, grounded in evidence |
| Fails when | The sample is wrong or the question is blunt | The interviewer leads, or stops too soon |
A number records the answer. Qualitative evidence reaches the layers beneath it, and those layers are where most of the actionable insight lives. Six things, in particular, only qualitative work can give you.
A survey can tell you that people stopped using a service. It cannot tell you why, because the reason was never one of the boxes you offered. Only a conversation, following the person's own account of the decision, surfaces the actual mechanism: that payment arrives on a Monday and must be spent before it disappears, that a rumour about reliability spread through a stage, that the nearest agent moved. The reason is almost never the one the questionnaire anticipated, which is exactly why it has to be asked openly.
Numbers measure change from the outside: income up, time saved, distance reduced. They cannot measure what that change means to the person, and meaning is often what determines whether a change lasts. "I am not chasing it all week; I know where I stand" is not a data point a survey can capture, but it explains retention better than any satisfaction score.
The deepest advantage. A survey can only measure the variables you built into it; it is structurally blind to the factor you did not anticipate. Qualitative research is open enough to be surprised. The unexpected barrier, the unintended benefit, the workaround nobody designed for, these show up in interviews precisely because interviews are not confined to a fixed list. Many of the most important findings in a study are things no one thought to put on the questionnaire.
A headline number averages over everyone it reached, and quietly excludes everyone it did not. Qualitative work can go and find the people the intervention was designed for but never touched, and ask them why. That is often the single most informative conversation in a study, and it is invisible to a survey of users, because non-users are not users.
Ask people to rate a list of features and you get their ranking of your categories. Ask them what they would change tomorrow and you get their priorities, in their own words, about the things that actually matter to them. The fix-it question surfaces a ranked list of real friction that no pre-set scale could have produced, because you did not know the friction existed.
Baseline and endline numbers show that something changed. Qualitative evidence shows the change as it was experienced: the genuine contrast between how life or work was done before and how it is done now. That contrast is what makes a finding legible to the people who have to act on it, and persuasive to the people who have to fund it.
There is a reason organisations drift toward the quantitative by default. Numbers are easy to collect, easy to put in a dashboard, and easy to defend. Outputs, participants reached, workshops delivered, units sold, are especially seductive, because they are fully within the programme's control and require no one to ask a hard question.
But easy-to-count is not the same as worth-counting. A programme can report impressive outputs and deliver no real impact at all; the reverse, modest activity that changes everything for the people it touches, is rarer but real, and no output metric will ever reveal it. When a study measures only what is convenient, it produces a confident account of the wrong thing.
Qualitative evidence is the corrective. It is how you find out whether the numbers mean what they appear to mean.
The strongest studies are not qualitative or quantitative; they are designed combinations, sequenced so each covers the other's blind spot.
Numbers first, then depth. A survey finds that adoption dropped in one region, and interviews then explain why. The numbers locate the question; the depth answers it. This is the most common shape, and it is powerful precisely because it points the qualitative work at the places that matter.
Depth first, then numbers. Interviews surface an unexpected factor, and a survey then measures how widespread it is. This is how you avoid measuring the wrong variables, by letting the fieldwork tell you what the important ones are before you scale up the counting.
Read together, on the same question. The survey says what happened and to how many; the interviews say why and what it meant. Confidence comes from the two agreeing. When they disagree, that disagreement is itself the finding, and usually the most important one in the study.
Transitions Lab leads from the qualitative, and quantifies around it. We do this because the questions that decide whether a technology or a programme actually works, who adopts and who is left out, why people switch or refuse, what a change means to the people living it, are questions numbers can describe but not explain. We use surveys and administrative and sensor data to establish the pattern at scale, and we use depth interviews to understand it. The result is evidence that is comparable where it needs to be, deep where it needs to be, and honest about what it can and cannot conclude.
For the depth technique itself, see the in-depth interview guide. For the quantitative spine that sits alongside, see the impact-tracking template. For the wider Impact Measurement service that pairs the two, and the Field Research capability behind it, follow the links.
If a decision in your work turns on understanding not just what changed but why, and for whom, that is exactly the territory qualitative evidence was built for. To discuss a study, see Contact.