|||
One of the most important part of any work is measuring whether we are doing the right thing. In general, social programs are notoriously bad at this.
Most funders do not ask for more than “activity or output” measures. For example, how many families visited the service? How many children did you see? Number of families who have completed the program? I have seen scenarios where measures like “number of children who visited the dentist” are used. That’s it. Nothing further is looked at as it is “hard” to measure and collect. How does this help anybody?. if my daughter visited the dentist, I will be interested in not only that she has visited but what is the outcome. Is her teeth healthy? Does she need anything else? What about her gums? When does she need to be back for another check up? Any suggestions and tips? However, for some programs the measure is whether there has been a visit.
In my view I see three kinds of measurements.
The first kind are improvement measures. These are the “what happened after you visited the dentist” kind. What can we do with the information that is useful to improve the program? Does that information tell us something more about the issue at hand. In the dental example, do we know how the “teeth” is and in subsequent visits can we find what is the trajectory of the improvement. In these we can use other ratios like “teacher-student” ratio for example, if that makes a difference.
As the image suggests, it is as simple as that. What has happened in the past, what is happening now and where is this going? If we can predict the improvement and then track it then it helps us with working with our assumptions.
The second kind are evaluative measures. These are done infrequently and tell us whether a particular program is working or not. Randomised control trails fit here. Realist evaluation also works for this. If you are serious, cost benefit analysis follows. Sometimes these are done the first time a service is provided and not repeated again. These are generally expensive, are useful to know what happened and nobody funds them. In my estimation, millions of dollars are spent on social programs that are not evaluated. Go figure.
Lots of services are being provided to many people across Australia and in many parts of the world. Why have the outcomes not improved? In a lot of cases we can’t answer that question because we don’t even both to evaluate.
The third kind are well-being measures. The whole point of services and programs are whether people and the communities that they live in are better off. Mortality rate, life span, educational attainments, quality of water, etc. These are the point of doing anything. However, the challenge is to understand the cause and effect. Which programs create which outcomes and what are the environmental and social conditions for well being. In some cases, the programs are powerful and in other cases they do not matter much on their own. In most cases, well being is beyond any one program. And in a lot of cases, general economics, social and cultural aspects of life governs well being.
First, the most important thing is to understand your world view. Do you view society as something in which your programs and services are part of it and only one part? Or do you view your services as an overly important part of society? Is the service within society and or the services are the main thing which effect society. Understanding this world view is possibly the first step.
Second, how do you see the cause and effect? Do you see the services, programs, welfare money, government support as the key to improving lives? Do you see that as what has created the prosperity of Australia for example till today? Or do you see other players like business, community, family structures, people’s individual motivations and aspirations as more important?
Third, for which set of people do we need what?
We need to think about “performance” across all these levels and think about why and what we do in that space.
