Constructivist Evaluation

 Posted By: Debbie King | March 19th, 2010

Success Works adopts a Constructivist approach to evaluation and to its other social policy work.
Constructivism is a way of understanding the world that is based on the belief that “truth” is not an objectively knowable thing but is individually and collectively constructed by the people who experience it . ₁ As constructivist evaluators we assume that there are many different ways of interpreting or understanding the intent and impact of a policy or program and that to understand a policy or program the evaluator must gather together those understandings from a diverse range of sources. For a constructivist evaluator the truth of the situation is what the people who experience it describe.
Constructivism is a research paradigm that suits a complex world. We start from an understanding that our world, and the programs and policies that exist within it, is made up of a series of complex adaptive systems in which each individual participant and each individual process is in a state of constant action and reaction to each other, so that its overall behaviour and direction is unpredictable. From complex adaptive systems, emergence is the production of new and more effective ways of working that are greater than the sum of the parts. To discover the emergent properties of a system, the evaluator needs to see the overall structure and understand what has been achieved at an overall systems level. Success Works uses a systems analysis and meta-evaluation₂ approaches for this purpose.
As constructivist evaluators we do not assume that we know what the content or impact of a program or policy will be. We listen closely to how it is described by those in a position to know or understand the program or policy. We look for common themes in these constructions and these become the findings in relation to the evaluation.
In constructivist evaluation qualitative research approaches (interviews, focus groups, workshops) are always used and our questions allow the participants to convey their understanding of the program, policy or initiative. Questions are broad and open-ended so that participants can convey their understanding and their meaning of the situation being investigated. We recognise that our backgrounds and age, gender and cultural perspectives will shape our understanding of what we hear, see and feel in our evaluations. We use team based approaches and diverse staff teams to the extent possible to overcome these biases. We also acknowledge that they exist.
We use project/program logic as a key tool in the development of our evaluation frameworks. This is a constructivist approach which seeks to determine the project/program ‘theory’ or ‘hypothesis’ based on the knowledge and experiences of those responsible for the project/program. The “truth” of the logic model is measured through the evaluation based largely on the knowledge and experiences of those involved or in receipt of the project or program.
Findings from our evaluation interviews are presented in terms of patterns and themes. We look for common themes in the interviews and focus groups we conduct. The process of triangulating findings is a critical process when analysing data within a constructivist paradigm, as it enables ‘shared truths’ to be identified. Success Works recognises that, for many evaluations, it is appropriate to consider relevant quantitative data (e.g. survey data, administrative data sets) as another ‘axis’ in the triangulation process; that is, the quantitative data represents a particular perspective on a set of issues. Conceptualising quantitative data in this way allows for a true ‘mixed methods’ approach to evaluation, with an integrated analysis based around critical issues and themes, rather than the research methods themselves. In addition to its role within the triangulation process, quantitative data is also an important source of information for determining the scale and scope of a particular program or policy.

₁ ^{The other major approach to evaluation is positivism or post positivism. These approaches start from an understanding that truth is objectively knowable (positivism) or able to be approximated (post-positivism).}
₂ ^{Meta-evaluation is the evaluation of a number of related programs or policies and identifying findings at a systems level.}