Part 2: The challenge of Complexity

In my last blog I touched on how our values influence the way we define a problem. Boyd Fuller, a Public Policy Theorist at the Centre for Governance and Public Management explains it like this:

“Parties with divergent frames may define problems quite differently. Some might, for example, see that slums are an unsightly blight which will then lead them to think of removing the slums and placing its inhabitants elsewhere. Others might see slums as a place where poor people work as a community to generate resources for self-reliance and see the problem more as one of providing the right policy environment for that innovation to flourish. Often, the differences in stakeholders’ stories of problem definition and appropriate solutions are missed by analysts and negotiators, who then attribute negative characteristics to the other side (unwilling to listen, not intelligent, aggressive) instead of understanding that the two sides are having two mostly independent conversations in which they argue about solutions (positions) without first clarifying what problem each side is talking about and what problem they should be solving as a group.”

Fuller’s analysis certainly rings true in the world of education: some practitioners (and policy makers) look at pupils who are struggling academically and argue that the solution is to offer them engaging pathways leading to alternative qualifications. Others argue for additional support to help them master the basics that stand in the way of academic success. Their disagreement is not only about how to solve a problem, it is also about the nature of the problem: are we failing to ensure all pupils have qualifications that will help them secure gainful employment, or are we failing to help all pupils access “the best that has been thought and written”?

This is just one dimension of the complexity that makes it tricky to pursue the ‘hypothesis driven’ approach to data gathering that I advocated in my last blog. Many of the challenges that education practitioners are grappling with are so complex that the appealing idea of identifying a problem (like poor academic achievement), generating a hypothesis about it (that it’s caused by a lack of reading skill), finding some information about it (testing pupils’ reading) and then tackling it (for example through extra reading support), is never quite that simple. 

One attendee at our recent roundtable therefore urged us to learn from research on “Wicked Problems”.

The idea of Wicked Problems originated in a 1973 article by two researchers in the field of Urban Planning, called Rittel and Webber. The concept caught on like wildfire and their original article has now been cited over 20,000 times.

According to Rittel and Weber, back in the early 70s people were beginning to realise the difficulties of defining problems and finding where, in a complex causal network, the problem originated. They argued that these Wicked Problems were fiendishly difficult to solve simply by deploying data and evidence. 

They enumerated the characteristics of Wicked Problems as follows: 

1. These problems are hard to define objectively and values come into the mix.
2. “There’s no stopping rule” - i.e. there’s no point at which you can say “right we’ve solved that problem.” Spending more, plugging in more resources or further analysing the problem will generally result in finding an even better solution. 
3. Judging how good the solution is involves deciding on a good or bad solution, not just a correct or incorrect one. In other words, even if beating children with sticks improved educational achievement, it might not be a good solution. 
4. The complexity of the issues means there will be extensive waves of consequences across the system. These are hard to predict and may not be known for a long time.
5. Every time you try to act, it changes the situation. Trial and error therefore become difficult. For example, if you try giving Saturday detentions when pupils don’t complete homework and discover that that approach doesn’t work, then the fact that you’ve tried that approach might change your relationship with the pupils and their families. That means you’re now facing a subtly different situation - so when you try another solution you’re tackling a subtly different problem.
6. There isn't a defined set of solutions to test. 
7. There are significant differences between each situation you encounter, making each problem unique. 
8. Problems are nested, in other words, each problem might be a symptom of another problem; poor behaviour might result from poor literacy, which might result from inequalities in early childhood experiences. 
9. How you define the problem also affects your preferred approach to solving it. 
10. Practitioners have no right to be wrong because they are expected to make things better and the consequences matter a great deal. 

There are various problems with Rittel and Weber’s argument and some researchers have since argued that Wicked Problems aren’t in fact distinctive. They also worry that when people say a problem is ‘wicked’ they may sometimes be using that as an excuse for not doing anything about it. However the issues that we deal with in education are undoubtedly incredibly complex, particularly in relation to the uniqueness of each situation, the inter-related and nested nature of the issues and the scope for unanticipated consequences; removing a pupil from class for an hour a day for one-to-one maths tutoring might improve their maths, but it can also precipitate a range of other consequences. Evidence that one-to-one tutoring improves maths outcomes for disadvantaged pupils therefore might not be enough to confirm that it’s the optimal solution to educational equity.

Given the complexity of the problems we’re addressing it’s no surprise that some of the blunter ways we use information don’t allow us to make the improvements we hope for. Perhaps the answer is therefore to look at each problem, or hypothesis and to ask the question “what level of simplicity or granularity do I need?” Thus, when seeking to understand how community context might shape how pupils engage in school, a high degree of complexity might be involved. The situation could therefore merit the “what should we know about your community” approach that I described in my last blog. Meanwhile for other questions - such as identifying misconceptions about a specific topic, more simple, dashboard style information from closed questions, could be sufficient. 

Perhaps it’s when we misjudge the degree of complexity involved in questions relating to educational equity that things go wrong - but, that’s not to say that better information doesn’t help us make better judgments. We just need to develop the right culture around data - which is what I’ll be exploring in my next blog.

This is part two of a 3-part blog series asking, 'How can information help teachers, youth practitioners and leaders map the way to educational equity?' which forms part of our Mapping The Way discussion series with Loic Menzies.

You can join the conversation by following along on Twitter using using the hashtag #MappingTheWay and tagging @LoicMnzs and @AssessNetwork! Plus, you can sign up for our next webinar and add your voice.