Can perceptions of risk be truthful?
For many of us, and particularly those in decision making roles, trying to understand the complex landscape we occupy now is getting harder. It is why, we draw on known ‘expertise’ when it comes to knowledge products and research to help inform our thinking.
But how can we ensure that the analysis we are relying on is truthful? How can we be confident that it is giving us an ‘accurate as possible’ sense of the world.
I have, as many others for many years, relied on global analysis in my work. As I have evolved in my expertise, knowledge and experience, I questioned whether ‘global reports’ can be truely independent, confident, and verifiable that speaks to the complexity and nuance of the world.
So I thought I’d do a bit of a nosy into two reports that I have drawn on in my career, to just test my hypothesis.
This is a “globally deployed online survey of the general population. Designed by our research team, the survey questionnaires are programmed, translated, localized, and completed in up to 28 countries around the world, with demographically representative samples reflective of each population.” The methodology consists of a 30 min online interviews, conducted over a period of 3 weeks across 28 countries, with 32,000 respondents, with approximately 1500 respondents per country.
Now at face value, this seems a reasonable sample size. But lets break this down:
- 28 countries out of 195 countries is less than 14% of countries surveyed
- 32,000 people globally out of a global population of 8.1 billion is…? Significantly less than 1% surveyed.
- These are the countries surveyed: Argentina Australia Brazil Canada China Colombia France Germany India Indonesia Ireland Italy Japan Kenya Malaysia Mexico Netherlands Nigeria Saudi Arabia Singapore S. Africa S. Korea Spain Sweden Thailand UAE UK U.S.
Lets break this last data point down further. The World Bank Group country classifications by income level for FY24 (the actual data set is available here): lists these 28 countries as follows:
- Australia, Canada, France, Germany, Ireland, Italy, Japan, Netherlands, Saudi Arabia, Singapore, Sweden, Spain, UAE, UK, US — High Income Countries
- Argentina, Brazil, China, Colombia, Indonesia, Malaysia, Mexico, South Africa, South Korea, Thailand — Upper Middle Income Countries
- India, Kenya, Nigeria — Lower Middle Income Countries (note that there is classification difference between Lower Income Countries and Lower Middle Income Countries)
So what does this mean? Of the 28 countries sampled, 11% are from Lower Middle Income Countries, 36% are from Upper Middle Income Countries, and 54% are from High Income Countries (note — I’ve rounded up the percentages)
Therefore — out of 14% of countries, with a sample size of less than 1% of the global population surveyed — 54% of which are from High Income Countries and 0% from Low Income Countries.
How can a report that surveys 14% of countries, and leaves out an entire demographic of states and people that live in fragile and vulnerable contexts — speak as a global authority on its findings, when its findings are skewed to a particular income level?
Edelman’s 2024 findings state that “developing countries lead on trust”. The countries that this includes are China, Indonesia, UAE, India, Saudi Arabia, Singapore, Thailand, Kenya, Malaysia, Mexico. All these countries are across all income levels — lower middle income, upper middle income, and high income.
So what constitutes them as being ‘developing’? That they are countries perceived to be ‘Global South’? This broad categorisation already is biased which means the analysis is not clean nor independent.
Most of the countries that are counted as most trusted, are widely perceived to be autocratic. It has been argued that these same countries end up exploiting Edelman’s findings to legitimise their reputation. Adam Lowenstein in an article in The Guardian on 24 November 2023 states that “Edelman has been less forthcoming about the fact that some of these same authoritarian governments have also been its clients”, and have been paid millions of dollars by those same governments, including Saudi Arabia and the UAE.
What isn’t disclosed, or does not appear to be factored into the data is that citizens’ or survey respondents from (perceived) autocratic countries, tend to not publicly criticise their government for fear of reprisal. The report also further establishes that businesses are more trusted than governments and are that they are most trusted to integrate innovation into society. Lets unpack this — innovation and business are factors rated for trust.
I would actually argue that innovation and business are integral if the aim is to make trust profitable. Which then makes complete sense as Edelman, fundamentally — is a PR company, focused on profit.
Lets now go to the World Economic Forum (WEF) Global Risk Report .
This report, done in partnership with two global insurance companies is often drawn on as a barometer of global risk. NB: Insurance companies define/view risk as something that could go wrong that would result in financial damage to key stakeholders
Before going further, I must stress that any ideas of current and emerging risk are fundamentally baselined on perceptions of risk. The WEF report cites itself as a risk perceptions report. This is an important footnote as how risk categories are then ranked are entirely based on the perceptions of people that are providing that ranking.
So who ranked the risks? Firstly, there were approximately 1490 surveyed globally (significantly less than the Edelman report). Again — 1490 out of a global population of 8.1 billion is … ?
Appendix B of the WEF 2024 report outlines that of those 1490 survey respondents, 61% identified as male. The highest percentage of respondents were from Europe (38%) and North America (18%), followed then by Latin America and the Caribbean at 9%.
To complement the global risk data, a further survey (see Appendix C of the report) was done with 11,000 business leaders in 113 economies to provide insight into ‘local concerns and priorities’. I want to highlight that this ‘Executive Opinion Survey (EOS)’ focused on business leaders. Not civil society leaders. Not government leaders. Not religious leaders. Not youth leaders. Not any other kind of ‘leader’ that exists in every community.
Essentially the majority of respondents are from Europe and North America, are business leaders, and predominantly identify as male (note: the report does not use the language of ‘identify as male’ but just gendered as male). In addition, the highest age bracket of survey respondents are firmly between 40–59 (i.e. very much middle aged).
So perspectives of global risk were obtained from arguably privileged groups. This explains then why categorisation of global risks were broad in this report (i.e. extreme weather events and not heat inequality or heat stress) are highlighted. Another data point — involuntary migration. This categorisation is immediately biased and plays into a right wing narrative that is untrue. Migration is not a risk nor a problem to solve. The factors that force people to move involuntarily, or their barriers to be safe and protected in their next destination are risks. But not migration itself.
The more abstract a risk data point, the more opaque its impacts and its subsequent risk mitigation. The more biased a data point, the more it influences mitigation/adaptation measures that are not fit for purpose and could possibly cause harm. When both the above become fact, then the burden of managing it falls downstream — to those that are most disenfranchised, and exposed.
So…what does this even mean? Firstly, I want to be clear — that both these reports are just fine. They provide a good point of view — but this is key: it is a specific point of view. It doesn’t diminish their findings, but it is important to balance their findings against whose perspectives those findings privilege.
If data and risk analysis does not account for positionality and bias, it merely is a representational view that in fact, leaves much of the world out of its scope. If the truth of these risks are only important to a certain group of people, how will we know how to be prepared for threats and harms that affect all of us?
What both these reports do though, is weave a connection between issues of trust and risk that are bound to weightings of capitalism. If trust as perceived by Edelman and accepted by the world, is linked to merely business and innovation, what we are testing then is the profitability of trust. If risk as perceived by WEF and accepted by the world, is linked to merely its financial damage, what we are testing then is the profitability of that risk, not the non-financial damage that it will cause to populations.
A lie, repeated often enough, gets accepted as truth.
Don’t accept data analysis as fait accompli. Question, interrogate, understand the motives, the agenda, who, where, how that analysis was done. Do not rely on just one type of data source — blend, cross-test, triangulate. Never assume that drawing meaning from complex data is easy. Do it the service it deserves.
We do ourselves in injustice and disservice when we are lazy in our risk and data analysis and merely broad brush stroke its insights. We question then why our actions don’t yield just, tangible results for all. Well I argue that its partly because we’re solving for the wrong problem, and for the wrong groups of people.
We end up trusting what we don’t fully know, and the long term impacts of doing that — are not borne by a small group, but instead all of us bear the brunt of failure.
Full Disclosure: I have been an advisor to the World Economic Forum in their stream of work on Technology and the 4th Industrial Revolution. I have also quoted from both the WEF Global Risk Reports and Edelman Trust Reports in the past.
