Is British democracy facing a legitimacy crisis?
One of the biggest and most debated questions in political science is whether there is a legitimacy crisis. Seminal, widely-cited texts explore the causes and consequences of this apparent crisis, but there is still little agreement on whether there even is a crisis of legitimacy. [1]
Such a question should, on first blush, be relatively easy to answer: just collect the responses to surveys and graph the averages over time. And still, there is little agreement over whether there is any decline in perceived legitimacy. What if this is perfectly reasonable: what if the data are actually showing you different answers, despite using essentially the same survey items?
Perceptions of democratic legitimacy are usually measured through surveys, primarily measuring things such as ‘satisfaction with democracy’ or ‘trust in parliament’. These questions ask the respondent, in various ways, to rate how satisfied they are with how democracy works in their country or how much they trust their country’s parliament (and other political institutions). Respondents are usually given either a binary or four/five-category response scales, such as ‘trust or do not trust’ or ‘very, fairly, not very, or not at all satisfied’. Almost all existing research on citizens’ attitudes towards their domestic system is measured like this.
In this post, we compare two highly respected, high quality, and regularly used surveys that use almost identical survey items to measure democratic satisfaction, a common indicator for democratic legitimacy. We show how the surveys not only provide different absolute levels, but contradictory trends in democratic satisfaction in Britain. We also show how the likely source is a rather innocuous change in wording.
The surveys
We use the British Election Study (BES) 1997–2017 and the Eurobarometer (EB) 1973–2017. The BES is fielded after every election, whilst the Eurobarometer is fielded every 6 months and within some ad hoc surveys (the ‘special Eurobarometers’). Each BES wave contains a sample size of approximately 3000, and the EB sample size is approximately 1000 per wave. Both surveys are restricted to England, Scotland and Wales.
Whilst the BES has six waves (years), coinciding with British general elections, the EB runs almost every year from 1973, excluding 1974, 1975, 1996 and 2008. Although there are often multiple waves in each year, we aggregate the responses according to year.
The survey items for democratic satisfaction are presented below and are very similar, particularly in the 1997 BES. Logically, these are identical survey questions and responses. Both the EB and BES are asking whether the respondent is satisfied with the way democracy works in Britain on a scale of 1–4. The main difference is that, in BES 2001–2017, the response is ‘a little/very dissatisfied’ rather than ‘not very/not at all’ satisfied.
Is there a legitimacy crisis in Britain?
How do these look over time? For simplicity, we collapse them into binary indicators, where 1 equals ‘very’ or ‘fairly’ satisfied, and 0 equals dissatisfied/not very or not at all satisfied. The trends are the same with the original response categories. We then plot the percentage of respondents satisfied within a given year.
The two graphs lead to the opposite conclusion. If one were to rely on the BES data (right), one would indeed find a relatively slow, linear decline in democratic satisfaction. On the other hand, using the Eurobarometer data (left), one would conclude that there’s been the opposite trend: increasing satisfaction with the way democracy works in Britain.
It may, however, be misleading since the Eurobarometer is asked so regularly relative to the BES. So, we create mean averages for each year in line with the BES. Even in the year with an essentially identical question and identical response categories (1997), there is a 13-percentage point difference in the absolute levels of democratic satisfaction. This may be down to when the BES is asked — just after an election, and a particularly emphatic one in 1997. More worrying, however, is the continuing divergence in the trends.
Of course, there are other potential measures for democratic legitimacy that we do not look at here and which could present clearer and more consistent pictures of levels and trends of legitimacy in Britain in recent decades (it seems, for instance, political trust shows a steady downward trend in both). However, satisfaction with democracy is a very commonly used measure of democratic support and legitimacy, and these stark discrepancies between reliable sources that survey nationally representative samples in the same country and across overlapping periods is a serious cause for concern. [2]
Question wording effects
What could this be down to? It’s well known how seemingly innocuous wording changes in surveys (either in the question or response scale) can drastically alter the responses. Given this, the slight discrepancy in questions might offer one reason for the different trends.
Fortunately, the British Election Study 2015 fielded both types of questions, with a self-completion questionnaire complementing the face-to-face interviewing. The self-completion questionnaire contains a question from the Comparative Study of Electoral Systems survey which is identical to the Eurobarometer question used above, while the interview includes the original BES question. This provides the best opportunity to take a look at the potential discrepancy in question wording.
The table below provides a tabulation of the two questions, with the (rounded) percentages of respondents falling into the different cells. 40% of those who respond they are a ‘little dissatisfied’ in the BES version of the question answered ‘fairly satisfied’ in the Eurobarometer version. More alarmingly, a full 18% of those who say they are ‘very dissatisfied’ answered either fairly or very satisfied. This becomes starker if, as we do in the above graphs, make the variable binary. Approximately a third (34%) of those who are coded as dissatisfied in the BES version are indeed coded as satisfied in the Eurobarometer version of the question. Overall, whilst the correlation between the two is statistically significant, they are only correlated at 55% — lower than would be expected from questions that are intended to measure an identical attitude.
Whilst we can’t say for sure, these discrepancies appear to be more than large enough to explain the divergent trends between the two data sets — although can’t explain the large difference in 1997. The question nonetheless remains about why such similar questions, aimed at the same underlying attitude, get significantly different results.
What we do know, however, is that the answer to whether there a growing democratic legitimacy crisis in Britain depends on which data you are looking at. This post shows the importance of using multiple indicators and data sets in this type of research and highlights the caution with which we should interpret any findings using any particular measure of satisfaction with democracy.
Posted with Viktor Valgardsson, PhD researcher at the University of Southampton
[1]
Dalton, Russell J. Democratic Challenges, Democratic Choices. Oxford: Oxford University Press, 2004.
Newton, Ken. “Trust and Political Disenchantment.” In Political Trust and Disenchantment with Politics: International Perspectives, edited by Christina Eder, Ingvill C. Mochmann, and Markus Quandt, 19–31. Leiden: Brill, 2014.
Norris, Pippa. Democratic Deficit: Critical Citizens Revisited. Cambridge: Cambridge University Press, 2011.
[2]
Anderson, Christopher J, and Christine a Guillory. “Political Institutions and Satisfaction with Democracy: A Cross-National Analysis of Consensus and Majoritarian Systems.” The American Political Science Review 91, no. 1 (1997): 66–81. https://doi.org/10.2307/2952259.
Grönlund, Kimmo, and Maija Setaïa. “Political Trust, Satisfaction and Voter Turnout.” Comparative European Politics 5 (2007): 400–422. https://doi.org/10.1057/palgrave.cep.6110113.
Linde, Jonas, and Joakim Ekman. “Satisfaction with democracy: A note on a frequently used indicator in comparative politics.” European Journal of Political Research 42, no. 3 (May 2003): 391–408. https://doi.org/10.1111/1475-6765.00089.
Quaranta, Mario, and Sergio Martini. “Does the economy really matter for satisfaction with democracy? Longitudinal and cross-country evidence from the European Union.” Electoral Studies 42 (2016): 164–74. https://doi.org/10.1016/j.electstud.2016.02.015.
