Explaining the different results in opinion polls on Irish unity

Before the Brexit referendum, repeated opinion polls suggested that half or more of those who voted for Irish nationalist parties in Northern Ireland, did not want to call an immediate referendum on Irish unity and if a referendum were called, they would vote against immediate change.[1] Less frequently identified in polling, are the middle ground voters who largely supported UK membership before Brexit, but who are now much more divided in their views. Indeed, on polling related to Brexit and the Northern Ireland Protocol, centre ground voters are much closer in their views to nationalists, than unionists and their views on Irish unity are now clearly related to a desire to retain as close as link as possible to the European Union.

Illustrating the political shift that has taken place following the Brexit referendum, a December 2018 poll found that:

• 35% of nationalists wanted a border poll to be held in 2019,
• 79% wanted one within 5 years, and
• 89% wanted a poll within 10 years.

In the same survey, 93% of nationalists said they would vote to leave the UK, and a further 5% of nationalists ‘probably would’, if the poll was held in 2019, in the context of a ‘no deal’ Brexit. [2]

A poll in September 2019, by Ashcroft Polling, confirmed these shifts in opinion in NI.[3] In that poll, 45% of respondents said they would vote to stay in the UK, and 46% said they would choose to leave and create a United Ireland – a lead of 51% to 49% for Irish unification when ‘don’t knows’ and those who say they would not vote are excluded. While within the margin of error, the political impact of even one reputable poll showing that Northern Ireland public opinion was so finely balanced marked a watershed.

A Sunday Times poll of January 2021 – carried out as part of a feature on the future of the UK union, showed a majority in Northern Ireland in favour of holding a poll on Irish unity and when asked how they would vote, 46.8% said to stay in the UK, 42.3% chose a united Ireland and 10.7% were unsure. In that poll a majority of those under 44 years of age supported Irish unity.

Moreover, among ‘constitutional neutrals’, that is those who did not vote for a nationalist or unionist party in any of the last 3 elections – 38% said they would vote for a United Ireland, 36% said they were unsure but would vote, and only 26% said they would vote to stay in the UK.[4]

Polls held during the EU-UK negotiations on Brexit, confirmed a strong linkage between the nature of the post-Brexit agreement and attitudes to a united Ireland. In a 2018 poll, 37% of ‘others’ in Northern Ireland wanted a border poll within 5 years, with 68% wanting this within 10 years. If there was no deal, 70% of such voters said they were certain or likely to vote for Irish unity, whereas if a hard Brexit did not proceed, this proportion dropped to 54%.[5]

Nonetheless there is still considerable variation in the results from different polling companies. Table one summarises the latest polls for NI (which have different methodologies and question wording).

While there is a good degree of consensus on the percentage support to stay in the UK, with 6 out of 8 polls in the 45% to 49% range, there are outliers in two of the Liverpool polls, which polled all respondents and not just voters or likely voters. With the exception of the two outlier results, the biggest variation is in the balance between definite supporters of unity – widely ranging from 29% to 46%, and don’t know / no reply ranging from 8% to 25%.

There are many reasons why polls might get different results, but one way to explore their accuracy is to look at how they estimated party support just before the most recent PR election (May 2022) – when voters did not have to vote tactically (tactical voting is more likely in a Westminster election). Using this approach, we see extensive variation in party support estimates.

The most evident feature of this comparison of polls is the relative accuracy of the Lucid Talk, Liverpool and Survation polls, and the enormous gaps between NILT and actual election results. Moreover, this large underestimation of Sinn Féin, in particular and an over estimation of Alliance, is a feature of NILT polls over the quarter century post GFA period, with some polls showing Sinn Féin support as low as 9%. The extent of NILT’s inaccuracy – showing the UUP more popular than the DUP, SF and SDLP with the same support and Alliance and Greens on 30.4%, instead of 15.4%, means that this series of polls should be used with extreme caution, if at all, to discuss political support or attitudes to constitutional change, either as a one off or over time and arguably be excluded from “poll of polls” aggregated results as they inaccurately skew the average.

The most recent Liverpool polls are similar to Lucid Talk in their estimates of support for Sinn Féin, the DUP, UUP and SDLP, but overestimate Alliance and Greens (20.9% in Liverpool, 17% in Lucid Talk, compared to an actual 15.4%).

Even if NILT is excluded, there is still a considerable variation between the three Liverpool polls, over the 2019 to 2022 period, and even between the latest Liverpool poll and Lucid Talk.

There are a few possible explanations for this variation.

Firstly, polling in post-conflict and deeply divided societies is challenging. People can be reluctant to express an opinion to strangers. Polls throughout the conflict under-represented SF and over-represented Alliance, for example. It is reasonable to assume that many respondents assumed that replying you ‘did not know’ or that you vote Alliance, or supporting middle ground policy options, were the safest responses.

Traditionally face-to-face, in-home, polling was seen as the ‘gold standard’ in attitude surveys and online surveys based on demographically representative panels were treated with more suspicion as people volunteered to be on the original database panel. However, people also choose whether to take part in a face-to-face survey and the debate on polling accuracy is changing, with many high-profile, reputable international polling companies now using representative online panels. In post-conflict societies, online methodologies seem to get fewer people refusing to reply to a question or answering that they ‘do not know’.

The Lucid Talk NI poll series uses an online methodology – increasingly common in polling internationally. It is not pure self-selection like a Twitter poll. People volunteer to be on a panel for a polling company, who will usually do commercial product surveys as well as political polls. Many companies offer discount vouchers to encourage people to register. The company will then send a survey to a selection of all those who have registered, which is representative of the population in terms of age, gender, class, location etc. The Managing Director of Lucid Talk has argued that this methodology produces a good representative sample, and can make larger sample sizes affordable. Almost 3,000 replies were analysed in their 2021 Sunday Times poll, compared to a sample size of 1,000 in most other surveys. Their approach also greatly reduces the number of people who answer “do not know” in response to sensitive questions. Lucid Talk argue that when you get very large percentages – into the 20%s – saying ‘do not know’, then it is not statistically reliable to simply assume they would either not vote or break down pro-rata, which is the common practice in polling when the ‘do not know’ category is much smaller.

The 2019 Liverpool poll was face-to-face, but their 2021 and 2022 polls were online. Yet while all three got similar results for those supporting a united Ireland, the level of support (29% to 33%) was much lower that Lucid Talk and Ashcroft polls (41% to 46%). The three Liverpool polls also produced quite different results on the pro-UK, versus don’t know. The percentage support against Irish unity in 2022 was very similar to Lucid Talk’s consistent results, but was lower than Liverpool had found in 2019 and 2021. What might explain these different outcomes?

Greater data is available on the 2019 Liverpool poll. Non-voters make up about one third of the sample. The survey asked people what the “long-term policy” for NI should be – and did not ask people if they would vote or how they would vote. It shows a higher percentage support for the UK (53%) and higher percentage ‘don’t knows’ / refused to answer (18%) than other polls.

Liverpool also included replies from the entire sample without asking people if they would actually vote. Turnout will certainly be high in a unity referendum, but it will not be 100%, and therefore what we need to survey is the opinions of likely voters in such a referendum and we need to ask them specifically how they would vote.

Less data has been published by Liverpool on the 2021 and 2022 surveys, as the polling company used, is not a member of the British professional association for polling companies, which require such publication. However, it seems that the sample has been purposively built around self-definition with approximately equal numbers of nationalists, unionists and ‘other / neither’. Self-definition has not been a stable category in other surveys and has changed very significantly from year to year with ‘others’ growing and then falling, in a manner which does not seem likely to reflect real social change in such short periods. If we look at voting behaviour, 15.5% of the public voted for parties without an explicit position on Irish unity versus the status quo in 2022, so creating a survey sample based on one third of ‘neither’, may under-represent supporters of a united Ireland.

The precise question wording is also important. The highest percentage support for a united Ireland is found when the question is a simple one of “how would you vote?”. If the question is framed more generally as to what the best policy would be (Liverpool 2019), or many options are offered, the support for unity seems to drop off.

Research in Scotland also suggests that the wording on the ballot paper / survey influences behaviour. There was a very extensive debate, managed by an independent electoral commission before the wording in Scotland was agreed by all parties – eventually they opted for the wording ‘should Scotland be an independent country?’ This wording was seen as more neutral by experts, following test surveys and focus group research, than alternative suggestions which sought to frame the question as “leave” the UK versus “stay” in the UK.[7]

Follow-up research in Scotland, in the aftermath of Brexit suggests that the words “leave” and “remain” have now become very loaded and using those words in a survey, reduces support for Scottish independence, compared to polls with more neutral wording. Almost all of the survey questions in Northern Ireland have framed the question as ‘remain in the UK’ versus ‘leave’ (Lucid Talk/BBC; Liverpool 2019), or ‘stay’ in the UK v ‘leave’ (Liverpool 2021). The Liverpool 2022 survey with a more neutral wording, and with a straightforward phrase “I would vote for a united Ireland”. However it still used the agree / disagree style of question, rather than simply asking how the respondent would vote. This formula however produced their lowest level of opposition to a united Ireland, compared to Liverpool’s other surveys where other question wordings were used. On the other hand, while the change to a more neutral wording in Lucid talk/ Sunday Times poll, also showed lower support for the status quo, than the question wording ‘remain v. leave’, and more don’t knows, it was only a marginal difference compared to other Lucid Talk polls.

Given the relatively high proportion of people in some polls who reply that they do not know how they would vote in a referendum, the political context of the period when fieldwork is carried out also seems to have an impact. When it appeared that the EU and UK would not reach agreement on a withdrawal agreement and that therefore a hard border on the island of Ireland, was very likely, polling in 2018 and 2019, produced high response rates in surveys in support of voting for a united Ireland.[8] In contrast, a 2021 Liverpool University poll, held after the EU put forward new proposals on the Protocol, which looked as though they might resolve the EU-UK dispute, asked people “If there was a border poll tomorrow, would you vote for Northern Ireland to stay as part of the United Kingdom or for a United Ireland?” This poll showed marginally higher than average support for staying in the UK and lower figures for don’t know and for ‘a United Ireland’.[9]

The Brexit / Protocol debates may also offer some sort of proxy insight into potential attitudes to a united Ireland, as a means to re-join the EU. Broken down by party support, an EU-UK deal on Northern Ireland with checks in the Irish Sea was supported by 98% of supporters of the two major Irish nationalist parties, Sinn Féin and the SDLP, 89% of Alliance Party voters, 86% of Green Party voters, but only 27% of UUP voters, and 5% of DUP voters.

By self-defined community membership, this represented approximately 93% of self-defined Irish nationalists, 20% of self-defined unionists, and 71% of those who do not self-define as nationalist or unionist.[10] The centre ground on EU/Protocol issues was much closer to the nationalist position than the unionist view and as discussed above, during periods where a ‘no deal’ outcome in EU-UK negotiations seemed likely, support for Irish unity among the centre ground increased.

Polls almost all ask people how they would vote if a poll was held ‘tomorrow’, as polling companies argue that people do not know what they will think in the future. However, in the case of a united Ireland, while the status quo is known to people, what a future united Ireland might look like is not known at this time, and as a consequence only a small minority want a referendum to be held ‘now’, while clear majorities north and south favour holding a referendum in a 5-to-10 year timeframe – after appropriate research and debate. The impact of greater information and detail will only be seen in time.

What is clear is that more sophisticated polling is needed, both north and south. The existing polls in the Republic of Ireland all suggest that a referendum would be carried by a large majority, but as the Irish parliament will have to take a lead on defining both the process of public debate and the proposed nature of a united Ireland – polling on the detail of specific proposals, ranging from the health system, to pensions and potential devolution, after they have been publicly debated, would be useful. In NI, there is no clear majority for a united Ireland at this time, but opinion is more finely balanced than ever before and there is a significant bloc of voters who will only decide after more detail is available to them. At present, the majority of credible polls of likely voters, show the percentage support for remaining in the UK in the mid-to-high 40s, and therefore the currently undecided will determine the final outcome.

John Doyle, Director, Dublin City University Institute for International Conflict Resolution