General Election 2017 projection and prediction, Election Day update

By Jeremy Blackwell

I started writing this article in totally different state of mind from the one I’m in right now. That was on Saturday, eagerly waiting for the latest polls and keeping track of all the campaign events in Scotland. Then the London Bridge attack happened. Once again terrorists sent the message that they hate our democracy and want to destroy it, especially by interfering in our democratic election process. And they hit London, a city I love despite all its faults.

Fear and hate will lead us nowhere except to a darker future. Today say no to fear and hate and get out to vote massively.

So I will start this last article of my General Election 2017 series with a picture of togetherness, hope and joy. Nicola Sturgeon hugging a wee lad who gave her a homemade card while she was campaigning in Ayr on Saturday.

As the last article, this one has to be the riskiest as we will know only a few hours from now if I was spot on, wildly off or just slightly off. I venture ‘slightly off’. I owe a lot of thanks to Jamie Smith for hosting my articles on his blog. Also lots of thanks to all those who fed me with information and all those who follow me on Twitter.

Of course next week I will perform a post-mortem. I will try to sort out what was right and what was wrong in the polls. And also what was right and what was wrong in my model itself (and the others too).

But first let’s see what scenarios we have right now for this election in Scotland.

Scenario #1: Polls are right

By ‘polls’ I mean here only the few full Scottish polls we had this year. In 2015 Scottish polls were pretty accurate and missed individual parties’ results by at most a share of a percent. So if they perform just as well this year, here are the vote shares we get, based on all full Scottish polls since mid-April. And the range of seats each party would get on such results.

This scenario would be quite good for the SNP, but other parties would not see all of their expectations fulfilled. Except if tactical voting is stronger than expected, which might happen in any scenario anyway.

Scenario #2: No Conservative surge

In this scenario, Conservatives fail to bag the extra votes needed to gain a significant number of seats. Tactical Unionist voting is kept to a minimum, most Labour voters remain faithful to their party and (most importantly) the swing from the SNP to the Conservatives in key regions remains low. That way we get a GE result quite close to the Scottish Parliament 2016 elections. This scenario leaves only seven seats tied or marginal.

This would be the dream scenario for the SNP. As James Kelly reminded us, what matters most in first-past-the-post elections is how far ahead the first party is. And this scenario maximizes the SNP’s lead over the Conservatives. Losses could be kept to a minimum, possibly as low as one or two in Borders and one or two in North East. But Labour still doing well makes gaining Edinburgh South less likely. Sorry for that, Jim.

Scenario #3: Maximum tactical voting

This is the mirror image of #2 and the nightmare scenario for the SNP. It would have the SNP roughly on current polling average. But on top of it, this includes a noticeable swing from the SNP to the Conservatives in key regions. If this is combined with full-blown tactical voting from Labour voters and some LibDems, it spells doom for many SNP MPs. Here we have nine seats tied or marginal, though obviously not the same ones as above. The swingometer has moved deeper into SNP-leaning territory and some unexpected losses may happen like East Dunbartonshire, Stirling, North East Fife or Ochil and South Perthshire.

Please note that this ‘doomsday scenario’ still has SNP winning 70-75% of Scottish seats. I know many governing parties in Europe who would welcome that kind of ‘defeat’. Anyway, after a series of major Conservative trainwreck appearances on various media during the campaign, this is obviously the unlikeliest of all.

Done with scenarios now JB, why not tell us how the actual polling data evolved?

There’s a little bit of my three scenarios is this week’s polls: not too good for the SNP, no real Conservative surge, tactical voting mostly (and surprisngly) favouring Labour with the ‘Corbyn surge’ still alive.

These results are nevertheless still better than could be feared for the SNP who maintain a 14% lead over the Conservatives. And even if Labour are doing better than expected (and very close to their 2015 result), the SNP’s lead in SNP-Lab competitive seats is such that no Labour gains are to be expected.

And now JB, why don’t you tell us what we can actually expect tonight?

The slight changes in polling average have some marginal impact on the seat ratings. Tied seats are now East Dunbartonshire, Edinburgh South West, Perth and North Perthshire, Stirling.

And the usual seat-by-seat projections (compared with ScotlandVotes and ElectoralCalculus) and my updated prediction.

I think the SNP will finally do well today, better than some pundits predict. But some symbolic seats will be lost. I still think local factors will see the SNP hold Perth and North Perthshire. But I am not too optimistic about the Edinburgh seats, contrary to my first predictions about them. Organized tactical voting from Unionist parties was tested here last year for the Scottish Parliament elections and it worked. I have a hunch a similar scheme is at work now and will produce fairly similar results: Labour hoding South wille the SNP lose West to LibDems and South West to Conservatives.

But in fact none of the psephologists knows exactky what will happen tonight, even if we like to pretend we do. Always believe the old saying ‘Timeo psephologistos et dona ferentes’. We are just as clueless as our readers are. Perhaps crystal ball and tea leaves would help. Or sacrificing a black goat. In the end, with all we can deduce from statistics and make a great show of it, the best we can come up with is the proverbial best educated guess. Here is what various predictor sites or individual predictors had in store this morning:

And here’s my ‘exit poll’, some hours ahead of the actual one. The best prediction I can offer with all available data and the obvious uncertainty that is part of every election. And that it matches the average of all predictors is pure coincidence. You know how I work so you can take my word for that.

So tonight on 10pm we will know. Don’t forget 48 seats is 80% of the Scottish representation in the House of Commons. So still a resounding victory and a mandate for the SNP, whatever you might hear elesewhere.

And until then…

Ach ‘s math dhomh bhith seo an drasd
A cur failt air a’ bhlas
‘San tir a tha cho you’re dhomh an diugh
Is a bha I nuair bha mi ‘nam phaisd

It is not in the stars to hold our destiny but in ourselves
Our doubts are traitors and make us lose the good we oft might win by fearing to attempt
Boldness be my friend

Saor Alba gu bràth

Jeremy Blackwell, 8 June 2017

General Election 2017 projection and prediction, Glorious 1st of June update

The Glorious First of June by Philip James de Loutherbourg (1795)
The Glorious First of June by Philip James de Loutherbourg (1795)

What’s new in the polls?

So far I never mentioned UK-wide polls as this is a Scottish blog. But this time I feel I have to because something unexpected is happening Doon Sooth. The UK-wide trends show it. Not that it means Jeremy Corbyn will be the next PM, but it might well end up with Corbyn’s Labour doing better than Blair’s New Labour in 2005. On a strongly left-wing manifesto. And that would be good news.

I’m fully aware that it won’t make any difference for Scotland, and won’t make any for England and Wales either as Tories are still hugely favoured to win this election. Labour gaining ground is a sign that the UK might be heading back to sanity after years of delusion. And there is still the possibility that the Conservative lead will shrink even further; and that the last week will see the UK enter hung Parliament territory.

*

And now back to ‘too wee, too poor, too daft’ Alba.

As weeks go, the last seven days were kind of a ‘septimana horribila’ for the SNP. With the manifesto launch postponed by a week because of the Manchester bombing, the national campaign was pretty much in limbo and failed to get the proper momentum at a crucial moment. The local campaigns went ahead regardless, are working hard, and seem to be doing well especially in battleground seats. And then a new full Scottish poll from SurveyMonkey (published by the Scottish Sun) found the party at its lowest in three years, ahead of the Conservatives by only 10%.

Facing such an outlier I had to give it some thought before deciding what I would make of it. I finally decided to ignore it. Not because it’s bad for the SNP but because the underlying methodology is unreliable and not abiding by British Polling Council (BPC) rules. For the record, well-established aggregator sites like UK Polling Report ignore SurveyMonkey results entirely. Martin Baxter at Electoral Calculus confirmed to me he won’t include it either as he only uses polls from BPC members.

So here are the updated trends and rolling average:

*

*

The most intriguing part of this week’s rolling average is that the SNP, Conservatives and Liberal Democrats all lost some votes. And all these votes have shifted to Labour, who are up 2% from last week. I find this surprising as it happens just after the uproar caused by Labour-Conservative coalitions in several Scottish Councils, most in defiance of Scottish Labour’s National Executive Committee decisions.

I don’t quite know what to make of it because it could mean one of two things: either voters approve the ‘SNP out’ coalitions (which I would find deeply disturbing because I had expected these situations to have the exact opposite effect); or it is just a by-product of the real Labour surge seen in the most recent UK-wide polls (and then it might change during the last week depending on which direction the UK polls go).

I also compared this week’s results with the rolling average a month ago when I started this series. The SNP (-1.2%), Conservatives (-0.6%), Liberal Democrats (-1%), Greens (-0.4%), and UKIP (-0.9%) are all down. Part of it is surely due to voters now factoring in that Greens and UKIP don’t field candidates in every constituency and reassessing their vote accordingly. But the swing away from the SNP, Conservatives and LibDems benefits Labour, now up 4.2% from a month ago. I find these results to be totally counter-intuitive; and fully expect the last week to have its fair share of surprises too.

And what does that change in the projection and prediction?

As can be expected, this week’s polling average projects into a weaker SNP result. It also makes the outcome more uncertain in a number of battleground seats (twelve now qualifying as ‘Tied’ or ‘Marginal’).

*

*

And finally my updated projection compared with two other models (Electoral Calculus and ScotlandVotes) and my updated prediction. This time I also identified the few seats who qualify as ‘Tied’ on Electoral Calculus’ seat-by-seat analysis.

*

*

I have altered my prediction to 50 SNP seats, the lowest so far. This week’s polling improved the Conservatives’ position in several marginal seats and made a number of SNP holds more unlikely. Coincidentally (or not) 50 seats is also what YouGov predict using their own model and voting intentions from a 50,000 UK-wide panel.

Seats to watch

To conclude my ‘Seats To Watch’ series, let’s have a look at two of the most important seats: Moray, Perth and North Perthshire. Both are held by SNP ‘superstars’ (Angus Robertson and Pete Wishart). Both are part of the ‘historic five’ seats that the SNP has held continuously for the last 20 years in various incarnations through boundary changes. Both are squarely in the danger zone on the current polling average. So will these two seats provide the SNP’s ‘Portillo moments’ (or ‘Balls moments’ if you want a more recent reference) on election night? Possibly, or possibly not.

It should be noted that the two sitting SNP MPs held their seats in recent years with a relatively low share of the vote (40% or below) and thanks to a fractured opposition. Both seats are no longer safe as they turned into the now classic SNP-Conservative one-on-one, with a much stronger Conservative vote than before endangering the SNP. And both are also prime targets in an Unionist ‘coordination scheme‘ revealed by STV.

In both cases I will provide some statistics on recent elections. These will clarify why my perspective on Moray has changed since my first article. Trends in both constituencies clearly show why the SNP should be worried and pay special attention to both constituencies. Basic statistics also show how the SNP vote (and to a lesser extent the Conservative vote) evolved on strikingly similar patterns in both seats. These were once SNP heartland. This year they might turn into Conservative base camp. Or not.

Moray

The Moray House of Commons (HofC) constituency covers the same area as the Moray Council. It is also the area covered by the Moray Scottish Parliament (SP) constituency except for one-and-a-half wards that are part of Banffshire and Buchan Coast. So here comparisons between the HofC, SP and Council votes seem relevant even if different elections follow different patterns. The widely similar trends in recent years are what matters.

Moray (and its almost predecessor seat Moray and Nairn) was a Conservative stronghold from 1923 to 1987, except between 1974 and 1979 when Winnie Ewing held Moray and Nairn for the SNP.

It is also worth remembering that (apart from the 2015 landslide) the SNP never did better than 44.5% in the current Moray (Margaret Ewing in 1992). Sitting MP Angus Robertson (again apart from 2015) was elected three times on less than 40%. So there are clear hints that 2015 was an outlier and the SNP is not as safe here as you might think at first glance.

Martin Baxter at Electoral Calculus gives the Conservatives a 58% chance of winning here. His demographic statistics also point to a more favourable profile than average for the Conservatives (older, larger proportion of ‘UK born’, equal split on EU membership). His projection of vote shares and mine are pretty close, with a Conservative lead of about 4% to 6%. Not coincidentally in my opinion, the Conservative lead over the SNP in last month’s Council elections was within that range.

Finally the most significant statistical result is that, unlike other parts of Scotland, there has been a real Conservative surge here recently. Conservatives nearly doubling their vote share in 2016 and more than doubling it in 2017 is obviously a bad sign for the SNP. The Unionist vote visibly coalescing around the Conservatives here also makes this seat a golden opportunity for a gain.

*

The SNP candidate is Angus Robertson, who obviously needs no elaborate introduction. As the party’s Group Leader in Westminster since 2007 and Depute Leader since last October, he has been one of the most visible SNP figures recently, second only to Nicola Sturgeon. Robertson was the SNP panelist in yesterday’s Election Debate instead of Sturgeon for genuine practical reasons. But the extra media exposure it offered him was obviously welcome as he faces a tough re-election.

Labour and the Liberal Democrats will obviously play third and fourth fiddle here, so I will not discuss the merits of their candidates, especially as both are complete unknowns to me.

The Conservative candidate is Doug Ross (this one, not that one), assistant football referee and list MSP for Highlands and Islands. Also the Conservative spokesperson for Justice in Scottish Parliament. Ross had his fifteen minutes of fame last September when he was away for a whole week on a ‘referee trip’ to Switzerland. He missed a Justice Committee meeting and also a key vote on the Council tax that the SNP government won while they should have lost it (63-63 with the Presiding Officer casting the tie-breaking vote in favour of the government as convention requires). At the time Kezia Dugdale’s vote ‘failing to register’ made headlines. But it wouldn’t have mattered if Ross had been present; then the government would have lost 64-63.

Though Ross supported ‘Remain’ during the EU referendum campaign, the newly-found ‘best Brexit deal’ talking points might go down well with voters in a constituency which supported ‘Remain’ by only 122 votes. He will of course campaign on ‘No to independence’ like all Scottish Conservative candidates. And probably also attack Robertson on the SNP’s record on devolved matters, just as Ruth Davidson did during the BBC’s abysmal Scottish leaders’ debate. That this is irrelevant in a General Election campaign probably doesn’t matter as a precedent has been set. And unfortunately it might very well work if Robertson falls for it as Nicola Sturgeon did.

I think the SNP will do a little better here than statistics say but not by much, even if I predict they will lose less votes here than the national average. I also have a hunch that even the incumbency factor will not be enough to overturn the projected Conservative lead. In the end I go with the statistical evidence and the high probability of a major upset, so reluctantly…

I rate it as a Conservative gain from SNP
Conservatives 47% (+16%), SNP 46% (-4%), Labour 6% (-4%), LibDems 2% (-1%)

Perth and North Perthshire

This constituency has an electoral history pretty similar to Moray’s. It and its ‘near-predecessor’ seats were held by the Conservatives from 1924 to 1997 except for a short Liberal Unionist interlude in 1935. Its near predecessor Perth and East Perthshire was also held by the SNP between 1974 and 1979, with Douglas Crawford as its MP. Then as North Tayside it was John Swinney’s seat from 1997 to 2001. When Swinney stood down to avoid double-jobbing as MP and MSP, Pete Wishart took over. As in Moray the SNP’s vote share under current boundaries was always below 40%, except in the 2015 landslide. And the 2015 result was pretty similar in both seats.

I based the comparisons with SP and Council elections on the Perthshire North SP constituency on one hand, and the eight wards covered entirely or mostly by the Westminster constituency on the other. Perthshire North does not exactly overlap Perth and North Perthshire as it does not include the whole of the city of Perth, but the boundaries are close enough to identify a trend.

I have to disagree with Martin Baxter on this one. His projection includes an ‘Other’ candidate while in fact there is none here. The 2015 Independent candidate Xander McDade (an opposition independent councillor on the current Conservative-led Perth and Kinross council) appears to be quite left-leaning on a number of major national policy issues, so I expect his absence this year to benefit the SNP.

*

Just like Angus Robertson, sitting SNP MP Pete Wishart does not need an elaborate introduction. As the keyboardist for Big Country and Runrig he was in the public’s eye long before his first election. Since then he has done his best to maintain a high profile and high visibility, even (or perhaps especially) when it implies taking provocative and controversial positions on some issues. His Twitter account draws a lot of attention since he was awarded ‘Parliamentary Tweeter of the Year’ in 2014.

The outcome here will probably be decided by the differential turnout between the rural wards (leaning Conservative) and Perth itself (leaning SNP). So I think the SNP’s decision to hold their rescheduled manifesto launch in Perth (instead of the original venue in Edinburgh) was also clearly devised to bring the constituency and Wishart into the spotlight at a key moment.

The Conservative candidate is Ian Duncan MEP. Duncan campaigned in 2014 on a platform of EU reform and was an early proponent of an EU membership referendum. But his blog posts both before and after the referendum were ambiguous, carefully worded to not explicitly reveal which side he was on. This might not go down too well here as Perth and Kinross voted 61% ‘Remain’, just 1% below the national average. Duncan is surely aware the ‘best Brexit deal’ talking points will mostly fall on deaf ears here and he was careful not to mention it when he announced his candidacy. Duncan might be an accomplished MEP but he does not strike me as much of a campaigner. Even his blog post about the second independence referendum sounded long-winded and bland. And did not even include the magic words ‘No To Independence’.

Labour’s David Roemmele and Liberal Democrat Peter Barrett will compete for a distant third place. Both of them can’t expect more than watching the SNP-Con one-on-one from the sidelines.

Statistically this seat is a tie, with the Conservatives slightly favoured. But even the Daily Record is very careful not to speculate on the outcome. Unlike Moray, I believe this one is close enough for the incumbency factor to work and get Wishart re-elected.

I rate it as a SNP hold
SNP 46% (-4%), Conservatives 45% (+12%), Labour 5% (-3%), LibDems 2% (-2%)

And this concludes my ‘Seats to watch’ discussions. Next update will be on Election Day so I will only discuss the broader picture: what will this GE deliver for Scotland? And, to stay on the cautious side, I will give one straight answer but also propose some possible alternative scenarios.

As we all know psephology is not an exact science, the accuracy of polls is in doubt, and even seemingly minor events can switch voters in sufficient numbers to make the outcome even more uncertain. I’ll deal with that next week.

Saor Alba gu bràth

Jeremy Blackwell, 1 June 2017

Jeremy BlackwellJeremy Blackwell is an analyst and statistician living and working in Edinburgh. You can follow him on Twitter at @WeAreThe59.

General Election 2017 projection and prediction, 25 May update

On Monday night Manchester was hit by a cowardly and brutal terrorist attack. It targeted young people who were together to enjoy themselves and have fun, just as previous terrorist attacks in Paris and Berlin did. It is yet another sign that those who want to destroy our democracy will stop at nothing.

We won’t back down. We will stand up united for freedom, life and joy.

Some have already tried to make political gain out of this terrorist act. Shame on them.

My thoughts with Manchester and all who lost loved ones or were injured in the attack.

And now back to business as usual, with a heavy heart.

On the reliability of polls

We already know that polls have a reputation for unreliability. The 2015 General Election is the best known example. The 2011 and 2016 Scottish Parliament elections also prove it. And hard evidence demonstrates that polls can often be off by much more than the margin of error. The methodology itself is sometimes flawed, especially the weighting of voting intentions relative to previous votes or other mysterious criteria. A pollster following British Polling Council guidelines doesn’t mean the results should be taken at face value. They all follow the guidelines and have all been wrong at some point in the past.

So let’s see what the projection of seats becomes on three possible scenarios:

  • Scenario A is based on last week’s polling average. SNP 44%, Conservatives 29%, Labour 16%, LibDems 6%.
  • Scenario B is SNP overestimated by 4%. Vote supposed to be SNP 40%, Conservatives 31%, Labour 17%, LibDems 7%.
  • Scenario C is SNP underestimated by 4%. Vote supposed to be SNP 48%, Conservatives 26%, Labour 15%, LibDems 6%.
A B C
SNP 51 41 58
Labour 0 1 0
Liberal Democrats 1 3 0
Conservatives 7 14 1

Note that even the worst case scenario would still give the SNP as many seats as Scottish Labour held from 2005 to 2015. Remember nobody then denied Labour was the dominant party and had a mandate to speak for Scotland. Whether Labour did it or not is a different story and I won’t discuss that right now.

Continue reading General Election 2017 projection and prediction, 25 May update

General Election 2017 projection and prediction, 18 May update

Teaser: this week’s update will definitely be better for the SNP than the previous ones.

Scottish Greens standing down: a game-changer?

Technically the Scottish Green Party are not standing down. They just will be fielding only three candidates, compared to thirty-two in 2015. Whatever the official explanations and carefully crafted talking points, the true political intention is clear: get out of the way in marginal constituencies and hope it will benefit the SNP. And then get something in return. But that’s me going cynical.

In 2015 Scottish Greens received 39,205 votes or 1.3% of the national vote (sixth behind UKIP). In the 32 constituencies where they stood their average vote was 2.6%. Lowest in Rutherglen and Hamilton West (0.5%) and Glasgow East (0.9%). Highest in Glasgow North (6.2%) and Edinburgh East (6.0%). They also lost their deposit in 29 constituencies and that’s certainly something they chose to avoid this year. But that’s me going cynical again.

Recent polls credited Scottish Greens with 2 to 6% of the vote, with pollsters assuming they would stand in all constituencies. On average they’re credited with almost 4%, double their 2015 vote share. Allowing for margin of error and uncertainty about how Green supporters will vote now, it probably leaves about 2 to 3% of the vote literally up for grabs for the SNP nationally.

At the time of the announcement my model identified 14 Scottish seats as marginals. Seven of them with the Conservatives favoured; five with the SNP and two the Liberal Democrats. The Scottish Green Party fielded candidates in nine of these constituencies in 2015 and will be fielding only one this year (Edinburgh North and Leith). So the SNP have a real opportunity either to switch some seats back to their column or to hold them on better margins than projected.

Will the Greens’ decision be the turning point of this campaign? Probably yes if the SNP play it smart from now on. Conservatives have already started denouncing a ‘Yes alliance’ which in fact doesn’t exist; because they know some seats might be out of their reach now. And ultimately it’s up to the SNP to make the best out of the better hand they have unexpectedly been dealt.

UKIP are also fielding fewer candidates this year than in 2015, and this should bring a few more votes to the Conservatives. I expect the impact to be less visible, because polls show UKIP in Scotland down from 2015 as in the rest of the UK. Their vote share was never really significant in Scotland and some of their 2015 voters seem to have shifted to the Conservatives already.

How did the polling data evolve?

Unfortunately we don’t have any new full Scottish poll again this week, but a large number of new Scottish subsamples from UK-wide polls, used with all the usual caveats. Now that the campaign has started, polls show a slight swing towards the SNP and against the Conservatives. This is visible in the trendlines and has started to make rolling average better for the SNP.

*

*

James Kelly’s ‘Poll of polls’ on Scot Goes Pop is even more favourable to the SNP. But he doesn’t use the same set of subsamples. I keep some older ones in my set that James has already discarded. They were distinctly less good for the SNP than the more recent ones. And, as I said before, when I have a choice I always choose the scenario that is least favourable for the SNP. Previous elections taught us that a cautious approach is often the wisest.

How do all these factors change the projection and prediction?

I have adapted my model to the new context. In an additional final step I chose to reallocate 60% of the potential Green vote to the SNP and 90% of the potential UKIP vote to the Conservatives in all constituencies where Greens or UKIP stood in 2015 and don’t stand this time. This is obviously an approximation and the real transfers will be more complex. But I had to make an assumption and this one seemed as good as any other.

The cumulative effect of the slight change in polling average and the reallocation of Green votes is to shift three marginal seats from the Conservative to the SNP column (Edinburgh South, Perth and North Perthshire, and Stirling). Aberdeen South and Moray remain in the Conservative column because of the reallocation of the 2015 UKIP vote but in both cases the Conservative lead is much reduced.

Martin Baxter at Electoral Calculus has also adapted his model to account for the actual Green and UKIP candidacies. He delivers the exact same number of seats as my model when fed with the same polling average data. But there are still a few differences in the allocation of seats to each party. ScotlandVotes is less favourable for the SNP but close nevertheless. I have a hunch their model has not been adapted in the same way as the others (yet) and still reflects what would happen with a much larger number of Green candidates.

Below are my updated projections (seats by SLLM—Safe, Likely, Lean, Marginal—rating, and seat by seat winner’s margin) and the full seat-by-seat projected results. Again I’m taking a big risk here as some of the detailed results might be way off, but it’s worth it.

*

*

*

And finally the summary of my updated projection (compared with what Electoral Calculus and ScotlandVotes would project) and my updated prediction.

*

*

In my prediction I switched three seats from the Conservative to the SNP column: Aberdeen South, East Renfrewshire, and Moray. I already explained my reasons for these three seats in previous posts, so I will not elaborate further.

Seats to watch

Dumfries and Galloway

Dumfries and Galloway is one of the top Conservative targets in this election. This seat and its (almost) predecessor Galloway and Upper Nithsdale have been held by all three major parties (SNP, Conservatives, and Labour) at some point in the past 24 years. So it can really be called a battleground.

The decline of Labour in recent years has benefited both the SNP and the Conservatives here. But recent trends point to Conservatives having the upper hand, with the SNP coming second both in 2016 and 2017. Even if Scottish Parliament and Council elections are not predictors of the General Election, they nevertheless give some hints on which direction the electorate is moving.

The SNP candidate is sitting MP Richard Arkless. He gained the seat from Labour in 2015 on a 25% swing and with a majority of 6,500. As a hint of things to come the Conservatives finished second and Labour was already relegated to third place. I didn’t find much to say about Arkless except what you find in his profile on the SNP website and his Twitter account. He comes out as a typical low-profile backbencher, deeply involved in his job as MP and certainly respected and well-liked by his constituents. But is that enough to win in a competitive election that has already turned nasty on the national level?

The Conservatives are fielding Alister Jack (a local farmer and businessman who will campaign on the usual ‘no to independence’ and ‘best Brexit deal’) and Labour Daniel Goodare (a local A&E doctor).

Goodare doesn’t stand a chance as Labour support has gone downhill here just as much as anywhere else in Scotland. Jack will try to build on Conservative success in the area in 2016 and 2017. Apart from the classic Conservative campaign themes, he will also try to rally the farmers’ vote. It already paid off in other rural areas of Scotland in 2016, regardless of the validity of the arguments, and is probably the key to a Conservative gain here.

I rate it as a Conservative gain from SNP
Conservatives 41% (+11%), SNP 39% (-2%), Labour 18% (-7%), LibDems 2% (-)

Edinburgh South

This one will obviously be the most watched of all Scottish constituencies in this General Election, being the last Labour seat in Scotland and an unexpected Labour hold in 2015. Over the last 40 years Edinburgh South has been first a weak Conservative seat and then a weak Labour seat. It is also one of the trickiest to predict as Edinburgh politics have a life of their own and have recently been full of upsets. So whoever wins on 8 June will probably do so by just a few hundred votes.

In 2015 the SNP clearly did not choose the best candidate here. This is obviously not the case this year. SNP candidate Jim Eadie was MSP for Edinburgh Southern from 2011 to 2016. During his term he played an important part in the Scottish Parliament’s work; first as Parliamentary Liaison Officer for First Minister Nicola Sturgeon and then as Convener of the Infrastructure and Capital Investment Committee. Eadie was unseated in 2016 by Labour’s Daniel Johnson though he gained 2,500 votes and 3% of the vote compared to 2011. His defeat was at the time widely attributed to massive Unionist tactical voting, which seems to have become quite common in Edinburgh recently. His nomination for the General Election has been interpreted as a clear sign the SNP intend to put up a real fight for this seat. The SNP can certainly count on Eadie’s high name recognition, personal status and strong campaigning skills to bring in the extra votes needed to gain the seat.

After a seemingly chaotic selection process, the Conservatives nominated Stephanie Smith, who has since been elected Councillor for the Liberton/Gilmerton ward. Previously she stood for MSP in Almond Valley in 2016 and finished a distant third. Other than this I admit I know nothing about her. I find it surprising that the Conservatives did not choose a more high-profile candidate such as Miles Briggs, who stood here in 2015. But this year Briggs is standing in the neighbouring Edinburgh South West, which I will discuss in my next update. The Conservatives probably expect that Smith will automatically benefit from their surge in Edinburgh over the last two years but they should not take it for granted. She could also benefit from UKIP fielding no candidate this year (they received 1.2% here in 2015).

And finally Ian Murray, the sitting Labour MP. Murray was first elected in 2010, taking over from retiring Labour MP Nigel Griffiths. Common wisdom at the time was that he would lose the seat (then a Labour-LibDems marginal) to the Liberal Democrats. But he won by 316 votes. Again in 2015 Edinburgh South was generally considered highly likely to switch to the SNP. But Murray held the seat by 2,637 votes, with some help from the SNP themselves. There’s definitely something of the ‘come-back kid’ in him.

Murray is a member of the Blairite (or ‘progressive center-ground’ as they call themselves) Progress group and it shows in his voting record. He has been the target of harsh criticism from within Labour after his highly publicised resignation from the Shadow Cabinet and also because of his perceived undermining of Jeremy Corbyn. Murray has won an award as ‘most responsive Scottish MP’, whatever that actually means. But he is certainly not as popular as he claims or would like to be, even within his own party. What was once dubbed ‘The Socialist Republic of Morningside’ may well not be as kind to him as it was two years ago.

This seat is technically a three-way marginal bordering on a three-way tie. Common wisdom is that Conservatives will gain it but I strongly doubt it. The Greens’ decision to stand down in this constituency will probably be less of a game-changer here than in other seats. They received 4.2% of the vote in 2015 and it’s likely to switch to both SNP and Labour.

So I will go out on a limb again and…

I rate it as a SNP gain from Labour
SNP 35% (+1%), Labour 33% (-6%), Conservatives 29% (+12%), LibDems 3% (-1%)

A little extra: the three constituencies where the Scottish Green Party stands…

Edinburgh North and Leith

I expected Greens to stand down in this one too, because it’s a marginal and the only one of the three where a Green candidacy could actually hurt the SNP’s prospects of holding the seat. Greens received 5.4% of the vote here in 2015 and can be expected to reach 8-10% this year under favourable circumstances. Such a result would reduce the SNP margin to probably 2 or 3% down from almost 10% in 2015. I don’t expect Greens to do much better than that. They do well in the eastern part of the constituency but significantly less in its western part; and their results here in the 2017 Council elections were not much different from 2012. But I think the SNP here would still welcome a fair share of tactical voting within the pro-Independence camp.

Falkirk

Greens intend to campaign here on ‘ban fracking’. The SNP have already explained at length why the current moratorium is the best choice; and how an outright ban would backfire as it would be challenged in court and would be overturned. Greens did not stand here in 2015 so there is little basis for comparison; only that they received 4.7% of the list vote for the Central region (which includes Falkirk) in 2016; and 3.6% in Falkirk Council elections this month. I expect their share of the vote in this GE to be fairly close to that.

Glasgow North

Patrick Harvie received 24% of the vote last year in the Glasgow Kelvin Scottish Parliament constituency. But it doesn’t say much about what might happen in this General Election. Greens did well in the Hillhead ward two weeks ago (26%; one councillor elected on first count) but not so well in other areas of the constituency. Greens received 6.2% of the vote here in 2015. So I don’t expect Harvie to do much better than 15% this time. That would have him competing with the Conservatives for third place, not with the SNP for first.

Next week: Edinburgh West, Edinburgh South West

Saor Alba gu bràth.

Jeremy Blackwell, 17 May 2017

Jeremy BlackwellJeremy Blackwell is an analyst and statistician living and working in Edinburgh. You can follow him on Twitter at @WeAreThe59.

General Election 2017 projection and prediction, 10 May update

By Jeremy Blackwell

Now that the Conservatives have won the Council elections in Scotland, and Ruth Davidson has taken Glasgow and won the French Presidential election, I will have to reconsider all my predictions. Just kidding, but in some cases what happened last Thursday offers a different perspective on what might (or might not) happen on 8 June. So let’s go.

How did the polling data evolve?

For the record, I have my doubts about polls. In 2011 they underestimated the SNP and failed to predict the SNP majority in the Scottish Parliament. In 2015 they were spot on and predicted the SNP landslide in Westminster. In 2016 they overestimated the SNP and failed to predict SNP losing their majority in the Scottish Parliament. So what is it going to be this time?

That being said, there have been few new polls in the last week and unfortunately no full Scottish polls. The trend, not unexpectedly, is again better for the Conservatives than for the SNP:

*

The rolling average also moved in favour of the Conservatives and against the Liberal Democrats. This obviously has an impact on the projections, and has also changed my prediction for a few seats.

*

Continue reading General Election 2017 projection and prediction, 10 May update

General Election 2017 projection and prediction, 4 May update

By Jeremy Blackwell

I will have an update on my projection and prediction every week on Thursday until Election Day. I will keep it short and mostly point to the changes since the last update. At the end of each update I will also discuss some specific points like ‘seats to watch’.

How did the polling data evolve?

The complete crosstabs for some of the weekend polls were late showing up. And there were two Tuesday polls to add. But the main trends have not changed significantly since Monday. For better visibility I have hidden the small dots that represent individual polls. The large dots on the left side are the actual 2015 results.

*

The rolling average barely changed with the new polls in:

*

Continue reading General Election 2017 projection and prediction, 4 May update

General Election 2017: which way will Scotland go?

By Jeremy Blackwell

You’re lucky this time you won’t have to go through the ‘A few words about me’ and the ‘Hits and misses’ parts. You already got them. So let’s go directly to the heart of the matter.

How does my model work?

Like all election projection models, my model works mostly on ‘Uniform National Swing’ (UNS). With UNS you have all seats moving in exactly the same direction as the national polling average—which never happens in real life.

So I have a second algorithm on top of the first one which I call ‘relative swing’. It factors in not just the usual UNS, but the way parties’ vote shares change relative to each other.

For example, you might start with the SNP on 50%, Labour on 24%, the Conservatives on 15%, and the Liberal Democrats on 8%; then later have the SNP on 45%, the Conservatives on 30%, Labour on 15%, and the Liberal Democrats on 5%. With UNS you simply subtract 5% from the SNP vote, 9% from the Labour vote, 3% from the Liberal Democrat vote, and add 15% to the Conservative vote, in every seat. (And get negative votes for some parties sometimes.)

But the truth is that it’s more complex than this. That’s what ‘relative swing’ takes care of, by taking into account the ‘multiplier effect’. With the above numbers you multiply the SNP vote by 0.9, the Labour and Liberal Democrat votes by 0.625, and the Conservative vote by 2.0 in every seat. (And sometimes get a vote total over 100%.)

So my model is a mix of both, and currently tuned on 80% UNS and 20% ‘relative swing’.

The first step as described above will sometimes deliver negative votes in a few cases. And in some seats the vote total doesn’t add up to 100%. The second step is to eliminate all the negative votes. This is done simply by automatically switching from the model parameters to ‘100% relative swing’. This being a multiplier, it guarantees there can’t be any negative results. Third and last step is to deal with seats where the vote total is not 100%. This is also done automatically by recalculating votes proportionally from what the second step delivers.

Is the model foolproof? Of course not. No statistical model is, especially when you consider tactical voting. That’s why we can get wrong projections even with accurate polls (see ‘Hits and misses’ in my previous article). But I’ve compared my results with those you can get using Electoral Calculus or ScotlandVotes and they’re pretty similar. Not identical, but pretty close (more on that later). So I guess their underlying algorithms are pretty similar to mine.

How do I feed the model?

The source data are all the polls that I can find. Ideally I would rely on full Scottish polls (that is polls fielded in Scotland only and with a sample size of over 1,000). There were plenty such polls before the 2015 General Election and they proved accurate, unlike UK-wide polling. In the graph below the small dots are individual polls. The large dots at both ends are the actual 2010 and 2015 results. The trendlines show the evolution of vote shares and how polls correctly predicted the SNP landslide.

*

We are not that lucky this year. The snap election took everyone, including the pollsters, by surprise and there are many fewer full Scottish polls: nine so far and only five in 2017.

*

I therefore also have to rely on Scottish subsamples of UK-wide polls. A subsample size is typically 100 to 150, so they have a much larger margin of error (MOE) than full polls. This is the reason why many people think they should be discarded from any analysis. In principle I agree with this point of view, but this year subsamples are the bulk of the data we have. And there is a simple way to deal with the larger MOE and still get valid projections when interpreting the data. Which I will explain in the next section.

Continue reading General Election 2017: which way will Scotland go?

Predicting Scottish Councils elections: an impossible task?

By Jeremy Blackwell

A few words about me

Jeremy BlackwellJeremy Blackwell, almost 26 now, known on Twitter as @WeAreThe59. My friends usually call me Jez or JB but Jeremy is OK—actually I like it better. Born in France and studied statistics and political science there.

Living in Craiglockhart since late 2015 with my French boyfriend Alban and our beloved dog Bosco. Also member of the SNP, so no ambiguity about where I stand. Started seriously working on polls and projections during the 2015 General Election campaign; loved it and carried on during the 2016 Scottish Parliament campaign. And now giving a try at the 2017 Scottish Council election.

Hits and misses

Psephology is not an exact science. Sometimes polls are wrong. Sometimes your method is wrong. It’s closer to ‘trial and error’ than to rocket science, whatever the pundits might tell you about it.

Like everyone I totally missed General Election (GE) 2015 because UK-wide polls were so off the actual result. But I got Scotland right because Scottish polls were really accurate. Got the 56-1-1-1 part right; only got the exact seats wrong. I thought the Labour seat would be Glasgow North East; not Edinburgh South. I also thought that the Tory seat would be Berwickshire, Roxburgh and Selkirk, not Dumfriesshire, Clydesdale and Tweeddale. Still, I got the Lib Dem seat right. That was the easiest one.

I did better on Scottish Parliament (SP) 2016. Everybody got it wrong because the polls were wildly overestimating the SNP vote. But I was the ‘least wrong’ of all, crediting the SNP with only 66 seats while others went up to 70 or even more.

Now to the heart of the matter

The main question is: can Scottish Council elections be predicted with the same level of accuracy as Westminster or Holyrood elections? And the answer is a resounding no. Standard or sophisticated statistical methods that work for Parliamentary elections mostly fail for Council elections because there are many more variables that interact in sometimes unpredictable ways.

The first factor is that there are many more independent candidates in Council elections than in Parliamentary elections, and they get a much bigger share of the vote:

  • 0.1% in the 2015 General Election in Scotland
  • 0.3% in the 2016 Scottish Parliament election
  • but 11.8% in the 2012 Scottish Council election

The second factor is that Council elections definitely follow a wholly different pattern from other elections, like Westminster (WM) and Holyrood (HR) elections did before 2015 when they finally aligned. Council elections are still very far from aligning on the WM/HR voting patterns. One reason is that we have 59 MPs, 73 constituency MSPs but 1,219 Councillors after the last Boundary Review. So personal factors have a heavier influence on the Council vote, while Parliamentary votes are closer to a divide purely on party lines that can be statistically projected.

The third factor (and probably the most important) is the voting system: STV (Single Transferable Vote). There are many technical explanations of STV to be found (just Google it and make your pick). All of them are technically right but all are wrong on one major point. STV is not and has never been PR (Proportional Representation). Quite the opposite, in fact. STV is the perfect vehicle for tactical voting, and is even worse in many cases than FPTP (First Past The Post). Partly because FPTP is basic and simple while many people find STV extremely complicated and don’t fully grasp its limits and implications, and also how it can be used to maximise tactical voting.

Continue reading Predicting Scottish Councils elections: an impossible task?