'Byelections traditionally favour anti-establishment candidates.' Photograph: Peter Macdiarmid/Getty Images  A few months ago we sat down with Nigel Farage to talk about Ukip's fortunes. The party was riding high in the polls, could look back on an impressive set of results at the 2013 local elections and was dominating Britain's political debate. But aside from all of this, we asked him: what exactly would Ukip need to achieve the goal that has proved so elusive – a seat in Westminster? Farage paused, stared out of the window for a moment and then replied: "Things like the local and European parliament elections are important in showing our strength. But we also need that bit of luck. Like a byelection."  With the resignation of the Conservative MP Patrick Mercer, Farage has once again got lucky. Ukip has been handed yet another opportunity to build its momentum ahead of the 2015 general election, even if Farage quickly ruled himself out of standing for the seat. Byelections are especially important for challengers such as Ukip because they have traditionally favoured anti-establishment candidates and often feature outsized swings against the governing parties. This is why Farage put byelection campaigns at the centre of his post-2010 strategy, learning from the Reform party in Canada and the Liberal Democrats, who identified these campaigns as a crucial step on the road to a wider breakthrough.  But how favourable is Newark in Nottinghamshire for Ukip, and why has Farage decided not to stand? For our book on Ukip and its support we ranked every constituency in terms of its demographic favourability for Farage's party, taking into account the local mix of social groups which Ukip attracts, and those which shun the party. Newark is ranked 248th, which puts it in the top half of favourable seats, but far from ideal demographic territory.  This is not to say Ukip could not do well; the seat includes lots of conservative-leaning, older white voters who are central to Ukip's revolt, and while the demographic context is not ideal, the political context certainly is: the byelection was triggered by the resignation of a scandal-tainted MP, and it is only the second byelection this parliament in a Conservative-held seat, providing a rare opportunity for voters to vent their feelings. It is also likely to take place immediately after a European parliament election that Ukip has a strong chance of winning.  There are, however, important weaknesses for Ukip in this seat. It has a large number of financially secure, middle-class voters who have so far been immune to the party's appeal, and who could function as a firewall for the local Conservatives. Also, Ukip's election record in the area, which Farage increasingly uses as a guide, is not as strong as he would like. When Farage says he decided not to stand because he does not have local connections, he is also talking about his party. Ukip lacks the local roots that it is quickly putting down in other areas. Newark is not like Eastleigh, for example, where Ukip has been painstakingly preparing the ground ahead of the 2015 campaign (and which it will again target in local elections this year). In terms of demographics and local party presence, Newark is nowhere near as favourable as the seats along the east coast that offer Farage and his party their best chance of a successful Westminster campaign.  Farage, then, has calculated that as party leader the opportunity in Newark is not worth the risk. Perhaps his decision was influenced by his experience in Buckingham in 2010, where despite much hype about his prospects in a shire Tory seat of the kind many in the media still associate with his party, he ultimately failed to impress. Or perhaps he has finally grasped Ukip's problems with expectation management; he knows that anything other than a Farage win in Newark would fuel speculation that he and Ukip are simply not up to the task of constituency battles, and are forever destined to be a minor player. A second- or third-placed Farage would jeopardise Ukip's entire 2015 general election campaign before it even began. When Farage finally does select a seat, it will be as Ukip-friendly as possible, and far more so than Newark.  This is not to say that Ukip does not have a chance in Newark, though coming from 3.8% in 2010 and finishing ahead of the Conservatives and Labour is an incredible challenge. Ukip's successes at the European and local elections on 22 May will create an ideal political context, but its lack of local history and the awkward demographic profile of the seat means that pulling off a historic upset will be difficult.  What seems more certain is that Farage will use the Newark campaign to continue with efforts of addressing Ukip's notable weakness among women and young people, and framing the party as a populist alternative. In Farage's absence expect a local, female or young Ukipper, as the party pushes back against the accusation that it is merely a "one-man band". But make no mistake, Ukip has got lucky once again, and it will throw everything into this campaign.
Robert Ford and Matthew Goodwin: Ukip matters: With its leader not standing, the party can frame itself as a populist alternative and not just a one-man band