Exit polls: How they are conducted, their relevance & accuracy

Experts say the trickiest part of exit polls is to convert vote share into seats and this requires statistical training.

Representational Image | PTI

New Delhi: With the final phase of polling for the 17^th Lok Sabha elections ending Sunday, all eyes are now on the much-awaited exit polls.

Even as questions about the accuracy of exit polls have been raised over the years, they continue to be watched keenly across the world.

In India, a number of private agencies and media organisations such as Today’s Chanakya, ABP-Cvoter, NewsX-Neta, Republic-Jan Ki Baat, Republic-CVoter, ABP-CSDS, News18–IPSOS, India Today-Axis, Times Now-CNX and Chintamani conduct the exit polls — each claiming to predict the decision of 900 million voters with maximum accuracy.

However, little is known about how these exit polls are actually conducted. Hours before they are released Sunday, ThePrint explains what exit polls are, the costs involved, how they are conducted, their limitations as well as relevance.

What are exit polls and how are they conducted?

Exit poll is a post-voting poll conducted just after a voter walks out after casting his or her vote. By contrast, some agencies also carry out post-poll surveys, which refer to a poll of voters taken between the period when the last ballot is cast and when the counting begins. Among the most well-known players in the country, only the Centre for the Study of Developing Societies (CSDS) carries out post-poll surveys.

To conduct an exit poll, a random sample size is first selected. The sample size could range anywhere between 20-25,000 voters to 7-8 lakh voters.

While it is not necessary that all 543 constituencies are surveyed in order to accurately predict the results – some agencies could survey as few as half the constituencies – it is important that the sample size is geographically, socially and demographically representative, said Pranav Gupta, a PhD student at University of California at Berkeley.

“A lot of times, people can get exit polls wrong because the sample size was not representative. For example, they only went to urban areas or did not ask Muslim voters or voters of a particular caste, etc.,” Gupta said.

The number of constituencies and the sample size to be surveyed are typically decided on the basis of the budget available with each agency. The budget can run up to several crores of rupees with each interview of voters costing anywhere between Rs 80-100 to Rs 300-400.

The tricky part – converting vote share into seats

Gathering data from across several constituencies and demographic groups is only a part of the exercise. The other, and perhaps greater challenge is to convert the vote share into seats, said Gupta.

“The survey can only give you a sense of vote share…But to translate that into seats is a mathematical and statistical exercise,” he said.

It is this conversion of vote share into seats that sets apart a psephologist from others, including journalists, who may claim to know the “mood of the nation.”

“Seat conversion is hard, but someone with a statistical training could do it,” said Rahul Verma, a fellow at the Centre for Policy Research.

Verma said while most pollsters in India often get direction of the results right, they are way off the mark on vote and seat prediction.

“These days, a lot of agencies claim that they have surveyed every seat in order to claim that they can predict each seat accurately. But that is not how it is necessarily done,” he argued.

As per the conventional practice, a method called the “uniform regional swing” is applied to convert vote share into seats, he said. For example, if in a certain region, the BJP has a vote share of 35 per cent, and the last time it had a vote share of 38 per cent, on the basis of the number of seats it had won with 38 per cent the last time, one makes an estimate of how many seats it would potentially lose with a loss of vote share of three per cent.

But this is no mean task.

Take this example given by Verma in his paper titled ‘Elections, Exit Polls and the Electronic Media.’

“The Congress won 19.3 per cent votes in 2014, but could win only 44 seats while the BJP in 2009 won 116 seats with 18.8 per cent vote share,” as per the paper.

How reliable are exit polls?

Exit polls can often be unreliable.

For example, in 2004 Lok Sabha elections, most exit polls had predicted a victory for the BJP-led National Democratic Alliance (NDA) government. Then in 2009, the exit polls had failed to predict the margin of victory for the United Progressive Alliance (UPA) government correctly. Since then, pollsters have got several state elections wrong.

According to Gupta, the reasons for the inaccuracy of exit polls include voters lying about who they actually voted for, the absence of random and representative sampling, biases in questioning or simply bad fieldwork. While data fudging on part of the agencies or channels in order to reinforce allegiance to a certain party can also be a reason, it remains in the realm of speculation, he argued.

The social and political value of exit polls

Verma also said the inaccuracy of exit polls is a misnomer.

“Often, the expectations we have from exit polls are just too high for any survey,” he said. “In any case, all election-related polls are not carried out for the sole purpose of seat projection and media entertainment.”

While for journalists and the public, the importance of exit polls ends with the brouhaha over accurate seat projection as well as the maddening competition and excitement around them, for psephologists, sociologists or those in the business of election campaign designing, exit polls have a lot more meaning.

“The information you gather through exit polls is not just who voted for whom, but also why they voted for them,” said Verma.

“This information is of great importance when it comes to providing data of social, political and public value,” he added.

Experts say the problem lies not with the ability of exit polls to predict the number of seats correctly, but the lack of scientific guidelines, methodological protocols and transparency in how they are conducted.

“The agencies do not disclose how they conduct the exit polls, their methods, the demographic break-up of their sample size etc.,” he said. “In the absence of this, it becomes very difficult for us to even use this information.”

Moreover, with no standardised guidelines, conducting exit polls is like a giant black box, Verma said. “Even if they get it right, how do we know it was not just guess work?” he said.

This is an updated version of the article.