More people in rural India defecate in open than what govt’s household survey shows: Study

India’s Demographic and Health Survey that asks household-level questions greatly underestimates open defecation among households with latrines.

Representational image | Flickr

On this day five years ago, the Narendra Modi government launched the Swachh Bharat Mission, a national sanitation campaign, which aims to eliminate open defecation in India by 2019. Many latrines have been constructed in rural India as a result of this campaign. Yet, the effect the SBM has had on reducing open defecation is still unknown.

Rural India is home to more than half of the world’s open defecation. Because the persistence of open defecation threatens gains in child health, the Sustainable Development Goals (SDGs) call for its elimination by 2030. India’s most recent Demographic and Health Survey (DHS), conducted between January 2015 and December 2016, estimates that 54 per cent of households in rural India defecated in the open, down from 75 per cent in the 2005 to 2006 DHS. This measure is based on a household-level question that asks about the behaviour of everyone in the household in the same question.

Recent evidence from studies carried out in India suggests, however, that it is common for individuals living in households with latrines to nevertheless defecate in the open. In rural parts of five north Indian states, Coffey et al found that 21 per cent of individuals defecated in the open, despite owning a latrine. In rural Tamil Nadu, Yogananth and Bhatnagar report that 54 per cent of respondents defecated in the open despite having a household latrine. In Odisha, Barnard et al found that less than half of members of households with latrines reported using their latrines at all times.

These findings suggest that open defecation among latrine-owning households is substantial. Since it is probable that latrine use is the socially desirable response to questions on sanitation behaviour, measures based on household-level questions, such as those from the DHS, will likely underestimate true open defecation in rural India, particularly among households with latrines.

Because open defecation is an individual behaviour, an individual-level survey question may be able to more accurately measure it compared with a household-level question, particularly among households with latrines. We designed this study to experimentally test this hypothesis in rural India. We aimed to investigate whether a balanced question about latrine use or open defecation for every member of a household finds different levels of open defecation compared with a household-level question.

This is the first study to experimentally vary survey methodology to improve on the measurement of open defecation currently being used. Our study contributes to this literature by comparing estimates of open defecation obtained from questions that can be administered in a large household survey, and highlighting potential sources of error in open defecation measurement.

Sample: mostly latrine-owning households in rural parts of four states

This study uses as its sampling frame the study areas of International Initiative for Impact Evaluation(3ie)’s Promoting Latrine Use in Rural India Thematic Window. This window has funded four independent research teams to conduct randomised control trials of distinct behavioural campaigns to promote the use of pit latrines in rural parts of Bihar, Gujarat, Karnataka and Odisha. The study areas are spread across India, representing different contexts and varying levels of rural open defecation.

Figure 1: Sample selection. Households that were started but not completed refers to households that refused at the beginning or part of the way through and households in which a suitable respondent was not available.

Figure 1 describes the sample selection. The villages visited in each state were randomly selected from the full set of villages included in the 3ie research teams’ studies. The research team led by Oxford Policy Management worked with World Vision in Bihar, the team led by the London School of Hygiene and Tropical Medicine worked with Coastal Salinity Prevention Cell in Gujarat, Eawag worked with Wateraid in Karnataka, and Emory University worked with the Rural Welfare Institute in Odisha. Data for our study were collected in 22–25 villages in each of the four study areas. In most areas, we sampled more villages than we actually visited in order to facilitate coordination with the research teams. Ninety-five villages were visited in total. On average, the survey team interviewed 25 households per village. Data collection took place between March and July 2018.

Randomisation and masking: random variation in latrine use questions at the household level

We randomly assigned the type of latrine use question administered in the survey at the household level. Roughly half of the households were assigned individual-level questions on latrine use. The other half were assigned a household-level question.

The individual-level questions asked for every household member age 5 or older whether the individual defecated in the open or used the latrine.

The rest of the households were assigned the household-level question used in India’s DHS: ‘What kind of toilet facility do members of your household usually use?’.

The primary outcome of interest is reported open defecation.

Results

The total sample consisted of 2,431 households, which were approximately equally divided across question type in each of the study areas. There were no significant differences on measures relevant for latrine use between households assigned different types of latrine use questions. Households in both groups had approximately the same number of household members, fraction female, fraction Hindu, educational attainment of the household head, and asset ownership of 13 assets, including mobile phone, electricity, radio, television, fan, mosquito net, bicycle, motorcycle, car, chair, gas stove, pressure cooker and shoes for everyone in the family. It is important for the validity of the results that the sample is balanced on religious composition, since studies have documented an association between household religion and latrine use.

Figure 2: Individual-level, balanced latrine use questions find significantly higher rates of open defecation than the household-level question. Figure shows means and CIs. Red, dashed lines indicate responses to individual-level, balanced latrine use questions, and blue solid lines indicate responses to the household-level question. Unit of observation is individuals for individual-level questions and households for household-level questions. CIs are computed using SEs clustered by village.

Figure 2 presents the main results of the study; it shows means and 95 per cent confidence intervals (CIs) from the individual-level (shown as dashed red bars) and the household-level (shown as solid blue bars) questions, for the full dataset and for different subsamples. Observations are individuals for the individual-level estimates, and households for the household-level estimates. In the full sample, and in all subsamples analysed in this figure, the individual questions find more open defecation. Moreover, the size of the difference in reported open defecation between question types is large and consistent. The first set of estimates shown in the figure uses the full sample. The second, third, fourth and fifth sets of estimates break the sample up by project area. The sixth set of estimates uses only households with latrines. No matter how the data are broken up, the individual-level, balanced latrine use questions find significantly higher rates of open defecation than the household-level question.

Measure open defecation at individual level

Our findings show that in our sample, individual-level, balanced questions find 20–21 (95 per cent CIs 16 to 25 for both estimates) percentage points more open defecation than the household-level question. This is both a statistically significant and practically important difference. This study presents compelling evidence that India’s DHS, which provides the most recent nationally representative estimates of open defecation for rural India, and other surveys that ask household-level questions, greatly underestimate open defecation among households with latrines.

We also found that the difference in reported open defecation between the two question types is significantly greater for households that received help to construct their latrines compared with households that did not. Among households that received help to construct their latrines, the individual-level questions find 28–29 (95 per cent CIs 22 to 35 for both estimates) percentage points more open defecation than the household-level question. This suggests that as more and more households receive government assistance for a latrine through the SBM, household-level questions will become even less accurate at estimating open defecation.

The larger difference in measured open defecation between the two question types among households that received help compared with those that did not is likely arising from higher rates of open defecation among households that received help to construct their latrines. There are several reasons that could explain why households receiving assistance may be less likely to use their latrines. First, these households are likely to have lower demand for latrine use, compared with households that built latrines on their own. Second, households that received help have latrines with pits that are on average 150 ft³ smaller than the pits of latrines in households that did not receive help. Because of concerns over ritual purity, rural Indians are less likely to use latrines with pits that need to be emptied manually every few years, like the latrines that are promoted and constructed by the government. Whether only one, both or other factors are leading to more open defecation among households that received help, the individual-level questions are better able to capture this open defecation than the household-level question.

We also find a statistically significant difference in reported open defecation between the two question types for males compared with females. The difference between the individual-level, balanced questions and the household-level question is 4 (95 per cent CI 2 to 7) percentage points more for males compared with females. This supports evidence that, conditional on latrine ownership, males are more likely to defecate in the open compared with females. Individual-level questions understate the difference in open defecation between the two sexes by less than the household-level questions. Compared with the difference in reported open defecation by receiving help to construct the latrine, differences by sex are not as large.

Measuring open defecation at the individual-level in a large household survey is doable and will provide a more accurate estimate of open defecation in rural India. Since reducing open defecation in India is important for meeting the SDGs, and since open defecation is an important factor contributing to poor health among children in India, it is important to measure its progress as accurately as possible.

Sangita Vyas is Research Fellow at the Research Institute for Compassionate Economics (RICE), and a PhD student of economics at the University of Texas at Austin.

This article is an edited excerpt from a coauthored research paper, ‘Measuring open defecation in India using survey questions: evidence from a randomised survey experiment,’ which was first published in BMJ Journals.