In June 2021, United Nations Development Programme (UNDP) India released an appraisal report on the Aspirational Districts Programme (henceforth ADP). ADP is a flagship programme by the Government of India which is dedicated to the development of 115 aspirational districts (a new nomenclature used for most backward districts) across 28 states in India. In its 65 pages long report, the UNDP India asserts that the ADP has resulted in sectoral growth and improvements in governance and administration. “The programme [ADP] is not only replicable within India, but also across the globe,” says Renata Dessallien, the UN Resident Coordinator in India in her message in the report, who has also touted the programme as a “global example in enlisting sub-national government, with multi-stakeholder partnerships”. UNDP India has drawn such conclusions on the basis of findings derived from its evaluative study of ADP. Last year, Ms. Dessallien was under fire from the union government for her statements. Government representatives had termed her remarks on Covid-19, migrant workers, and Hathras rape “highly objectionable” and “unwarranted”.
In its study, the UNDP team have analysed quantitative data, qualitative data and also reviewed the best practices carried out under ADP. Before I discuss the report further, let’s understand what ADP is.
Exactly what ADP is?
The ADP was launched by the NDA government in January 2018 with the view to alleviate regional inequalities with respect to development. Across 28 states, 115 most backward districts have been selected for the implementation of ADP. The Ministry of Home Affairs identified 35 districts on the basis of “density of violence”. It includes districts like Bastar, Bijapur, Dakshin Bastar Dantewada from Chhattisgarh and Latehar, Lohardaga from Jharkhand. Different ministries of the Government of India selected 55 other districts on the basis of “severe deficiency or sensitivity”. Araria & Aurangabad from Bihar and Kalahandi & Kandhamal from Odisha are such districts.
Initially, NITI Aayog had selected 117 districts for ADP. However, 5 districts of West Bengal never joined the programme. Hence, UNDP India has taken 112 districts into consideration while making this report. Change in nomenclature of these districts from ‘Backward’ to ‘Aspirational’ has a backdrop. Prime Minister Modi suggested the change in his address during the launching ceremony of ADP on January 5, 2018.
Thematic areas of development in these districts are based on the Human Development Index (HDI). Factors such as long and healthy life, education and decent standard of living comprise HDI according to UNDP. ADP considers sectors such as health and nutrition, education, agriculture & water resources, financial inclusion & skill development, and basic infrastructure as thematic areas of development. To oversee the performance across these sectors, further break up has been done in the form of 49 Key Performing Indicators (KPIs) and 81 data points. The following table shows weightage and number of indicators of each sector:
| Sectors | Weightage | No. of Indicators |
| Health and Nutrition | 30% | 13 |
| Education | 30% | 08 |
| Agriculture and Water Resources | 20% | 10 |
| Financial Inclusion and Skill Development | 10% | 11 |
| Basic Infrastructure | 10% | 07 |
| Composite Index | 100% | 49 |
(Source: Aspirational Districts: Unlocking Potentials)
According to its programme document, ADP follows the “3C approach” which are Convergence, Collaboration and Competition. Here, convergence means to run multiple schemes (both union and state schemes) in coordination; collaboration of central, state-level ‘Prabhari’ officers & District Collectors; and competition among districts to perform better. To inspire competition among districts, more weightage has been given to health & nutrition and education sector compared to others. Since, sectors such as agriculture, financial inclusion, and infrastructure are largely in the hands of state and union governments, district offices have little to contribute there. District offices have more scope for intervention and improvement of the performances in sectors such as health, nutrition and education. This is how it is intended for districts to compete with each other.
However, Rashtriya Sam Vikas Yojana (RSVY) and Backward Regions Grant Fund (BRGF), which are similar to the ADP, were launched during the previous union governments in 2004 and 2007 respectively. Later, RSVY subsumed into the BRGF programme and the BRGF programme has been delinked from the budgetary support of the Union government since 2015-16. One major feature which distinguishes ADP from these two former programmes is “Real-Time Monitoring”. A dashboard on the website Champions of Change allows regular update of district wise data by district officials and concerned ministries regarding programme implementation. On the basis of data received, the dashboard automatically calculates a delta ranking of performance of each district. ADP claims in its programme document that the dashboard and the delta ranking inspire competition.
Data collection and analysis
There are problems with the data used by the UNDP India team for their analysis of the programme. In its press release for the appraisal report, the UNDP India states that it is an “independent appraisal report” of the ADP. However, the source from where the UNDP India team has collected quantitative data for its report is the same ‘Champions of Change’ website run by NITI Aayog which has been discussed earlier in this piece. This puts a question mark on the “independent” nature of the UNDP India’s appraisal report. The following are other issues with the data and analysis in the report.
Insufficient data
The indicator-wise data used for analysis are insufficient. In education sector, data regarding three indicators with 10 data points are not available, which are 23.4% of the composite index of total sectors. Education sector carries 30% of total weightage. In health sector, data regarding four indicators with 7 data points are not available, which are 5.7% of the composite index. Health sector too carries 30% of the total weightage. In agriculture sector, data regarding three indicators with 7 data points are not available, which are 4.5% of the composite index. Agriculture sector carries 20% of total weightage. Cumulatively, these are 33.6% of the overall composite index, which means a little over one-third of the total data is missing for the analysis. With this extent of dearth of data and the absence of independent verification, a fair analysis cannot be expected. It is difficult to grasp how this appraisal report ascertained growth in above-mentioned sectors under ADP.
However, the UNDP India team has used imputation method to substitute the missing data for the analysis. It is a statistical process through which missing data is estimated on the basis of existing data. In this case, most of the missing data for the year 2020 have been imputed from the year 2019 by the UNDP India team. The imputation method is considered reliable, but the appraisal report has not discussed the use of the method in more detail.
Analysis of data
Coming to the analysis of the quantitative data, the UNDP India team has mentioned two methods, namely Net Resilience Index; and Difference in Difference (henceforth DiD) method. Preparing a Net Resilience Index is an exercise to see the overall performance of a district under ADP. This index was prepared by making comparisons of resilience and vulnerability scores of each district. The purpose of this index is to show the resilience of a district to sustain the development despite its vulnerabilities. Districts, which improve positive indicators such as percentage of elementary schools having standard pupil-teacher ratio compared to their negative indicators such as percentage of underweight children, are considered as resilient in development terms.
Bias in data analysis and collection
DiD method is used for impact evaluation of a programme, which requires two sets of group for the analysis. One is the treatment group which receives interventions made under the programme and the other is the control group which is of the same characteristics but without the intervention. Here, the treatment group consists of the 112 Aspirational Districts. For the control group, 113 Non-Aspirational Districts have been chosen for the Health & Nutrition sector and 112 Non-Aspirational Districts for the Financial Inclusion sector. DiD method is known to be biased. As Pamela Jakiela discusses it in her recent paper, “…it [DiD] can be used for retrospective evaluation in a wide range of settings where pre-treatment data is available… Recent research demonstrates that such estimates can be severely biased – and may even be incorrectly signed…”
For qualitative data collection, the UNDP India team conducted interviews with the persons responsible for implementation of the programme. It includes District Magistrates or District Collectors, Prabhari officers, knowledge partners, development partners, Aspirational District Fellows and UN volunteers working in these districts. A few interviews were also conducted with district magistrates of non-Aspirational Districts. These persons are mostly administrative representatives, consultants, and volunteers. The report doesn’t mention an interview or any other kind of qualitative data collection with a beneficiary of the programme. For an evaluative study about the impact and efficacy of an ongoing development programme, it is beyond comprehension. It is obvious that those persons, who are responsible for the implementation of the programme, would have biases and speak positively about the impact of the programme. Claims about the impact of the programme could have been cross-checked and supported by knowing the views of beneficiaries. The UNDP India appraisal report fails to do that.
A constructive and critical evaluation of a development oriented programme like ADP is quite important. It helps to observe effectiveness of the programme and points out the scope for improvement too. Moreover, an independent evaluation is more reliable because of its unbiased approach. UNDP India team fails to stand on its claim to produce an independent report, since it has used government data for the analysis. Inconsistency and gaps persist in data which have been used for the evaluation. Though statistical method like DiD method has been used to overcome this problem but the method can give biased result. Furthermore, the absence of beneficiaries’ views is another reason to question the impact of ADP as claimed in this appraisal report.
Aditya Anand is an independent development practitioner. He can be reached at [email protected]
