India’s declining rank in global indices due to ‘serious problems’ in methodology: Analysis | India News

NEW DELHI: The decline in India’s rankings on a number of global opinion-based indices are due to “cherry-picking of certain media reports” and are primarily based on the opinions of a group of unknown “experts”, a recent study has concluded.
A new working paper titled “Why India does poorly on global perception indices” found that while such indices cannot be ignored as “mere opinions” since they feed into World Bank’s World Governance Indicators (WGI), there needs to be a closer inspection on the methodology used to arrive at the data.

The findings were published by Sanjeev Sanyal, member of the Economic Advisory Council to the Prime Minister (EAC to PM) and Aakanksha Arora, deputy director of (EAC to PM).
In the report, the authors conducted a case study of three opinion-based indices: Freedom in the World index, EIU Democracy index and Variety of Democracy.
They drew four broad conclusions from the study:
1) Lack of transparency: The indices were primarily based on the opinions of a tiny group of unknown “experts”.
2) Subjectivity: The questions used were subjective and worded in a way that is impossible to answer objectively even for a country.
3) Omission of important questions: Key questions which are pertinent to a measure of democracy, like “Is the head of state democratically elected?”, were not asked.
4) Ambiguous questions: Certain questions used by these indices were not an appropriate measure of democracy across all countries.
Here’s a look at the three indices examined by the study:
Freedom in the World Index
India’s score on Freedom in the World Index — an annual global report on political rights and civil liberties — has consistently declined post 2018.
It’s score on civil liberties was flat at 42 till 2018 but dropped sharply to 33 by 2022. It’s political rights score dropped from 35 to 33. Thus, India’s total score dropped to 66 which places India in the “partially free” category – the same status it had during the Emergency.
The study found that only two previous instances where India was considered as Partially Free was during the time of Emergency and then during 1991-96 which were years of economic liberalisation.

“Clearly this is arbitrary. What did the years of Emergency, which was a period of obvious political repression, suspended elections, official censoring of the press, jailing of opponents without charge, banned labour strikes etc, have in common with period of economic liberalisation and of today,” the study asked.
It concluded that the index “cherry-picked” some media reports and issues to make the judgement.
The authors further found that in Freedom House’s latest report of 2022, India’s score of the Freedom in the World Index is 66 and it is in category “Partially Free”.
“Cross country comparisons point towards the arbitrariness in the way scoring is done. There are some examples of countries which have scores higher than India which seem clearly unusual. Northern Cyprus is considered as a free territory with a score of 77 (in 2022 report). It is ironical as North Cyprus is not even recognised by United Nations as a country. It is recognised only by Turkey,” the authors noted.
Economist Intelligence Unit
In the Economist Intelligence Unit (EIU) Democracy Index, published the research and consulting arm of the firm that publishes the Economist magazine, India is placed in the category of “Flawed Democracy”.
Its rank deteriorated sharply from 27 in 2014 to 53 in 2020 and then improved a bit to 46 in 2021. The decline in rank has been on account of decline in scores primarily in the categories of civil liberties and political culture.
The authors found that list of questions used to determine the outcome was “quite subjective”, making objective scoring difficult.

They said that the responses for about 45 questions are from experts but the report does not reveal the number, nationality, credentials or even field of expertise of these experts.
Moreover, the authors noted that since the latest public opinion poll has not been conducted after 2012, it implies that the score for India is based only on expert opinions since 2012 till today.
The authors observed that India’s ranking improved partially after the withdrawl of the controversial farm laws last year, with the report saying that “victory of the protesters showed that there are mechanisms and institutions in place to allow government accountability to the electorate.”
But the authors questioned how report took political position on agriculture policy of a country into account, calling it “strange”.
Varieties of Democracy (V-DEM) index
An analysis of the V-DEM scores show that while India did well on objective parameters such as share of population with suffrage, its scores on various subjective subindices have declined sharply since 2014.
The authors noted India has been termed as an “electoral autocracy” in the 2021 report, same as it was during the period of Emergency.
“Clearly this is totally arbitrary as the years of Emergency which was a period of obvious political repression, suspended elections, censored press, etc have been put on par with today’s situation,” the authors surmised.
The authors’ analysis of the reports of V-DEM indicated that articles from media were “cherry-picked” and judgements have been made on that basis.
“For instance, the report notes that there has been a decline in the autonomy of the election management body. It mentioned that ‘The overall freedom and fairness of elections (Elections free and fair) also was hard hit, with the last elections held under Prime Minister Modi’s reign in 2019, precipitating a downgrading to an electoral autocracy.’ The report does not provide solid basis for coming to this conclusion,” the authors said.
Conclusion
According to the authors, there are “serious problems” with the methodology used in these perception-based indices.
They said that the common thread in all these indices is that they are derived from the “perceptions or opinions of few experts”.
“These institutions do not provide any transparency on how the experts were chosen or even their expertise or nationality (expect in case of V-DEM where they clarify that they chose some experts from each country from different fields),” they said in their report.
The authors suggested that the Indian government should request the World Bank to demand greater transparency and accountability from these institutions.
Meanwhile, independent Indian think-tanks should be encouraged to do similar perception based indices for the world in order to break the monopoly of a handful of western institutions,” they said.