Updated: December 2023
World Economics has developed the Global GDP Data Quality Ratings to review the usefulness of official GDP data of individual countries
The Ratings currently cover five factors to determine data quality.
Each factor is evaluated to provide country scores which are then normalised using the standard deviation of the data for each factor and combined into the DQR score using an average aggregate to reflect the importance of each of the individual factors.
World Economics has developed the Global GDP Data Quality Ratings to review the usefulness of official GDP data of individual countries. The Ratings currently cover five factors to determine data quality. Each factor is evaluated to provide country scores which are then normalised using the standard deviation of the data for each factor and combined into the DQR score using an average aggregate to reflect the importance of each of the individual factors.
These five factors used to judge data quality are:
It should be noted that there is not infrequent variation between what the World Bank and IMF list as the most recent Base Year and/or most recent SNA in use, and what countries themselves claim to be using. This is sometimes caused by often unavoidable time lags in the International organisations being informed of changes that have taken place locally and sometimes simple error is involved. Whatever the reasons, World Economics takes some trouble to find out what is the on-the-ground reality behind the figures. If we also fail to reflect the latest changes occasionally, we apologise in advance, but hope the data in this report is as correct and timely as is possible to achieve.
Base Year (Range from 1990 to Chained)
Constant price estimates of GDP use the inflation adjusted price of goods and services relative to a particular year, known as a base year, to weight the volume components of output. But since the structure of production and relative prices over time are dynamic, the structure of the prices of products and the industries surveyed in the base year become less relevant over time. For some rapidly changing products (such as the smartphone in recent years) rapid technological change and relative price falls make any kind of comparison fraught with difficulty.
What is clear is that using data from 10 or 20 years ago (as many countries do) as a basis for calculations of the size and shape of economic activity, is unlikely to produce reliable estimates of GDP. In countries that revise base dates, very significant increases are usually recorded, highlighting just how inaccurate data is that has been produced using out of date base years.
When GDP is revised and the base year is updated, it allows the statistician to reweight the relative importance of the different sectors of economic activity, and further change or reconsider the methods and data sources.
The United Nations recommends updating base years every five years, although, most developed countries now adopt the practice of chaining, where relative prices are updated every year. The more out of date a country’s base year, the more inaccurate are estimates of GDP and the lower a country’s score in the World Economics Data Quality Ratings (DQR).
The base year score for each country in the DQR is a number between 0 and 100, with 100 indicating that a country is using a chaining system, where base years (or relative prices) are updated every year). Information on individual country’s base year is taken from the World Bank’s World Development Indicators (WDI), IMF’s World Economic Outlook Report, United Nations and National Statistics Offices. Base year points for use in the data ratings are then calculated by taking the range of base years used and applying a sliding scale based on the number of years out of data. 100 indicates Chained or the latest possible base year where the oldest base year used (Bolivia, 32 years) is assigned the lowest score of 0. All variations between these years are deducted points for each year out of date.
System of National Accounts: (SNAs used range from 1968 to 2008)
National income measurement is governed by a global standard: the United Nations System of National Accounts (SNA) - an internationally agreed standard set of recommendations on how to compile and measure economic activity and facilitate international comparability of economic statistics. The first SNA was published in 1953 and there have been three revisions SNA 1968, SNA 1993, and SNA 2008.
The longer it takes a country to update its SNA the less reliable the data becomes, particularly when used for economic comparisons to a country with a more recent SNA version. In the World Economics Data Quality Ratings, the newer the SNA version, the higher a country’s score.
The score for the SNA component is based on a scale of 0-100, with 100 points given to countries using the latest SNA version. Information on individual country’s SNA is taken from the World Bank’s World Development Indicators (WDI), IMF World Economic Outlook Report, United Nations and National Statistics Offices.
Informal Economy: (ranges from less than 7.3% to over 72.0% of GDP)
In many poorer countries, a very large swathe of activity can remain uncounted and even in wealthy countries, some informal activities remain outside the national accounts. But due to the nature of much informal work, ranging from housework, farming through to gambling, prostitution, drug dealing, and smuggling, calculations of the value of such activities are extremely difficult. The existence of such large amount of informal activity is so economically important that to leave it unrecorded in the official national accounts is unsatisfactory.
There have been many attempts to estimate the size of parts of the informal economy. The World Economics Data Quality Ratings employs estimates for the World Economics Quarterly Informal Econony Survey.
In constructing the data, the higher the size of the informal economy, the lower a country’s factor score. A DQR score of 100 means that a country has the lowest rate of informal sector activity as percentage of GDP.
Resources Available for Producing National Accounts Data
The quality of national income estimates depends to some extent on the statistical capacity and the resources available to national statistics offices. The United Nations System of National Accounts has put a global standard in place but the challenge for a local national statistics office is to produce a measure of the economy, usually with limited resources. Statistical capacity, or the ability to adhere to the global standard, depends critically on the resources and information available at any given time and place.
All other things being equal, there are a priori grounds to believe that poorer economies will have lower-quality statistics. The statistical capacity and economic resources in national statistics offices therefore matter a great deal in terms of data availability and quality of economic statistics. Data availability is subject to the number of trained staff and the level of resources available for collecting, processing and analysing the data.
As an illustration of the importance of resources in the collection of data, few have shown with greater clarity the nature of the problem than Morten Jerven in his book Poor Numbers, 2013, based on actual visits to statistics offices in Africa. To quote Jerven: “This book has shown that the most basic metric of development , GDP, should not be treated as an objective number but rather as a number that is the product of a process in which a range of arbitrary and controversial assumptions are made. As a result the metric should be used with the utmost care. The quality of this number depends on the state of the system that produces the statistics and this system is deficient in many poor countries.” This problem is not confined to Africa but is evident in countries on all continents.
The score for this component of the DQR is derived from the World Bank Statistical Capacity Index. We use this index as a proxy for assessing the availability of economic resources in national statistics offices. In theory, the larger the resources devoted to statistics offices, the better the quality of statistics. That is, the higher the index, the higher the country’s score. This is a proxy measure.
Government Interference
Governments interfere with the production and dissemination of basic economic data in many ways. Attempts in Greece to prosecute and potentially jail Andreas Georgiou, the man hired by the IMF to sort out the corrupt mess of Greek economic data is perhaps the most egregious recent example. Read: The Case of Andreas Georgiou.
The Greek instance might appear to be an extreme special case. But unfortunately there are also many occurrences of serious Government interference in the production of economic data in the Americas.
For example, in the paper (On Measuring Hyperinflation; Venezuela's Episode, by Hanke), records the following:
The Banco Central de Venezuela (BCV), like many central banks, has followed a pattern that Oskar Morgenstern elegantly documents in his classic work On the Accuracy of Economic Observations. Indeed, the BCV has failed to report data that would reflect poorly on the government, and when it has reported inflation statistics, it has lied and doctored the data. Instead of reporting Venezuela’s ‘real’ open rate of inflation, the BCV has attempted to measure suppressed inflation.
Venezuela imposes a thick blanket of price controls and a maze of subsidies over the economy. List prices are artificially held down. Yet these suppressed prices are the ones that, in principle, the BCV attempts to measure and use to construct a price index for calculating the inflation rate. But this metric misses the mark. Arbitrage opportunities prevail under the Venezuelan regime of price controls and subsidies, because there is a gap between the items under price controls and the prices of those goods and services that are actually exchanged on the black market. And it is in the black market and underground economy that most of Venezuela’s economic activity occurs. In consequence, there is a huge gap between the official inflation rate, which is based on artificially suppressed prices, and the ‘real’ open inflation rate.
And similarly, a paper by Ariel Coremberg "Measuring Argentina's GDP Growth; Myths and Facts" makes the following points:
Governments and Government agencies manipulate GDP data directly in many ways, for example through the calculation of price indexes such as the GDP deflator which impact on GDP per capita data. They can and do stop publishing important data prior to elections. They try to abolish independent statistics bureaus. They try to add questions that will bias responses to Census data. They leave in place price indexes known to be unreliable and impacting heavily and negatively on crucial pensions systems.
This is not only a problem evident in poor countries, although countries with autocratic systems probably suffer to a greater extent. Sometimes the transgressions are deliberate, and sometimes due to incompetence or lack of resource.
Government corruption also infect all parts of an economy and its accurate measurement in systematic ways. Often a direct result of the government’s concentration of economic or political power, corruption manifests itself in many forms such as bribery, extortion, nepotism, patronage, embezzlement, and graft.
For example, excessive and redundant government regulations provide opportunities for bribery or graft. In addition, government regulations or restrictions in one area may create informal markets in another. As a result, governance and the informal economy are often correlated.
All these potential ways of corrupting data are difficult to measure directly. We have adopted a general measure of Governance as a proxy for Government interference. The score for this component of the DQR is derived directly from the World Economics Governance Rankings for over 155 countries.
The Governance score is based on a 100-point scale in which a score of 100 indicates very good levels of governance and a score of 0 indicates very poor governance.
Differences in reliability are highlighted by combining the five factors discussed using a simple average. The base year, SNA version, the research based Informal Economy, Capacity Indicator and Governance are weighted equally with 20% each.
The five factors are combined into the GDP Data Quality Ratings Index. The data are then ranked by quartile with consideration given to logical index groups) and each country receives a summary rating from A to E grade.
The summary ratings are described below:
Grade A: As Good as it Gets Countries in this grade all have up-to-date Base Years or use the chaining methodology and all employ the most up to date SNA 2008 standard for measuring GDP. They have lower levels of unmeasured informal economic activities with a median estimate of just over 14 per cent of official GDP and generally high levels of governance. The quality of the economic data in ‘A’ graded countries is “as good as it gets” and can be used for most purposes, although the usual caution must be taken given less than full implementation of the latest standards, difficulties in estimating government output and the services sector.
Grade B: Good A number of countries in this grade have out of date base years ranging from 3 to 8 years and nearly one in five still employ the outmoded SNA 1993 standard for measuring GDP. The average level of unmeasured informal economic activities as a proportion of official GDP at 28 per cent is over twice the estimate for ‘A ‘ranked countries with a range from 12 to 53 per cent. Governance levels on average are twice as high as in category ‘A’ countries. The quality of economic data in ‘B’ graded countries is generally good but should be used with some caution, it means that it provides a reasonable guide of GDP for most purposes , but it should not be used to make direct comparisons to countries ranked with A grade data. Particular attention should be made to the quality of government data and price indexes.
Grade C: Use with caution The majority of the countries in this grade have out of date base years apart from a few which use chaining. The lag ranges from 4 to 18 years with an average of 7 years out of date. Two-thirds of the countries still employ the SNA 1993 standard for measuring GDP which means that the average level of unmeasured informal economic activities as a proportion of official GDP is high at 32 per cent with a range from 15 to 60 per cent. Governance levels remain fair. The quality of economic data in ‘C’ graded countries should only be used as a general guide, but some attempts are being made by many countries to improve accuracy.
Grade D: Poor Most countries in this grade have out of date base years and many countries still employ the SNA 1993 standard for measuring GDP which means that the average level of unmeasured informal economic activities as a proportion of official GDP is high. Governance levels are lacking. The quality of economic data in ‘D’ graded countries provides a poor guide.
Grade E: Extremely Poor All of the countries in this grade have out of date base years with a lag ranging from 5 to 32 years with an average of 15 years out of date. Virtually all of the countries are using SNA 1993, apart from seven a few that still use SNA 1968. The average level of unmeasured informal economic activities as a proportion of official GDP is very high at 37 per cent and ranges from 15 to 67 per cent. Governance is poor with levels much lower than in category D countries. The quality and reliability of economic data in ‘E’ graded countries is extremely poor and official GDP data should not be used for any purpose.