Search:

World Economics Data

Ratings of GDP Accuracy in the Americas

World Economics Research

The quality of economic data in North and South America is very variable. The United States Canada and Chile publish consistently good quality data, ranking high by international standards. We believe there are significant quality issues with data produced in Bolivia, El Salvador, and Brazil. And at the extreme end of the scale, data produced in Bolivia and Haiti is likely to be of little value in describing prevailing economic conditions.

    See the Data Quality Ratings table for the Americas


WHY GDP QUALITY IS VERY IMPORTANT

GDP data have been described as “The World’s Most Powerful Numbers“1. We wouldn’t take issue with that description.

GDP data are, without question, of crucial importance. GDP data are used by Governments everywhere to help make policy decisions affecting millions; to direct governmental aid flows; and in many other ways. GDP data are used by banks, financial organisations of all kinds and businesses the world over to help make investment decisions, as a base for base corporate strategy, and for many other tasks.

Yet few countries achieve the standards set internationally by the United Nations, or keep up to date with recommended best practice. Even when data meets UN standards, most measures of GDP are flawed conceptually, in the way they are collected and presented, to an extent that few realise (further papers will review the conceptual problems underlying GDP data).

This paper sets out only to review which GDP data sets in the Americas are reliable in relation to basic standards, including base years used (basing data on a 25 year old snapshot of economic activity isn’t likely to produce accurate data); which UN ordained System of National Accounts (SNA) they use (some still use 1993 standards – a pre Amazon, Facebook and Google era); how much informal economic activity may remain uncounted (maybe up to 50%); and various other proxy indicators of possible inaccuracy. From our ratings we believe it is possible to understand which countries GDP data are as good as it gets, what data should be used only with the care, and those which are potentially highly misleading. Similar papers are available covering Africa, Asia, Europe and the Middle East.

MEASURING GDP QUALITY

World Economics has developed the Global GDP Data Quality Ratings to review the utility of official GDP data of individual countries. The Ratings currently cover six factors to determine data quality. Each factor is evaluated to provide country scores which are then normalised using the standard deviation of the data for each factor and combined into the DQR score using a weighted aggregate to reflect the importance of each of the individual factors.

These six factors used to judge data quality are:
  1. Base Year used to calculate the GDP data (chained or years out of date)
  2. Standard of National Accounts (SNA) applied
  3. Estimated Size of the Informal Economy
  4. A Proxy for Resources Devoted to Measuring Economic Activity
  5. A Proxy measure for likely Government Interference in Economic Data production
  6. A Proxy variable for Regulatory Hurdles faced by enterprises generally
It should be noted that there is not infrequent variation between what the World Bank and IMF list as the most recent Base Year and/or most recent SNA in use, and what countries themselves claim to be using. This is sometimes caused by often unavoidable time lags in the International organisations being informed of changes that have taken place locally and sometimes simple error is involved. Whatever the reasons, World Economics takes some trouble to find out what is the on-the-ground reality behind the figures. If we also fail to reflect the latest changes occasionally, we apologise in advance, but hope the data in this report is as correct and timely as is possible to achieve.


KEY VARIABLES AND METHODOLOGY

Base Year (Range from 1984 to Chained (2018))

Constant price estimates of GDP use the inflation adjusted price of goods and services relative to a particular year, known as a base year, to weight the volume components of output. But since the structure of production and relative prices over time are dynamic, the structure of the prices of products and the industries surveyed in the base year become less relevant over time. For some rapidly changing products (such as the smartphone in recent years) rapid technological change and relative price falls make any kind of comparison fraught with difficulty.

What is clear is that using data from 10 or 20 years ago (as many countries do) as a basis for calculations of the size and shape of economic activity, is unlikely to produce reliable estimates of GDP. In countries that revise base dates, very significant increases are usually recorded, highlighting just how inaccurate data is that has been produced using out of date base years.

When GDP is revised and the base year is updated, it allows the statistician to reweight the relative importance of the different sectors of economic activity, and further change or reconsider the methods and data sources.

The United Nations recommends updating base years every five years, although, most developed countries now adopt the practice of chaining, where relative prices are updated every year. The more out of date a country’s base year, the more inaccurate are estimates of GDP and the lower a country’s score in the World Economics Data Quality Ratings (DQR).

The base year score for each country in the DQR is a number between 0 and 100, with 100 indicating that a country is using a chaining system, where base years (or relative prices) are updated every year). Information on individual country’s base year is taken from the World Bank’s World Development Indicators (WDI), IMF’s World Economic Outlook Report, United Nations and National Statistics Offices. Base year points for use in the data ratings are then calculated by taking the range of base years used and applying a sliding scale based on the number of years out of data. 100 indicates Chained or the latest possible base year where the oldest base year used (Madagascar, 34 years) is assigned the lowest score of 0. All variations between these years are deducted multiples of 2.9 points for each year out of date.


System of National Accounts: (SNAs used range from 1968 to 2008)

National income measurement is governed by a global standard: the United Nations System of National Accounts (SNA) - an internationally agreed standard set of recommendations on how to compile and measure economic activity and facilitate international comparability of economic statistics. The first SNA was published in 1953 and there have been three revisions SNA 1968, SNA 1993, and SNA 2008.

The longer it takes a country to update its SNA the less reliable the data becomes, particularly when used for economic comparisons to a country with a more recent SNA version. In the World Economics Data Quality Ratings, the newer the SNA version, the higher a country’s score.

The score for the SNA component is based on a scale of 0-100, with 100 points given to countries using the latest SNA version. Information on individual country’s SNA is taken from the World Bank’s World Development Indicators (WDI), IMF World Economic Outlook Report, United Nations and National Statistics Offices.

Table 1: System of National Accounts, DQR Points Available


Informal Economy: (ranges from less than 7% to over 65% of GDP)

In many poorer countries, a very large swathe of activity can remain uncounted and even in wealthy countries, some informal activities remain outside the national accounts. But due to the nature of much informal work, ranging from housework, farming through to gambling, prostitution, drug dealing, and smuggling, calculations of the value of such activities are extremely difficult. The existence of such large amount of informal activity is so economically important that to leave it unrecorded in the official national accounts is unsatisfactory.

There have been many attempts to estimate the size of parts of the informal economy. The World Economics Data Quality Ratings employs estimates for 2015 provided by the IMF Working Paper: WP/18/172.

In constructing the data, the higher the size of the informal economy, the lower a country’s factor score. A DQR score of 100 means that a country has the lowest rate of informal sector activity as percentage of GDP.

The United States of America, for example with just 7% of economic activity attributed to the informal economy receives a factor score of 100, while Panama with 60% of activity counted as informal scores 12.


Resources Available for Producing National Accounts Data

The quality of national income estimates depends to some extent on the statistical capacity and the resources available to national statistics offices. The United Nations System of National Accounts has put a global standard in place but the challenge for a local national statistics office is to produce a measure of the economy, usually with limited resources. Statistical capacity, or the ability to adhere to the global standard, depends critically on the resources and information available at any given time and place. 

All other things being equal, there are a priori grounds to believe that poorer economies will have lower-quality statistics. The statistical capacity and economic resources in national statistics offices therefore matter a great deal in terms of data availability and quality of economic statistics. Data availability is subject to the number of trained staff and the level of resources available for collecting, processing and analysing the data. 

As an illustration of the importance of resources in the collection of data, few have shown with greater clarity the nature of the problem than Morten Jerven in his book Poor Numbers, 2013, based on actual visits to statistics offices in Africa. To quote Jerven: “This book has shown that the most basic metric of development , GDP, should not be treated as an objective number but rather as a number that is the product of a process in which a range of arbitrary and controversial assumptions are made. As a result the metric should be used with the utmost care. The quality of this number depends on the state of the system that produces the statistics and this system is deficient in many poor countries.” This problem is not confined to Africa but is evident in countries on all continents.

The score for this component of the DQR is derived from the United Nations Human Development Index (HDI). We use the HDI as a proxy for assessing the availability of economic resources in national statistics offices. In theory, the larger the resources devoted to statistics offices, the better the quality of statistics. That is, the higher the HDI, the higher the country’s score. This is a proxy measure. We know of no research that measures the precise relationship between the HDI and the reliability of economic data. However we believe there is a strong likelihood based on common sense (and research such as that undertaken by Jervens and others) that countries with a high HDI are likely to produce more accurate data.


Government Interference
Governments interfere with the production and dissemination of basic economic data in many ways. Attempts in Greece to prosecute and potentially jail the man hired by the IMF to sort out the corrupt mess of Greek economic data is perhaps the most egregious recent example. 

The Greek instance might appear to be an extreme special case. But unfortunately there are also many occurrences of serious Government interference in the production of economic data in the Americas.

For example, a recent paper (On Measuring Hyperinflation; Venezuela's Episode, by Hanke), records the following:

The Banco Central de Venezuela (BCV), like many central banks, has followed a pattern that Oskar Morgenstern elegantly documents in his classic work On the Accuracy of Economic Observations. Indeed, the BCV has failed to report data that would reflect poorly on the government, and when it has reported inflation statistics, it has lied and doctored the data. Instead of reporting Venezuela’s ‘real’ open rate of inflation, the BCV has attempted to measure suppressed inflation. 

Venezuela imposes a thick blanket of price controls and a maze of subsidies over the economy. List prices are artificially held down. Yet these suppressed prices are the ones that, in principle, the BCV attempts to measure and use to construct a price index for calculating the inflation rate. But this metric misses the mark. Arbitrage opportunities prevail under the Venezuelan regime of price controls and subsidies, because there is a gap between the items under price controls and the prices of those goods and services that are actually exchanged on the black market. And it is in the black market and underground economy that most of Venezuela’s economic activity occurs. In consequence, there is a huge gap between the official inflation rate, which is based on artificially suppressed prices, and the ‘real’ open inflation rate.

And similarly, a paper by Ariel Coremberg "Measuring Argentina's GDP Growth; Myths and Facts" makes the following points:

  • Since 2007, official economic statistics in Argentina, particularly on consumer inflation and GDP, have been subject to political manipulation.
  • This paper reproduces Argentine national income from 2007 using standard methods and original sector data and finds that declared GDP is 12.2% higher in 1993 prices due to political intervention.
  • The paper finds that the distortion is mainly due to changes in accounting methodology across industries and not to changes in inflation estimates.
  • The reproduced GDP data dispels the myth that Argentina has been the fastest growing South American economy in recent years.
Governments and Government agencies manipulate GDP data directly in many ways, for example through the calculation of price indexes such as the GDP deflator which impact on GDP per capita data. They can and do stop publishing important data prior to elections. They try to abolish independent statistics bureaus. They try to add questions that will bias responses to Census data. They leave in place price indexes known to be unreliable and impacting heavily and negatively on crucial pensions systems.  

This is not only a problem evident in poor countries, although countries with autocratic systems probably suffer to a greater extent. Sometimes the transgressions are deliberate, and sometimes due to incompetence or lack of resource. 

Government corruption also infect all parts of an economy and its accurate measurement in systematic ways. Often a direct result of the government’s concentration of economic or political power, corruption manifests itself in many forms such as bribery, extortion, nepotism, patronage, embezzlement, and graft. 

For example, excessive and redundant government regulations provide opportunities for bribery or graft. In addition, government regulations or restrictions in one area may create informal markets in another. As a result, corruption and the informal economy are often correlated. 

All these potential ways of corrupting data are difficult to measure directly. We have adopted a general measure of corruption as a proxy for Government interference .The score for this component of the DQR is derived directly from Transparency International’s Corruption Perceptions Index (CPI), which measures the level of perceived corruption in 175 countries.

The CPI score is based on a 100-point scale in which a score of 100 indicates very little corruption and a score of 0 indicates rampant corruption. That is, in the DQR, the lower the level of corruption, the higher a country’s score. Similarly, the higher the level of corruption, the lower a country’s score. This factor varies from the United States of America with a score of 77 to Haiti scoring only 9 when standardised into the Data Quality Ratings.


Regulatory Hurdles

The quality of economic information available in an economy is related both to the supply of data generated by businesses to customers and suppliers and to the incentives for companies to distort the information provided to authorities for tax assessments, court appeals, property purchases, licences or other transactions involved with the state. The lower the degree of government intervention and the regulation of business activity the greater the amount of unbiased information supplied by enterprises about their economic activities. In consequence a negative relationship would be expected to exist between the quality of economic data and the level of regulation in an economy.

The score for this component of the DQR is derived directly from the World Bank Ease of Doing Business Index (EODBI), which measures the level of regulation affecting businesses in 190 countries. The EODBI score is based on a 100-point scale in which a score of 100 indicates very little regulation and a score of 0 indicates a completely regulated business environment. That is, the lighter the role of the government in business and the more the government interferes with business or makes it difficult to set up and operate a business, the lower a country’s score.


CALCULATING WORLD ECONOMICS DATA RATINGS

Differences in reliability of economic data across the Americas is highlighted by weighting and combining the six factors discussed. The three variables given most weight (25% each) are the objective Base year and SNA Indexes, together with the research based Informal Economy values. The three proxy measure used (Resources available; Government Interference and Regulatory hurdles) are collectively weighted by 25%.

DQR values calculated for these factors are shown in Table 2 below where it can be seen that the quality, reliability and believability of economic data across the Americas differs widely. The DQR score varies theoretically between 0 and 100, the lowest and highest scores a country can earn. 

In the DQR Ratings Table countries are ranked from best in the region – United States with a global rank of 7 – to the worst – Haiti with a global rank of 154, the worse in the world..
 
Table 2: The Americas Data Quality Ratings 

    The Data Rankings for all 154 countries can be found in the Global DQR Report.



References

1. Fioramonti, Lorenzo. (2013). Gross Domestic Problem: The Politics Behind the World's Most Powerful Number.

2. Medina L. and Schneider F., Shadow Economies Around the World: What Did We Learn Over the Last 20 Years?;. International Monetary Fund, Working Paper No. WP/18/17. https://www.imf.org/~/media/Files/Publications/WP/2018/wp1817.ashx