World Economics - Insight , Analysis and Data

World Economics - Measuring the World Economy

The Data Quality Index (DQI)

Updated: 1st February 2017
Which Country’s GDP data you can trust?
(note: most you cannot)

Speed Read
Gross Domestic Product (GDP) data is flawed to an extent few realise. It is used in many ways, without thought as to its accuracy. It is used to apportion funds from international organisations, to direct governmental aid flows, to make corporate investment decisions, to influence rating agency decisions and much more. In short, GDP data is massively influential, but in reality is totally inadequate for the demands made of it. Very few countries achieve the high standards which such important data should meet. It is therefore not a trivial task to find some way to check which countries data can be trusted, and which are seriously misleading.

World Economics has developed a revolutionary way to gauge the quality of individual countries GDP data: the Data Quality Index (DQI). The DQI currently covers 5 factors of importance in determining data quality: base years used, national accounts standards used, size of the informal economy, resources devoted to measuring economic activity, and the intensity of corruption. Currently 154 countries are covered by the Index.

The following sections of this report provide a detailed account of the concept and methodology used to compute the scores for each of the five factors included in the Index. Each factor is given an equal weighting, but these weights can be changed by users to reflect their own views on the importance of individual factors, or for particular uses of the Index.


Download DQI Data Download Full DQI Dataset

 
The Importance of Base Years (or how to raise GDP by 60% overnight)
In November 2010, the Ghana Statistical Service announced that the country’s GDP had risen by 60%. This was not fraudulent use of the statistical service to demonstrate growth, but a sensible step to move Ghana’s GDP data calculations to a more recent base year. The new base year of 2006 allowed the accountants to add data for entire sectors of the economy. For example, in 1993, the mobile phone and the internet had not arrived in Ghana, yet today it is widely accepted that the majority of Ghana’s population are mobile phone and internet users.

The base year estimate is of crucial importance to the accuracy of national income data. Real growth in the activity of an economy is estimated by comparing GDP at current and at constant prices. Constant price estimates use the price relative to a particular year, known as a base year or benchmark year, to weight the volume components of production. But since the structure of production and relative prices over time are dynamic, the pattern of relative prices and the industries surveyed in the base year become less relevant over time. When GDP is revised and the base year is updated, it allows the statistician to reweight the relative importance of the different sectors, and further change or reconsider the methods and data sources. However, many developing economies are currently measured using outdated base years which if updated could lead to a collective upward revision in GDP of 21% (World Economics, 2015).

The United Nations recommends as best current international practice to update base years every five years, although, most OECD countries now adopt the practice of chaining, where relative prices are updated every year. But while chaining allows continual updates to be made to the structure of production and consumption, it also requires considerable expenditure on resources by statistical offices.

The more up-to-date the base year, the higher a country’s score in the World Economics Data Quality Index. For more details see Methodology.

 
The System of National Accounts (things have changed since 1968!)
The System of National Accounts (or SNA) used to compile GDP can have a major influence on the calculation of the size of GDP. Many statistical offices in developing countries are currently using a national income accounting standard, which if updated could add significantly to GDP levels (see World Economics, 2015).

National income measurement is governed by a global standard: the United Nations System of National Accounts (SNA)3. The SNA is the internationally agreed standard set of recommendations on how to compile and measure economic activity4. It was established by the United Nations Statistical Commission (UNSC) to facilitate international comparability of economic statistics.5 The first UNSC guide to the SNA was published in 1953. There have since been three revisions to recommend SNA standards: in addition to SNA 1953, there are also SNA 1968, SNA 1993, and SNA 2008, all approved by UNSC.6

The UNSC formally adopted SNA 2008 in 2009 and some of the recommendations of the new standard have been implemented by various countries. Significant differences between SNA 1993 and SNA 2008, for example, include the treatment of government accounts, capital expenditure, intellectual property and the measurement of the informal sector and illegal activities, areas which are particularly important in developing and emerging-market countries.7

Furthermore, the SNA, like its European counterpart, which is called the “European System of National and Regional Accounting” (ESA) and received official approval in 1996 in a European Council regulation, contains a huge body of mutually consistent concepts, definitions and classifications for measuring economic activity and several economic phenomena. In practice, it is also used as a base of reference for the production of sectoral and territorial economic statistics.8

However, the application of the SNA is not mandatory and the adoption by a country of a standard such as those mentioned above does not imply that all of the recommendations are implemented. If two countries adopt the same SNA standards, this means that their economic statistics are broadly comparable in terms of the definitions used and the accounting methodologies applied. But the longer it takes a country to update its SNA the less reliable the data becomes for economic comparisons to a country with a more recent SNA version.9

The DQI includes a factor for the SNA version in use. Thus, the newer the SNA version, the higher a country’s score. Similarly, the older the SNA, the lower a country’s score. For details of the scoring see Methodology.

 
The Informal Economy (up to 40% of GDP can be unaccounted)
Although attempts have been made to capture the size of the informal sector (see Section 2 above), in poorer countries, a very large swathe of activity can remain uncounted. Even in wealthy countries, informal activities (such as household cleaning remains outside the national accounts). According to Schneider and Williams (2013) the informal economy in poorer countries is typically between 25 and 40 per cent of national income and can represent up to 70 per cent of non-agricultural employment. The existence of such large amount of informal activity is so economically important that to leave it unrecorded in the official national accounts is unsatisfactory. But due to the nature of much of informal work, ranging from housework, farming through to gambling, prostitution, drug dealing, and smuggling, calculations of the value of such activities are extremely difficult.

There is still no consensus on how to define the informal economy.10 Schneider and Williams (2013) argue that one of the broadest definition is: those economic activities and the income derived from them that circumvent or otherwise avoid government regulation, taxation, or observation.11 Nevertheless, measurement of these activities are notoriously difficult as they are deliberately hidden from official transactions. Survey measures, for instance, tend to understate the size of the informal economy because even in the most careful circumstances, people do not like to admit illicit work. Also, the several approaches used in practice tend to give different results.

Even when using quite simple models, it can be difficult to predict whether informal market activities increase with tax. The effect of rising taxes on the extent of informal market activities depends, among other things, on how willing taxpayers are to risk evading tax. As a rule, the willingness to take a chance could be expected to increase with income (and vice versa), simply because the more a person earns, the more he can afford to pay (a fine) if his informal activities are discovered. Moreover, it can be seen that informal economy work is particularly prevalent in certain sectors such as agriculture where social security contributions are not required and formal employment arrangements are not compulsory. It is widely recognised that in developing economies the production standards of agricultural statistics are weak and unreliable mainly because of resource constraints in statistical agencies. Data are frequently subject to political factors.12 For example, policies that one might adopt to tackle the informal economy might be very different in countries where a legal infrastructure is lacking compared with where the legal infrastructure exists.13

The DQI score for this factor is derived from Schneider and Williams (2013) who estimate the size of the informal sector as a percentage of national income for 162 countries from 1999 to 2007. To do this, the latest data available (or the 2007 year) is used for each country. Where data was not available for that year the data available prior to that year is used. The higher the size of the informal economy, the lower a country’s score. Similarly, the lower the size of the informal economy, the higher a country’s score.

 
The Limited Resources of National Statistics Offices (and the tendency of international organisations to invent data)
The quality of national income estimates depends to some extent on the statistical capacity and the resources available to national statistics offices.14 The United Nations System of National Accounts has put a global standard in place but the challenge for a local national statistics office is to provide/produce a measure of the economy. Statistical capacity, or the ability to adhere to the global standard, depends critically on the resources and information available at any given time and place. All other things being equal, there are a priori grounds to believe that poorer economies will have lower-quality statistics. A poorer economy will have relatively fewer available resources to fund the functions/activities of an official/national statistics office.15 The capacity to gather information has been particularly constrained in African states16 because of the relatively higher proportion of the world’s poorer nations that are situated in the continent.17 However, the problem of statistical capacity, in Africa, has been further exacerbated by a loss or distortion of data due to wars, political instability and corruption, while the structure of economies and the nature of property rights means that large unrecorded informal sectors exist. In many African states, the unrecorded informal sector is so large and therefore so economically important that to leave it unrecorded partially invalidates any data produced. For large sectors of such economies there is little or no information and the figures involve a great deal of guesswork.18

Nevertheless, the problem is more serious than the shortcomings that can arise from individual national statistics offices feeding inaccurate data into the public domain – there are also serious inconsistencies in the data on African economies published in international economic databases.19 For example, Alwyn Young (2012) notes that while the popular Penn World Tables purchasing power parity dataset version provided real income estimates for sub-Saharan African countries, in 24 of those countries it did not have any benchmark study of prices.20 In a similar vein, he notes that although the online United Nations National Accounts database provides GDP data in current and constant prices for 47 sub-Saharan countries for each year from 1991 to 2004, the UN Statistical Office, which publishes these figures, had, as of mid-2006, actually received data for only just under half of these 1,410 observations and had, in fact, received no constant price data whatsoever on any year for 15 of the countries for which the complete 1991-2004 online time series are published.21

In an extensive research/study of African economic statistics, Morten Jerven (2011) cites a response by the IMF to one of his enquiries, which summarises the basic problem:

“...there is a need to strengthen source data for national accounts. However, these countries are poor with inadequate revenue to fund regular ongoing data collections for NAS purposes. In general, there is a need to strengthen data collection for agriculture, fishing, informal sector and services activities. There is also scope to improve prices collections (agriculture, producer prices). Compilation staff and resources are also constrained, due to budget, limiting the range and quality of the statistics produced. Existing staff need further training and development. The constraints in compilation are reflected in the limited dissemination.” Jerven (World Economics, 2011:45).22

Statistical capacity and economic resources in national statistics offices therefore matter a great deal in terms of data availability and quality of economic statistics. Data availability is subject to the number of trained staff and the level of resources available for collecting, processing and analysing the data.

The score for this component of the DQI is derived from the United Nations Human Development Index (HDI). We use the HDI as a proxy for assessing the availability of economic resources in national statistics offices. In theory, the larger the resources devoted to statistics offices, the better the quality of statistics. That is, the higher the HDI, the higher the country’s score. Similarly, the lower the HDI, the lower the country’s score. See Methodology.

 
Government Interference in GDP Calculations (lies, damn lies and GDP calculations)
Governments can either manipulate GDP data directly or through the calculation of price indexes such as the GDP deflator. Governments in countries undergoing severe inflation have a long history of hiding the true extent of their inflationary problems which are often reflected not in official data, but in their inability to maintain a stable domestic currency.23 So called “troubled currencies” are associated with elevated rates of inflation, and in some extreme cases, hyperinflation. In many cases, governments fabricate inflation statistics to hide their economic problems. In the extreme, countries simply stop reporting inflation data. As Professor Steve Hanke puts it “official economic data from countries with troubled currencies often amount to nothing more than ‘lying statistics’ and should be treated as such.”24 A current snapshot is shown in the table below, illustrating the gap between black-market exchange-rate data for three “troubled” currencies and the implied inflation rates for each country.

Table: Troubled Currencies: Exchange rates and Inflation
Source: Cato Institute

One example of political intervention in the production of inflation statistics has been witnessed in Argentina. In 2007, the Argentine administration decided to interfere with the calculation of the official Consumer Price Inflation Index (CPI) estimated by the National Statistics Institute (INDEC) and a few months later, the wholesale price index (WPI) was also modified, as well as the official Household and Employment, Manufacturing Survey. This has had a positive upwards bias on GDP and there have been accumulating gaps between the official estimation of inflation and alternative ones.

Argentine GDP are estimated using volume index indicators and a base year of 1993, not by deflating the value of production or value added at current prices by chaining. The extent to which the political intervention has biased GDP upwards has been recorded by Coremberg (2013) who applied the standard SNA 1993 methodology to the main basic series that constitute GDP in Argentina to produce a reproduced ARKLEMS25 series from 1993 to 2012 to compare against official GDP. The main result of this procedure was a series that replicates almost exactly the official Argentine GDP growth from 1993 to 2007. After that year, however, an important gap appears. This gap increases over time as is shown in the Figure below.

Official GDP shows a positive gap of 12.2% in 2012 with respect to reproduced GDP. During the period of political intervention in the production of official statistics, reproduced Argentina GDP grew by 15.9% between 2007 and 2012 (3% annually), a fall in the rate of growth of GDP with respect to the earlier period 2002-2007 of 47% (8.1% annual rate). However, after intervention official figures show a higher, almost doubled rate of GDP growth between the years 2007-2012: 30% (5.3% annual rate).

Figure: Argentina GDP 1993-2012, volume index 1993=100
Source: Coremburg (2014)

Another problem is government corruption which can also infect all parts of an economy and its accurate measurement in systematic ways26. Often a direct result of the government’s concentration of economic or political power, corruption manifests itself in many forms such as bribery, extortion, nepotism, cronyism, patronage, embezzlement, and graft. For example, excessive and redundant government regulations provide opportunities for bribery or graft. In addition, government regulations or restrictions in one area may create informal markets in another. As a result, corruption and informal economy are positively correlated.27 That is, countries with more corruption and bribery have larger informal economies. However, the relationship between corruption and informal economy appears to rely on the national income levels as well as the effectiveness of the legal system.28

The score for this component of the DQI is derived directly from Transparency International’s Corruption Perceptions Index (CPI), which measures the level of perceived corruption in 175 countries. The CPI score is based on a 100-point scale in which a score of 100 indicates very little corruption and a score of 0 indicates a very corrupt government. That is, the lower the level of corruption, the higher a country’s score. Similarly, the higher the level of corruption, the lower a country’s score.

 
Measurement of Government (measuring the unmeasurable) 
Most of the output of the public sector is not sold at market prices which causes a measurement problem. Traditionally, only inputs to the government sector such as salaries were reflected in the National Accounts. This treatment meant that the output of the government sector in terms of defence, welfare, education, and health etc. was measured by final expenditure which by definition imposed zero productivity growth for the sector. There have been a number of improvements undertaken over time within the OECD-Eurostat countries to move towards output based measures of government output, but these are not universal and have been made on a country by country basis. Most other regions use compensation based measures of government output making intercountry comparisons almost meaningless. This problem is recognised, if not solved by the World Bank’s rolling Intercountry Comparison Programme (ICP):

“The input approach was used in 2005 based on government salaries for a number of occupations. Because of quality differences, productivity adjustments were made in Asia, Africa, and Western Asia. Regional linking factors were computed from compensation data from 75 countries representing all regions including Eurostat-OECD. The linking factors were computed without adjustments for productivity and independently of the regional PPPs.”


 
Measurement of Financial Sector Output (or what exactly is the output of a bank!)
Measuring the financial sector is a well-known difficult problem in the national accounts.29 According to Andrew Haldane and co-authors, “the recent history of banking appears to be as much mirage as miracle, which seems to affect all countries’ GDP."30 One study of the United States concludes: “Making conservative assumptions, we show that the current official method overestimates the service output of the commercial banking industry by at least 21% between 1997 and 2007.”31 Also, a 2010 study by Colangelo and Inklaar suggests that for the Eurozone, adjusting for banks’ risk-taking would reduce the measured output of the financial sector by 25-40 percent. If the same factor were applied in the United Kingdom, the measured contribution of the financial sector would have been 6-7.5 percent of GDP in 2008, rather than 9 percent claimed.32 These figures are staggering: the size of the financial sector in recent years has been overstated by at least one-fifth, maybe even by as much as one-half.

 
Methodology
The World Economics’ Data Quality Index, on a 0 to 100 scale, measures the quality and reliability of national income data (or GDP estimates) in 154 countries. The DQI index by calculated by weighting the five normalised indicators which include base years, the SNA, the informal economy, lack of statistical resources, and corruption (these default weights can be changed to reflect views on the importance of individual factors). In addition, the DQI is rounded off to one decimal place.

Minimum and maximum values are calculated in order to transform the indicators expressed in different units into indices between 0 and 1 from which component indicators are standardised.

Base Year: The more up-to-date the base year, the higher a country’s score in the World Economics Data Quality Index. The base year score for each country is a number between 0 and 100, with 100 indicating that a country is using a chaining system, where base years/(or relative prices) are updated every year). Information on individual country’s base year is taken from the World Bank’s World Development Indicators (WDI) Metadata, IMF World Economic Outlook Report, United Nations and National Statistics Offices.


System of National Accounts (SNA): The newer the SNA version, the higher a country’s score. Similarly, the older the SNA, the lower a country’s score. The score for the SNA component is based on a 0 to 100 scale, with 100 points given to countries using the latest SNA version. Information on individual country’s SNA is taken from the World Bank’s World Development Indicators (WDI) Metadata, IMF World Economic Outlook Report, United Nations and National Statistics Offices.


The Informal Economy: The DQI score for this factor is derived from Schneider and Williams (2013) who estimate the size of the informal sector as a percentage of national income for 162 countries from 1999 to 2007. To do this, the latest data available (or the 2007 year) is used for each country. Where data was not available for that year the data available prior to that year is used. The higher the size of the informal economy, the lower a country’s score. Similarly, the lower the size of the informal economy, the higher a country’s score. A score of 100 means that a country has the lowest rate of informal sector as percentage of GDP.


Quality of statistics: We use the United Nations Human Development Index (HDI) as a proxy for assessing the availability of economic resources in national statistics offices. In theory, the larger the resources devoted to statistics offices, the better the quality of statistics. The HDI covers 188 countries. The latest data available (or the 2014 year) is used for each country. The HDI scores range from 0.34 (minimum) to 0.94 (maximum). That is, the higher the HDI, the higher the country’s score. Similarly, the lower the HDI, the lower the country’s score.

Corruption: The score for this component of the DQI is derived directly from Transparency International’s Corruption Perceptions Index (CPI), which measures the level of perceived corruption in 175 countries. The CPI score is based on a 100-point scale in which a score of 100 indicates very little corruption and a score of 0 indicates a very corrupt government. That is, the lower the level of corruption, the higher a country’s score. Similarly, the higher the level of corruption, the lower a country’s score.



  1. Ghana Statistical Service (2010) Information Paper on Economic Statistics: Rebasing of Ghana’s National Accounts to Reference Year 2006, 10 November. Available online 
  2. See: http://www.worldeconomics.com/Papers/GDPEUR_64ec7254-de22-472d-838c-d17489391707.paper 
  3. See Jerven (2013:9-10). 
  4. See: http://unstats.un.org/unsd/nationalaccount/sna.asp 
  5. See: http://www.worldeconomics.com/Papers/GDPAmericas_7fbab7d6-44fd-4f40-b4bb-233fb5aeaab5.paper 
  6. See Jerven (2013:9-10). 
  7. See: http://www.worldeconomics.com/Papers/GDPAmericas_7fbab7d6-44fd-4f40-b4bb-233fb5aeaab5.paper 
  8. See: http://www.oecd.org/std/41746745.pdf 
  9. See: http://www.worldeconomics.com/Papers/GDPAmericas_7fbab7d6-44fd-4f40-b4bb-233fb5aeaab5.paper 
  10. See: http://www.africaresearchinstitute.org/blog/africas-gdp-conundrum-in-conversation-with-morten-jerven-by-jonathan-bhalla/ 
  11. These include illegal activities such as drug dealing and manufacturing, prostitution, gambling, smuggling, fraud, human trafficking and weapon trafficking. Also, activities within family household such as tax evasion through unreported income from self-employment, wages, and assets from unreported work related to legal services and goods. 
  12. See: http://www.worldeconomics.com/Papers/Agricultural_96b727d6-fdf0-468e-b937-13680d2f7d6d.paper 
  13. Schneider and Williams (2013: 60-61). 
  14. See: http://www.worldeconomics.com/Papers/AfricaGDP_2c4addf3-b795-44f2-8d30-23b9e22f284e.paper 
  15. See Jerven (2013:2). 
  16. See Jerven (2013:3). 
  17. See: http://www.worldeconomics.com/Papers/AfricaGDP_2c4addf3-b795-44f2-8d30-23b9e22f284e.paper 
  18. See Jerven (2013:12-13). 
  19. See: http://www.worldeconomics.com/Papers/AfricaGDP_2c4addf3-b795-44f2-8d30-23b9e22f284e.paper 
  20. The surveys needed to collect these information are carried out infrequently. 
  21. See Young (2012). 
  22. See: http://www.sfu.ca/~mjerven/PDFs/WEC%2012(4)%20Jerven.pdf or http://www.worldeconomics.com/Papers/Billion_f4168a63-84c1-4bb8-b22b-a2291d757e7a.paper 
  23. See: http://www.cato.org/research/troubled-currencies 
  24. See http://www.cato.org/research/troubled-currencies 
  25. Specifically Argentina KLEMS where KLEMS (K-capital, L-labour, E-energy, M-materials, and S-purchased services) refers to broad categories of intermediate inputs that are consumed by industries in their production of goods and services. 
  26. See:http://www.heritage.org/index/book/chapter-1 
  27. See Hibbs and Piculescu (2005) 
  28. See: http://www.amazon.com/New-Paradigm-International-Business-Proceedings/dp/9812874984 
  29. Coyle (2014:98). 
  30. See: https://harr123et.files.wordpress.com/2010/07/futureoffinance5.pdf 
  31. See: http://www.bostonfed.org/economic/wp/wp2008/wp0804.pdf 
  32. See: https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp1204.pdf?329e15dbdef0f905a688a2df0a84ed23 





© Copyright World Economics Ltd. 2017