Understanding Commercial Property Price Indexes

Mick Silver - October 2013

Speed Read
  • Constructing commercial property price indexes is problematic since properties within each sector such as offices, retail, industrial are heterogeneous and transactions are irregular.
  • Appraisal-based indexes avoid the problem of limited transaction data, but they provide unduly smoothed and lagged representation of commercial property price inflation.
  • An international effort is under-way in the USA and Europe to collate and compare data on real estate price indexes for use in economic policy, particularly financial stability.
  • Major concern is that existing indexes will be used in country and multilateral country analysis with limited understanding of their deficiencies.

The compilation of commercial property price indexes (CPPIs) is highly problematic. Commercial properties include offices, retail, industrial and residential (owned or developed for commercial purposes) properties. Even within each of these categories properties are heterogeneous and transactions irregular thus complicating comparisons of average transaction prices for a fixed-quality bundle of properties over time – the type of methodology that would be applied in compiling a consumer price index. Even where matched (repeat) transactions can be used, the population of properties sold more than once in the period of the index can be very limiting and unrepresentative of the total population of commercial properties. Further, between any two transactions for the same property, the ‘quality’ of the property may change and mechanisms to remove the effects of capital renovations and include depreciation of the properties are often limited. The problem of limited transactions on heterogeneous properties is worse when measurement really matters, as we go into and during recessions.1

The provision of accurate CPPIs is recognised as important for economic analysis, monetary policy, financial stability and prudential supervision. The International monetary Fund (ImF)/Financial stability Board (FsB) G20 data Gaps Initiative (dGI), endorsed by the G20 Finance ministers and Central Bank Governors, and the ImF’s International monetary and Financial Committee (ImFC), includes CPPIs in its 20 recommendations on data gaps (see Heath 2013). The importance of CPPIs as an economic indicator, as summarised by Kanutin (2013), is to assess and undertake analysis of:

  • financial stability/soundness
  • mortgage lenders’ exposure to risk and risk management by mortgage providers
  • inflation pressure for macroeconomic analysis
  • developments of the commercial property market and its connection to the construction sector (as an indicator of economic activity)
  • the relation of commercial property prices to fundamentals, including the detection of bubbles, length and duration of downswings in economic cycles, and
  • use as a deflator in national accounts, specifically in relation to measures of changes in the non-financial corporations sector’s wealth.

Kanutin (2013) also outlines work by the European Central Bank’s (ECB) Working Group on General Economic statistics (WGGEs) including tentative results for experimental quarterly indicators for CPPIs for the EU, the Euro area and 13 individual countries. These indicators are based on valuation data supplied by a commercial data provider ‘enhanced’ using estimates of transaction prices, based on the relationship between transaction prices and appraisals. Initial results are expected to be ready for publication as ‘experimental statistics’ at the end of 2013, an important milestone. Yet the resulting estimated CPPIs nevertheless are deemed experimental, and further development is both necessary and planned. such is the challenge and caution required in this most difficult area of measuring price changes using limited transaction prices on heterogeneous properties.

An alternative approach to compiling CPPIs that avoids the problem of limited transaction data is appraisal-based indexes, also referred to as ‘valuation indexes’. These indexes measure changes over time in the sum of appraisals (by experts) of the value of defined holdings of properties. In principle, appraisals are undertaken each period on well-defined groups of properties, thus effectively allowing comparisons over time to be made of the sums of the valuations of matched properties. such regular revaluations of properties may be a requirement of membership of a ‘trade’ association. The reliability of the index would naturally reflect the reliability of the appraisal system and its implementation. In practice, valuations may take place once a year with quarterly indexes compiled from interpolated valuations or valuations undertaken by the holding property company, as opposed to an outside expert. Well-documented and undesirable features of appraisal-based CPPIs include their tendency to provide an unduly smoothed and lagged representation of commercial property price inflation (Clayton et al. 2001; Geltner et al. 2003; Devaney & Diaz 2011).

Using published information on their methodology, we consider in the following two sections these two main types of CPPI, appraisal-based and transaction-based indexes in the context of two such indexes compiled for the United states (Us): the national Council of Real Estate Investment Fiduciaries (nCREIF) Property Index and the Real Capital Analytics (RCA). Following this, we look at such indexes for Europe and, in the final section, draw conclusions.

Appraisal-based indexes: issues and an example from the United States
An appraisal-based commercial property index compares valuations in one period with another; it is the ratio of the aggregate (usually sum) of appraised values between the two periods. Two important limitations with appraisal-based indexes are their dampening or smoothing of market price volatility and the tendency of appraisals to lag market prices. Further, not all properties are regularly or properly reappraised every period; for example, properties may be reappraised only annually, with quarterly data either interpolated or estimated by the responding property companies, as opposed to external experts. Respondents have a tendency to repeat or marginally update the last ‘authorised’ valuation; Geltner and Fisher (2007, p. 3) refer to this as a ‘stale-appraisal effect’ that adds further lag to the index. There is also an implicit assumption that the mix of properties remains constant between the comparisons and, as discussed below, this may not generally be the case.

A major appraisal-based CPPI series used in the Us is the national Council of Real Estate Investment Fiduciaries (nCREIF) Property Index (NPI). It is a quarterly time series of the total rate of return of individual commercial real estate properties acquired in the private market for investment purposes.2 The index covers office, retail, industrial and apartment/hotel properties. The total return of a property includes both net operating income and the capital return, as if properties were purchased at the beginning of the quarter and sold at the end of the quarter with the investor receiving all net cash flows during the quarter. At the end of each quarter, (nCREIF data contributing-)members submit their appraisal of their properties’ fair market values and net operating income (noI) from which the aggregate NPI is compiled. The appraisals are subjective valuations made by an appraisal firm or the manager/owner of the property.

Since all properties in the NPI have been acquired, at least in part, on behalf of institutional investors – the great majority being pension funds – there is a potential sample selectivity bias because pension funds favour trading in larger properties. The coverage for 2012 quarter 4 was 7,270 properties with a total market value of $319.95billion. Giventhat Florance et al. (2010) estimated the 2009 total value of commercial real estate as close to $9 trillion, the coverage can be regarded as limited, although it may be representative of changes. Geltner and Fisher (2007, endnote 9) note that:

in 2006 the NPI included less than $30 billion of property sales, whereas the Real Capital Analytics Inc. (RCA) database recorded over $300 billion of commercial property sales tracking only sales greater than $2.5 million. As of the end of 2006, the NPI consisted of approximately 5,000 properties worth a total of about $250 billion, whereas JP Morgan Asset Management’s Real Estate Universe report estimated the total value of US commercial real estate at that time to be some $6.7 trillion, or over 25 times the nCREIF population value although this included corporate or non-traded real estate and small ‘mom and pop’ properties as well as the larger properties tracked by the RCA database.

NPI measures its index as a ratio of the sums of valuations (prices) of included properties between the current and prior quarters – a Dutot index.3 The overall change between two successive months forms a link and, by further successive multiplication, the links form a (chained) index. There are two issues that merit noting for this calculation. First, the numbers of properties in the NPI change as properties are bought and sold, and new contributors are added. The NPI does not appear to form short-term relatives as Dutot indexes of the prices of matched properties between the periods compared. It is important to the index that changes in an aggregate of valuations of matched properties be used; otherwise changes in the quality mix of the sample might be measured as price changes. If, for example, a sample of 2,000 were used in may 2013 for a property group and similarly for June, but in July 50 properties were dropped, the May to June comparison should be based on matched samples of 2,000 and the June to July comparison on matched samples of 1,950. The issue is not insurmountable. Valuation indexes using Investment Property databank (IPD) by the European Central Bank:

maintain a constant coverage for five consecutive quarters. This allows year to year percentage change series to be calculated which have the same underlying assets included in the calculation and thus ensuring that movements in the index in the analysis period are due to price movements rather than asset level shifts [due to divestment or new portfolios joining the sample]. (Kanutin 2013, p. 8)

Second, simple algebra4 shows valuation changes to be implicitly weighted by the relative valuation in the reference month of each binary monthly comparison, i.e. link. since the index is made up of chained links, the (albeit reference month) valuation or weight adjusts with the index. larger properties in terms of relative reference-month valuation have a greater impact on the index than smaller properties. If this is considered undesirable, the chaining gives a cumulative drift to the index.

some mention should be made of the heterogeneity of commercial property. As noted above, we include quite disparate buildings in different locations across the Us, serving quite different needs, including industrial, office, apartments and retail properties. Retail properties, for example, are quite diverse; nCREIF further categorises retail properties by region and type for each region, the typology including the following.

Neighborhood Center (RN): provides for the sale of daily living needs of the immediate area. Typical area is 30,000 to 150,000 square feet with at least one anchor tenant.

Community Center (RC): in addition to convenience goods, provides for the sale of goods such as apparel or furniture. Typical area is 100,000 to 350,000 square feet with two or more anchor tenants.

Regional Mall (RR): provides a variety of goods comparable to those of a central business district in a small city, including general merchandise, apparel and home furnishings, as well as a variety of services and perhaps recreational facilities. Two or more full-line department stores anchor a total area of 400,000 to 800,000 square feet.

Super-Regional Mall (RS): provides an extensive variety of shopping goods comparable to those of the central business district of a major metropolitan area. The anchors are three or more full-line department stores, with total area in excess of 800,000 square feet.

Fashion/Specialty Center (RF): typically 80,000 to 250,000 square feet with no dominant anchors, consisting of higher-end fashion oriented tenants.

Power Center (RP): typically 250,000 to 600,000 square feet with three or more anchor stores that occupy 75–90% of the total area. Anchor stores are ‘category-dominant’ home improvement stores, discount department stores, warehouse clubs and off-price stores.

Theme/Festival Center (RE): anchored by restaurants or other entertainment facilities, and oriented toward leisure and tourist-oriented goods and services. Typically 80,000 to 250,000 square feet.

Outlet Center (RO): typically 50,000 to 400,000 square feet of manufacturer’s outlet stores, with a primary trade area radius of 25 to 75 miles.

Single-Tenant (RT): freestanding.

Naturally there will heterogeneity within each type within and across regions/states of the US.

Transaction-based indexes have the advantage of being based on actual market prices as opposed to subjective appraisals. However, transaction based CPPIs can suffer from small sample sizes, especially when it matters, at times of economic downturn. Further, there is the need for quality-mix adjustments, as the sample of transactions in one period are of a different quality than the next. The two principal methods used to control for such quality-mix changes are the repeat-sales method and hedonic regression equations, though see Holmgaard (2013) for an application of the sales Price Appraisal Ratio (sPAR) in compiling commercial property price indexes for Denmark, and Shimizu et al. (2013) for estimates of hedonic quality-adjusted commercial property price indexes using real estate investment trust (REIT) data for Tokyo.

Transaction-based indexes: issues and an example from the United States

A major transaction-based CPPI for the Us is produced by Real Capital Analytics (RCA CPPI).5 It extends from 2000 to end of 2012 and focuses on relatively high-value transactions. RCA data from 2000 covered transactions of $5 million plus, but in 2005 this increased to $2.5 million plus.

The RCA CPPI uses a repeat sales regression thus restricting the sample to properties that have had more than one transaction during the period. For each quarter, data are collected on sales and if a record of an earlier transaction for the property is identified, the two transactions are paired and treated as a repeat sale. A repeat sales price index is estimated by including data on the prices of transaction pairs in a regression on dummy variables on the time of each sale (Shiller 1991). By limiting the sample to price comparisons of transactions-pairs of the same properties, some control is established over changes in the quality mix of transactions. The primary disadvantages are: (1) the quality of a repeat purchase may depreciate, with wear and tear, or appreciate, with renovations; (2) there is increased sampling error due to relatively small sample sizes and potential sample selectivity bias – properties not sold or new properties sold once are excluded; the sample would comprise an unduly higher proportion of atypical houses sold more frequently and exclude atypical houses sold less (see mason and Pryce 2011); (3) there are implications for the estimator due to an asymmetric and positive variance of the error term in the repeat sales regression; longer gaps between sales may require less weight; alternative assumptions regarding this relationship can have a major impact on the index (Leventis 2008); and (4) as new transaction pairs become available with the addition of new historical data, the index may be subject to a volatile revision history.

Three relatively major modifications to the basic repeat sales methodology have been applied by RCA (details in Geltner & Pollakowski 2007). First, as widely used in repeat sales property indexes, is the use of a weighted least squares estimator that gives less weight to transaction price comparisons over longer time periods, thus correcting for Heteroskedasticity related to the length of the period of the comparison. second, ridge regression is applied to results that have near zero first-order autocorrelation; this is because negative serial correlation reflects excess short-term volatility and negative serial correlation, and positive serial correlation reflects a lag and dampened volatility, neither of which are desirable. Third, a time-weighted dummy variable specification is used for the period of comparison. Usually this is a zero-one dummy variable, but in this case it is the proportion of the period during which the property was held by the investor.

The RCA CPPI database benefits from a number of filters, including minimum time periods between sales to exclude ‘flipped’ properties, portfolio transactions, excessively old data, data with incomplete information, built before first sale and extreme returns, and to ensure comparability in usage and size. Appraisal indexes may benefit from reported data on capital improvements, unlike repeat-sales indexes, though the data filters may remove properties that have undergone major improvements.

An alternative major transaction-based CPPI in the Us is compiled by Costar. It applies equal weight to each property and uses the repeat sales method to control for quality-mix changes over time. After the filtering process, the final dataset had a total of 85,428 repeat-sale observations covering the period 1996–2010.6

A major concern with transaction-based indexes is their limited sample size compared to appraisal-based indexes, and even more so in the lead up to, and during, recessions. In any period, in principle, an appraisal can be conducted on every commercial property. However, the sample size of a CPPI is limited to those properties transacted in that period and, for the repeat-sales approach, the sample is further constrained to properties that have had more than one sale during the sample period and, further, not been deleted by the aforementioned filters. on the other hand, a transaction index has the virtue of being based on actual market prices and has none of the smoothing and lagged properties of appraisal-based indexes.

While CPPIs are disseminated for the overall commercial property market, it may be useful for economic analysis to examine what may be quite different changes in sub-markets of the index. RCA CPPI series can be distinguished by four types of transaction (apartment, industrial, office and retail) in specified geographical areas (by six regions, 20 states and 35 cities). This benefit of more homogeneous comparisons for these sub-indices, arising from the use of more granular data, will be at a cost of very small sample sizes at these detailed levels.

As noted above, econometric analysis has established the lagging and smoothing feature of appraisal-based indexes against transaction-based indexes. However, as demonstrated by Figure 1, while trends and turning points for the two types of series are similar, appraisal-based indexes show Us commercial property price inflation to be markedly higher over the period 2000Q1 to 2012Q4.

And in Europe...

The issues in the compilation of, and data sources for, European CPPIs are well reported by Kanutin (2013) in the context of meeting European Union’s policymakers’ data needs in this area; much of the following is based on this account. The ECB undertook in may 2010 a stocktaking exercise aimed at examining what data on commercial property prices were available in each of the EU member states. data sources were invariably private companies whose measures included 41 data sources across 20 countries – however, there was much variability, with only ten countries having data on ‘total return’ and eight countries on ‘capital value growth’. For each measure, the manner in which the valuations were undertaken varied between countries.

The ‘total return’ is calculated as the change in capital value, less any capital expenditure incurred, plus net income, expressed as a percentage of capital employed over the period concerned. Total value is then arrived at by estimating the value of future income (i.e. using discounted cash flows for the rental income).

A valid concern in this methodology is that the future cash flow projections and yields used for discounting are not harmonised across markets and are instead chosen by individual valuers. nonetheless, within a market segment it is not believed that the approach taken within a particular country would deviate significantly. (Kanutin 2013, p. 8, ff.11)

But of course, between sector/country comparisons, used for multilateral country economic analysis, may deviate significantly.

As part of the stocktaking exercise, end-users were asked their views as to their needs; the relatively uniform response was commercial property price index based on transaction prices; valuation indexes were, as noted by Kanutin (2013, p. 4), ‘a second best option’.

As an interim approach, the ECB has built a data set of valuation indexes based on valuation data from the IPD, though in a relatively limited number of cases where ‘better’ data existed in a country, this has been used to replace or complement the IPD data. IPD data cover the retail, office, industrial and residential (apartment) properties, wherever they are held in professionally managed portfolios. IPD national quarterly valuation series are currently available to the ECB for Austria, Belgium, Denmark, France, Germany, Ireland, Italy, the Netherlands, Norway, Poland, Portugal, Spain, Sweden, Switzerland and the UK. The longer-term approach is that the European statistical system compiles CPPIs.

Limitations do exist in the valuations used. of particular concern to the compilation of a quarterly valuation-based indicator is that, of the 17 countries for which IPD data are used, property valuations for 12 are on an annual basis, and for the remaining countries, on some mix of monthly, quarterly, bi-annual and annual periodicities; only valuations for Ireland are just quarterly. linear interpolations based on year-end valuations are used to generate quarterly valuation series.7 such interpolations carry over to the ‘transaction-linked’ method; though see Picchetti (2013) for an outline and examination of the issue using Brazilian data.

The ECB distinguish two approaches to estimate national CPPIs: IPD valuation indexes and a ‘transaction linked’ method. The latter is the result of regressing the log of actual transaction prices for traded properties, when available, on the appraised capital value for the preceding two quarters, and dummy variables on country and property type. The estimated coefficients are used to generate predicted transaction prices for non-traded properties. The results of this hybrid valuation/transaction procedure are used to compile CPPIs for each country by property category type. The idea of utilising valuation data to improve the limited coverage of transactions is a useful concept (see also the Handbook on Residential Property Price Indexes).8 However, at its root are transaction data that, especially in recessions, may dry up. Kanutin (2013, p. 11, ff.12) draws attention to only four recorded IPD commercial property transactions for Switzerland in 2011Q4, well below its historical level. The prediction intervals for the transaction prices based, for non-traded properties, on a model using this data would naturally be very wide.


The importance of commercial property price inflation to economic analysis was outlined above and is evidenced by their inclusion in the G20 dGI. A major concern is that valuation indexes and CPPIs will be used in country and multilateral country analysis with limited understanding of their deficiencies.

Following on from this, the very real problems in their measurement were also discussed, using the Us and ECB’s initiative for European countries, and aggregates thereof, for illustration. The response to the measurement problem is encouraging: a recent active body of research has emerged from work by national statistical offices, international organisations and the academic community, on practical methods to deal with the issues raised in this article. most important is the cooperation between data providers and analysts; much of the research work undertaken is on commercial property valuation and price index series provided by the mainly private-sector organisations, such as IPD, nCREIF and RCA. Research and international standards on CPPIs have been promoted by the dGI, in part under the aegis of the IWGPs. notable achievements include: (1) an international conference in may 2012, hosted by the ECB and series of papers outlining principles, practice and ways forward (http://www.ecb. europa.eu/events/conferences/html/20120511_cppi.en.html); (2) the current preparation of an international Handbook on CPPI measurement, with Eurostat as the lead agency (outlined in Passerini 2013); (3) a separate session on CPPIs at the meeting of the United nations International Working Group on Price Indexes, the OttowaGroup, in may 2013 (http://www.dst.dk/da/sites/ottawa-group/agenda.aspx); (4) a forthcoming special topic session at the August 2013 World Congress of statistics; (5) the hands-on development of such series for EU countries by the ECB, as outlined in the previous section; and (6) cooperation between private data providers and researchers and statistical offices. There may be no easy solution, but it will not be for lack of serious research that can, at the very least, identify and surmount some methodological pitfalls and confront others with eyes wide open.


Acknowledgements are due to Jeff Fisher (RCA), David Geltner (MIT), Brian Graf (ImF) and Kim Zieschang (ImF) for helpful advice. *The views expressed in this article belong solely to the authors. nothing contained in this article should be reported as representing ImF policy or the views of the ImF, its Executive Board, member governments, or any other entity mentioned herein.


Clayton, J., Geltner, D. & Hamilton, S.W. (2001) Smoothing in commercial property valuations: evidence from individual appraisals. Real Estate Economics, 29, 3, pp. 337– 360.

Devaney, s. & Diaz, R.m. (2011) Transaction based indices for the UK commercial real estate market: an exploration using IPD transaction data. Journal of Property Research, 28, 4, pp. 269–289.

Florance, A.C., Miller, N.G., Spivey, J. & Peng, R. (2010) Slicing, dicing, and scoping the size of the Us commercial real estate market. Journal of Real Estate Portfolio Management, 16, 2, may–August, pp. 101–118.

Geltner, D. & Fisher, J. (2007) Pricing and index considerations in commercial real estate derivatives. Journal of Portfolio Management, 33, 5, pp. 99–118.

Geltner, d. & Pollakowski, H. (2007) A set of indexes for trading commercial real estate based on the real capital analytics transaction prices database. MIT Center for Real Estate working paper, release 2.

Geltner, d., MacGregor, B.d. & schwann, G.m. (2003) Appraisal smoothing and price discovery in real estate markets. Urban Studies, 40, 5/6, pp. 1047–1064.

Heath, R. (2013) Why are the G20 data Gaps Initiative and the sdds Plus relevant for financial stability analysis? ImF working paper series, WP/13/6.

Holmgaard, J. (2013) Commercial property price indices in Denmark. Paper presented at the 13th OttowaGroup meeting, 1–3 may, Copenhagen, Denmark.
Available online at: http://www.dst.dk/da/sites/ottawa-group/agenda.aspx (accessed 18 July 2013).

Kanutin, A. (2013) ECB progress towards a European commercial property price index. Paper presented at the 13th OttowaGroup meeting, 1–3 may, Copenhagen, Denmark. Available online at: http://www.dst.dk/da/sites/ottawa-group/agenda.aspx (accessed 18 July 2013).

Leventis, A. (2008) Revisiting the differences between the oFHEo and s&P/ Case-shiller house price indexes: new explanations. office of Federal Housing Enterprise Oversight, January. Available online at: http://www.fhfa.gov/webfiles/1163/ oFHEosPCs12008.pdf.

Mason, P. & Pryce, G. (2011) Controlling for transactions bias in regional house price indices. Housing Studies, 26, 5, 1 July, pp. 639–660.

Passerini, P. (2013) developing commercial property price indicators. Paper presented at the 13th OttowaGroup meeting, 1–3 may, Copenhagen, Denmark. Available online at: http://www.dst.dk/da/Sites/ottawa-group/agenda.aspx (accessed 18 July 2013).

Picchetti, P. (2013) Estimating and smoothing appraisal-based commercial real-estate performance indexes. Paper presented at the 13th OttowaGroup meeting, 1–3 may, Copenhagen, Denmark. Available online at: http://www.dst.dk/da/sites/ottawa-group/ agenda.aspx (accessed 18 July 2013).

Shiller, R.J. (1991) Arithmetic repeat sales price estimators. Journal Housing Economics, 1, 1, pp. 110–126.

Shimizu, C., Diewert, W.E., Nishimura, K. & Watanabe, T. (2013) Commercial property price indexes for Tokyo: estimating quality adjusted commercial property price indexes using J-REIT Data. Paper presented at the 13th OttowaGroup meeting, 1–3 may, Copenhagen, Denmark. Available online at: http://www.dst.dk/da/ Sites/ottawa-group/agenda.aspx (accessed 18 July 2013).

Silver, m. (2012) Why house price indexes differ: measurement and analysis. ImF working paper WP/12/125.

Silver, m. & Heravi, s. (2007) Why elementary price index number formulas differ: price dispersion and product heterogeneity. Journal of Econometrics, 140, 2, pp. 874– 883.

1. Similar problems arise with residential property price indexes (RPPIs) (silver 2012), though even during recessions the number of transactions remains reasonable for estimation purposes and the heterogeneity is not as marked as with commercial property.

2 . Detailed information is given in the FAQs on nCREIF’s website: http://www.ncreif.org/faqsproperty.aspx.

3. The properties of this index are given in silver and Heravi (2007) and contrasted with a (superior) ratio of geometric means – a Jevons index.

4. For Vit denoting the valuation of the ith property in period t, the valuation ratio, for example, for the April to may link of the index is:

5. The RCA CPPI is advertised at: https://www.rcanalytics.com/Public/rca_cppi.aspx; for details on methodology see: presentation by david Geltner under ‘mIT/CRE Historical development of CPPI’ at http://web.mit.edu/ cre/research/credl/. subsets of 20 national level indexes are co-branded with moody’s Investors service as the moody’s/RCA CPPI. details of the methodology for this subset and, thus the more granular RCA CPPI, are given in Geltner and Pollakowski (2007).

6. Costar Commercial Repeat-sale Indices: methodology. Available at: http://www.costar.com/uploadedFiles/ About_Costar/CCRsI/articles/pdfs/CCRsI-methodology.pdf

7. Kanutin (2013), on which this is based, makes no mention of an extrapolation procedure required for generating timely indexes.

8. Available online at: http://epp.eurostat.ec.europa.eu/portal/page/portal/hicp/methodology/hps/rppi_handbook.