Auditing Automated Valuation Model Case multivariate regression model

✒ Auditing Automated Valuation Model Case: multivariate regression model for residential property Auditing automated valuation model – case: multivariate regression model for residential property 1

Authors ✒ Auditing Automated Valuation Model Seppo Koponen Jamie Donovan Realia Management Oy Vice President MRICS, Hyp. Zert(MLV) AKA, KHK, LKV Realia Management Oy Analyst Valuation Advisory Authors Mikael Postila Orava Funds Plc Head of Research, Investment Analysis & Tenant Services Kauko Viitanen Aalto University Professor, Head of the Department, Real Estate Economics and Valuation, Department of Real Estate, Planning and Geoinformatics FRICS Auditing automated valuation model – case: multivariate regression model for residential property 2

Realia Management Oy ✒ A part of Realia Group Auditing automated valuation model – case: multivariate regression model for residential property 3

Realia Management Oy ✒ A part of Realia Group Valuation Advisory Valuation services, Tailored advisory and consulting support services, Market research services ❞ Realia Group is Nordic’s largest management and consulting group specialized in residential and commercial property transactions and in managing residential and commercial assets ❞ Auditing automated valuation model – case: multivariate regression model for residential property 4

➢ The case study • Explorative case study • Introspective • We emphasise process description, exploring ways to improve and pinpoint issues we have encountered. Academic Positive/Descriptive Objective Subjective Normative Practice Auditing automated valuation model – case: multivariate regression model for residential property We are not in a position to give an objective, qualitative assesment of the success of the case 5

Description of Orava Residential REIT in-house AVM Purpose • • • Monthly valuation of portfolio for board meetings Valuation for quarterly interim and end of year financial statement Valuation of proposed investments Regulation • • • Reasoning behind in-house model • • Timeliness: all property assets can be valued at any given moment, without waiting external valuation. Company board should be aware of developments at least monthly. Cost effectiness: assume a REIT has 100 properties, which are valued monthly – update of valuation á € 1000 12*100*€ 1000 = 1, 2 M€ !!!. In-house valuation team of 1 -2 people is sufficiently large for this. AVM objectivity According to IFRS a company must value its assets at fair value. According to rather new IFRS standards company management must assess fair value – i. e. • mgmt must make its own assessment even though an external valuer (IVS, RICS, …) provides some Verification calculation. • Values are compared semi-annually against desktop Finnish law does not require external valuation of chartered valuers (AKA/KHK) Finnish FSA and CPA (at the moment Pw. C) have • Valuation methodology is audited annually by approved approach external chartered valuer Auditing automated valuation model – case: multivariate regression model for residential property 6

Description of Orava Residential REIT in-house AVM Methodology • • • Specification Multivariate hedonic regression / OLS chosen since it 1) can be easily explained to investors and board members, 2) is simple, no need for high level econometric expertise, and 3) can be verified and possible mistakes in estimation / data validation process are easy to find Problems: data related issues are difficult to eliminate using regular OLS (better, but more difficult methods include: GLS, Kalman filter, GWR, neural networks, …) Separate model done for every property Estimation and value calculation duplicated (-> spotting human errors) • • Model specification changed only if a significant estimation/data issue is found Changes are tested for the whole portfolio before applying Same universal specification is used for every property Specification should be 1) theoretically sound, 2) fit data (yield reasonable valuations compared to external valuation) and 3) minimise s. e. of regression Data • Asking prices (www. oikotie. fi –portal); correction term calculated quarterly using Statistics Finland transaction price statistics Auditing automated valuation model – case: multivariate regression model for residential property 7

? What is an audit? Is this an audit? • A second opinion. However, audit checks against audit criteria – a more objective and systematic approach to assurance. • “systematic, independent and documented process for obtaining audit evidence (3. 3) and evaluating it objectively to determine the extent to which the audit criteria (3. 2) are fulfilled” (ISO 19011: 2011). • According to ISA 500 and ISA 620, the AVM auditor can function as the reporting entity’s expert or the primary (financial statement) auditor’s expert. (International Standards on Auditing) AVM valuations AVM Auditor Audits of Financial Statements Auditing automated valuation model – case: multivariate regression model for residential property Public 8

Piecing standards and guidelines together ISO (International Organization for Standardization) IAAO IVS (International Association of Assessing Officers) (International Valuation Standards) IAASB RICS IFRS (International Auditing and Assurance Standards Board) ✒ Auditing framework Publications (draft) (International Financial Reporting Standards) (IVS) AVM standards A good overview of applicable standards Auditing automated valuation model – case: multivariate regression model for residential property 9

Piecing standards and guidelines together ✒ Access to source material Knowledge of regression methodology Established Audit Criteria Auditing is made possible Valuation and local market Expertise Auditing automated valuation model – case: multivariate regression model for residential property 10

Audit process of the Orava AVM ✒ The tenets of an audit • Largely a question of pass or fail • Towards standardisation • ”You know what you get” Auditing automated valuation model – case: multivariate regression model for residential property 11

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Beginning work • • • Competency Other qualifications (independency) Documentation Audit plan Audit criteria Auditing automated valuation model – case: multivariate regression model for residential property 12

Examples criteria used in Orava Audit ✒ “For the whole portfolio, irrespective of the size of the portfolio, the sum of individual asset values must be within 5 % of the sum of asset values as valued by an independent valuer. ” “For a sub-portfolio, the sum of values must be within 7. 5 % of the sum of values as valued by an independent valuer. “ “Single property assets (combination of multiple apartments at the same address) must be valued within 15 % of the equivalent valuation by an independent valuer. Of the entire set of property assets, at least 80 % must pass this criteria. ” Auditing automated valuation model – case: multivariate regression model for residential property 13

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Beginning work • • Competency Documentation Audit plan Audit criteria Auditing automated valuation model – case: multivariate regression model for residential property 14

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ The audit report structure • • • Object and Contents of External Audit Regression Models in Property Valuation Automated Valuation Orava Automated Valuation Process (audit) Auditing Results and Audit Statement Auditing automated valuation model – case: multivariate regression model for residential property 15

Scope of External Audit used in Orava Audit ✒ “The purpose of the audit is to ascertain independently the balanced and quality and the true and fair treatment of the data and the results in respect to all parties involved. ” “The object of audit is the automated property valuation model by Orava Fund” “The purpose of the automated valuation model is to define a market value for the properties owned by Orava Fund. The audit of the model is thus limited to the use of the model for the defining of fair value (market value) for the afore mentioned portfolio at the time of audit. “ “The audit is based on the data and information obtained from Orava and other sources, in part verified against each other. ” Auditing automated valuation model – case: multivariate regression model for residential property 16

Scope of External Audit used in Orava Audit ✒ Definition of Audit “The purpose of the audit is to ascertain independently the balanced and quality and the true and fair treatment of the data and the results in respect to all parties involved. ” Object of Audit “The object of audit is the automated property valuation model by Orava Fund” Purpose and Scope “The purpose of the automated valuation model is to define a market value for the properties owned by Orava Fund. The audit of the model is thus limited to the use of the model for the defining of fair value (market value) for the afore mentioned portfolio at the time of audit. “ Sources “The audit is based on the data and information obtained from Orava and other sources, in part verified against each other. ” Auditing automated valuation model – case: multivariate regression model for residential property 17

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ The audit report structure • • • Object and Contents of External Audit Regression Models in Property Valuation Automated Valuation Orava Automated Valuation Process (audit) Auditing Results and Audit Statement Auditing automated valuation model – case: multivariate regression model for residential property 18

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Observing the AVM creation/implementation process • • Data collection and pre-processing Modelling post-processing and valuation AVM reporting Auditing automated valuation model – case: multivariate regression model for residential property 19

Example table used in Orava Auditing automated valuation model – case: multivariate regression model for residential property ✒ 20

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Observing the AVM creation/implementation process • • Data collection and pre-processing Modelling post-processing and valuation AVM reporting Auditing automated valuation model – case: multivariate regression model for residential property 21

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Considering implications • Regression analysis is prone to its own problems as do all methodologies for automatic price estimation • Full evaluation requires understanding of the methods used in AVM – audit should not be used merely as a stamp? • Market/Fair Value definition is not problematic, however the use of AVMs for price indices is a more touchy subject. • Assumptions! • These are some of the considerations faced when doing an (AVM) audit. Auditing automated valuation model – case: multivariate regression model for residential property 22

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Preparing the Audit Report and Statement • Limitation of auditor liability • Short form and long form • Consider the audience! Auditing automated valuation model – case: multivariate regression model for residential property 23

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality ✒ Self-assesment of audit quality • ISO-standards 19011 • IAASB handbook vol. 1, section for International standards on quality control • IAASB publication “Framework for audit Quality “, Feb 2014. Auditing automated valuation model – case: multivariate regression model for residential property 24

Most notable references • ISO 19011: 2011. (2011) ‘Guidelines for auditing management systems’. The International Organization for Standardization. • IVS 2013. (2013) ‘International Valuation Standards’. International Valuation Standards Council. • IVSC 2013. (2013) ‘A Guide to the Audit Process for Professional Valuers’. International Valuation Standards Council. • IAASB. (2014/02) ‘A framework for audit quality – key elements that create an environment for audit quality’. International Auditing and Assurance Standards Board Auditing automated valuation model – case: multivariate regression model for residential property 25

Orava REIT AVM Audit 2013 Auditing automated valuation model – case: multivariate regression model for residential property 26

Contact �� All input welcome! Jamie Donovan Helsinki, Finland �� Phone: +358 40 718 9135 Auditing automated valuation model – case: multivariate regression model for residential property jamie@donovan. fi 27

Some important terminology • Audit systematic, independent and documented process for obtaining audit evidence (3. 3) and evaluating it objectively to determine the extent to which the audit criteria (3. 2) are fulfilled (ISO 19011: 2011). • Internal/external audit • Compliance • Assurance Unqualified / qualified statement High / low confidence of Assurance • Auditing criteria set of policies, procedures or requirements used as a reference against which audit evidence is compared (ISO 19011: 2011). Auditing automated valuation model – case: multivariate regression model for residential property 28

Variables of Orava Residential REIT in-house AVM • Size: 3 rd degree polynomial • Age: 3 rd degree polynomial • Decor/apt condition: 4 classes – very good, fair, poor (dummy) • Apartment sauna / proxy for equipment (dummy) • Quarterly time dummies (8 quarters) • Lot ownership dummy / min. 20 observations for leasehold in data • Location – level 1: zip code dummies • Location – level 2: 1 sqkm or 4 sqkm plane around the property • Plane can tilt both south-north & east-west • 4 sqkm plane used if obs individual properties < 15 in 1 sqkm plane • When there are known transaction prices for valued property they are assigned a ’property dummy’. These are converted into asking prices using ’negotiation coefficient’. • Apartment specific dummies may be used when verified apartment specific asking prices exist. • Rental house factor is assigned to all properties for which ownership exceeds 50% and construction year is before 1990 with no transactions. • • rental house dummy is calculated using average of property dummies (above) from other models Negotiation coefficient (currently 6, 21% for cities with population < 100 000 and 4, 94% above 100 000. ) Auditing automated valuation model – case: multivariate regression model for residential property 29