Model Validation For Derivative Pricing The First Line

  • Slides: 12
Download presentation
Model Validation For Derivative Pricing: The First Line of Risk Management Defense

Model Validation For Derivative Pricing: The First Line of Risk Management Defense

Defining Model Risks • Model Risk is a collection of several related issues surrounding

Defining Model Risks • Model Risk is a collection of several related issues surrounding the behavior of a derivatives pricing model. The recent financial crisis and the attendant regulatory crackdown has compelled a renewed interest in this field. This has driven deeper thinking on the many possible contributors to model risk.

Types of Model Risk in Derivative Pricing • Implementation Errors (bugs) • Dynamic Misspecification,

Types of Model Risk in Derivative Pricing • Implementation Errors (bugs) • Dynamic Misspecification, i. e. – Omission of important risk factors – Model performs poorly in new market regime • Poor choice of calibration instruments • Poor choice of calibration method – Bad weighting choice in objective function (e. g. absolute vs. relative price errors) – Excessively local solvers failing to find the best solution – Poor initial guess for solver • Poor choice of hedge instruments

Types of Model Risk, cont’d • Poor choice of hedge ratio calculation method –

Types of Model Risk, cont’d • Poor choice of hedge ratio calculation method – E. g. Continuous derivatives vs. various discrete difference methods • Poor choice of proxy data • Data Quality problems – feeding the models with the wrong data • Poor Documentation – a misunderstanding of the model can lead to passing incorrect data to the model, i. e. data quality problems

What can we do about these risks? • Implementation Risks – A mathematical library

What can we do about these risks? • Implementation Risks – A mathematical library separate from the financial library, to be called by the financial library, instead of writing inlined or bespoke mathematical functions for each option pricer. – A strong testing regime, containing: • Mathematical identities for all pure math functions • Limiting or Special cases • Tests should be run on MORE than just a few typical cases – A test domain should be defined and a large number of cases from the test domain should be selected at random, different ones each night. The tests need to cover high, and low values, in every possible combination, of all parameters. • Multiple methods – A strong testing regime pays you back with easier debugging, by ruling out places where bugs can be, by allowing developers and bugfixers to try out fixes on unfamiliar code more fearlessly.

Implementation Risks – Examples of Possible Tests Instrument and Model Specific Tests 1) Parallel

Implementation Risks – Examples of Possible Tests Instrument and Model Specific Tests 1) Parallel Reimplementation 2) Limiting Cases or Identities for Payoff (e. g. Knock-in + Knock-out = Vanilla, Barrier Vanilla as Barrier infinity) 3) Limiting Cases or Identities for Model (e. g. Heston Black-Scholes, Romano-Touzi Mixing Theorem) 4) Convergence and convergence rate of method 5) Zero and infinite volatility limits, to catch payoff specification errors.

Model Validation Tests – Mathematical Correctness Model and Instrument Independent, “Generic” Tests 1) Delta-Vega

Model Validation Tests – Mathematical Correctness Model and Instrument Independent, “Generic” Tests 1) Delta-Vega Smoothness Tests Is there a “jitter” or oscillation mixed in with the pricing that throws off greeks even if model price is accurate? 2) Calibration Round Trip Tests Does our calibration method re-price the calibrated instruments accurately? When a calibration problem is set up with a known solution, can the calibrator find it? 3) Convergence Tests Does our pricing converge, and is the production price close to the converged value?

Model Validation Tests – Financial Correctness • Calibration Stability Tests Does our model have

Model Validation Tests – Financial Correctness • Calibration Stability Tests Does our model have similar calibration parameters from day-to-day? Does our model price the same vanilla similarly from day-to-day? • Calibration Error Tests Does our model, when calibrated to vanillas from the market, price all the calibration instruments accurately? • Pn. L Attribution Tests Does our model account for changes in Pn. L of the instrument due to changes in market data well without calibration? If we use this model to hedge, will our hedges hedge our changes in value? (Similar to Calibration Stability Test) • Variance of Hedged Portfolio Test What is the variance of the hedged portfolio (similar to Pn. L Attribution Test) • Cost of Hedging Test Does the initially calculated price accurately reflect the eventual cost of hedging?

Model Validation Tests – Test Definitions • Calibration Stability Tests We calibrate a model

Model Validation Tests – Test Definitions • Calibration Stability Tests We calibrate a model to vanilla instruments on a series of dates, and price a test option. We then perturb the market data, and re-price the test option. The change in the price should be smaller than the perturbation. This test is to be run over market data from Many different market regimes, preferably at least 10 years worth. • Calibration Error Tests We calibrate a model every day, and record the sum of squared errors. We may then compare this to other models, or to the same model in other periods. • Pn. L Attribution Tests We compute the price and hedge-ratios of the model every day throughout the life of the option, starting at a day in history, and record the Pn. L difference as reported by re-calibration and re-pricing, on the one hand, and using the Greeks and changes in underlying on the other. We graph this function, and look for spikes in this graph.

Model Validation Tests – Test Definitions • Variance of Hedged Portfolio Test We compute

Model Validation Tests – Test Definitions • Variance of Hedged Portfolio Test We compute the price of the portfolio and its hedge together, every day. This should barely move, and its movement is a measure of residual risk due to the model’s imperfect description of the market. We compute the sum of squared errors of the one-day change in Pn. L of the hedged portfolio, and report this as the residual risk • Cost of Hedging Test We compute the hedge every day, and consider the losses or gains in this trading strategy by itself. The theory of Black-Scholes says that the price of the option should be equal to its cost of hedging. We then compare the cost of hedging to the computed price of the option at inception.

Questions to ask in your organization • How old is my code – for

Questions to ask in your organization • How old is my code – for what percentage of code are the original authors still in the organization? • How much time do my developers spend debugging, vs. new code development? • Are there known bugs that no one knows how to solve? • How much are you relying on “work-arounds”, instead of true bugfixes? • How much do my traders trust the model? How much manual intervention does it require? • How much bleed are hedgers seeing? How much variance from market prices? • How volatile is my portfolio? • Are my calibrations accurately repricing the market? • Are my calibrations stable? • What kinds of data-tagging does the code do to identify things like “which volatility is coming from where”, to prevent data errors?

Thank you Visit us online at: www. numerix. com News and updates: blog. numerix.

Thank you Visit us online at: www. numerix. com News and updates: blog. numerix. com Derivatives discussion board: forums. numerix. com