Model Risk – Key Concepts
A model is “a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories,
techniques, and assumptions to process input data into quantitative estimates.” Model risk is the risk associated with using financial models that are inherently flawed. It occurs when a model used to measure a value does not perform as planned. It is a subset of operational risk; it mainly impacts the firm that develops and utilizes the model.
Models are generally judged by their predictive ability. By the nature of their existence; all models are incorrect, or flawed to a certain degree. Common sources of model risk include:
- Incorrect calibration – issues with input parameters, including inputs based on unsuitable sample periods, outdated inputs, and those measured with error.
- Implementation risk – error from user-defined inputs. For example, if multiple users of the same model are asked to self-define or estimate an input, the model will produce different results based on the inputs.
- Incorrect model application – using a model to predict something that it wasn’t designed to predict. Another example of model misapplication is using an outdated model.
- Incorrect specification – omitting a key risk factor or using an incorrect distribution. Not incorporating transaction costs, taxes, liquidity factors, etc., are also examples of model misspecification.
- Programming and data problems – issues include programming bugs, poor logic, and incorrect algorithms. An increase in model complexity leads to an increase in programming problems. Data problems arise when using poor quality data – “garbage in, garbage out”.
Model risk may not be completely eliminated, however it may be mitigated. Below are a few methods for protecting against model risk.
- Document and track model development, implementation, and performance
- Back-test / stress-test the model
- Do not ignore minor issues with the model, as they may be indicative of larger issues
- Avoid using overly complex models unless the situation absolutely requires it
- Update the model regularly to reflect recent trends
- Use an effective model validation process
- Establish a framework that implements sound risk management procedures
Sources:
OCC: Sound Practices for Model Risk Management
http://www.occ.treas.gov/news-issuances/bulletins/2011/bulletin-2011-12a.pdf
