Model Validation

What is a Margin of Conservatism?

This question comes up a lot in my field of work, and unfortunately, there is no rigorous answer. Everybody seems to have a different (albeit loosely similar) definition of a margin of conservatism. However, we all need a definition, as it drives much of the model development and validation work at financial institutions. Furthermore, AIRB risk parameter estimation models require a margin of conservatism,…

Weights of Evidence and the Information Value

In order to determine which variables demonstrate a high level of predictive power, we calculate the binned Weights of Evidence (WoE) and the associate Information Value (IV). Generally speaking, the IV provides a measure of how well an explanatory variable  is able to distinguish between binary responses (e.g. "good" versus "bad") in some target variable. The idea is if a variable  has…

Age-Period-Cohort Model for Transition State Forecasting

You may be familiar with transition state models that are governed by the Age-Period-Cohort (APC) modelling framework. If you are, you've likely been witness to the recent insurgence of APC models in the literature -mainly being used to forecast mortgage and bond performance. The content related to APC modelling can be dense (literally, not figuratively) and you may find yourself confused when the the…

Herfindahl–Hirschman Index

The Herfindahl–Hirschman Index is a measure of the size of subgroups in relation to its containing group. It is often used in econometrics to measure the market share of companies (subgroup) in relation to an industry (group). For simplicity, it can be interpreted as a measure of competition or concentration. The measure itself is quite simple , given by the expression:…

Population Stability Index

One way to measure 'shifts' in the proportion of observations within subgroups is by the Population Stability Index (PSI). When a sample population is classified into various subgroups, one might want to establish whether those subgroups are stable with respect to a base population. The PSI is quantified by the formula: where and refer to the proportion of observations belonging to the subgroup…