Scorecards

Scorecard (statistical or expert) is a model whose aim is to predict certain events – most often client’s creditworthiness (i.e. probability that a client will fail to repay a loan). This can be estimated by applying a set of scores, which are subsequently used for ranking and/or sorting clients in different categories by their riskiness.

Scorecards can be applied in many different instances, such as:

• providing loan products and leasing services (assessment of clients taking their actual or expected exposure into account – identification of clients prone to bankruptcy etc.),
• providing special offers (identification of clients positively sensitive to special offers),
• customer retention (identification of clients susceptible to shifting to a competitor or decreasing their orders),
• employee retention (determination of probability that an employee will give a notice within a given period).

Scorecards are employed in financial institutions as an integral part of credit risk assessment where they can be used on a stand-alone basis or in conjunction with other processes (e.g. mapping to the rating scale). In non-financial institutions, scorecards can be used to determine credit limits.

Advanced Risk Management offers the following services with respect to scorecards:

• Development of scorecards
• Validation of scorecards

Development of scorecards

The development of applicational and behavioural scorecards is carried out in a close cooperation between ARM and its customers. The methodology selected for the purpose of the scorecard calculation, its preliminary results and subcomponents are presented to the customer and discussed during the development process. Therefore, the development of scorecards fully reflects the customer’s needs and usually consists of the following steps:

• determination of the aim of the scorecard and of the required quality (criteria for the assessment of quality and accuracy of the scorecard, number of grades used for the assessment taking into account the scorecard’s purpose, selection of time horizon for the model creation etc.),
• making a long list of scorecard characteristics, which may be potentially suitable for the model,
• narrowing the long list of characteristics to a short list based on:
  • quantitative degree of relevance of employed variables (information value, Weight of Evidence, AIC, chi-square, Gini coefficient...),
  • correlation analysis (correlated variables are not recommended to be included),
  • experience and opinions of experts,
  • stability of characteristics both over time and with respect to a particular client segment,
• draft model:
  • definition of assumptions for scorecard development,
  • selection of methodology and verification of its assumptions,
  • analysis of possible restrictions of a scorecard.
• model building:
  • determination of a data sample and its parameters,
  • splitting the data on a development sample and a validation sample,
  • calibration of the scorecard parameters,
  • assessment of the scorecard predictive power,
  • drafting a detailed scorecard documentation.
• scorecard implementation:
  • setting up a tool in which the model will be used,
  • drafting a user manual,
• mapping of the credit score to a selected rating scale or a scale of selected rating agency.

Up

Validation of scorecards

The validation of scorecards involves a set of processes and activities which attempt to prove the suitability and relevance of scorecards. ARM offers help with the process of validation, which typically covers the following areas:

• assumptions and restrictions of the scorecard:
  • verification of the assumptions regarding composition of input data (in the initial phase for the development and validation samples, subsequently for the input data),
  • verification of the assumptions regarding the structure of the modelled variables,
  • verification of the quality of input data,
• input characteristics:
  • assessment of the stability of characteristics over time,
  • assessment of the stability regarding client segments,
  • verification of methodology for handling missing values and errors,
  • assessment of methodology for data transformation,
• methods:
  • verification of suitability of selected statistical methods/expert approach for the given task,
  • logical correctness of scorecard,
• calculations:
  • evaluation of the correctness of utilized calculations,
  • data backtesting, verifying that deviations between scorecard predictions and observed values (reality) are within a predefined range,
• scorecard quality:
  • assessment of the scorecard predictive power based on quantitative criteria (e.g. Gini coefficient, AIC etc.),
  • assessment of the stability of predictions over time,
  • the extent to which expert judgement is used to adjust resulting scores (overrides),
• scorecard documentation:
  • assessment that scorecard documentation contains an appropriate level of detail,
  • assessment that the scorecard outcome is regularly monitored.
• compliance with regulatory requirements and relevant internal methodologies.

ARM’s engagement in the process of scorecard validation results in a comprehensive report assessing various aspects of scorecards. The report lists all scorecard shortcomings and their sources identified within the validation process, as well as suggestions on how to fix them.

Up