Ten Years of Validation Reflects a Higher Standard of Performance

November 8, 2016         By: Mike Dautner

VantageScore Solutions, in accordance with its mission, annually validates VantageScore credit score models and, to encourage transparency and aid in model governance, publishes the results for all to see.

To commemorate its ten-year anniversary, VantageScore published its most recent validation results in a paper that also presents insights gleaned from a decade of credit score model validations.

VantageScore started the validation process as it always does, by pulling 15 million credit files from consumers completely random. Each model is validated by comparing predictive performance for both originations and existing-account management applications in the bankcard, auto, and real estate industries. VantageScore models’ performance is also benchmarked against in-house models developed by the three national credit reporting companies (CRCs), Equifax, Experian and TransUnion.

Some highlights of the paper include:

• The VantageScore 3.0 model continues to outperform the CRC in-house scores for all major industries and business applications. Additionally, VantageScore 3.0 delivers superior performance stability during both peak and nonpeak recession timeframes.

• Over the last ten years, the VantageScore models have consistently outperformed all CRC in-house models.

• VantageScore 3.0 continues to score 30 to 35 million additional consumers more than conventional credit scoring models.

• In part as a result of the highly conservative post-recession lending and risk environment, all VantageScore credit score models discussed in this paper are currently performing very strongly.

• 2013-2015 VantageScore performances in comparison with CRC in-house models:

• VantageScore 3.0 delivers consistently superior performance compared with the CRCs’ in-house credit score models.

To demonstrate its unparalleled consistency in scoring consumers across CRCs., VantageScore 3.0 was used to score a sample of one million consumers with credit files at all three CRCs. For consumers with real estate trades, 80.8 percent of the sample had scores that fell within 20 points of each other across the three CRCs, and 93.2 percent of the sample had scores within 40 points across the CRCs.

A key question for lenders as they consider whether to upgrade to newer credit scoring models is: “How will a recently developed model perform in more volatile risk periods?”

To answer this question, VantageScore validated all three of its models on consumer originations performance during a period of economic turbulence, from 2007 to 2009: VantageScore 1.0, which was developed pre-recession (2003-2005); VantageScore 2.0, which was developed mid-recession (2006-2009); and VantageScore 3.0, which was developed post-recession (2009-2012). As expected, VantageScore 2.0 performed optimally because it was developed on the same recession timeframe, but VantageScore 3.0 exhibited tremendous strength and stability, thanks to a methodology that blends two development windows, and uses 45 million consumer credit files (nearly two times the sample size used on earlier models), to uncover a broader and deeper array of behaviors for consideration in model design and calibration.

VantageScore has been at the forefront of the industry for the past decade, especially when it comes to model validation.

In summation, a decade of credit score model validations has shown that modern credit scoring models are remarkably robust. Even in the midst of recession, these models effectively continued to rank order. VantageScore models, in particular, have continued to outperform competitor models on all key dimensions.