A bit of background

Launching a new credit card is no easy task! It requires bank relationships, a robust compliance program, advanced technology and stellar operations. These are among the myriad reasons that no one has been able to create a major new credit card company successfully in a generation. Given the technological advancements of the last thirty years, the industry is primed to be disrupted. That’s where we come in. Cardless helps consumer brands launch credit cards so they can engage their superfans with special experiences and rewards.

One constant in the credit card industry is the need to underwrite applicants to determine (1) who is creditworthy and (2) what is the right product for a given approved applicant. The incumbent issuers have a major advantage in this area: they can use their mountains of data and gigantic teams of analysts, engineers and scientists to make very educated and precise credit decisions. As we will need to underwrite well to survive, we are faced with the monumental task of building a credit model and underwriting system without the benefit of having a single customer. In this post we’ll discuss the process, data science techniques and outcome of building a credit model from scratch.

Finding What Data to Use

So how do we underwrite applicants with limited internal data? The best way is to build a synthetic dataset containing previous credit card applicants that most resemble what we expect to see when we launch our Cavs card. Therefore, we purchased historical, anonymized archived credit bureau data containing over 2,700 credit attributes. We bought this data on 1.25 million applicants who received similar bankcard products to the Cavs card between April 2017 and June 2018 with a two year performance window.

Once we created the synthetic dataset, we set to work building a credit model. Our dependent variable was whether the accountholder had a serious delinquency on their credit card within their first two years of card ownership. We then split our sample into a build set, containing 2017 applicants and held out Q1 2018 and Q2 2018 tranches to serve as out-of-time validation samples. This distinction was especially important because of the performance window associated with our validation samples. Q1 2018 applicants’ performance window covered April 2018-March 2020 while the window for Q2 2020 applicants covered July 2018-June 2020. Therefore, while our build samples were unaffected by COVID and the accompanying recession, we saw some COVID impacts in the Q1 2018 data and more pronounced impacts in the Q2 2018 tranche. We will cover how we treated this impact later.

Finding the Most Important Attributes

Next, we needed to identify which of the 2,700 attributes were most important at predicting credit risk and remove the attributes that were least predictive. To do this, we bootstrapped a Lasso regression 500 times by selecting a random 50,000 records for each run.

Before we could run our regression, we needed to standardize our input variables. If we didn’t, an attribute with a large range of values, such as “account balance” would have an outsized impact on the model versus a variable with a smaller range of values, such as number of credit inquiries. We used the StandardScaler module in the sklearn.preprocessing package within Python to standardize our attributes to reflect the number of standard deviations above or below the mean value of each attribute.

Next, we ran the Lasso regression as described above. For each run of the regression, any coefficient from a non-predictive attribute was lowered to zero and we recorded whether each attribute was predictive in each run of the regression. Once we completed the 500 iterations, we kept any attribute that was predictive more than 20% of the time. As validation, we checked to see how well correlated the number of model runs was against the average coefficient when the attribute was predictive and saw a high degree of correlation:

This narrowed our attribute list down to 101. Next, we used the MontonicBinning function in Python’s xverse package to bin the surviving attributes. Binning attributes is essential to weed out noise within the attributes. The MontonicBinning function ensures that the bins chosen are effective enough to split performance while containing enough records to keep a meaningful sample size in each bin.

Next, we calculated the Information Value (IV) of each attribute to determine the individual impact each attribute has on our dependent variable. We then looked to find the optimal IV cutoff between the number of attributes and chose a cutoff of 0.1.

Building and Validating the Model

With our final list of attributes selected, we replaced each attribute bin with a Weight of Evidence (WoE) score to indicate the impact each bin has on our dependent variable, which in this case was the likelihood of repayment. In the example below, an attribute value of 6 would be replaced with a value of 0.304 when we train our logistic regression model.

Finally, we used GridSearch from the sklearn.model_selection package to select the best parameters to train a logistic regression model based on the WoE score for each attribute. At last, we have a working credit model! We named our model the Pioneer Model to represent the fact that this is Cardless’ first step out of many in predicting credit performance.

However, our work was far from done! While the Pioneer Model predicted risk on our build sample, we needed confidence that our model would still predict risk well once we begin having Cavs card applicants.

Our first step was to validate the Pioneer model on our two out-of-time samples from the Q1 2018 and Q2 2018 tranches. Even when taking the COVID impacts described earlier into consideration, the out-of-time samples had similar AUC and KS Scores to our build sample. Additionally, we saw greater predictive power from the Pioneer Model than using Vantage Score which gave us confidence that the Pioneer Model did in fact have incremental value.

With this added confidence in Pioneer’s performance, we needed to determine how we were going to implement the model into underwriting decisions. First, we layered in a multiplier to our model score to account for the impacts of COVID. While the Q2 2018 data helped give us a directional idea for how COVID would impact credit performance, we felt the actual impacts might be more severe and chose a multiplier more conservative than what the out-of-time samples suggested. We will continue to purchase ongoing performance data and adjust our multiplier as necessary. Additionally, we built guardrails into our credit policy to artificially lower credit lines and raise interest rates for population segments where the Pioneer Model predicted better outcomes than Vantage Score.

Conclusion

While the Pioneer Model is crucial to help Cardless underwriting get off the ground, it is just our first step. Having Cardless’ own performance data will be much more telling of our credit performance than synthetic archive data is so we will rebuild the model in the future once we have our own internal data. Additionally, we will onboard alternative sources of data in the coming months that should help us improve our underwriting decisions.

Lastly, the Pioneer Model would not have been possible without the help and support of Cardless’s employees and advisors as well as our business partners. Thank you to Michael, Lisa, Sam, Long, Frank, Cathy, Abhi, Veronica, Peter, Briana, Hannah, Todd, Brian and John!