← SIMRANJAISWAL.INCASE · STUCK MONEY · 06 · MACHINE LEARNING

Who pays late?

A repayment-propensity model on 30,000 real accounts: six months of payment history in, a ranked chase list with an economic cut-off out. Calibrated, checked for leakage and fairness, exported to plain JavaScript — and scoring live on this page.

◈ REAL PUBLIC DATA · UCI, 30,000 ACCOUNTS · CODE, MODEL AND EVALUATION OPEN
0.78ROC-AUC · HELD-OUT 6,000 ACCOUNTS
0.56PR-AUC · AGAINST A 22% BASE RATE
3.1×TOP-DECILE LIFT · MISS RATE 70% VS 22%
±0.01AUC SPREAD ACROSS SEX AND AGE GROUPS
01 · THE BUSINESS QUESTION

A collections team can call two hundred accounts a day. Which two hundred?

Every ledger with credit in it faces the same morning: more accounts in arrears than hands to chase them. The ladder in case 02 decided what to send and when; the arrears path in case 04 decided who must not be chased. What neither did is decide who to call first. That is a ranking problem, and ranking is where a model earns its place: not to replace judgement, but to put the two hundred most likely-to-slip, highest-exposure accounts at the top of the list before anyone picks up a phone.

02 · THE DATA

Thirty thousand real accounts, six months of "how late, how much billed, how much paid", and whether the next payment was missed.

The UCI Default of Credit Card Clients dataset (Yeh & Lien, 2009): 30,000 credit-card accounts from a Taiwanese bank, April to September 2005. It is used here because it has exactly the shape of a receivables ageing history — monthly repayment status, bill and payment amounts — and a clean, real outcome. 22.1% of accounts missed the next payment.

FIELDWHAT IT ISUSED AS
LIMIT_BAL, AGECredit limit (NT$) and ageLimit is a feature; age is reported by group, never a feature
PAY_1 … PAY_6Repayment status each month: −1 paid in full, 0 revolving, 1–9 = months lateThe ageing history — the model's core
BILL_AMT1 … 6Statement balance each monthExposure, utilisation, trend
PAY_AMT1 … 6Amount actually paid each monthShare of the previous bill paid, zero-payment months
SEX, EDUCATION, MARRIAGEDemographicsExcluded from the model; sex used only for the fairness check
default payment next month1 = the next payment was missedThe target
03 ·TRACEHYGIENE, LEAKAGE AND THE PROTOCOL

Before any model: what would fool it, and what would fool us.

Three things had to be settled first. Undocumented codes — the education and marital fields contain values the codebook never defines; they were folded into "other" rather than dropped. Time order — every feature is built from the six months before the outcome month; nothing from the scored month leaks in. And the protocol — a stratified 80/20 hold-out set touched exactly once, with five-fold cross-validation on the training half for every comparison, so the reported numbers are not the numbers the model was tuned on.

# features built only from history known before the month being predicted (train.py)
pay = d[PAY].clip(lower=-1)                       # -2 (no spend) and -1 (paid in full) both mean "not late"
f["late_now"]     = pay["PAY_1"].clip(lower=0)
f["months_late"]  = (pay > 0).sum(axis=1)
w = np.array([6, 5, 4, 3, 2, 1])                  # the last month counts most
f["late_recency"] = (pay.clip(lower=0).values * w).sum(axis=1) / w.sum()
ratio = np.where(prev_bill > 0, paid / np.maximum(prev_bill, 1), 1.0)   # share of last bill actually paid
f["pay_ratio_mean"] = np.clip(ratio, 0, 2).mean(axis=1)

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, stratify=y, random_state=42)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)   # every comparison happens inside the training half
04 ·AGEFEATURES FROM THE AGEING HISTORY

Fifteen features, all of them things a collector would recognise. The recency-weighted lateness score does most of the work.

WHAT THE MODEL LEANS ON
GRADIENT-BOOSTING FEATURE IMPORTANCE (SHARE OF SPLIT GAIN) · ONE SIGNAL — HOW LATE, WEIGHTED TO THE LAST MONTH — CARRIES MOST OF IT; MONEY FEATURES ADD THE REST.
FEATUREMEANINGWHY
late_recencyMonths late each month, weighted 6…1 from latest to oldestRecent lateness predicts; lateness five months ago mostly doesn't
late_now · max_late · months_lateHow late right now; the worst month; how many late monthsLevel, severity and persistence — three different things
late_trendLatest status minus the status two months earlierGetting worse is not the same as being bad
util_now · util_meanBill as a share of the limit, now and on averageHeadroom — the classic credit signal
pay_ratio_now · pay_ratio_meanShare of the previous bill actually paidBehaviour, not status: paying 5% every month is a pattern
zero_pay_months · paid_total_6mMonths with no payment at all; total paid in six monthsDisengagement is the strongest tell before a miss
bill_now · bill_trend · limitExposure and its directionWhat is at stake if the model is right
05 ·RANKTHE MODEL, AND WHETHER TO BELIEVE IT

Gradient boosting beats a good logistic baseline by 2.6 AUC points. More importantly, its probabilities mean what they say.

MODELCV AUCTEST AUCTEST PR-AUCBRIER
Logistic regression (scaled, C = 0.5)0.7690.7550.5230.140
Gradient boosting (220 trees, depth 3, lr 0.05)0.7850.7810.5640.134
DISCRIMINATION

ROC — the model against a coin toss

HELD-OUT SET · AUC 0.781 · THE DIAGONAL IS RANDOM.
THE HONEST CURVE AT A 22% BASE RATE

Precision–recall

PR-AUC 0.564 · THE FLAT LINE AT 0.22 IS "CHASE EVERYONE".
CAN YOU TRUST A 30%?

Calibration — predicted vs observed, by decile

QUANTILE BINS · POINTS ON THE DIAGONAL ARE HONEST PROBABILITIES · BRIER 0.134.
WHAT A RANKED LIST BUYS YOU

Miss rate by model decile

TOP DECILE MISSES 70% OF THE TIME; BOTTOM DECILE 4% · BASE RATE 22%.

Fairness — reported, not assumed

Sex and age are not features. The model is still checked against them on the held-out set, because a feature like "share of bill paid" can carry demographic signal indirectly. It scores each group about as well as the whole, and its mean score tracks each group's actual miss rate:

GROUPNAUCACTUAL MISS RATEMEAN SCORE
06 ·SEALTHE SCORER · RUNNING IN YOUR BROWSER

Describe an account. The exported model scores it here — same trees, same probabilities as the Python — and explains itself.

LOADING MODEL…

WHY · CONTRIBUTIONS TO THE LOG-ODDS, BY FEATURE
THE SCORE IS THE MODEL'S; THE ACCOUNT IS YOURS. CONTRIBUTIONS ARE PATH ATTRIBUTIONS ACROSS ALL 220 TREES AND SUM EXACTLY TO THE MODEL'S OUTPUT. RANK IS AGAINST 1,000 HELD-OUT ACCOUNTS, BY PROBABILITY × CURRENT BILL.

Then the economics: where to stop calling

A probability is not a decision. Rank accounts by probability × exposure, contact the top k%, and every contact costs something while only some would-have-missed accounts pay because you called. Move the two levers below; the model's job is to make the best k visible.

NET RECOVERY BY CHASE DEPTH ON 1,000 HELD-OUT ACCOUNTS · POLICY PARAMETERS ARE ILLUSTRATIVE INPUTS, NOT FINDINGS.
07 ·WATCHTHE MODEL CARD, AND WHAT WOULD MAKE IT WRONG
CARD
Intended usePrioritise which accounts in arrears a human contacts first. Not an automatic decision about any customer; exclusions (vulnerability, disputes, kept plans) run before any score, as in case 04.
Training dataUCI credit-card clients, Taiwan, April–September 2005, 30,000 accounts, 24,000 used for training.
Not valid forAny ledger it was not fitted on. B2B invoices, utilities, 2026. The method transfers; the model does not. Re-fit on your own history first.
Known failure modesNew accounts with under three months of history (features degenerate to zero); accounts whose bills are zero (utilisation undefined, clipped); behaviour shifts after a policy change (a new reminder ladder changes the very patterns the model learned).
MonitoringMonthly: calibration by decile against realised misses; population-stability index on late_recency and pay_ratio_mean; AUC by group. Re-fit when calibration drifts past ±5 points in any decile or PSI exceeds 0.2.
ExplainabilityPer-account path contributions (shown above), which sum exactly to the model's output; global importance by split gain.
08 · THE TAKEAWAY

A good collections model is a ranked list with a cut-off, not a verdict. Six months of "how late, how much, how paid" carries most of what there is to know; a small, calibrated, explainable model turns it into the order to call people in, and the economics say where to stop. Everything above it — the exclusions, the ladder, the tie-out — stays human. The model just makes sure the humans start in the right place.

A MACHINE-LEARNING PROJECT ON PUBLIC DATA, NOT CLIENT WORK. DATASET: UCI MACHINE LEARNING REPOSITORY, "DEFAULT OF CREDIT CARD CLIENTS" (YEH & LIEN, 2009) — REAL, ANONYMISED, TAIWAN 2005. ALL METRICS ARE ON A HELD-OUT SET TOUCHED ONCE; THE IN-BROWSER MODEL REPRODUCES THE PYTHON MODEL'S PROBABILITIES TO WITHIN 1E-6. CHASE ECONOMICS USE ILLUSTRATIVE COST AND RECOVERY PARAMETERS YOU CAN CHANGE. CODE (TRAIN.PY), MODEL.JSON, EVALUATION.JSON AND A README ARE PUBLIC AT /ML/WHO-PAYS-LATE/.

Have an arrears book and a team that can't call everyone? A ranked list on your own history is a Seal-sized piece of work: two to four weeks, fixed fee, model card included. It starts with a TRACE of the ledger it will rank.

Start with a TRACE → train.py README model.json