A challenger model is an independent model built to verify or compete with a bank’s primary risk model, giving supervisors and internal governance teams a second opinion on whether the main model’s outputs are reasonable. Regulators expect banks to have one because model risk, the risk that a flawed model produces incorrect outputs that lead to bad decisions, is now treated as a formal risk category on par with credit and market risk. The sections below unpack what challenger models do, how they differ from validation, which models need them, and where banks typically run into difficulty.
Why do regulators require banks to have a challenger model?
Regulators require banks to have challenger models because relying on a single model without independent verification creates a blind spot. If the primary model contains a flawed assumption, a miscalibrated parameter, or a methodology that no longer fits current conditions, there is no automatic check to catch it. A challenger model provides that check by producing outputs through a different approach, making it possible to identify material differences and investigate their cause.
The regulatory push behind this requirement has grown significantly over the past few years. The Bank of England PRA’s SS1/23, which came into effect in May 2024, places model risk explicitly on par with credit and market risk and requires it to be fully integrated into a bank’s overall risk management framework. That means independent challenge is not an optional best practice, it is a governance requirement with supervisory teeth.
Basel IV is adding further pressure. As banks recertify or rebuild their Internal Ratings-Based models to meet new output floor constraints and enhanced documentation standards for PD, LGD, and EAD models, regulators are scrutinizing the governance surrounding those models more closely than before. More documentation, more independent challenge, and more lineage tracking are now expected as part of the process. A challenger model is one of the most direct ways to demonstrate that independent challenge is genuinely happening.
The underlying logic is straightforward: models are built to reflect conditions that existed when they were scoped and validated against data that was current when independent review began. By the time a model is deployed, the environment has already shifted. A challenger model gives the bank and its supervisors a structured way to test whether the primary model is still producing outputs that make sense in the current environment.
What is the difference between a challenger model and model validation?
Model validation and challenger models both serve the goal of independent oversight, but they operate differently. Model validation is a formal, periodic review process that assesses whether a model is conceptually sound, correctly implemented, and fit for its intended purpose. A challenger model is a parallel, running alternative that continuously or periodically produces competing outputs to benchmark against the primary model.
Think of validation as a structured audit that happens at defined points in a model’s lifecycle, typically before deployment, after significant changes, and on a regular review cycle. It examines documentation, methodology, data inputs, assumptions, and outcomes. Validation produces a report and may result in conditions or restrictions on the model’s use.
A challenger model, by contrast, is operational. It runs calculations using a different methodology or different assumptions and produces outputs that can be compared directly to the primary model’s results. The comparison itself is informative: if the two models produce materially different numbers, that triggers investigation. If they consistently agree, that gives the bank and its supervisors greater confidence in the primary model’s outputs.
The two practices are complementary rather than substitutes. Validation tells you whether the model was built correctly. A challenger model tells you whether it continues to perform correctly relative to a credible alternative. Banks operating under SS1/23 or equivalent frameworks are expected to have both in place for material models.
What types of models require a challenger in banking?
Not every model in a bank’s inventory requires a dedicated challenger. The expectation is generally proportionate to a model’s materiality, how much capital, provisioning, or regulatory output depends on it. In practice, the models most commonly subject to challenger requirements are credit risk models, particularly those used for regulatory capital and impairment calculations.
The most common categories include:
- PD models (Probability of Default): Used in both IFRS 9 Expected Credit Loss calculations and Basel IRB capital calculations. These are high-materiality models where errors have direct consequences for provisioning levels and capital adequacy.
- LGD models (Loss Given Default): Determine the severity of loss if a borrower defaults. LGD assumptions have a significant effect on ECL and RWA outputs, making them a natural candidate for independent challenge.
- EAD models (Exposure at Default): Estimate how much a borrower owes at the point of default, particularly relevant for revolving credit facilities and off-balance-sheet exposures.
- Behavioral models: Models that estimate how customers will behave, for example, prepayment models or drawdown models, where assumptions can shift materially as economic conditions change.
- Stress testing and macro-scenario models: Used to project portfolio performance under adverse conditions. Given that these models inform capital planning and regulatory stress test submissions, supervisors expect robust independent challenge.
The principle to apply is: if the model’s output feeds a regulatory submission, a capital ratio, or a material provision, a challenger model is likely expected. Supervisors in the EU, the UK, and increasingly in Asia-Pacific jurisdictions are applying this expectation consistently.
How does a challenger model actually work in practice?
In practice, a challenger model is a second model that runs on the same input data as the primary model but uses a different methodology, different assumptions, or a different functional form. The outputs of both models are then compared, and material differences are investigated to understand which model is better capturing the underlying risk dynamics.
The process typically follows a structured pattern:
- Define scope and methodology: The challenger model’s approach is chosen to be meaningfully different from the primary model. If the primary model uses a logistic regression for PD estimation, the challenger might use a machine learning approach or a different set of macro variables.
- Run both models on shared data: The challenger runs on the same portfolio data and time period as the primary model. Using the same input data ensures that differences in output are attributable to methodology rather than data discrepancies.
- Compare outputs: The outputs, PD estimates, ECL figures, RWA calculations, are compared at portfolio level and, where relevant, at segment or obligor level. Statistical tests and visual diagnostics are used to assess whether differences are within acceptable tolerance.
- Investigate material differences: Where the challenger produces materially different results, the bank investigates the source of the divergence. This might reveal a problem with the primary model, confirm that the challenger is capturing something the primary model misses, or identify data issues affecting both.
- Document and report: The comparison, findings, and any actions taken are documented as part of the model governance record. This documentation is what regulators review to confirm that independent challenge is substantive rather than procedural.
One practical challenge is timing. Running a challenger model requires that both models can execute quickly enough to produce comparable outputs within the same reporting cycle. When primary models run on overnight batch cycles, fitting a challenger into the same window adds complexity. Banks moving to real-time or near-real-time calculation environments find it considerably easier to run challenger scenarios dynamically rather than treating them as a separate, scheduled exercise.
What are the biggest challenges banks face when running challenger models?
The biggest challenges banks face with challenger models are data consistency, governance overhead, and the speed gap between when a model change is needed and when the challenger can be updated to reflect it. Each of these creates practical friction that can undermine the quality of independent challenge even when the intent is sound.
Data consistency between primary and challenger models
For a challenger model comparison to be meaningful, both models need to run on the same data. In practice, many banks operate primary and challenger models in separate environments, different teams, different tools, different data extracts. When data is extracted at different times or processed through different pipelines, differences in output can reflect data discrepancies rather than genuine model differences. This makes the comparison harder to interpret and creates an audit question when results diverge unexpectedly. BCBS 239 requires that every aggregated risk number be traceable back to its source transaction, model, and assumptions on demand. Meeting that standard across two separate model environments is significantly more difficult than meeting it within a single governed execution layer.
Governance overhead and documentation burden
Running a challenger model is not just a technical exercise, it generates documentation requirements. The methodology choice, the comparison results, the investigation of material differences, and any actions taken all need to be recorded in a form that satisfies supervisory expectations. When this documentation is produced manually, it adds meaningful overhead to teams that are already managing primary model development, validation cycles, and regulatory reporting deadlines. The governance burden is manageable when challenger models run infrequently, but as the pace of model change increases, driven by Basel IV recertification requirements and the need to respond to macroeconomic shifts faster, the documentation workload scales with it.
Keeping the challenger current
A challenger model that was calibrated twelve months ago and has not been updated since is a weaker form of independent challenge than one that reflects current conditions. When primary models are updated to incorporate new macro variables, recalibrated parameters, or revised methodologies, the challenger needs to evolve in parallel to remain a credible benchmark. The model change cycle, development queue, independent review, governance approval, was designed for annual or quarterly release cadences, not for the current environment where geopolitical events, central bank pivots, and credit market dislocations can make core assumptions obsolete within weeks.
These three challenges compound each other. A bank with fragmented data environments, heavy manual documentation processes, and slow model change cycles will find it genuinely difficult to run challenger models that provide substantive independent challenge rather than a compliance checkbox. Addressing them requires thinking about model governance as an integrated capability rather than a set of separate processes.
This is exactly the kind of problem we built the .NXT platform to solve. When primary and challenger models run on the same governed data layer, with versioning and audit trails generated automatically as part of each execution, the data consistency problem and the documentation burden both shrink considerably. Challenger scenarios can run in a sandbox environment without requiring a separate data extract, and the comparison between primary and challenger outputs is available in real time rather than after a separate reconciliation exercise. If you want to see how this works in practice, talk to our team, we are happy to walk through a concrete example relevant to your model environment.
Frequently Asked Questions
How do we decide which methodology to use for our challenger model?
The challenger model’s methodology should be meaningfully different from the primary model’s approach, not just a recalibration of the same technique. If your primary PD model uses logistic regression with a fixed set of macroeconomic variables, a credible challenger might use a survival analysis framework, a gradient boosting model, or a structurally different variable set. The goal is to ensure that if the primary model has a systematic blind spot, the challenger is unlikely to share it. Methodology selection should be documented and justified as part of the challenger’s governance record.
How often should a challenger model be updated or recalibrated?
A challenger model should be updated whenever the primary model undergoes a material change, and independently reviewed at least as frequently as the primary model’s regular validation cycle. In practice, significant macroeconomic shifts, such as a rapid rate hiking cycle or a credit market dislocation, should also trigger a review of whether the challenger’s assumptions remain credible. A challenger that was calibrated over a year ago and left static provides progressively weaker independent challenge as conditions evolve, which is a concern regulators are increasingly flagging during supervisory reviews.
What counts as a 'material difference' between primary and challenger model outputs, and what should we do when we find one?
There is no universal regulatory threshold that defines materiality, so banks are expected to set their own tolerance bands based on the model type, portfolio size, and the downstream impact of the output. Common approaches include percentage divergence thresholds at portfolio level combined with segment-level diagnostic checks. When a material difference is identified, the bank should investigate the source systematically: determine whether the divergence stems from a methodology difference, a data discrepancy, or a genuine signal that one model is outperforming the other, then document the findings and any remedial actions taken regardless of the conclusion.
Can a machine learning model be used as a challenger for a traditional statistical model, and will regulators accept it?
Yes, machine learning models can serve as challengers, and regulators do not categorically prohibit their use in this role. However, explainability is a key concern: if the challenger produces materially different outputs, the bank needs to be able to explain why, and a black-box model makes that investigation harder. Using an interpretable ML approach, such as gradient boosting with SHAP values, or pairing the ML challenger with robust model documentation, addresses most regulatory concerns. The PRA and ECB have both published guidance indicating that ML models are acceptable where governance, explainability, and validation standards are met.
Do smaller or less complex banks need challenger models, or is this only a requirement for large institutions?
The formal expectation for challenger models is proportionate to model materiality and institutional complexity, so smaller banks with simpler model inventories are not held to the same standard as G-SIBs. However, any institution using IRB models for regulatory capital, IFRS 9 ECL models for provisioning, or stress testing models for capital planning should expect supervisory scrutiny of its independent challenge arrangements regardless of size. Regulators have been clear that proportionality does not mean absence of challenge, it means the depth and formality of the challenger framework should match the risk the model represents.
What is the most common mistake banks make when setting up a challenger model program?
The most common mistake is treating the challenger model as a documentation exercise rather than a genuine analytical tool, which means it gets built, compared once at validation, and then left unchanged while the primary model evolves. This produces a compliance artifact rather than a credible benchmark. A close second is running the challenger on a different data extract than the primary model, which means output differences can never be cleanly attributed to methodology alone. Both mistakes are flagged regularly in supervisory feedback letters and model risk review findings.
How should challenger model results be reported to senior management and the board?
Challenger model results should be summarised in model risk reporting at a frequency that matches the model’s materiality and output cadence, typically quarterly for high-materiality credit risk models. The report should highlight cases where challenger and primary outputs diverged materially, the investigation findings, and any actions taken or pending. Boards and risk committees do not need granular technical detail, but they do need enough information to satisfy themselves that independent challenge is substantive and that identified differences are being resolved in a timely way. SS1/23 specifically expects board-level engagement with model risk as a formal risk category, which means challenger model outcomes should feed into that reporting line.
Related Articles
- What does the output floor mean for bank capital requirements?
- What are common COREP reporting compliance challenges?
- What is a regulatory reporting connector and how does it work?
- How do you stress test derivatives portfolios?
- What are the main challenges in integrated risk management?
This content was generated with the help of AI and it may contain mistakes
