Pub. 6 2016 Issue 1

T H E ARIZONA DEPARTMENT OF FINANCIAL INSTITUTIONS (AZDFI) MONITORS AND FOLLOWS THE FEDERAL DEPOSIT INSURANCE CORPORATION (FDIC), THE FEDERAL RESERVE BANK (FRB), AND THE OFFICE OF THE Comptroller of the Currency (OCC) auditing and examina- tion trends (Examination, n.d.). Reviewing the validation of internally developed and vendor models deployed at large banks is an FDIC and OCC priority. A strong belief exists that regulators will begin model validation reviews at mid- size banks. Model Validation Model validations help reduce model misuse and accuracy risk that can result in financial losses and reputation damage. A model validation should identify data, design, develop- ment, testing, deployment, and monitoring risks (Supervisory, 2011). The validation may not reveal that a model is optimal. However, a thorough validation will result in actionable, risk- reducing, recommendations. Components of model validations include conceptual soundness, data quality and timeliness, design and develop- ment processes, outcome analysis, and monitoring. Executive and detailed validation reports contain a critical evaluation of each component (Burns, 2005). Leaders should be able to read and understand validation documents without an interpreter or a statistics book. Benefits Financial regulators require that banks use external audit- ing resources (External Audit, 1996). Outside resources or internal groups, separate from the model development team, can perform the model validations and create the documents for regulatory review. Passing regulatory reviews is not the only reason to perform model validations. Other benefits of model validations include 1) gaining executive trust, 2) understanding data source risks, 3) identifying fair lending issues, 4) monitoring disparate impact, and 5) identifying model optimization opportunities. More than one banking leader has questioned their models’ accuracy and business value. Easy to understand validation reports that contain an overview, results, and recommendations can reduce the trust gap. Validations help discover prohibitive model inputs such as race, religion, sex, and gender. Using prohibitive inputs to influence decisions is illegal and can result in fines and reputa- tion damage. As fair lending monitoring increased, banks increased the use of model driven, automated decisions. Even with auto- mated decisions, disparate impacts can exist. Documenting adverse decisions via reason codes can help reduce disparate impact risk. Model validations should identify issues and provide recommendations that can resolve compliance issues, enhance models, and create best practices. What are the Benefits and Challenges of Model Validations for Banks? BY DR. ERIC GOLLA 18 www.azbankers.org