During a panel at Fair Isaac’s Interact conference last week, a banker from Abbey National in the UK suggested that part of the credit crunch was due to the use of the FICO score. Unlike other panelists, who were former Fair Isaac employees, this gentleman was formerly of Experian! So there was perhaps some friendly rivalry, but his point was a good one. He cited an earlier presentation by the founder of Strategic Analytics that touched on the divergence between FICO scores and the probability of default. The panelist’s key point was that some part of the mortgage crisis could be blamed on credit scores, a point that was first raised in the media last fall.
The FICO score is not a probability.
Fair Isaac people describe the FICO score as a ranking of creditworthiness. And banks rely on the FICO score for pricing and qualification for mortgages. The ratio of the loan to value is also critical, but for any two applicants seeking a loan with the same LTV, the one with the better FICO score is more likely to qualify and receive the better price.
Ideally, a bank’s pricing and qualification criteria would accurately reflect the likelihood of default. The mortgage crisis demonstrates that their assessment, expressed with the FICO score, was wrong. Their probabilities were off. (more…)
Ian Ayres, the author of Super Crunchers, gave a keynote at Fair Isaac’s Interact conference in San Francisco this morning. He made a number of interesting points related to his thesis that intuitive decision making is doomed. I found his points on random trials much more interesting, however.
In one of his examples on “The End of Intuition”, a computer program using six variables did a better job of predicting Supreme Court decisions than a team of experts. He focused on the fact that the program “discovered” that one justice would most likely vote against an appeal if it was labeled a liberal decision. By discovered we mean that a decision tree for this justice’s vote had a top level decision as to whether the decision was liberal, in which case the program had no further concern for any other information. (more…)
In comments to a recent post concerning the acquisition of Haley Systems by Ruleburst, James Taylor suggested that a “decision-centric” perspective is necessary for business rules to become mainstream. In subsequent correspondence, I questioned whether fixating on decisions would achieve his objectives for enterprise decision management. EDM hopes to integrate business intelligence (e.g., predictive analytics) with point decision making so as to improve decision making over time. This is a natural step beyond the typical point decision making application of business rules, such as in a stateless web service that returns a simple decision, such as a score, price or simple yes/no. But it is a narrow perspective on the broader confusion between business rules and business process that has been holding back the mainstream.
For years, smart people have been searching for a razor to determine what logic they should “code” in process versus as rules (e.g., using a BRMS versus their BPM platform). At first glance, the decision-centric approach seems to have the answer. Simply put a decision node in your business process diagram and let the BPM tool orchestrate the decision implemented as a stateless web service!
Unfortunately, this alluring answer is all too often inadequate or impractical. The business rule vendor has effectively transfered responsibility for managing state (i.e., information collection and provisioning) into the business process diagram and orchestration tools – or code. The result is implementation complexity, limited user communities, cost overruns and failures. That will certainly hold back mainstreaming a bit!
A better answer is coming. Complex event processing anticipates that business processes and decision making can be stateful, as Paul Vincent explains briefly but well here. When CEP is supported by knowledge capture, management and automation tools such as the better BRMSystems provide, the lines between process specification and decision specification will further blur beyond the adequacy of the decision-centric advisory. Expect this to happen in 2008.