Probabilities are Better than Scores

During a panel at Fair Isaac’s Interact conference last week, a banker from Abbey National in the UK suggested that part of the credit crunch was due to the use of the FICO score.  Unlike other panelists, who were former Fair Isaac employees, this gentleman was formerly of Experian!  So there was perhaps some friendly rivalry, but his point was a good one.  He cited an earlier presentation by the founder of Strategic Analytics that touched on the divergence between FICO scores and the probability of default.  The panelist’s key point was that some part of the mortgage crisis could be blamed on credit scores, a point that was first raised in the media last fall.

The FICO score is not a probability. 

Fair Isaac people describe the FICO score as a ranking of creditworthiness.  And banks rely on the FICO score for pricing and qualification for mortgages.  The ratio of the loan to value is also critical, but for any two applicants seeking a loan with the same LTV, the one with the better FICO score is more likely to qualify and receive the better price.

Ideally, a bank’s pricing and qualification criteria would accurately reflect the likelihood of default.  The mortgage crisis demonstrates that their assessment, expressed with the FICO score, was wrong.  Their probabilities were off.

Super Crunchers: predictive analytics is not enough

Ian Ayres, the author of Super Crunchers, gave a keynote at Fair Isaac’s Interact conference in San Francisco this morning.   He made a number of interesting points related to his thesis that intuitive decision making is doomed.   I found his points on random trials much more interesting, however.

In one of his examples on “The End of Intuition”, a computer program using six variables did a better job of predicting Supreme Court decisions than a team of experts.  He focused on the fact that the program “discovered” that one justice would most likely vote against an appeal if it was labeled a liberal decision.    By discovered we mean that a decision tree for this justice’s vote had a top level decision as to whether the decision was liberal, in which case the program had no further concern for any other information. 

Agile decision services without XML details

Externalizing enterprise decision management using service-oriented architecture orchestrated by business process management makes increases agility and allows continuous performance improvement, but…

How do you implement the rules of EDM in an SOA decision service? 

Behind the CEP curtain – it’s about time, not the cache

TIBCO is the CEP vendor most focused on the market for business rules, as reflected in Paul Vincent’s post here.  Although I agree with Paul that rule vendors are not currently offering enough in terms of support for long-running processes, the conclusions that he draws in favor of considering a CEP alternative to a BRMS are not compelling yet.

Paul said that rules don’t address the following that are addressed by CEP:

1. BAM (business activity monitoring) and the other BPM (business performance management)
2. Complex-rule processing
3. Customer-centric (portfolio-based) decisions / policies

I am sure Paul was just being flippant, but you may notice that there is a bit of a war going on between CEP, BPM and rules right now. 

Externalization of rules and SOA is important – for now

James Taylor’s notes on his lunch with Sandy Carter of IBM and the CEO of Ilog prompted me to write this.  Part of the conversation concerned the appeal of SOA and rules to business users.  Speaking as a former vendor, we all want business people to appreciate our technology.  We earn more if they do.  They say to IT “we want SOA” or “we want rules” and our sale not only becomes easier, it becomes more valuable.  So we try to convince the business that they are service-oriented, so they should use SOA.  Or we tell the business that they have (and make) rules, so they should use (and manage their own) rules.   And rules advocates embrace and enhance the SOA value proposition saying that combined, you get the best of both worlds.  This is almost precisely the decision management appeal.  Externalize your decisions as services and externalize rules from those services for increased agility in decision making.   This is an accurate and appropriate perspective for point decision making.  But it doesn’t cover the bigger picture that strategic business people consider, which includes governance and compliance. 

Nonetheless,

Effective SOA and business rules have one requirement (or benefit) in common: externalization.  

The externalization of services from applications

CEP crossing the chasm into BPM by way of BRMS

Complex event processing (CEP) software handles many low-level events to recognize a high-level event that triggers a business process.  Since many business processes do not consider low-level data events, BPM may not seem to need event processing.  On the other hand, event processing would not be relevant at all if it did not occasionally trigger a business process or decision.  In other words, it appears that:

• CEP requires BPM but
• BPM does not require CEP

The first point is market limiting for CEP vendors.  Fortunately for CEP vendors, however, most BPM does require event-processing, however complex.  In fact, event processing is perhaps the greatest weakness of current BPM systems (BPMS) and business rules management systems (BRMS), as discussed further below.

Agile Business Rules Management Requires Methodology

Don’t miss the great post about his and Ilog’s take on rule and decision management methodologies by James Taylor today (available here). 

Here’s the bottom line:

1. Focus on what the system does or decides.
   • Focus on the actions taken during a business process and the decisions that govern them and the deductions that they rely on.
   • In priority order, focus on actions, then decisions, then deductions.
2. Don’t expect to automate every nuance of an evolving business process on day one – iterate.
   • Iteratively elaborate and refine the conditions and exceptions under which
     • actions show be taken,
     • decisions are appropriate, and
     • deductions hold true

Another way of summing this up is:

Try not to use the word “then” in your rules!

Do check out the Ilog methodology as well as the one I developed for Haley that is available here.  The key thing

Process vs. Decisions

In comments to a recent post concerning the acquisition of Haley Systems by Ruleburst, James Taylor suggested that a “decision-centric” perspective is necessary for business rules to become mainstream.  In subsequent correspondence, I questioned whether fixating on decisions would achieve his objectives for enterprise decision management.   EDM hopes to integrate business intelligence (e.g., predictive analytics) with point decision making so as to improve decision making over time.  This is a natural step beyond the typical point decision making application of business rules, such as in a stateless web service that returns a simple decision, such as a score, price or simple yes/no.  But it is a narrow perspective on the broader confusion between business rules and business process that has been holding back the mainstream.

For years, smart people have been searching for a razor to determine what logic they should “code” in process versus as rules (e.g., using a BRMS versus their BPM platform).  At first glance, the decision-centric approach seems to have the answer.  Simply put a decision node in your business process diagram and let the BPM tool orchestrate the decision implemented as a stateless web service! 

Unfortunately, this alluring answer is all too often inadequate or impractical.  The business rule vendor has effectively transfered responsibility for managing state (i.e., information collection and provisioning) into the business process diagram and orchestration tools – or code.  The result is implementation complexity, limited user communities, cost overruns and failures.  That will certainly hold back mainstreaming a bit!

A better answer is coming.  Complex event processing anticipates that business processes and decision making can be stateful, as Paul Vincent explains briefly but well here.  When CEP is supported by knowledge capture, management and automation tools such as the better BRMSystems provide, the lines between process specification and decision specification will further blur beyond the adequacy of the decision-centric advisory.   Expect this to happen in 2008.