In preparation for a post later this month, about the ‘usefulness’ of risk heat maps …
Against the most basic of common sense, some people [most of the ones working in the areas where this should matter a lot, near life-or-death levels of a lot, e.g., banking supervision and compliance] still hold onto ‘heat maps’ and similar outright oversimplicity.
Even when pointed out over and over again where the errors are, these people seem not to see the latter, and cling onto what they had received from … some other ones, equally flawed in their prefrontal cortex but with apparent authority for no detectable reason. The argumentam ad baculum… Just because someone has a big stick, doesn’t make that person right – look at all the dictators around the world; almost all the continents have at least one (and I don’t mean Mexico or Canada).
David Freedman (in Nassim Taleb’s Black Swan) listed their standard arguments for using the flawed model(s) still, despite their stupidity. Typically, your (?) standard arguments. I’ve added some rebuttals:
- Yes we know all that. Nothing’s perfect.
But if something is completely bonkers, built on quicksand, you will sink with it. - The assumptions are reasonable.
No they’re not. And as if that’s enough. The model is still nonsense and has nothing to do with reason let alone reasonableness. - The assumptions don’t really matter.
[Ah, the flip side of the previous; so you acknowledge that one was nonsense] So, why have them, then? Or admit they do matter and you better have the right assumptions, and have them right and met. - The assumptions are conservative.
Conservative to which side? Why not use best estimate assumptions? - You cannot prove the assumptions are wrong.
No, you prove they are right … Ah, you fail systemically there. For a reason. And I don’t need to prove anything about assumptions, just show that they’re the wrong ones. - We only do what everyone else does.
‘Everyone’ it isn’t, and a lot of people [way too many as it is > 0] commit suicide – I don’t encourage it but if that is your wish… ‘Everyone else’ is no guarantee for reasonableness, you don’t even know what everyone else is doing, and when you know they’re doing the wrong thing, why follow? You jump off a cliff if they do? And then find out they prepared with parachutes, maybe. - The decision maker is better off with us than without us.
No. You actually mislead the decision maker, which is an offense and may constitute fraud. - The models are not completely useless.
Yes they are. Otherwise, you could pick the parts that are not-useless, and glue them together while adding not-useless parts as needed. Since you don’t, you have models that are useless. - You gotta make the best of the data you’ve got.
When that’s not enough, it’s simply not enough. When the decision maker jumps off the cliff as you propose (s)he should, having a handkerchief doesn’t substitute for a parachute, even if (s)he’s making the best of what (s)he has available. And the error is yours; see item 7. - You need assumptions to make progress.
You need valid assumptions. That would completely obliterate your models (‘ validity). You don’t need to make progress, neither in a Wrong direction that you take when using flawed assumptions, nor at all after you’ve crafted a valid model (which you didn’t). - The models deserve the benefit of the doubt.
No. Why? If is demonstrably False, it deserves no life. - Models and assumptions don’t do any harm so why bother …?
Oh they do do harm, a lot! They wreck organisations, hence they wreck the lives of countless employees and their families, for at least two generations.
So, let’s not pretend like procrastination is a good thing or so. At best, you are stumbling along. Change, or fall.
On the positive side:
[You may end up as a piece of Art; Zuid-As Ams]
2 thoughts on “RM with admitted-flawed models”