I See the World in Black and White

Once again, thanks to "Revolutionary" Dave for finding something interesting in the world of quant. The post-mortem is too long for the average trip to the loo, but the appendix on Value at Risk (VaR) which is the metric to be modeled, can be read. VaR is just a number of dollars assumed to be lost under varying circumstances (wiki piece). Dave's take on the Excel/R conflict is spot on.

The trader who had instructed the modeler to develop the new VaR model (and to whom the modeler reported at the time), CIO Market Risk, and the modeler himself also believed that the Basel I model was too conservative -- that is, that it was producing a higher VaR than was appropriate.

This is the heart of the continuing quant problem: unlike natural/mechanical processes, human processes (accounting and finance especially, since they only exist as games with arbitrary rules) are subject to "unexplained" perturbations. This is what Black Swans are. Black Swans exist in nature, the archetype being the Alvarez/Yucatan asteroid. The difference is that such natural Black Swans are exogenous to the process, while human Black Swans are largely endogenous; they happen because participants can and do change the rules for self-benefit. Such manipulation is generally impossible to model.

The Model Review Group performed only limited back-testing of the model, comparing the VaR under the new model computed using historical data to the daily profit-and-loss over a subset of trading days during a two-month period. The modeler informed the Model Review Group that CIO lacked the data necessary for more extensive back-testing of the model (running the comparison required position data for the 264 previous trading days, meaning that a back-test for September 2011 would require position data from September 2010). Neither the Model Review Group nor CIO Market Risk expressed concerns about the lack of more extensive historical position data.

Again, using more historical data may only have been useful for face-validity purposes. As always, the fiddlers fiddled:

There is some evidence the Model Review Group accelerated its review as a result of this pressure, and in so doing it may have been more willing to overlook the operational flaws apparent during the approval process.

While Excel is really not appropriate to any activity beyond vanilla accounting, the failure here (as with The Great Recession) was the result of Rand-ian self-interest. The real Adam Smith posited an economy in which each actor is not only autonomous, but also bereft of market power to impose collateral damage. Finance is all about collateral damage; that's how they earn it.

Here, it gets really interesting:

On January 26, the Model Review Group discovered that, for purposes of a pricing step used in the VaR calculation, CIO was using something called the "West End" analytic suite rather than Numerix, an approved vendor model that the Model Review Group had thought was being used. The Model Review Group had never reviewed or approved West End, which (like Numerix) had been developed by the modeler.

Turns out that Numerix is built on Excel (at least in part): "All Numerix functions are available via Microsoft ® Excel® as an add-in".

If you've ever inherited an analysis application executed in 1-2-3/Excel macros, you have my condolences been there done that.

In sum: while the use of Excel, even if it's also used by a major analytics vendor, is problematic; the cause, as with The Great Recession, was corruption and not only the Suits in this case. What's especially galling is that these quants are often some sort of math-y Ph.D., yet use Excel in haphazard ways. The poor will always be with us.