Two weeks ago, we published a post on how analytics aren’t so smart after all. It has been a sensation (only our Cat Modeler’s Guide to the Protection Gap has been read more this year), so it is only natural that we pursue the topic further.
The blog that lit up a little corner of the web was about the limitations of analytics, specifically flood risk analytics. It turns out there is some interesting science and folklore about the topic, so let’s take a look.
Analytics are an attempt to measure an immeasurable phenomenon. Cat risk analytics (flood, quake, hail, etc.) are perfect examples, as they aspire to determine the chance of something happening in a given location during a given timeframe – it is not possible. However, by evaluating a combination of different types of information, they can begin to produce results that are useful for underwriters who need those unknowable answers.
For those with a more scientific inclination, below is Anscombe’s Quartet. In 1973, the English statistician Frank Anscombe created a collection of four Cartesian datasets that had equivalent means of X, means of Y, variances of X, variances of Y, correlations, and linear regressions. What’s so peculiar about that? Check out the datasets.
Statistically speaking, they all share equivalent defining traits. Graphically, they are completely different from each other. Anscombe created this quartet to illustrate the importance of graphing data for evaluation instead of depending on the stats, but it also illustrates the fallacy of depending too much on analytics.
Anscombe’s lessons for cat underwriters might include the importance of looking at a risk on a map, and using as many different datasets as possible (there is no Anscombe’s Quartet). Actuaries and accumulation folks should also take note of the great influence of the outliers in the third and fourth sets. But the most important lesson is that underwriters need to understand their analytics, especially the weaknesses. Writing flood risk based on return periods from a single flood map is much like depending on the statistics to evaluate the Quartet – it looks right and makes sense, but it’s deeply flawed.
Since all folk tales, and most mathematical theorems, have a moral, let’s go with: Don’t be a blind underwriter using a few stats to build a portfolio – goodness knows what you’ll be left holding.