Differences

The basic principle of analytics

The most basic principle of analytics is that the insight is to be found in the differences from the norm.

Analysis is the process of breaking a complex topic or substance into smaller parts to gain a better understanding of it.

Wikipedia

As you break apart a thing that needs to be understood  it becomes easier to make comparisons and spot differences that can lead to insight. There is an old saw that says “When a dog bites a man, that is not news. But if a man bites a dog, that is news.” The former is not news simply because it happens so often. It is the norm, if you like. This is illustrative of the point.

An occurence is not interesting (that is to say there is no insight to be had) if it happens so often that it is no different from any other occurence. The insight is in the differences not in things being the same.

Insight can be defined as actionable data together with an understanding of the effects of potential actions. A process for creating insight is :

  1. Find the differences
  2. Understand the reason for each difference
  3. Harness the understanding to make changes and so improve things

If every widget is 50% efficient except one that is 25% efficient the first thing to be done is to find out what is different about the lower efficiency widget. To do this identify all the other differences that might provide insight as to why the widget is less efficient. Once the reason has been found (say a different tension in the drive belt) the next step is to understand the difference and see if it can be exploited. If for instance tightening the drive belt a notch takes the widget to 37.5% efficient and two notches takes it to the same as all the other widgets (50%) then what does tightening it three notches do? Does it break the widget, lower the efficiency or has a way of improving efficiency been found that can then be applied to all widgets?

It is for this reason that it is best to try and vary as few things as possible whilst trying to understand a phenomenen. It may be that there are several (call it three) differences between the less efficient widget and the other widgets. First test each of the differences independantly and only when the effects are understood is it worth trying to combine the effects of more than one variable (*). This allows the cause of any differences to be well understood and so harnessed to improve things.

The principle of the insight being in the differences is so basic that it is often overlooked. It has been encoded into human beings as a basic survival trait and is thought so natural that people do not analyse it and so do not understand it. Without understanding the basics it is easy to be misled. As other principles of analytics are discussed the basic principle that the insight is in the differences will come up time and again both implicitly and explicitly.

Analysis is about finding a pattern that provides information that can be used. Patterns are simply anything that is not random. Randomness or white noise becomes homgenous and self similar if looked at from a distance. A pattern is therefore about the difference from a homgenous background. Mankind is an animal that finds patterns to survive and so one of our basic features is to look for differences from the norm. Beware of false positives where a pattern seems to exist and does not.

(*)Later an article on the startups.evison.com blog will talk about multivariate testing and the tamaguchi method for selecting combinations of variables to test when there is a large field but most problems are tractable using very few variables and a more exhaustive understanding can be acheived.