Greg Douma
At the height of World War 2, Allied airplanes were being shot out of the sky at an unbelievable rate, with the US losing 50-100 planes per day. Desperate for a solution, some officers within the War Department proposed adding armor to all bombing aircraft, so they would sustain less damage and hopefully could return home.
But if they added armor to all areas of the airplane, the plane would become too heavy to fly any missions.
So to what areas of the aircraft do they add the armor - the engines, wings, cockpit, fuselage, tail?
Deciding they needed more data to make the best decision, the War Department proceeded to inspect all airplanes that had survived their missions and returned home. Army men took count of all the artillery holes in the different sections of the aircraft, and this is what they found:
approximately 70% of the artillery damage was in the wings
approximately 16% in the middle fuselage
approximately 13% in the tail
Using this data, the military officers in charge of the project proposed adding armor to the wings of every bomber in the air fleet, in order to reduce damage and prevent further costs and death.
Makes sense, right?
Thankfully, before taking any action, the War Department consulted a statistician named Abraham Wald. Wald didn't need much time with the data before he simply told the War Department they were wrong. But how could that be, when they were making logical, objective decisions based only on the data?
It was because data is only as good as the people interpreting it, and Wald realized what the others had somehow left out of their equation: the thousands of aircraft (and pilots) that never returned.
Those planes that ended up as wreckage behind enemy lines, or sitting at the bottom of the lakes, rivers, and ocean, or were blown to pieces in the sky - those lost aircraft never made it into the data, simply because they could never be inspected.
So the question still remained: where to add the plane-saving armor?
The only data available to Wald was where every returning plane was hit: in the wings, the fuselage, and the tail. And not the engines or cockpit. Therefore, relying on the laws of probabilities, Wald was able to deduce that these planes were left intact because they were not hit in the cockpit or engines. Therefore, the planes that were shot out of the sky to never return must have been hit in those crucial areas.
Wald convinced the War Department that all bombers must be armored in the cockpit and engine areas. This decision ultimately proved to be a successful solution and saved thousands of aircraft (and lives) over the remaining course of the war.
Survivorship bias is real, and is the most prevalent bias we encounter with our data-driven clients.
So what lessons can we, as Retail Professionals, learn from this?
Lesson #1 Data is Priceless
Wald was able to find the best solution to the War Department's problem because the military had done all that work of inspecting every bullet hole in every airplane. So do the hard work of recording every single incident of loss in your buildings - whether attempted or successful, theft or otherwise.
Lesson #2 Data Can Tell Conflicting Stories
The War Department and Wald arrived at two opposite solutions after reading the exact same data sets. Similarly, you could compare two stores with vastly different amounts of defeated sensor tags found in fitting rooms and say that one store clearly has more theft, or one store is throwing out the sensor tags they find, or has an ORC group that steals items with the tags attached. You could take two different LP Teams with vastly different apprehension statistics and say that one team is clearly more effective at arresting shoplifters, or the other team is clearly more effective at preventing shoplifting. Any combination of the two, or neither of the two, could be true because:
Lesson #3 Data Can Hide Puzzle Pieces
Wald was able to find the best solution to the War Department’s problem because he knew he didn’t have all the data, and he knew which pieces of the puzzle were missing.
The NRF continues to publish annual surveys that state shoplifting is the top cause of shrinkage. The reason for that is simple: that’s the only data that many retailers have. Stores with $1 Million in total shrink will report that they lost $30k to shoplifters, and in the next moment will say that that’s their largest source of loss. If you can only speak to 3% of your total loss, common sense would tell you that 97% of your loss is completely unknown or uninspected.
Lesson #4? To be continued....
