Big Data, Knowledge, and Hurricanes

This is a short analysis of WalMart's response to hurricanes, in order to explore some of the differences between Data, Information, and Knowledge.

Hurricane Irene 2011 by Walmart on Flickr

One of the most common and most famous "big data" stories is around Walmart and it's analysis of sales data.  

According to the New York Times in 2004 -

A week ahead of the storm's landfall, Linda M. Dillman, Wal-Mart's chief information officer, pressed her staff to come up with forecasts based on what had happened when Hurricane Charley struck several weeks earlier. Backed by the trillions of bytes' worth of shopper history that is stored in Wal-Mart's computer network, she felt that the company could "start predicting what's going to happen, instead of waiting for it to happen," as she put it.

The experts mined the data and found that the stores would indeed need certain products - and not just the usual flashlights. "We didn't know in the past that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane," Ms. Dillman said in a recent interview. "And the pre-hurricane top-selling item was beer." 

I would disagree with the New York Times. I think that what the data-mining experts found was Information in the form of a Correlation. The Knowledge would come in knowing what to do with that information (Knowledge, after all, being what enables correct action).

What would you do, dear reader, if you were a Walmart Executive who had been given this information on beer and pop tart sales? What action would you take?

• The New York times suggests that the stores would be stocked with extra supplies of these items when hurricane warnings were first announced. 
• This article suggests that store managers were told to put their Pop-Tarts near the store entrances during hurricane season (Pop Tarts, for the non-US reader, are  a brand of rectangular, pre-baked pastries made by the Kellogg Company. Pop-Tarts have a sugary filling sealed inside two layers of cardboard-like pasty, and are designed to be cooked within an electric pop-up toaster, therefore being an unlikely snack when power outages are forecast. Or perhaps you dip them in your beer).
• You could even consider raising the margin on these items, following the principles of supply/demand pricing.

The problem with the three approaches above is that the public is very suspicious of anything that might be interpreted as Profiteering. 

If Walmart is perceived to be making additional profit through public concern about imminent hurricanes, then the publicity backlash could be damaging. The Information found as a correlation within the big dataset must be treated with care.

As Peter Drucker said, "Information only becomes knowledge in the hands of someone who knows what to do with it" and luckily Walmart has the knowledge they need to use this information wisely.

What Walmart actually do 

Walmart have built what they call their own "best practice" for disaster relief.  In the US this includes

• Making sure their own stores are reopened as soon as possible 
• Providing merchandise for emergency responders, such as bottled water for first responders, office supplies for command posts, flashlights and batteries for search and rescue crews
• Donating pallets of dry food, soft drinks, flashlights, clothes, blankets and childrens board games
• Hosting massive barbecues in their parking lots, to feed citizens with no domestic power

 As a result WalMart is seen as a lifesaver, and it's disaster response procedure has been compared favourably with that of FEMA. They have won the hearts of the public, and of the administrators.

Walmart are winning on three counts. They have plenty of data, they analyse this data to derive information, and they have knowledge (based on experience and codified in best practice) that allows them to take the correct actions. It is that knowledge that allows them to know what to do with the information they receive.