Shoppers don't exist in a spreadsheet

There seems to have been a lot of debate around number-crunching recently, including the need for "big data" to be able to have any chance of competing in today's market. Dunn Humby continue to offer ever more complex models and tools, there's an endless queue of data suppliers at my door and a worrying obsession with getting more data, more quickly and more often.

But does complicated data actually tell us anything? No matter how rich and detailed the data is - and we have lifestyles, life stages, affluence, ethnicity, regionaliy, mission, shopper behaviour and even actual sales to choose from - it's always using historical perform to try and predict the future. Unfortunately, shoppers don't act in a rational and logical manner - if they did, there'd be quite a few thousand less jobs in retail and FMCG overnight. I've worked on many ranging projects from different angles over a number of years and every one has had at least one major oversight caused by putting too much emphasis into the output of a spreadsheet and too little trust of retail instinct.

This manifested itself in being virtually lynched in a trial store in Sheffield for delisting Henderson's relish to seeing a SKU our modelling told us had 4 weeks cover on shelf sell out in 2 hours. Data suggested that a relatively expensive small bottle of fresh fruit call Innocent wouldn't sell while a tiny caffeine-packed chocolate bar called Go-co would, while the reverse was true. That was 10 years ago when data wasn't as all conquering and we found out what would actually work by taking a punt and giving them both a go.

But a reliance on data is gradually eroding the old retailer instinct. Young buyers now just point at a spreadsheet and seem unwilling or unable to take a gamble or even grasp the concept of "breadth not depth" when creating a range. Multiples' c-stores have appalling ranges created by just picking from a list of SKUs not even ranked by sales, but by profit contribution. 

And then there's clustering.

On one of my categories has 11 different clusters for a convenience channel in a retailer. Whilst shoppers do clearly shop differently along affluence, mission even ethnic lines, creating a cluster of stores almost becomes a self-fulfilling prophecy and also assumes that everyone within a group behaves the same. At worst, an obsession with meeting a clusters' needs means you reduce the core range which serves everyone's needs. It assumes that because you are from an Asian background, you eat sweeter foods, it assumes that if you live in a lower affluence area that you will only buy the cheapest products and never ever trade up. It assumes because you're relatively wealthy, you want Bolivian baked beans in sundried tomato sauce rather than Heinz originals. Well, it does if you just read the data.

Aldi and Lidl don't use this data, don't cluster and are doing ok without it. They are the masters of identifying a core range and applying it broadly. The concept of the core range elsewhere seems to be lost - do you believe anyone in Co-op actually KNOWS what their core range is after several of various failed format options? Ranging processes have changed - most retailers have moved from a top-down approach to a bottom-up, but there's still relatively little time spent on the smaller store plans. The biggest plans may deliver the biggest sales, but your core range reaches all your customers so arguably should have the most time spent on them. This isn't just about looking at the top performers, even looking at complicated composite ranks - although both things are valuable inputs. It's about getting to know your shoppers, meeting your shoppers face to face, watching them in stores and learning about the choices made in front of the fixture. But most of all, it's about empathy - we're all shoppers, yet we seem to forget that when there's some sexy new charts on a PowerPoint waved in front of us.

Now, I'm no Luddite, in fact I'm quite a geek with data myself. However, I never forget that data alone cannot be used to make good solid retailing decisions. Compared to when I first started looking at formats 15 years ago, the available data is so much more advanced and often genuinely excited. But it is crazy to think we'll ever get to a position where it can fully replaced retail nouse.

That doesn't mean it won't - as each generation comes through more used to data, there's less and less real retailing knowledge out there, not helped by a generation of store managers who are great implementers of centralized plans yet have no ability or training to trade on their own.