Averages and Realtime data – how to and how not to

Recently, I bought a new car as my old faithful chariot was becoming uneconomical to keep running. Having put my hard earned cash in the car dealer’s hand, I was the proud owner of a very nearly new motor!

I had an opportunity to drive 200 miles to go to see a client soon after the purchase, so I topped up the car’s tank and zeroed my super-duper trip meter before I set off.  My trip meter tells me my average fuel consumption for the duration of a trip as well as in real-time.  This is great for a data nut like me.

Brimming with fuel and enthusiasm, I floored it out of the gas station (up to but not exceeding 30mph officer) and my average consumption read an instant 18mpg!  Crumbs – my car must be really thirsty, right?  Nope.  10 miles down the road, my average consumption is now reading 43mpg – cool.  I check again 1 mile later and its up to 44mpg…at this rate I’ll be generating fuel soon!

I had to use the ‘loud pedal’ again later to ‘make some progress’.  Whilst doing this I looked at the real-time consumption – back down to 11mpg…wow, really bad…Having said that, later on, I was coasting into a 50mph limit and the real-time was back to 99.99mpg – awesome!

At the end of the trip I managed 43.9mpg – pretty happy overall.

So, what does this have to do with internet marketing and optimisation?  Well, having started out looking at a tiny data set, I drew a number of hasty and flawed conclusions about the performance of my car.  Imagine sending out a marketing email to your customers.  What can you tell about the performance of the email when 5 customers have clicked on the email and been to your site?  NOTHING!  The data set generated by the traffic is too small to draw any meaningful conclusions.

You CAN use smaller data sets (perhaps generated in realtime) to compare, say for example, two batches of emails sent to like-segments of traffic to see which one generates a stronger immediate response.  Treat the smaller data set with respect and contextualise the conclusions to be safe.

So, I saw by the end of the drive that I had a representative fuel consumption level for a variety of conditions – this is something I can budget for.  I can use this large data set to draw high confidence conclusions that help me plan my trip.  You can do the same with larger bodies of data.  Plan your business and make decisions based on data that you can be confident are representative of your true business performance.