Framing a question – get more value from your agency
Albert Einstein is often credited with the following quote:
“Everything Should Be Made as Simple as Possible, But Not Simpler”
Smart chap. We agree!
In the context of digital analytics, we consider a model to be a deliberately simplified representation of reality. For example, define business model:
See how the model is composed of very few moving parts. A business is a complex beast, but the model simplifies reality to fit into our understanding.
Following on from the Einstein quote, and seemingly contradictory, George Box is credited as having said:
“Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so over-elaboration and over-parameterization are often the mark of mediocrity.”
So, models are good and even though they’re all wrong, a good one can be useful. A good (useful) model is clear, precise and concise. A good model tells us what we need to know and nothing else.
This is a good approach a framing a business question. Reading ‘Keeping up with the Quants’ by Tom Davenport you will see that framing a question is a very important part of an analysis project. It’s fundamental to a successful relationship with an agency.
When might you choose to employ an analyst or agency to help resolve business issues?
- Something happened that was unexpected, and you want to know why.
- Something didn’t happen that should have, and you want to know why.
- You want to do something (test or campaign), but you’re not sure how or when is best.
- This could be a useful opportunity to capitalise on in the future, so you want to know the magnitude of the effects (good/bad or otherwise).
When you engage with an agency for analytical work, frame your question. Build a model to describe your question.
How do you build a model and frame a business question though? Easy to say, but much harder to do.
Some simple heuristics will help:
- Be relevant and specific – what is this about? Nothing else matters.
- Context is important. Leave out unnecessary cruft
- Identify important variables – what changes are you interested in?
- Describe expected effects – what should happen to your business?
- Be timely – what’s the time frame you’re interested in?
- Be realistic – don’t cook pie in the sky unless you have the recipe!
Consider a role reversal. Agencies will use the classic ‘elevator pitch’ method to win your business. 30 seconds of spiel to grab your interest and then they’ll poor honey in your ear (metaphorically…hopefully) to get your mark on a contract.
Then the expectations are reversed. Employ the same winning tactic on the agency. Pose the business question as an elevator pitch to introduce (frame) the question and then jointly tease the business problem into a project to work on.
Now, this doesn’t mean:
“My GA data seems wrong. Can you take a look?”
“My test tanked…”
“Your campaign was awesome! What next?”
“Let’s try a blue button instead.”
“The SEO is down. I need more copy. Can you give me some numbers?”
Such opening gambits will take you down a rocky road to failure. Try these approaches instead:
“The back end systems are showing surprisingly positive results for last month. Can we find out which channels are driving such an uplift to learn more?”
“It seems the data refutes the test hypothesis. What can we learn from the test to refine it?”
“Your campaign was awesome! Can we find out why it performed so well last month for mobile users?”
“From our last user survey, they reported that the buttons on the sign up form are hard to see against the background. Perhaps this is a good test?”
“Organic volume dropped 15% year on year. Can we investigate Webmaster tools? Has traffic dropped overall or is paid advertising giving a better return?”
None of these examples are in any way complete briefs but they are far better introductions to business questions/problems.
The focus is now on the analyst to work with you to refine your requirements and guide you to make data driven decisions that help you grow your business. Posing the question fully with framing massively increases the likelihood of success.