This blog post has been written as part of a project I am working on to produce a series of short books that will act as guides to different aspects of market research. The specific post looks at two key aspects of writing up market research results, i.e. differentiating between the ‘facts’ and the judgement/opinion elements, and using past, present, and future tense to make reporting clearer and more actionable.
I am very keen to hear other people’s views on the advice in this post – all contributors to this series will, of course, be listed and thanked.
Market research results consist of two elements, which we can loosely call:
- Facts
- Judgement/opinion
We can argue about the meaning or existence of facts, but in this case I am talking about the material revealed by market research that is not disputed. For example, we might find that 75% of the sample said they were male.
The term judgment (or opinion, or insight) covers things such as:
- How good/appropriate you think the research was.
- What you think the research means. For example, you might discover that trial is an important driver of purchase.
- What you think the client should do. For example, you might recommend that the client launch product A with campaign X and target it at young professionals.
- Predictions. For example, you might conclude that the demand for this product will decline over the next 18 months.
Separating the facts from judgement
When writing up the results (as a presentation, report, or even in an email) it is important to separate the facts from the judgement/opinion. This is especially true if the opinion turns out to be wrong to help you identify what went wrong. Knowing the difference between fact and opinion is an important part of the process of creating recommendations and predictions.
In some cases the client is focused on the facts, but in many cases it is the judgement that creates the real value of market research – treating it all as facts can devalue the process of adding insight, interpretation, and recommendations.
The facts tend to be in the past tense
Market research is typically based on using surveys, discussions, or observations. In almost all cases these produce information (facts) that are in the past. In a survey we might ask how often somebody shops online, or what advertising they have seen, or their satisfaction with their last service experience. Even if we ask research participants to make predictions (e.g. “How likely are you to buy a new smartphone next year?”) those predictions were made in the past.
When you report facts, use the past tense to help identify them as facts and to remind the reader that they tend to be a ‘rear-view mirror’ view of the world. For example, for example we would not say “Customers prefer the red option to the blue option.”, we would say something like “The sample of customers said they preferred the red option to the blue option.”
In many cases the facts can be best represented via charts or tables to show that the information is data, as opposed to opinion. For example:
“In response to ‘Which of these two options do you prefer?’ the responses were:
- Red 67%
- Blue31%
- Neither 2%”
Most research facts are hearsay
Observational research (for example passive data collected from mobile devices or websites) is about what people have actually done. However, most research, at the moment, is based on survey responses and things said in discussions (for example in a focus group or during an online discussion) and may not be true in an objective/scientific sense. There are several reasons why what people say may not be ‘factually correct’, including:
- Confusion, sometimes the questions market researchers ask are not clear to every research participant, which can result in the participant entering data that is not correct. For example, if we ask how many rooms are in your home, some participants might include the bathroom, whilst others might not – unless the instructions are very clear.
- Mistakes, sometimes participants make a mistake. They might click the wrong option, or they may simply make a mistake when recalling something, for example what supermarket did you visit most recently.
- Social desirability bias, some questions tend to elicit answers that make people appear ‘better’. For example, if you ask people if they eat too much, drink too much, exercise regularly, or visit adult entertainment sites, many participants will give answers that do not correctly reflect their behaviour.
- Being poor witnesses to our own motivations and beliefs. Writers such as Daniel Kahneman Mark Earls have highlighted that people are often unaware of why they do things. Asking people questions like “How motivated by price are you?”, “Which attribute was most important in your decision?”, or even “Will you buy this product in the future?” are likely to elicit inaccurate results.
The best way to flag up that the ‘facts’ are hearsay is to insert words such as ‘said’ into that section of the reporting. For example say, “75% of participants said they would buy the product in the future.”, rather than “75% will buy the product.”
Clients are focused on the future, not the past
Generally, the people who buy, commission, read, and use market research results are looking for guidance in what they should do next. For example, should they launch a new product, change a service, shift resources, etc.
The guidance they are seeking is typically opinion (i.e. is not simply a reporting of the data as collected) and it needs to be focused on the future. Sometimes, the guidance might focus on describing the current market, but usually with the intention of using that information to manage the future.
This dichotomy between a client’s need (future focused) and research results (firmly based in the past) highlights the importance of judgement. It is judgement/opinion that that translates the data into insight and from there into actionable recommendations.
Use current and future tense for opinion
To signal to the client that the conclusions and recommendation are judgment or opinion-based, and to make them more action orientated, use the current and future tense.
Use the current tense to describe the world as you believe it is today.
Use the future tense for:
- Predictions.
- Recommendations.
- Next steps.
For example, after conducting a concept test you might report the following:
The Facts: 1000 customers were interviewed (to the normal, representative specification) and 25% said they would definitely buy the product and a further 40% said they would try it. [past tense, third-person, passive]
What it means: Compared with other studies and our benchmarks we believe this is a very good result, showing there is a demand for this sort of product. [Current tense, first person] If the product is launched in the near future, and is consistent with the proposition tested in the research, we would expect it sales in the first 12 months to be in the region X to Y. [future tense, first person]
Recommendations: From a customer demand point of view we would recommend launching the product, and launching it before either demand changes or some other product takes its place. If there are legal, marketing, or logistic issues that have not been fully resolved, then these should be tackled as a priority. The implementation plan should include further research to ensure that the product being launched aligns with the concept that has been tested. [future tense, first person]
Algorithms are opinion too!
A sales forecast from a proprietary concept screening program, or a brand index score from a brand tracker, or a sentiment score from a social media monitoring system are not facts.
I would recommend adopting the following conventions when reporting results and forecasts based on algorithms:
- The data produced the following scores – past tense.
- This means A, B, C – current tense.
- Therefore we think that you should do X and we predict Y will happen – future tense.
The balance between facts and judgement
There is no general right and wrong answer to the question how to balance the facts and the judgement.
For some clients and for some projects, the main deliverable is the collection of facts. For example, many tracking projects are focused on the week-by-week data, with the changes in the numbers being highlighted. When a project is mostly about the facts, the reporting will normally be structured around those facts, for example:
- A summary of what was done, the key results, and key conclusions/recommendations.
- The key facts (for example the tracking data as charts or tables).
- A set of conclusions and recommendations.
- The detailed data – perhaps as a separate document, file, or database. This might include the questionnaire and the cross-tabs for example.
However, in many cases it is the judgment that is the key component. If the project is mostly about the analysis and interpretation of the research, then the reporting should reflect this, for example:
- Executive summary, what does the client need to know and what do you think they should do.
- A minimal description/background – comprising only those things that a reader should know before assessing your conclusions, recommendations, and predictions.
- The judgement/opinion piece.
- The detailed data – usually as a separate document, file, or databases.
Your Thoughts?
As I said at the start of this post, I am very keen to hear other people’s views on the advice in this post. For example:
- Is it clear?
- Do you agree with the basic proposition?
- Would you add, delete, or amend anything?
