Synthetic audience research is testing ideas, scripts and messages against a panel of AI-based personas before investing in production and media. It is fast, so it suits the early stage when you choose between directions and sharpen messages. Academic research shows it can reflect trends at the group level, but also that it tends toward superficial and overly positive answers, so it complements market research with real people rather than replacing it.
In brief
- כלי למיון, לא לחיזוי. Synthetic personas are good at identifying weak directions and raising questions, and less good at predicting behavior precisely.
- הראיות מעורבות. Studies have found good agreement with human data at the group level in some cases, and significant deviations in others.
- ההקשר המקומי חשוב. Studies have found that language models tend to reflect mainly Western populations, so personas need to be built and checked against knowledge about the local audience.
- המקום הנכון הוא לפני ההפקה. Test between the idea and the shoot, and after going live keep measuring against a real audience.
What is synthetic audience research?
Synthetic audience research is a method in which marketing materials are presented to a panel of personas created with a language model, and their responses are analyzed as you would analyze a research group. Instead of recruiting participants, you build a simulated audience meant to reflect the audience segments relevant to the brand.
An AI persona is a simulated consumer profile, with characteristics such as age, region, family status, consumption habits and attitudes, which a language model embodies when answering questions. A synthetic panel is a collection of such personas, built to cover the diversity of the target audience.
In practice, you present the panel with an idea, script, headline or message, and get responses, explanations and a comparison between alternatives. You can hold an in-depth interview with a single persona to understand what lies behind its response, or simulate a group discussion in which different personas respond to each other. The output is a picture of what works, for whom and why, more than a single number.
What does AI-based market research rely on, and how does it work?
AI market research relies on language models' ability to reproduce patterns of attitudes and preferences that exist in the texts they were trained on, when given a detailed description of a particular person. The more the description is based on real data, the more useful the answers.
One of the most cited studies in the field, by researchers from Brigham Young University and posted as a preprint on arXiv in 2022, tested the idea systematically. The researchers gave GPT-3 backstories of real respondents from American election surveys, and asked it to answer as they would. They called the method silicon sampling, meaning building a simulated population from profiles of a real population.
They also proposed an important concept: algorithmic fidelity, the extent to which the relationships between attitudes, ideas and demographic characteristics in the model reflect those in human groups. According to the findings, the model often reproduced the group picture closely, and human evaluators had difficulty distinguishing between texts it wrote and texts written by people. However, the agreement was weaker for some groups, and the researchers emphasized that it is assessed at the group level and not at the level of the individual, and that the method is mainly suited to preliminary testing and developing hypotheses.
What does research say about the limits of synthetic respondents?
Research points to three main limitations: inaccurate answers to quantitative questions, a bias toward Western populations, and a tendency toward superficial and overly positive answers. Each of them affects how the method should be used.
Accuracy on quantitative questions
A 2023 Harvard Business School working paper compared willingness-to-pay estimates obtained from language models with estimates from studies with people. For some products the model's estimates were close to those of participants, and for others the gap was large, and sometimes the trend even reversed. When the researchers trained the model on previous surveys, it improved at estimating new product features in the same category and with the same audience. In a new category, or when comparing customer segments, the training didn't improve the results. So the authors recommend using language models as a complement to research with people.
Who is actually represented
A study called "Which Humans?", posted as a preprint in 2023 by Harvard researchers, examined the similarity between language model answers and psychological data from different cultures. The researchers found that the models answer mostly like people from societies the research literature calls WEIRD: Western, educated, industrialized, rich and democratic. The further a population is from this profile, the less similar the model's answers are to its own. In a market with diverse audiences like Israel, this means segments far from the profile the model knows best need to be examined with special care.
Depth and over-positivity
Nielsen Norman Group, a user experience research and consulting firm, compared synthetic users in 2024 with three studies with real participants. The synthetic users tended to be enthusiastic even about ideas real participants responded to coolly, gave general answers, and had difficulty distinguishing between what really matters to them and what matters less. NN/g's conclusion is that the method helps formulate hypotheses and prepare research with people, for example trying out an interview guide in advance, and that when presenting the results you should state explicitly that they come from a simulation.
What is the difference between traditional market research and synthetic audience research?
Traditional market research measures real people and provides evidence for a decision, while synthetic audience research is faster and suits early exploration and screening between directions. The question is what fits each stage.
| Criterion | Traditional market research | Synthetic audience research |
|---|---|---|
| Speed | Days to weeks, depending on recruitment and fieldwork | Results in a short time, and the test can be repeated immediately |
| Depth | Real reactions, including surprises and emotion | Detailed explanations, but sometimes superficial and uniform |
| Diversity | Limited to the participants actually recruited | Many segments can be covered, subject to persona quality |
| Predictive reliability | Direct evidence, subject to sample quality | Weak on quantitative questions and niche audiences |
| Main biases | Sample bias, moderator influence | Over-positivity, bias toward Western populations |
| When to use | Big decisions, pricing, demand, validation | Screening directions, sharpening messages, preparing real research |
In many cases the right approach is a combination. Synthetic research narrows many directions down to a few promising ones and sharpens questions, and research with people, or an in-market test, validates the decisions that are truly expensive.
When does synthetic audience research help with creative testing?
Synthetic creative testing helps when you need to choose between several directions, sharpen wording or uncover a risk early, before investing in production and media. It is strongest at comparison, and weaker at absolute scores.
- בחירה בין כיוונים. When there are several ideas on the table and you need to decide which to invest in.
- חידוד מסרים. To understand which wording is clearer, more credible and more relevant for each audience segment.
- זיהוי סיכונים. To uncover early a message that may be misinterpreted by a particular audience.
- הבנת ה"למה". In-depth interviews with personas help you understand what lies behind a response, not just whether it is positive or negative.
- הכנה למחקר אמיתי. To try out a questionnaire or interview guide before going into the field.
There are also cases where it's better not to rely on it alone. Decisions about price and demand, very specific niche audiences, and topics where the response depends on deep personal experience require data from real people. In those cases synthetic research can help formulate the questions, but not answer them.
How do you integrate synthetic audience research into the process without falling into the traps?
Integrate it as a defined screening stage: a narrow question, a comparison between alternatives, and validation with a real audience for the big decisions. That way you get the speed without attributing a certainty to the results that they don't have.
- Define the question: what you want to decide, and for which audience.
- Make sure the personas are based on real knowledge of the audience, not on stereotypes.
- Test several directions side by side, and look at the gaps between them, not just each one's score.
- Ask for reasons and objections, not just ratings, to balance the tendency toward positivity.
- Sharpen the chosen direction according to the insights, and test again if needed.
- Move into production with a better-grounded brief, and keep measuring against a real audience in market.
Two more rules keep the use of the method fair and transparent. First: present synthetic results as insights that require validation, not as the opinion of consumers. Second: read every research finding in the context of the model and the market in which it was tested. The studies mentioned here tested certain models in certain contexts, for example GPT-3 against American election surveys, and as of the article's last update the field continues to evolve quickly.
How we approach this at CONNECT
At CONNECT, part of the Bruckner Yaar Levi advertising group, we use synthetic audience research as a screening stage before production, not as a substitute for measurement. It helps us arrive at the creative meeting with fewer directions and more reasoning, and write a better-grounded production brief. After going live, the data from the real audience is what decides.
For this we work with AudienceLab, a system developed in the Bruckner Yaar Levi group for early testing of ideas, scripts and messages against a panel of 200 Israeli synthetic personas. It includes in-depth interviews, a group discussion room and a Go, Revise or No-Go recommendation for each direction. Its insights are a simulation, and we weigh them alongside professional knowledge and data from a real audience. More about AudienceLab.
Frequently asked questions
What are AI personas?
AI personas are simulated consumer profiles that a language model embodies when answering questions. Each persona has characteristics such as age, place of residence, family status, consumption habits and attitudes. A collection of such personas creates a panel that simulates the target audience, to which you can present ideas, scripts and messages. The quality of the answers depends largely on the quality of the data the personas were built on.
Can AI market research replace a survey or a focus group?
Not entirely. Academic studies have found that language models can reflect group trends in some cases, but also that they make mistakes on quantitative questions such as willingness to pay, and tend toward positive and superficial answers. So AI market research suits the early stage, screening directions and sharpening questions, while big decisions such as pricing, demand or a product launch still require data from real people.
How accurate is synthetic audience research?
Accuracy varies greatly by topic, audience and how the personas were built. Studies show better agreement at the group level than at the individual level, and on questions of direction and comparison more than on quantitative questions. Agreement is weaker for audiences far from the populations the models know best. So it's best to treat the results as well-founded hypotheses, and validate the important ones.
When is it worth testing creative before production?
It's worth testing creative before production when there are several possible directions, when the production or media budget is significant, or when the message touches a sensitive topic that may be misinterpreted. At this stage a change costs little, and after the shoot it costs a lot. Early testing helps choose a direction, sharpen wording and identify risks, and afterwards you keep measuring the campaign against the real audience.
Is synthetic audience research suitable for the Israeli market?
It can be, provided the personas are built on real knowledge of the Israeli audience and not only on what the model knows in general. Academic research found that language models mainly reflect Western populations, so in a diverse market like Israel it's especially important to test segments that differ from one another. A panel built specifically for the local audience can help narrow this gap, but it is not a substitute for validation with real people, and the important findings still need to be verified.
ראו גם: Technology & AI in Marketing · המדריך ל-AI בשיווק · חוב מותג · תכנון מדיה שנתי · Technology & Tools
Sources: Argyle et al., Out of One, Many: Using Language Models to Simulate Human Samples (arXiv) · Brand, Israeli & Ngwe, Using LLMs for Market Research (SSRN, HBS Working Paper 23-062) · Atari et al., Which Humans? (Harvard CID) · Nielsen Norman Group – Synthetic Users: If, When, and How to Use AI-Generated Research
רוצים לבדוק את הקמפיין הבא לפני ההפקה? השאירו פרטים.