This blog post originally ran in QCRA Views. You can read the unabridged version, with references, here.

As research professionals, our success relies on delivering powerful insights. These insights help organizations solve challenges and achieve their goals. The more consistently we deliver, the more valuable we become.

However, the search for insights isn’t new. People have pursued understanding and innovation for millennia. From scientists to lawyers and detectives, many professionals share this mission. By studying their methods, we can, therefore, learn valuable lessons.

The Proven Framework of Steps to Generate Insights

In 1939, advertising executive James Webb Young introduced a five-step process for generating insights. This method, rooted in innovation science, offers a clear roadmap:

1. Gather raw materials.
2. Work the materials over in your mind.
3. Let your subconscious incubate the ideas.
4. Experience the “aha” moment when insights surface.
5. Shape and refine the ideas for practical use.

Next, let’s explore each step in detail to understand how to apply them.

Step 1: Gather Raw Materials

James Webb Young emphasized gathering two types of materials: those related to your problem and general knowledge. This step highlights the importance of diverse inputs.

The more varied your sources, the richer your insights. Drawing from multiple disciplines or analogous markets can lead to groundbreaking ideas. For instance, polymath scientist Hermann von Helmholtz exemplified this by applying knowledge across fields, from physics to medicine.

To begin, ask yourself:

• Have I explored diverse perspectives?
• Am I considering outside data sources or lessons from other industries?

Step 2: Work the Materials Over

Once you’ve gathered materials, the next step is to analyze and connect them. Examine the problem from different angles and synthesize your findings.

Young described this step as exploring relationships. Write down tentative ideas, even if they seem incomplete. Furthermore, this process helps pave the way for the final insight.

Key tips:

• Turn facts over in your mind.
• Look for unexpected connections.
• Document your thoughts for future refinement.

Step 3: Incubate the Ideas

Sometimes, stepping away from the problem can unlock creative solutions. For example, taking a walk, playing music, or even sleeping can trigger insights.

Studies confirm that walking boosts creativity. Similarly, comedian and writer John Cleese advocates for unplugging to let the unconscious mind work. By doing this, you allow ideas to percolate.

To make the most of this phase:

• Schedule breaks during your workflow.
• Allow your subconscious time to process ideas.
• Take a walk or engage in unrelated activities.

Step 4: Experience the “Eureka” Moment

The “aha” moment often comes suddenly, bringing clarity and confidence. Historical examples abound, such as Archimedes in his bathtub or Isaac Newton in an orchard.

These moments are the result of preparation, analysis, and incubation. They mark the connection of seemingly unrelated ideas into a meaningful insight.

Characteristics of an insight include:

• Sudden appearance
• A positive emotional response
• Confidence in its validity

Step 5: Shape and Refine the Idea

After the insight surfaces, the final step is to refine it for practical application. Share it with others for feedback. Collaboration often reveals overlooked possibilities and strengthens the final concept.

Young advised not to hold ideas too tightly. Instead, feedback helps expand and improve them.

To refine your idea:

• Seek input from peers with diverse expertise.
• Test its feasibility.
• Address biases and confirm assumptions.

Checklist for Better Insights

To integrate this framework into your workflow, use this checklist:

• Have you gathered diverse and relevant materials?
• Did you explore multiple hypotheses and connections?
• Did you schedule time for unconscious processing?
• Did you step away to let ideas incubate?
• Have you shared the idea for feedback and refinement?

Embrace the Process and Improve Outcomes

In conclusion, by following these steps, you can consistently generate deeper, more impactful insights. Adjusting your workflow to include diversity, incubation, and collaboration will enhance the quality of your ideas. Therefore, use this method on your next project and watch the difference it makes.

This blog post originally ran in QRCA views. You can read the un-abridged version, complete with references, here.

Have you ever reached a conclusion that seemed logical, but later turned out to be wrong?

As humans, we are hardwired to find patterns and make quick decisions. While this skill is valuable, it can sometimes lead to incorrect conclusions.

When working in medicine, wrong assumptions can have fatal consequences. In policing and law, they can lead to wrongful convictions. In insights, they can result in bad advice that harms businesses.

So how can we avoid these pitfalls? By adopting two proven strategies: diversity and doubt. Seeking diverse perspectives and questioning assumptions leads to deeper understanding and more meaningful insights.

Learning from Other Sense-Makers

Doctors, lawyers, detectives, intelligence analysts, and scientists are all sense-makers. Like market researchers, they gather information, analyze it, and make decisions.

Over time, their work has revealed effective strategies for reducing bias. Let’s explore three key methods they use to battle unconscious bias.

Three Ways to Battle Bias

1. Teamwork

Collaboration exposes blind spots and helps avoid faulty conclusions. In medicine, teamwork shows up in practices like hospital rounds and peer reviews.

2. Doubt

Decision scientists have found that questioning assumptions and attempting to disprove them is highly effective for reducing bias.

3. Checklists

A checklist ensures that teamwork and doubt are consistently applied. It provides a structured process to prompt critical thinking and minimize errors.

Why Our Brains Seek Patterns

Before diving deeper into these strategies, it’s essential to understand our natural biases.

“Our brains are belief engines: evolved pattern-recognition machines that connect the dots and create meaning out of the patterns that we think we see,” writes Michael Shermer in The Believing Brain.

This evolutionary trait helps us work efficiently. However, it also means we often rely on expectations of patterns—even when they aren’t real.

Read the phrases in this diagram.

Simple, right? Paris in the spring. Once in a lifetime. Bird in the hand.

But look more closely at the text in the triangles: the articles “the” and “a” appear twice in each phrase. Our brains focus on the expected sequence and ignore the evidence that doesn’t fit that pattern.

Understanding Confirmation Bias

Humans are wired for confirmation bias, defined by the Oxford English Dictionary as the tendency to seek or favor information that supports existing beliefs while rejecting what challenges them.

This bias isn’t new. Julius Caesar captured its essence when he observed that humans are “quick to believe that which they wish to be true.”

Spotting confirmation bias in others is easy. For example, product managers may dismiss focus group feedback that contradicts their assumptions, labeling participants as “not the right people.” However, identifying bias in ourselves is far more challenging.

To tackle this, we need strategies to recognize and address bias. One of the most effective tools is teamwork.

Teamwork: Leveraging Diversity to Battle Bias

Why Diversity Matters

Diverse teams are essential for overcoming bias. Groups with varied backgrounds, experiences, and perspectives expose blind spots and challenge shared assumptions.

The National Research Council’s report, Intelligence Analysis for Tomorrow, emphasizes this:
“When people with heterogeneous backgrounds work together, their perspectives emerge in different ways, allowing more knowledge and solutions to emerge.”

Diversity can take many forms, including:

• Subject-matter expertise
• Functional background
• Personal experience
• Mission perspective

Practical Steps for Researchers

For researchers, especially those working remotely or solo, collaboration may require extra effort:

• Invite stakeholders from sales, CX, operations, and other departments to a 45-minute insight session.
• Schedule 10–15-minute one-on-one calls with team members for their unique input.
• Seek external perspectives through reviews, articles, and social media analysis, including opinions from both fans and critics.

Yes, collaboration takes effort. But the cost of missing crucial insights is far higher.

Doubt: Shifting Perspectives for Better Decisions

The Power of Doubt

The impact of doubt is transformative, even if at first it might seem negative. As Voltaire said, “Doubt is an uncomfortable condition, but certainty is a ridiculous one.”

Doubt relies on metacognition—awareness of your own thought processes. In medicine, this has proven lifesaving.

Drs. Eoin O’Sullivan and Susie Schofield, in Cognitive Bias in Clinical Medicine, highlight the importance of asking, “What else could this be?” This practice forces consideration of alternative diagnoses and improves decision-making.

Doubt in Forecasting

In intelligence forecasting, doubt also improves accuracy. Philip Tetlock and Dan Gardner, in Superforecasting, found that asking people to question their initial judgment and make a second estimate significantly increases accuracy.

Doubt is powerful but not always intuitive. That’s where checklists come in.

Checklists: A Process for Battling Bias

Why Checklists Work

In order to combat bias, checklists can be a powerful tool. They provide a structured process to prompt critical thinking and ensure diverse perspectives are considered.

Here is a simple checklist that, when employed, reminds us to have diversity and doubt when engaging  with teams.

If we can say “yes” to each of these simple questions, we can be even more sure that our learnings will help and not hinder our clients.

You might think to yourself: “Seriously, though, a checklist, really?”

Why Do We Resist Checklists?

For many professionals, the idea of using a checklist feels like an insult. It may seem to suggest that we’re incapable of doing our job without one.

That’s exactly what renowned surgeon and author Atul Gawande thought initially. Gawande, known for his groundbreaking book  The Checklist Manifesto: How to Get Things Right has dedicated much of his career to improving healthcare systems and surgical outcomes through checklists.

Atul Gawande’s Journey with Checklists

The Evidence Behind Checklists

Gawande’s research emphasizes how checklists ensure that critical steps aren’t missed. They help prevent errors, enhance accuracy, and improve teamwork. In healthcare, this approach has saved countless lives and improved surgical outcomes.

Doubting the Value of Checklists

Despite the evidence, Gawande was skeptical when implementing a surgical checklist in his own practice. He described his initial doubts:

“Did I think the checklist would make much of a difference in my cases? No. In my cases? Please. To my chagrin, however, I have yet to get through a week in surgery without the checklists leading us to catch something we would have missed.”

One week alone revealed three critical catches across five surgeries—all thanks to the checklist.

Why Checklists Work

Catching What We Miss

Checklists work because they force us to confront overlooked details. As Gawande discovered, even the most experienced professionals benefit from this structured approach.

Overcoming Resistance

Many of us believe we’ve already internalized essential steps. However, when we stop and honestly go through a checklist, we often realize we’ve missed something important—or could improve our process.

How Checklists Lead to Better Insights

Addressing Bias

Generating insights is not as simple as it seems. Bias, whether unconscious or otherwise, can easily cloud judgment. Without purposeful effort, its effects often go unnoticed.

Leveraging Diversity and Doubt

Combining checklists with diverse perspectives and critical questioning helps mitigate bias. Diverse teams reveal blind spots, while doubt encourages us to challenge assumptions. So, having diversity and doubt it essential to improving insights.

Putting Checklists to the Test

Try It for Yourself

Don’t just take Gawande’s word for it—try a checklist in your own work. Whether you’re conducting research, analyzing data, or leading a team, a checklist can help you avoid errors and deliver better results.

Battling Bias for Better Insights

Checklists, combined with diversity and doubt, are essential tools for generating meaningful insights. They take time and effort to implement but are invaluable in avoiding errors and helping teams achieve success. Therefore, by using a checklist you not only improve your process but also ensuring you’re delivering accurate, reliable outcomes for your clients.

Learn more about Angus Reid Group

The Art of Asking Questions Meets the Science of Reliable Answers

Crafting insightful questions is an art. But ensuring the data you get is reliable? That’s pure science. Reliable insights depend on consistent responses—when you ask the same question of different but identical samples, the answers should align. If they don’t, it’s a red flag indicating a deeper issue with your data source.

Unfortunately, data reliability in the insights industry is under siege. While fraudsters, click farms, and disengaged respondents have long been a concern, the rise of AI-driven bots has created an unprecedented crisis. These bots infiltrate sample exchanges, wreaking havoc on data integrity. The result? Insights that are far from reliable.

The good news is that you can take steps to protect your research. Start by asking your sample provider these essential questions:

• Where do the respondents in your sample come from?
• What evidence supports the reliability of your data?
• How do you detect and eliminate AI-driven bots and other bad actors?

Understanding Respondent Sources: Why It Matters

The origin of your sample is critical to data quality. If your respondents aren’t the right fit, your insights won’t be accurate. For example:

• Loyalty Program Bias: Is the sample sourced from loyalty card members, who might have shopping habits skewed by their program affiliations?
• Sample Exchange Risks: Are respondents randomly redirected from other surveys they didn’t qualify for?
• AI-Driven Fraud: Are bots completing your surveys just to claim incentives?

At Angus Reid Group (ARG), we mitigate these risks through careful recruitment. We attract respondents via targeted advertising for the Angus Reid Forum, rigorously profile them, and require participation in engagement surveys before they’re accepted into our community. This process ensures a high standard of respondent quality, making our data reliable and actionable.

Evidence of Reliability: Proving Data Consistency

Before trusting a sample provider, demand evidence of reliability. Can they demonstrate that consistent questions yield consistent answers across different samples? This is a hallmark of dependable data.

At ARG, we continuously test the reliability of our data through our omnibus surveys. These surveys rotate a set of simple questions over time, allowing us to measure consistency. Our results show minimal variation—well within the expected range for nationally representative samples of 1,500 adult Canadians.

The takeaway? Boringly straight lines on a graph signal boringly reliable data. And in research, reliability is never boring—it’s essential.

Combatting AI-Driven Threats: Protecting Data Quality

AI-driven bots have introduced a new layer of complexity to sample integrity. Addressing this requires robust detection and prevention methods. Ask your provider:

• How do you identify bots, speeders, and straight-liners?
• What safeguards are in place to ensure responses come from real, engaged individuals?

At ARG, we’ve developed proprietary fraud detection methods combining AI, algorithms, and carefully designed question types. These tools help us identify and eliminate bots and bad actors, ensuring the reliability of our data.

You can see from this graph that the answers are consistent—with any variation being within what you’d expect with nationally representative samples of 1500 adult Canadians. Those straight lines are boring, but they are boringly reliable.

Ask any potential supplier to provide similar evidence of reliability. If they don’t have it, ask yourself why not?

As the old proverb says: “Trust, but verify.”

Battling Bots: Protecting Your Research from AI-Driven Threats

In the ever-evolving world of research, ensuring reliable data has become more challenging than ever. While speeders—respondents who rush through surveys—and straight-liners—those who answer with repetitive patterns—are relatively easy to spot, AI-driven bots present a whole new level of complexity. These bots mimic human behavior, making them much harder to identify.

To combat this, you need to ask the right questions of your sample provider. Transitioning from traditional detection methods to advanced AI-focused tools is essential in staying ahead of the game.

Why AI-Driven Bots Are a Growing Problem

In the past, low-quality responses like “asdfg” or “yuio” in open-ended questions made identifying fraudulent participants straightforward. But now, bots have become smarter, producing responses that appear nuanced and intentional. This evolution in technology has raised the stakes for researchers seeking reliable data.

Transitioning from simplistic detection methods to advanced bot-detection strategies is no longer optional—it’s a necessity.

Key Questions to Ask Your Sample Provider

When vetting a potential sample provider, always ensure they are prepared to combat AI-driven bots. Here are the essential questions you should ask:

1. What tools are you using to detect bots?
   Providers should have advanced methods in place to detect sophisticated bot behavior.
2. How are you adapting as bots evolve?
   Detection tools must evolve alongside the ever-changing capabilities of AI-driven bots.

Transitioning from a static approach to a dynamic, proactive strategy is critical to maintaining data quality.

How ARG Battles Bots: A Multi-Dimensional Approach

At Angus Reid Group (ARG), we use a comprehensive and constantly evolving toolkit to protect data quality. Our strategies include:

• Advanced Pattern Detection Algorithms: These identify subtle irregularities in response behavior.
• Knowledge Traps: Designed to catch bots, these are questions bots cannot answer correctly but real people can.
• Hidden Questions: These are invisible to human respondents but detectable by bots, helping us identify bot behavior.
• Timestamp Analysis: Tracks how quickly respondents complete the survey to flag speeders.
• AI-Driven Open-Ended Analysis: Looks for duplicate responses and cross-references with web sources for authenticity.
• Duplicate IP Detection: Flags multiple responses from the same IP address.
• Continuous Innovation: Our approaches evolve to stay ahead of emerging bot tactics.

By combining these tools, ARG takes a multi-dimensional approach to determine which responses are trustworthy and which need to be removed.

The Foundation of Research: Reproducibility

Transitioning to a discussion of broader research principles, reproducibility is the cornerstone of reliable insights. If the quality of your sample is compromised, the entire foundation of your research becomes shaky. To ensure robust data, you must verify the reliability of your sample source.

Three Essential Questions to Ensure Reliable Data

To protect the integrity of your research, ask these three critical questions of your sample provider:

1. Where do your respondents come from?
   Understanding the recruitment process is crucial for ensuring you’re reaching the right audience.
2. What evidence supports your data reliability?
   Providers should demonstrate consistent results across multiple surveys and samples.
3. How do you detect and eliminate bots and other bad actors?
   Knowing their strategies to address fraudulent responses will give you confidence in their data quality.

With good answers to these inquiries, you can be sure the answers to your questions will be reliable too. Trust, but verify.

Questions? Contact ARG today! 

How AI is Revolutionizing Market Research

The landscape of market research is quietly undergoing a profound transformation, driven by the power of Artificial Intelligence (AI). This shift represents more than just technological progress; it is a complete reimagining of how organizations analyze and leverage data. What was once the domain of specialists is now accessible to all, as AI democratizes data analysis and turns it into a universal, interactive language.

From Tool to Catalyst

Far from being a mere tool, AI serves as a catalyst for change. Traditionally complex and siloed, data has become more accessible, enabling every department to engage with it meaningfully. As a result, team members across various functions are becoming capable data interpreters. Meanwhile, specialists use AI to conduct deeper analyses at scale, pushing the boundaries of what’s possible.

Redefining the Way We Work

The influence of AI goes beyond increasing efficiency; it redefines how we approach our work. By simplifying data and encouraging collaboration, AI fosters a sense of purpose and engagement across teams. This transformation inspires creativity and cultivates a culture where challenges are viewed as opportunities for innovation. Ultimately, AI helps teams find joy in problem-solving while driving impactful results.

Transforming Teams and Processes

The integration of AI into research teams revolutionizes their ability to uncover hidden insights. This technology enhances both the depth and efficiency of their findings, allowing for more precise and actionable client deliverables.

In panel management, AI simplifies complexities like participant recruitment, response rate optimization, and satisfaction analysis. It serves as a guiding compass, streamlining intricate processes and improving outcomes.

Additionally, operational, marketing, and sales teams benefit from AI’s ability to refine workflows, foster stronger client engagement, and adapt to market trends with agility. These departments now operate with greater precision and creativity, leveraging AI to innovate continuously.

Addressing AI’s Challenges

Like any transformative technology, AI has its limitations. Occasionally, it may produce inaccuracies or “hallucinations.” However, as an evolving tool, AI continually learns and improves, making it increasingly reliable over time. By embracing this partnership between human expertise and AI capabilities, organizations can navigate these challenges effectively while driving progress.

A Shared Vision of Innovation to Revolutionize Market Research

At Angus Reid Group and across the broader market research industry, AI is redefining the journey toward insight and innovation. By making data insights accessible to all, it empowers teams to collaborate more effectively and engage deeply with their work.

This evolution is not just a technological upgrade—it’s a shift in how organizations connect with data and generate actionable insights. AI inspires a future where data informs and unites, weaving a comprehensive tapestry of understanding that is both accessible and transformative.

The revolution is here, and it’s centered on empowering people. With AI as an ally, organizations can lead with insights, innovate with purpose, and shape a brighter, more informed future for all.

Learn more about Angus Reid Group – Research Leaders