Can Your Customers Spot AI Creators in Your Ads? (We Ran the Test)

A while back I ran an AI creator in a client ad and quietly waited for someone to call it out in the comments. Nobody did. What spooked me was that I'd been bracing for the opposite.

I'd convinced myself the audience would smell it instantly. The slightly-too-smooth face, the voice that lands a touch flat, the eyes that don't quite track. I was sure the comments would fill up with "this is AI" within the hour. I'd even half-prepared a reply.

That reply never got used. And the lesson sat with me, because it ran against everything I assumed about trust and ads.

So we turned it into a proper test. Here's what we found.

The setup: a real spot-the-AI exercise

The idea isn't original. There's a well-known demo where a brand drops six creators into one ad, half of them AI-generated, half of them real people, and asks you to guess which is which. Most people get it wrong. I'd seen it and quietly assumed it was a stitched-up party trick.

So we replicated it with our own creative, on a real account, with real spend behind it.

We took a homewares client sitting at a ~$55 AOV and built two versions of the same testimonial-style ad. Same script. Same hook. Same product beats. One used a real UGC creator we'd worked with before. The other used an AI-generated creator built off the identical script.

Then we did two things. We ran them both in the account as a clean split so we could compare the numbers. And we watched the comments on each like hawks, logging every reply that mentioned anything looking off, fake, or "AI".

I genuinely didn't know which way it'd go.

What the comments actually said

Here's the thing that surprised me most. Across thousands of impressions on the AI version, we got close to zero comments calling it out as AI. Not one in the first few days. A handful later that we couldn't even cleanly attribute to the AI tell versus general scepticism ("looks like an ad").

To put that in perspective, the real-creator version got the normal mix of ad scepticism too. People are wary of any testimonial now, AI or not. The baseline distrust was roughly the same on both.

That lines up with something one media buyer put really bluntly: when people are scrolling the feed, they're in their least alert state. They're on the sofa, in bed, half-watching. They're not auditing your creator's jawline for render artefacts. If a trained marketer has to squint to catch it, a tired consumer mid-scroll has no chance.

So the premise behind the spot-the-AI panic, that customers will notice and feel betrayed, mostly didn't hold at the comment level. The trust erosion I was bracing for didn't show up in the replies.

But comments are only half the story. Sentiment isn't performance.

What the numbers said (and where AI lost)

The split told a more interesting story than the comments did.

On this particular ad, the real creator beat the AI one on the metric I care about most, which is whether people stick around long enough to buy. The AI version pulled a slightly cheaper cost per click. The real one converted better once they landed.

My read: the AI creator was good enough to stop the scroll and earn the click, but the real one carried a bit more conviction through to the decision. On a considered purchase, that gap matters.

I'd seen this pattern flagged before I ever ran the test. A few experienced buyers have said the same thing out loud. They haven't yet found an AI testimonial they'd happily run as a single standalone ad. Not because audiences catch it, but because raw single testimonials of any kind have softened over the last couple of years. AI or human, a lone talking-head testimonial just doesn't carry an ad the way it did.

Where the AI version genuinely won was earlier in the funnel and inside mashups. Short clips of an AI creator, cut into a sequence with proven-performing footage, product shots, and on-screen text, held up fine. Nobody flinched. And it cost us a fraction of the time.

That's the real split. Not "AI versus real". It's "where in the ad does this clip sit".

When AI beats a real creator, and when it's the reverse

Here's my take after running this on a few accounts now.

AI tends to win when:

• You need volume of variations fast. Ten angles by Friday, not three. A real shoot can't keep up.
• The clip is a building block, not the whole ad. Hooks, B-roll, a 6-second beat inside a mashup. The AI does its job and disappears.
• You're testing unproven angles you don't want to spend a creator's day rate on. Validate the message cheaply, then commission the real version of the winner.
• The product is an impulse buy where conviction matters less than stopping the scroll.

A real creator tends to win when:

• It's the hero standalone testimonial that has to carry the whole ad on its own.
• The purchase is considered or higher-priced, where a bit more human conviction nudges the conversion.
• The angle leans on genuine, specific lived detail that AI tends to flatten into generic praise.
• You're building a creator relationship you'll come back to for the brand, not just one ad.

The mistake I made at the start was framing it as a loyalty test. Real good, AI suspect. In reality it's a casting decision. You're matching the tool to the job, the same way you'd cast a face to a brief.

What I'd actually do in your account

Don't launch an AI creator as your one big standalone testimonial and expect it to outperform a strong human one. That's the spot where, in our test, the real creator still won on the number that matters.

Do use AI to fill your content library. Generate variation clips, hooks, and B-roll, then weave them into mashups with footage you already know performs. That's where it earned its keep for us with no downside in sentiment.

And run your own version of the test before you trust anyone's verdict, mine included. Build the same ad two ways, split them cleanly, then watch both the comments and the conversion. Your audience, your price point, and your category will tell you more than any demo will.

The thing I'd flag is how easy it is to fool yourself either way. Panic that customers will revolt, and you leave a cheap, fast tool on the table. Assume AI is a free lunch, and you'll quietly hand your hero ad to a clip that can't carry it.

If you'd like a hand reading which of your creatives are doing the heavy lifting and which are just filling space, a Signal/Noise Audit pulls your creative history apart angle by angle and shows you where the real and the synthetic each belong. No pressure either way. Mostly I'd just love to know what your own spot-the-AI test turns up, because every account I've run it on has surprised me a little.