4 ways AI prompt tracking data gets misread

SEOGami

Regular Member
Jr. VIP
Joined
Feb 28, 2024
Messages
236
Reaction score
163
Tom Capper laid out four prompt-tracking mistakes on this Moz Whiteboard Friday, and the useful part is the measurement, not the product pitch.

The big one is treating a citation as the win. His phone example cites PCMag while brand names sit in the answer. If you are Apple, apple.com missing from the source line is not a loss. These replies barely send clicks. Being named is the awareness play, and a sub-brand like iPhone can matter more than who got the link.

Second, do not score it like a rank. First mention is context. Over a large prompt set I would rather know what share of answers mention the brand at all, and what the wording around it actually says.

Third is volume. A site tracking tens of thousands of keywords should not watch 50 prompts and call that coverage. Answers move more day to day than SERPs. A local or very narrow business might be fine at 50. Match the scale you already use for keywords.

Last, do not dump head terms in. Almost nobody opens ChatGPT and types "laptops". He cites the old stat that about 15% of Google queries are brand new each day, and guesses prompts are closer to 90% one-offs. Pick topics, then write prompts people would actually ask.

SEOGami
 
I measure this from the publishing side, and the count misleads in the same way. I log citations per engine and per prompt; over one prompt set an engine returned 400+ citation events where a second returned zero, same answers, same mentions. That ranking is the engine, not us. What moved decisions was the share of answers where the name appears in the text, split by intent, and whether a price sits beside it. Vendor-written prompt sets also inflate whatever the vendor sells.
 
prompt tracking data is messy lol, i take those numbers with a grain of salt unless the sample is huge imo
 
The point about citations not always being the main win is interesting. A brand mention in the actual answer can matter more than getting a link in the sources. Prompt tracking needs to measure visibility not just citations.
 
I measure this from the publishing side, and the count misleads in the same way. I log citations per engine and per prompt; over one prompt set an engine returned 400+ citation events where a second returned zero, same answers, same mentions. That ranking is the engine, not us. What moved decisions was the share of answers where the name appears in the text, split by intent, and whether a price sits beside it. Vendor-written prompt sets also inflate whatever the vendor sells.
imo, the price next to the brand is worth checking too. If an AI answer mentions your brand with a price that's months out of date, the dashboard might count it as a win, but a potential customer could get the wrong impression.
 
Keep a fixed set of prompts for weekly comparisons. Changing the prompt set makes it harder to tell whether brand visibility has actually improved.
 
Back
Top